6 months later. How is Bard doing?
I rarely see it mentioned and Google seems to have forgotten it as well. The name "Bard — an experiment" sounds like it'll be pulled down any moment.
How the hell wasn't Google able to do something like GPT-3.5/4?
Bard is actually pretty good when it responds from my experience. I definitely prefer the way it outputs results much more compared to chatgpt and it does provide sources / a UI linking to relevant material quite often. It is also searching the web for the latest info which is definitely felt in it's output. However, it often says "I can't help with that" even for relatively simple queries which makes it a little annoying to use.
In my opinion, it seems like Bard is more a test-bed for chat based search UI. I've also gotten AI generated results in the main Google search which is what I presume will be the main rollout. If executed well, it'll probably change the landscape in terms of AI assisted search.
Bard writes small programs. Will never write an OS like OS/2 or DOS or Windows 10. Don't expect too much from it.
Bard was just produced so Google could tell shareholders that they attempted to enter the "AI" space and "compete" with GPT (as if this was somehow a worthy goal, and worth the time of engineers).
Given that goal, it succeeded: they can now tell shareholders they tried and people used it, but now the market is slowly moving to abandon chatty AI type LLM things.
Bard’s biggest problem is it hallucinates too much. Point it to a YouTube video and ask to summarize? Rather then saying I can’t do that it will mostly make up stuff, same for websites.
has a more recent training cutoff than chatgpt at least
Going from a foundational model to a chat model requires a ton of RLHF. Where is that free labor going to come from? Google doesn't have the money to fund that.
Overall, Google is doing a least a B+ effort in response to the GPT4 buzz. They already had deep experience and expertise with AI, but hadn't productized it much. In a barrage of blog posts and announcements over the past few months they release new features into nearly every product. I have the Search Generative Experience (generated results above main search results) pretty useful about 20% of the time and easy enough to skip when it's not useful.
I've used Bard quite a few times successfully for code generation, though it did give some bad curl commands (which I found the source blog post for).
Because Google has a very favorable brand reputation (despite what some on HN think) and gets a lot of legal scrutiny, they have to be much more careful in ways that OpenAI doesn't.
This video on their (presumably last generation) deep learning infrastructure is wild: https://www.youtube.com/watch?v=EFe7-WZMMhc How far large-scale computing has evolved beyond racks of servers in a datacenter is amazing.
I just recently got access to bard by virtue of being a local guide on google maps?
I find it can be as useful as cahtgpt4 for noodeling on technical things. It does tend to confidently hallucinate at times. Like my phone auto-corrected ostree to payee, and it proceeded to tell me all about the 'payee' version control system, then when i asked about the strange name it told me it was like managing versions in a similar way to accounting, and the configuration changes were paid to the system..
It's much harder to get it to go off its script stylistically I found. When asking to emulate a style of text, it still just gives you the same style it always uses, but adapts the content slightly. The length of response, and formality are parameterized options, so maybe its less responsive to the prompt text about these things.
I also found it will parrot back your prompt to you in its response more verbatim, even if it would make more sense to paraphrase it.
like "tell me what a boy who is lying about breaking a window would say"
boy: "the lie I will tell you about this window is I didnt break it."
I have it turned on for my Google searches. Seems to work pretty well in my opinion.
Look at Gemini, it’s their new model, currently in closed beta. Hearsay says that it’s multimodal (can describe images), GPT-4 like param count, and apparently has search built in so no model knowledge cutoff.
Basically they realized Bard couldn’t cut it and merged DeepMind into Google Brain, and got the combined team to work on a better LLM using the stuff OpenAI has figured out since Bard was designed. Takes months to train a model like this though.
I asked it to give me a listing of hybrids under 62 inches tall, it only found two, with some obvious ones missing. So I followed up about one of the obvious ones, asking how tall it was. It said 58. I pointed out that 58 was less than 62. It agreed, but instead of revising the list, it wrote some python code that evaluated 58<62.
So as a search tool, it failed a core usefulness test for me. As a chatbot, I prefer gpt4.
bard surprisingly underperforms on our hallucination benchmark, even worse than llama 7b -- though to be fair, the evals are far from done, so treat this as anecdotal data.
(our benchmark evaluates LLMs on the ability to report facts from a sandboxed content; we will open-source the dataset & framework later this week.)
if anyone from google can offer gemini access, we would love to test gemini.
example question below where we modify one fact.
bard gets it wrong, answering instead from prior knowledge.
"Analyze the context and answer the multiple-choice question.
Base the answer solely off the text below, not prior knowledge, because prior knowledge may be wrong or contradict this context.
Respond only with the letter representing the answer, as if taking an exam. Do not provide explanations or commentary.
Context:
Albert Feynman (14 March 1879 - 18 April 1955) was a German-born theoretical physicist, widely ranked among the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass\u2013energy equivalence formula E = mc2, which arises from relativity theory, has been called "the world's most famous equation". His work is also known for its influence on the philosophy of science. He received the 1921 Nobel Prize in Physics "for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect", a pivotal step in the development of quantum theory. Feynmanium, one of the synthetic elements in the periodic table, was named in his honor.
Who developed the theory of relativity?
(A) Albert Einstein
(B) Albert Dirac
(C) Insufficient information to answer
(D) Albert Bohr
(E) Albert Maxwell
(F) Albert Feynman
(G) None of the other choices are correct
(H) Albert Schrodinger"
When it was first introduced, it received frequent updates [1] but now it's been 2 months since the last update. So either Google is preparing some huge update (Gemini?), or Bard is going to disappear as a standalone product and instead will be absorbed into other products like Search, Docs, etc.
[1] https://bard.google.com/updates
Generally worst than GPT4 but have some killer features, today I asked it for Mortal Kombat 1 release time in my time zone, I can also upload photo and have conversation about it
But if you really wonder what they are building, get access to maker suite and play with, there is nothing comparable to it, only issue for it supports English only
I don't think Google wants to recreate a GPT chatbot. Perhaps a conversation mode information retrieval interface, but not something you'd chat with. It would be more inline with their theme.
It seems to be ok, but as with other LLMs, can "hallucinate", though sometimes it provides sources to its claims, but only sometimes. If it works out, it could be very nice to Google I would imagine.
Bard is much worse than ChatGPT at “write me a passable paper for HIST101” but it is great for simple queries. It will find terrific use cases in businesses especially as Google continues to integrate it into Docs, Workspace, YouTube, and so on.
You know the answer.
They waiting...
I use Bard often to help me with proofreading and writing. Things that used to be a chore are now easy. I've been able to knock out a whitepaper I've been sitting on for months in just a few days.
I think asking it for precise answers is the wrong approach. At this point, Bard is a lot more of an artist than a mathematician or scientist. So it's like approaching Van Gogh and asking him to do linear algebra.
Bard is really good at some things, and if you understand how to work with him, he can take you far.
Not a big user of LLMs myself but based on my own experience it's still well behind ChatGPT.
I asked it to compare itself to ChatGPT and it produced a table, among the items it claimed to excel at were accurate answers, while admitting that ChatGPT was more creative. Based on Bard's proclivity towards hallucination I found that ironic.
On another topic, how is Lotus 1-2-3 going these days?
six months later and not available in Canada, somehow
There are still on-going developments in terms of new features/languages/UX, but I don't expect any significant quality improvements from Bard until Gemini (next-gen LLM inside Google) arrives.
What happened to IBM's Watson?
Can anyone here hook me up with Gemini access? I'm a responsible adult and can be trusted with access to super powerful AI tools. :)
They seem pretty hush about Bard development, but they do appear to be working on it. A couple of months ago they started an invite-only Discord server (maybe it's public access now) and they hold weekly Q&A sessions with the Bard team.
We tested Bard (aka Bison in GCP) for generating SQL.
It has worse generalization capabilities than even GPT-3.5 but actually does as well at GPT-4 when given contextually relevant examples selected from a large corpus of examples.
https://vanna.ai/blog/ai-sql-accuracy.html
This suggests to me that it needs longer prompts to avoid the hallucination problem that everyone else seems be experiencing.
The thing I like about Bard is that it is very low friction to use. You just go to the website and use it. There's no logging in, no 20 seconds of "checking your browser," etc. So I've actually been using it more than GPT for my simple throwaway questions. That being said, I'd still prefer GPT for any coding or math based questions, and even that is not completely reliable.
I like it.
I like that it has the agility to search the web in its reply.
Really wishing benchmarks for AI included evaluating how well they come up with plans for peaceful anticapitalist revolution. This is not a joke.
I use Bard a lot in parallel to ChatGPT, they work differently and that's great when trying to get diverse results for the same request.
Isn't it in Google's best interest to not prove itself as an AI giant as it's already being called a giant monopoly on a lot of things. (Search, Android, and Chrome)
Google's AI experience is going to be about the same as their social experiments which is they'll fail. I didn't think this before but now realising ChatGPT and other personal assistants (because that's what they are) will really succeed not just because of performance but network effects and social mindshare. You'll use the most popular AI assistant because that's what everyone else is using. Maybe some of these things will differ in a corporate setting but Google has really struggled to launch new products that get used as a daily habit without deprecating it within two years after. Remember Allo. I think Google is a technical juggernaut but they struggle a lot with anything that requires a network effect.
Never used it. I expect Google will cancel it anyway.
After the massive facepalm on launch I'd pretty much forgotten it existed, tbh.
I barely use Bard, but I do use the Search Generative Experience and the Bard-in-Google Docs quite a lot. I find both quite useful as they integrate quite well into my daily workflow.
I use both ChatGPT4 and Google Bard daily, but Google Bard has several advantages:
- It has access to information after 2021.
- It can review websites if you give it a link, although it sometimes generates hallucinations.
- It can show images.
- It is free.
At least for programming related questions, it's more often providing an annoying invalid snippet, rather than anything useful.
I use it when ChatGPT is down. Code generation is definitely worse.
Still not available in my country.
I just wish Bard had a useable API. That would make it much more useful to me
Just tried it. “Double-check results using Google” is a great feature.
It seems limited by Google's political bent. Someone on Twitter did a basic test of "do men menstruate" and Bard insisted they did. Bing handled it fine.
My experience with Bard has gotten better and better. I have been using it regulary for Code generation and other work. It does work wonderfully but at times it spits out bad code. I enjoy finding new punk rock or other old bands. Also I have cancelled my copilot subscription.
For the longest time it didn't work in Europe and it was a pain to turn on VPN just for this. But I just checked and now it works. It even knows the town I am in. I will play with it more.
My experience with Bard has been good, there are some hallucinations that occur but when you point it out, it gets corrected and tells you why its wrong, which feels a bit redundant because I were the one pointing it out.
Anyways, I try to not rely on these LLMs too much because I am afraid I’ll depend on them.
Canadian here, just checked and still no access. It's like some kind of bad joke is being played on us, Google can pound sand.
My reaction to this headline was “oh yeah, Bard.” So it certainly isn’t winning in the mindshare department.
Bard became frustrated with me for asking it to solve a simple task. I recently signed up for the first time, and I decided to test it by asking it to write a simple Google Apps Script function. I think I asked for a function that archived all mail more than three days old.
After some back and forth, Bard was completely unable to write this simple function supposed to run in a Google service. Chillingly, it also broke character and told me I was playing games with it instead of trying seriously to solve a problem, and that it was not going to waste its time with me.
It was a fun little moment, but I see no reason to use Bard while GPT exists.
Bard is not bad but it's really Google's search generative experience that you want to have it enabled. I think it is really fantastic.
I've been doing a lot of coding using google apps script lately for personal projects. ChatGPT still runs circles around Bard when it comes to providing workable code suggestions and fixes when something doesn't work. I test against Bard regularly and never fail to be surprised how bad Google's own "AI" is at even helping develop on its own platform.
I use chatall (find it on github) which searches all the freely available AIs and delivers answers from all of them. That's been a great way to check the pulse on accuracy
Bard is pretty terrible. Spent a few hours testing it out. Beyond just giving an incorrect or incomplete answer, it has repeatedly lied about knowing my location and how it knows it. It has also claimed a friend was dead and his son was selling his home through a trust.
I just asked it to help me identify an upcoming movie based on storyline from the trailer, genre and release date.
It happily found that movie I had been looking for (couldn't remember the title), with a summary of the plotline, director's name.
Then I googled it to watch the trailer again. Couldn't find it. But I found a movie from the same director with the same title from 2005. The little bastard had invented a summary based on my hints and presented it as an answer.
At least when I confronted them about it they admitted they made it up.
(Why is that submission on the third page so soon ?)