Yes, I think it should be compared to GPT 3.5. Google doesn't really have a public contender to GPT 4 yet but this is what the Gemini project is intended to be, by the newly formed/reformed Google DeepMind team.
I don't think Bard can even perform at GPT 3.5 level, at least, not in my experience. Bard hallucinates pretty bad and you can't really be sure whether it's correct or not. Now, that's a given with any LLM, but in my experience, Bard usually gives you at least 1 wrong sentence in any of the results it produces whereas GPT 3.5 is mostly correct at surface level questions, but tends to hallucinate if you try to dig in deeper.
But, Bard can give you images as an output as well with the links to those images which may or may not be the correct links and GPT 3.5 can't do that. Still, in any general case, I'd say GPT 3.5 is way more reliable than Bard.
I'm really excited about Gemini. It's supposed to have some sort of planning and problem solving ability[2][3]. If it is like Tree of Thoughts [1], then I'm so hyped!
Also take a peek at which institution most of the ToT authors are at.
It may be 3.5-like, however when I tried it, it seems like it has a lot more handcuffs applied for “safety,” and I don’t mean safety from Skynet and Terminators. I mean things like it refuses to speculate about what medical condition might cause some given symptoms. “Sorry, I’m just a language model.” GPT had no problem giving it an educated guess.
I personally find this type of “safety“ to be patronizing and insulting, but I’m sure there are people who would prefer government regulation banning the use of language models for various and sundry “inappropriate” questions in order to protect humans who have no common sense. Anyway, in this condition, Bard’s a no for me.
> I mean things like it refuses to speculate about what medical condition might cause some given symptoms.
Are you sure about this specifically? I’ve recently had zero trouble getting GPT-4 to give potential diagnoses for a given set of symptoms, though perhaps it’s an issue of prompting.
