This announcement seem to have killed so many startups that were trying to do multi-modal on top of ChatGPT. The way it's progressing with solving use cases with images and voice, not too far when it might be the 'one app to rule them all'.
I can already see "Alexa/Siri/Google Home" replacement, "Google Image Search" replacement, ed-tech startups that were solving problems with AI using by taking a photo are also doomed and more to follow.
In retrospect, such startups should have been wary: they should have known that OpenAI had Whisper, and also that GPT-4 was designed with image modality. I wouldn't say that OpenAI "telegraphed" their intentions, but the very first strategic question should have been, "Why isn't OpenAI doing this already, and what do we do if they decide to start?"
Yeah I remember watching that and thinking oh I know a cool app idea. What if you just take a video of what food is in your kitchen and Chat GPT will create a recipe for you. I go to the docs and that was literally the example they gave.
I think the only place where plugins will make sense are for realtime things like booking travel or searching for sports/stock market/etc type information.
I have a home-spun version of ChatGPT that uses function calling to connect to my emails, calendar, and notes. This is really useful because I can say "Bob just emailed me to set up a call. Respond to Bob with some available times from my calendar."
It would hard to be more explicit than doing a demo of multi-modality in GPT-4, and having an audio API that is amazing and that you can use right now, for pennies.
It would be interesting to know if this really changed anything for anyone (competitors, VCs) for that reason. It's like the efficient market hypothesis applied to product roadmaps.
It is interesting that these startups did not recognize that the image modalities already existed, as evidenced by their initial GPT-4 announcement underneath “visual capabilities”
[1].
Talking to Google and Siri has been positively frustrating this year. On long solo drives, I just want to have a conversation to learn about random things. I've been itching to "talk" to chatGPT and learn more (french | music theory | history | math | whatever) all summer. This should hit the spot!
Voice assistants have always been a half complete product. They were shown off as a cool feature, then they were never integrated so they were useful.
The two biggest features I want are for the voice assistants to read something for me, and to do something on google/Apple Maps hand free. Neither of these ever work. “Siri/ ok google add the next gas station on the route” or “take me to the Chinese restaurant in Hoboken” seem like very obvious features for a voice assistant with a map program.
The other is why can I tell Siri to bring up the Wikipedia page for George Washington but I can’t have Siri read it to me? I am in the car, they know that, they just say “I can’t show you that while you’re driving”. The response should be “do you want me to read it to you?”
Sometimes google assistant will answer a query I thought for sure it would fail on with a really good answer and other times it will fail the most basic of commands. It's frustrating.
I've replaced my voice google assistant searches with the voice feature of the Bing app. It's a night and day difference. Bing voice is what I always expected from an AI companion of the future, it is just lacking commands -- setting tasks, home automation, etc.
I got sick of searching Google for in-game recipes for Disney Dreamlight because most of the results are a bunch of pointless text, and then finally the recipe hidden in it somewhere.
I used Bing yesterday and it was able to parse out exactly what I wanted, and then give me idiot-proof steps to making the recipe in-game. (I didn't need the steps, but it gave me what I wanted up front, easily.) I tried it twice and it was awesome both times. I'll definitely be using it in the future.
It almost sounds like their assistant and their search engine have the same problem! Years of SEO optimized garbage has polluted search and the data streams it feeds to their other products. I have a concern that soon the mess will turn into AI-optimized trash, with what is essentially data poisoning to get the AI to shovel the fake content instead.
> I got sick of searching Google for in-game recipes for Disney Dreamlight
You mean these? Took me a few seconds to find, not sure how an LLM would make that easier. I guess the biggest benefit of LLM then is for people who don't know how to find stuff.
Yes, but each time, I only actually care about 1 recipe, and it's easier to just search for that recipe than find a list of recipes and then search through that.
Bing made it even easier.
Also, I've found some of those lists to be missing some recipes.
precisely this. once someone figures out how to get something like GPT integrated with actual products like smart home devices and the same access levels as siri/google assistant, it will be the true voice assistant experience everyone has wanted.
My prediction on this is eventually the LLMs will just write and execute scripts directly to control things.
Imagine if iOS had something like apple script and all apps exposed and documented endpoints. LLMs would be able to trivially solve problems that the best voice assistants today cannot handle.
Then again none of the current assistants can handle all that much. "Send Alex P a meeting invite tomorrow for a playdate at the Zoo, he's from out of town so include the Zoo's full address in the invite".
"Find the next mutual free slot on the team's calendar and send out an invite for a zoom meeting at that time".
These are all things that voice assistants should have been doing a decade ago, but I presume they'd have required too much one off investment.
Give an LLM proper API access and train it on some example code, and these problems are easy for it to solve. Heck I bet if you do enough specialized training you could get one of the tiny simple LLMs to do it.
That’s extra frustrating because Siri absolutely had that functionality at some point in the past, and may even still have it if you say the right incantation. Those incantations change in unpredictable and unknowable ways though.
I still don't understand how you can talk to something that doesn't provide factual information and just take it at face value?
The other day I asked it about the place I live and it made up nonsense, I was trying to get it to help me with an essay and it was just wrong, it was telling me things about this region that weren't real.
Do we just drive through a town, ask for a made up history about it and just be satisfied with whatever is provided?
> What LLMs have made me realize more than anything is that we just don't care that much the information we receive being completely factual.
I find this highly concerning but I feel similar.
Even "smart people" I work with seem to have gulped down the LLM cool aid because it's convenient and it's "cool".
Sometimes I honestly think: "just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine...".
> just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine...
It's the same issue with Google Search, any web page, or, heck, any book. Fact checking gets you only so far. You need critical thinking. It's okay to "learn" wrong facts from time to time as long as you are willing to be critical and throw the ideas away if they turn out to be wrong. I think this Popperian view is much more useful than living with the idea that you can only accept information that is provably true. Life is too short to verify every fact. Most things outside programming are not even verifiable anyway. By the time that Steve Jobs would have "verified" that the iPhone was certainly a good idea to pursue, Apple might have been bankrupt. Or in the old days, by the time you have verified that there is a tiger in the bush, it has already eaten you.
There's a lot of truth in this comment and a lot that I wholeheartedly agree with.
When I spend time on something that turns out to be incorrect, I would prefer it to be because of choice I made instead of some random choice made by an LLM. Maybe the author is someone I'm interested in, maybe there's value in understanding other sides of the issue, etc. When I learn something erroneous from an LLM, all I know is that the LLM told me.
The issue is far more serious with ChatGPT/similar models because things that are laughably untrue are delivered exactly the same as something that's solidly true. When doing a normal search I can make some assessment on the quality of the source and the likelihood the source is wrong.
People should be able "throw the ideas away if they turn out to be wrong" but the problem is these ideas unconsciously or not help build your model of the world. Once you find out something isn't true it's hard to unpick your mental model of the world.
> Once you find out something isn't true it's hard to unpick your mental model of the world.
Intuitively, I would think the same, but a book about education research that I read and my own experience taught me that new information is surprisingly easy to unlearn. It’s probably because new information sits at the edges of your neural networks and do not yet provide a foundation for other knowledge. This will only happen if the knowledge stands the test of time (which is exactly how it should be according to Popper). If a counterexample is found, then the information can easily be discarded since it’s not foundational anyway and the brain learns the counterexample too (the brain is very good in remembering surprising things).
That presumes the wrong information is corrected quickly. What about the cases when that doesn't happen? Aren't you often finding out things you thought were true from years ago are wrong?
You weigh new information by how confident you are in it. You try to check different sources, you maintain an open-mind, etc. In that, ChatGPT is just an additional low-reliability source of information.
I just verify the information I need. I find it useful as a sort of search engine for solutions. Like, how could I use generators as hierarchical state machines? Are there other approaches that would work? What are some issues with these solutions? Etc. By the end I have enough information to begin searching the web for comparisons, other solutions, and so on.
The benefit is that I got a quick look at various solutions and quickly satisfied a curiosity, and decided if I’m interested in the concept or not. Without AI, I might just leave the idea alone or spend too much time figuring it out. Or perhaps never quite figure out the terms of what I’m trying to discover, as it’s good at connecting dots when you have an idea with some missing pieces.
I wouldn’t use it for a conversation about things as others are describing. I need a way to verify its output at any time. I find that idea bizarre. Just chatting with a hallucinating machine. Yet I still find it useful as a sort of “idea machine”.
The smart people I've seen using ChatGPT always double check the facts it gives. However, the truth is that RLHF works well to extinguish these lies over time. As more people use the platform and give feedback, the thing gets better. And now, I find it to be pretty darn accurate.
I don't know. The other day I was asking about a biology topic and it straight up gave me a self-contradicting chemical reaction process description. It kept doing that after I pointed out the contradiction. Eventually I got out of this hallucination loop by resetting the conversation and asking again.
I see this conversation pretty frequently and I think the root of it lies in the fact that we have mental heuristics for determining whether we need to fact check another human because they are a bullshitter, an idiot, a charlatan etc, but most people haven’t really developed this sense for AIs.
I think the current state of AI trustworthiness (“very impressive and often accurate but occasionally extremely wrong”) triggers similar mental pathways to interacting with a true sociopath or pathological liar for the first time in real life, which can be intensely disorienting and cause one to question their trust in everyone else, as they try to comprehend this type of person.
I think this post-factual attitude is stronger and more common in some cultures than others. I'm afraid to say but given my extensive travels it appears American culture (and its derivatives in other countries) seems to be spearheading this shift.
I think it's because Americans, more than nearly all other cultures, love convenience. It's why the love for driving is so strong in the US. Don't walk or ride, drive.
Once I was walking back from the grocer in Florida with 4 shopping bags, and people pulled over and asked if my car had broken down and if I needed a ride, people were stunned...I was walking for exercise and for the environment...and I was stunned.
More evidence of this trend can be seen in the products and marketing being produced:
Do you need to write a wedding speech? Click here.
Do you need to go get something from the store? get your fat ass in the car and drive, better yet, get a car that drives for you? Better than this, we'll deliver it with a drone...don't move a muscle.
Don't want to do your homework? Here...
Want to produce art? Please enter your prompt...
Want to lose weight? We have a drug for that...
Want to be the authority on some topic? We'll generate the facts you need.
I've also identified convenience as a core factor. Another dynamic at play is this:
As convenience in a domain becomes ubiquitous or at least expected among consumers, they quickly readjust their evaluation of "having time for X" around the new expectation of the convenient service, treating all alternatives as positive opportunity cost. This would explain a lot of those folks who are upset when it's suggested that they don't need Amazon, Instacart, etc. in their lives if they are to do something about their contributions to mass labor exploitation.
Of course these conveniences quickly become ubiquitous in large economies with a glut of disposable income, which encourages VCs to dump money into these enterprises so they're first to market, and also to encourage the public to believe that the future is already here and there's no reason to worry about backsliding or sustainability of the business model. Yet in every single case we see prices eventually rise, laborers squeezed, etc. A critical mass of people haven't yet acknowledged this inevitability, in no small part due to this fixation on convenience at the expense of more objective, reasoned understandings (read: post-truth mindset).
I agree with this, but I think there is a deeper level which explains this. And that is convenience is a product. The thing that truly defines how corporations in America have shaped our culture is that everything is turned into a way to sell you something.
This is a fairly perpetual discussion, but I'll go for another round:
I feel like using LLM today is like using search 15 years ago - you get a feel for getting results you want.
I'd never use chatGPT for anything that's even remotely obscure, controversial, or niche.
But through all my double-checking, I've had phenomenal success rate in getting useful, readable, valid responses to well-covered / documented topics such as introductory french, introductory music theory, well-covered & non-controversial history and science.
I'd love to see the example you experienced; if I ask chatGPT "tell me about Toronto, Canada", my expectation would be to get high accuracy. If I asked it "Was Hum, Croatia, part of the Istrian liberation movement in the seventies", I'd have far less confidence - it's a leading question, on a less covered topic, introducing inaccuracies in the prompt.
My point is - for a 3 hour drive to cottage, I'm OK with something that's only 95% accurate on easy topics! I'd get no better from my spouse or best friend if they made it on the same drive :). My life will not depend on it, I'll have an educationally good time and miles will pass faster :).
(also, these conversations always seem to end in suffocatingly self-righteous "I don't know how others can live in this post-fact free world of ignorance", but that has a LOT of assumptions and, ironically, non-factual bias in it as well)
> I feel like using LLM today is like using search 15 years ago - you get a feel for getting results you want.
I don't think it's quite the same.
With search results, aka web sites, you can compare between them and get a "majority opinion" if you have doubts - it doesn't guarantee correctness but it does improve the odds.
Some sites are also more reputable and reliable than others - e.g. if the information is from Reuters, a university's courseware, official government agencies, ... etc. it's probably correct.
With LLMs you get one answer and that's it - although some like Bard provide alternate drafts but they are all from the same source and can all be hallucinations ...
How expensive could it be? Google Bard, a free service, offers the drafts for free. Just do the comparison on the user’s machine if the LLM provider is that cheap.
P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no?
The drafts have to be evaluated either by a human or llm. Doing that for every request does not scale when you have millions of users.
>Just do the comparison on the user’s machine if the LLM provider is that cheap.
This is not possible. Users don't have the resources to run these gigantic models. LLM inference is not cheap. Open ai, Google aren't running profit on free cGPT or Bard.
>P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no?
It's not a problem of randomness. a temp of 0 doesn't reduce hallucinations. LLMs internally know when they are hallucinating/taking a wild guess. randomness influences how that guess manifests each time but the decision to guess was already made.
> LLMs internally know when they are hallucinating/taking a wild guess.
No they don’t. If they did we would be able to program them to not do so.
I would argue that wild guesses are all LLMs are doing. They practically statistically guess their way to an answer. It works surprisingly well a lot of the time but they don’t really understand why they are right/wrong.
P.S. LLMs are kind of like students who didn’t study for the test so they use “heuristics” to guess the answer. If the test setter is predictable enough, the student might actually get a few right.
"In particular, we find that LMs often hallucinate differing authors of hallucinated references when queried in independent sessions, while consistently identify authors of real references. This suggests that the hallucination may be more a generation issue than inherent to current training techniques or representation."
"SelfCheckGPT leverages the simple idea that if a LLM has knowledge of a given concept, sampled responses are likely to be similar and contain consistent facts. However, for hallucinated facts, stochastically sampled responses are likely to diverge and contradict one another."
Exactly this! This is my experience also. Your point about "well covered & non-controversial" is spot on. I know not to expect great results when asking about topics that have very little coverage. To be honest I wouldn't expect to go to an arbitrary human and get solid answers on a little covered topic, unless that person just happened to be topic expert. There is so much value in having the basics to intermediate levels of topics covered in a reliable way. That's where most of commercial activity occurs.
I think a key difference is that humans very rarely sound convincing talking about subjects they have no clue about.
I've seen the hallucination rate of LLMs improve significantly, if you stick to well covered topics they probably do quite well. The issue is they often have no tells when making things up.
Joe Rogan has made tons of money off talking without providing factual information. Hollywood has also made tons of money off movies "inspired by real events" that hallucinate key facts relevant to the movie's plot and characters. There's a huge market for infotainment that is "inspired by facts" but doesn't even try to be accurate.
You listen to Joe Rogan with the idea that this is a normal dude talking not an expert beyond martial arts and comedy.
A person who uses ChatGPT must have the understanding that it's not like Google search. The layman, however, has no idea that ChatGPT can give coherent incorrect information and treats the information as true.
Most people won't use it for infotainment and OpenAI will try its best to downplay the hallucination as fine print if it goes fully mainstream like google search.
Give people more credit. If you're using an AI these days, you have to know it hallucinates sometimes. There's even a warning about it when you log in.
There's a contingent of the population passing videos around on tiktok genuinely concerned that AIs have a mind of their own
no I will not give the public credit, most people have no grounding to discern wtf a language model is and what it's doing, all they know is computers didn't use to talk and now they do
I'll give tech people credit, but non-tech people I'm not so sure. A good example is the cookie permissions or app permissions. A great number of non-tech people don't even know or care what they mean.
You gotta stop bucketting people like that. People may not know terms "cookie permissions" or "app permissions" but they sure as fuck understand the idea of user tracking or handing a company access to your mic/camera. And to say they don't care about these things is simply not true.
now that you mention it, a big "for entertainment purposes only" banner like they use to have on all the psychic commercials on tv would not be inappropriate. it's incredible that LLMs are being marketed as general purpose assistants with a tiny asterisk, "may contain inaccuracies" like it's a walnut contamination
Not sure what's being incredible here. GPT-4 is a stellar general-purpose assistant, that shines when you stop treating it as encyclopedia, and start using it as an assistant. That is, give it tasks, like summarizing, or writing code, or explaining code, or rewriting prose. Ask for suggestions, ideas. You can do that to great effect, even when your requests are underspecified and somewhat confused, and it still works.
I just wish they were advertised for generative tasks and not retrieval tasks. It's not intelligence, it's not reasoning, it's text transformation.
It seems to be able to speak on history, sometimes it's even right, so there's a use case that people expect from it.
FYI I've used GPT4 and Claude 2 for hundreds of conversations, I understand what its good and bad at; I don't trust that the general public is being given a realistic view.
In my experience, LLVMs are not about being provided facts. They are about synthesizing new content and insights based on the model and inputs.
Rather than asking it about facts, I find it useful to derive new insights.
For example: "Tell me 5 topics about databases that might make it to the front page of hacker news." It can generate an interesting list. That is much more like the example they provided in the article, synthesizing a bed time story is not factual.
Also, "write me some python code to do x" where x is based on libraries that were well documented before 2022 also has similarly creative results in my experience.
You will now be able to feed it images and responses of the customers. Give it a function to call complementaryDrink(customerId)
Combine it with a simple vending machine style robot or something more complex that can mix drinks.
I'm not actually in a hurry to try to replace bartenders. Just saying these types of things immediately become more feasible.
You can also see the possibilities of the speech input and output for "virtual girlfriends". I assume someone at OpenAI must have been tempted to train a model on Scarlett Johansson's voice.
Hopefully people know not to ask others for factual information (unless it's an area they're actually well educated/knowledgeable in), but for opinions and subjective viewpoints. "How's your day going", "How are you feeling", "What did you think of X", etc, not "So what was the deal with the Hundred Year's War?" or whatever.
If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning.
> If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning.
That's on them. I mean, people need to figure out that LLMs aren't random strangers, they're unfiltered inner voices of random strangers, spouting the first reaction they have to what you say to them.
Anyway, there is a middle ground. I like to ask GPT-4 questions within my area of expertise, because I'm able to instantly and instinctively - read: effortlessly - judge how much to trust any given reply. It's very useful this way, because rating an answer in your own field takes much less work than coming up with it on your own.
No individual is "most people". Most of the time I spend talking to people in real life is with people whose professional expertise, hobbies, and other sources of knowledge I know at least roughly. I have an idea how good they are at evaluating what they know and how honest they and whether they are prone to wishful thinking.
I'm curious if you're using GPT-4 ($)? I find a lot of the criticisms about hallucination come from users who aren't, and my experience with GPT-4 is it's far less likely to make stuff up. Does it know all the answers, certainly not, but it's self-aware enough to say sorry I don't know instead of making a wild guess.
You can also prompt it to hold back if it doesn’t know, which seems to make a difference. It’s part of my default prompt, and since I added it I haven’t had any overt hallucinations. Definitely invalid code, but not due to crazy errors. Just syntax and inconsistent naming mostly.
I verify just about everything that I ask it, so it isn’t just a general sense of improvement.
Why would anyone pay for something if the free trial doesn’t work? “Hey, you know how we gave you a product that doesn’t quit work as you expect and is super frustrating? Just pay us money, and we’ll give you the same product, but it just works. Just trust us!”
GPT-4 is not the same product. I know it seems like it due to the way they position 3.5 and 4 on the same page, but they are really quite separate things. When I signed up for ChatGPT plus I didn't even bother using 3.5 because I knew it would be inferior. I still have only used it a handful of times. GPT-4 is just so much farther ahead that using 3.5 is just a waste of time.
Would you mind sharing some threads where you thought ChatGPT was useful? These discussions always feel like I’m living on a different planet with a different implementation of large language models than others who claim they’re great. The problems I run into seem to stem from the fundamental nature of this class of products.
Context: had a bunch of photos and videos I wanted to share with a colleague, without uploading them to any cloud. I asked GPT-4 to write me a trivial single-page gallery that doesn't look like crap, feeding it the output of `ls -l` on the media directory, got it on first shot, copy-pasted and uploaded the whole bundle to a personal server - all in few minutes. It took maybe 15 minutes from the idea of doing it first occurring to me, to a private link I could share.
I have plenty more of those touching C++, Emacs Lisp, Python, generating vCARD and iCalendar files out of blobs of hastily-retyped or copy-pasted text, etc. The common thread here is: one-off, ad-hoc requests, usually underspecified. GPT-4 is quite good at being a fully generic tool for one-off jobs. This is something that never existed before, except in form of delegating a task to another human.
I use ChatGPT for all sorts of things - looking into visas for countries, coding, reverse engineering companies from job descriptions, brainstorming etc etc.
It saves a lot of time and gives way more value than what you pay for it.
A human driving buddy can make up a lot of stuff too. Have an interesting conversation but don't take it too seriously. If you're really researching something serious then take a mental note to double check things later, pretend as if you're talking to a semi-reliable human who knows a lot but occasionally makes mistakes.
I always thought a better future would be full of more and more distilled, accurate, useful knowledge and truthful people to promote that.
Comments like yours make me think that no one cares about this...and judging by a lot of the other comments, I guess they don't.
Probably going to be people, wading through a sea of AI generated shit, and the individual is supposed to just forever "apply critical thinking" to it all. Even a call from ones spouse could be fake, and you'll just have to apply critical thinking or whatever to workout if you were scammed or not.
There aren't any real world sources of truth you can avoid applying critical thinking to. Much published research is false, and when it isn't, you need to know when it's expired or what context it's valid in.
Because it doesn't always make up stuff. Because I'm a human and can ask for more information. I don't want an encyclopedia on a podcast. I want to "talk" to someone about stuff. Not have an enumerated list of truths firehosed at me.
If the next version has the same step up in performance, I will no longer consider inaccuracy an issue - even the best books have mistakes in them, they just need to be infrequent enough.
You make it sound like business shenanigans, but the truth is, it's a natural fit for now, as performance of LLMs improves with their size, but costs of training (up-front investment) and inference (marginal, per-query) also go up.
I've wanted a ChatGPT Pod equivalent to a Google Home pod for a while! I have been intending to build it at some point. I am with you, talking to Google sucks.
"Hey Google, why do ____ happen?" "I'm sorry, I don't know anything about that"
But you're GOOGLE! Google it! What the heck lol
So yeah, ChatGPT being able to hear what I say and give me info about it would be great! My holdup has been wakewords.
It increasingly feels to me like building any kind of general-use AI tool or app is a bad choice. I see two viable AI business models:
1. Domain-specific AI - Training an AI model on highly technical and specific topics that general-purpose AI models don't excel at.
2. Integration - If you're going to build on an existing AI model, don't focus on adding more capabilities. Instead, focus on integrating it into companies' and users' existing workflows. Use it to automate internal processes and connect systems in ways that weren't previously possible. This adds a lot of value and isn't something that companies developing AI models are liable to do themselves.
> building any kind of general-use AI tool or app is a bad choice
Maybe not if you rely on models that can be ran locally.
OpenAI is big now, and will probably stay big, but with hardware acceleration, AI-anything will become ubiquitous and OpenAI won’t be able to control a domain that’s probably going to be as wide as what computing is already today.
The shape of what’s coming is hard to imagine now. I feel like the kid I was when I got my first 8-bit computer in the eighties: I knew it was going to change the world, but I had little idea how far, wide and fast it would be.
There are plenty of OS models being released - there's going to be a steadily increasing quantity + quality of models you can run locally. I don't think it's a good place to compete.
> Instead, focus on integrating it into companies' and users' existing workflows. Use it to automate internal processes and connect systems in ways that weren't previously possible
why wouldn’t a company do that themselves e.g. how inter come has vertically integrated AI? any examples?
It's classic build vs. buy. Companies tend to build their own products and use third party software for internal tools.
Just look at Salesforce AppExchange - it's a marketplace of software built on top of Salesforce, a large chunk of which serves to integrate other systems with Salesforce. LLMs open up the ability to build new types of integrations and to provide a much friendlier UI to non-developers who need to work on integrating things or dealing with data that exists in different places.
I don't think anybody following OpenAI's feature releases will be caught off guard by ChatGPT becoming multi-modal. The app already features voice input. That still translates voice into text before sending, but it works so well that you basically never need to check or correct anything. Rather, you might have already been asking yourself why it doesn't reply back with a voice already.
And the ability ingest images was a highlight and all the hype of the GPT-4 announcement back in March: https://openai.com/research/gpt-4
one of the original training sets for the BERT series is called 'BookCorpus', accumulated by regular grad students for Natural Language Processing science. Part of the content was specifically and exactly purposed to "align" movies and video with written text. That is partly why it contains several thousand teen romance novels and ordinary paperback-style story telling content. What else is in there? "inquiring minds want to know"
> This announcement seem to have killed so many startups that were trying to do multi-modal on top of ChatGPT.
Rather than die, why not just pivot to doing multi-modal on top of Llama 2 or some open source model or whatever? It wouldn’t be a huge change
A lot of businesses/governments/etc can’t use OpenAI due to their own policies that prohibit sending their data to third party services. They’ll pay for something they can run on-premise or in their own private cloud
I’ve got one eye on https://www.elto.ai/. I was pitching something I like better earlier this year (I still think they’re missing a few key things), but with backing from roughly YC, Meta, and God, and a pretty clear understanding that robustness goes up a lot faster than capability goes down?
I wouldn’t count out focused, revenue-oriented players with Meta’s shit in their pocket out just yet.
Took me a while to realise I can just type search queries into ChatGPT. e.g. simply "london bridge history" or whatever into the chat and not only get a complete answer, but I can ask it follow-up questions. And it's also personalised for the kinds of responses I want, thanks to the custom instructions setting.
ChatGPT is my primary search engine now. (I just wish it would accept a URL query parameter so it could be launched straight from the browser address bar.)
Trying that example, I’d much prefer just going to the Wikipedia page on London Bridge than trying to guess what phrases ChatGPT will respond well to in order to elicit more info. It’s initial response for me didn’t even mention one of the most interesting facts that people lived and worked on the bridge.
YMMV. For my case on software development, I don't even look on stackoverflow anymore.
Just type the tech question, start refining into what is needed and get a snippet of code tailored for what is needed. What previously would take 30 to 60 minutes of research and testing is now less than a couple of minutes.
And I don't have to wade through Stack Overflow and see all the times mods and others have tried to or succeeded in closing down very useful questions.
I know there are a lot of google programmers out there, but was using search engines for programming ever a good idea? Don’t get me wrong, I’ll look up how to do absolutely simple things every day but I basically always look in the official documentation.
Which may be why I’ve been very underwhelmed by GPT so far. It’s not terrible at programming, and it’s certainly better than what I can find on Google, but it’s not better than simply looking up how things work. I’m really curious as to why it hasn’t put a more heavy weight on official documentation for its answers, they must’ve scraped that a long with all the other stuff, yet it’ll give you absolutely horrible suggestions when the real answer must be in its dataset. Maybe that would be weird for less common things, but it’s so terrible at JavaScript that it might even be able to write some of those StackOverflow answers if we’re being satirical, and the entire documentation for that would’ve been very easy to flag as important.
Yes there are and it's infuriating. Colleague of mine had problems with integrating some code into an app that was built on a newer version of a framework because "there aren't a lot of examples yet". One web search and I found the frameworks own migration guide detailing the exact differences that would need to be accounted for.
Well, yes. Point is, GPT-4 read the entire StackOverflow and then some, comprehended it, and now is a better interface to it, more specific and free of all the bullshit that's part of the regular web.
It depends on the subject but search engines are on the decline. With so many fake website written by AI I can only see it get worse.
The most extreme I can think of is when I want to find when a show comes out and I have to read 10 paragraphs from 5 different sites to realize no one knows.
> The most extreme I can think of is when I want to find when a show comes out and I have to read 10 paragraphs from 5 different sites to realize no one knows.
I found that you can be pretty sure no one knows if it’s not already right on the results page. And if the displayed quote for a link on the results page is something like “wondering when show X is coming out?”, then it’s also a safe bet that clicking that link will be useless.
You learn those patterns fast, and then the search is fast as well.
Google still thinks I want to click on the sites I haven't clicked on in a decade even though they are first results. Search engines have a long way to go to catch up to GPT
> The most extreme I can think of is when I want to find when a show comes out
Yeah, I find that queries which can be answered in a sentence are the worst to find answers from search engines because all the results lengthen the response to an entire article, even when there isn't an answer.
Agreed except ChatGPT (3.5 at least, haven't tried 4) is unable to provide primary sources for its results. At least when I tried, it just provided hallucinated urls
who would have thought that few years ago, just goes to show that a Giant like Google is also susceptible when they stop innovating. The real battle is going to be fought between these two as Google's business is majorly dependent on search ads.
True. Although the training is on a snapshot of websites, including q&a like stackoverflow. If these were replaced too, where are we heading? We'll have to wait and see. One concern would be centralization/ lack of options and diversity. Stackoverflow started rolling AI on its own, despite the controversial way it did (dismissing long time contributors); it might be correctly following the trend.
Personally I prefer stackoverflow and such,
because I can see different answer including wrong
or non-applicable ones which don't solve my exact problem.
Last I heard, OpenAI was losing massive amounts of money to run all this. Has that changed?
Because past history shows that the first out of the gate is not the definitive winner much of the time. We aren't still using gopher. We aren't searching with altavista. We don't connect to the internet with AOL.
AI is going to change many things. That is all the more reason to keep working on how best to make it work, not give up and assume that efforts are "doomed" just because someone else built a functional tool first.
also, I did not know until today's thread that OpenAI's stated goal is building AGI. which is probably never going to happen, ever, no matter how good technology gets.
which means yes, we are absolutely looking at AltaVista here, not Google, because if you subtract a cult from an innovative business, you might be able to produce a profitable business.
Not only "Alexa/Siri/Google Home" but Google Search [ALL] itself. Google was a pioneer in search engines adding a page ranking / graph layers as a meaning but technologies such as ChatGPT could add a real layer of meaning, at least improve current Google Search approach. The future of search seems more conversational and contextual.
BTW, I expect these technologies to be democratized and the training be in the hands of more people, if not everyone.
To some extent yes, for generic multi-modal chat-bots this could be a problem, but there are many apps that provide tight integration / smooth tooling for whatever problem they are helping to solve, and that might be valuable to some people -- especially if it's a real value generating use case, where the difference between 80% solution from ChatGPT and 95% solution from a bespoke tool matters.
hobbyists and professionals on /r/localllama subreddit are having an existential crisis
most of them accurately detect it is a sunk cost fallacy to continue but it looks like a form of positive thinking... and that's the power of community!
This is good news - those ai companies have been freed to work on something else, along with the ai workers they employ. This is of great benefit to society.
I can already see "Alexa/Siri/Google Home" replacement, "Google Image Search" replacement, ed-tech startups that were solving problems with AI using by taking a photo are also doomed and more to follow.