If you were to take a handful of trending tech from the past decade, and plot peak hype against utility, I'd wager ChatGPT and related technologies would rank pretty favourably. Crypto and VR have gone through similar hype cycles but the utility falls short.
ChatGPT has it's shortcomings, but I've found myself returning to it over and over again. Then consider what this technology will look like in a few years time and the conclusion is that, imo, the hype is justified.
I've been trying to use ChatGPT productively by asking it instead of looking up docs or search Duck Duck Go.
I'm becoming more and more disillusioned. I was sceptical at first, then thoroughly impressed by all the blog posts, but actually using it is extremely frustrating. It's really good at composing answers that look plausible, but it just has no idea what you want.
The only thing it is truly good at is composing unnecessarily verbose responses to spam emails.
I've had fairly good results in doing zero shot classifications, or asking it to summarize structured data in a human friendly way (e.g. a weather report).
I've found its knowledge to be flawed if you poke enough into it though that feels more like it doesn't "understand" the facts that are embedded into the words.
However, using it to do the classification natural language, extract certain pieces of information from the query, and then convert the structured data back to natural language it shows itself to be very capable and able to be "better sounding" than rules based summaries of that data.
I've also found it to be quite capable at rewriting poorly written questions (e.g. Stack overflow questions - yea, I know, banned and all) to remove extraneous parts ("I'm a noob", "Thanks in advance"), fix grammar, and change the tone to be more professional... and creating a good summary title.
I’ve had the opposite experience. While at times it’s just wrong, it’s usually been pretty right even when I loosely phrase my request and ask it things Kagi or other IR systems wouldn’t be able to grok into meaningful results. Often it’s responses are also direct and succinct (even with all its boilerplate blah blah) and faster to process than clicking through 50 blogs and navigating SEO garbage, or distilling documentation to find the specific answer I need from it. Where it is insufficient it provides enough context and relevant search terms to construct a better IR query on a search engine.
I have definitely had it get stuff just flat out wrong, but it’s easy enough to pick it out and I degenerate to old practice of searching and filtering in those cases.
As has been posted about here many times in different ways, as they graft IR and reasoning systems onto chatgpt I expect these problems to disappear. I find it amazing how dismissive folks are of a tech that is barely a few months old. It’s like a whole generation of nerds have lost the capacity to dream.
You’re nuts, I use it to generate all my employee reviews, props etc. I used it to come up with high level interview questions. I used it to generate a JD. I used it to write a podcast. I could not disagree with you more. It’s… smart and witty and honestly almost feels like a real person sometimes rather than a token generator.
I had this feeling too, so I began feeding text from my documents and videos into the same GPT engine.
It results in much more relevant and grounded answers(well... grounded to whatever you documents claim, for better or worse). If you want to try it, there's a demo at fragen.co.uk
A lot of people outside of tech haven't heard of it yet. It is absolutely going to get overhyped but I would say it is currently underhyped and if you think this is hype then you're going to have a fun 2023.
Even Ryan Reynolds is using it to write advertising copy, https://youtu.be/_eHjifELI-k, I would say it's pretty well known. Not to mention all the get rich quick with ChatGPT nonsense that's popping up, that's squarely aimed at non-tech people.
Dude it was on the local news here last week. I donno where you are at but everyone I know has heard of it, even the people who don't own any computer beyond a smartphone in 2023.
I work at a company where we process vast amounts of retailer advertising data, and one of the things we need to do is tag images and extract data from deals. Although humans are still involved in this, at this point they are mostly checking the work of our ML models. This dramatically increases the speed with which we are able to do this - and this speed is one of the things that makes us attractive to our clients.
Though, I wouldn't be surprised that if overall, people have spent more time on compute than have made from the models. There are companies that treat data science properly and are doing it well - but in the previous financial era people were inclined to throw money at things without clear criteria for success.
This is BS. ML targeted ads were the first application to make money and that happened a long long time ago (maybe 15 years at this point?). Nowadays there are a lot of products where ML is an essential feature.
For the tweet, note the hyperbole and mocking of AWS more than the mocking of ML. The entire chain of tweets is mocking the cloud with hyperbole and a grain of truth.
Thats possible, however it seems like most folks who are disappointed in it are trying to use it inappropriately.
For me, it has turned out to be a massive productivity enhancer, when used to augment an existing set of skills rather than replace it. I think of it as a "cognitive workhorse" that can do the crappy parts of generating lots of stuff that I can go back and edit later. I much prefer this method over having to do the initial generation myself. Its worked great for me for both writing technical(ish) docs and for generating code.
Much of the time though I feel like it only works as well as it does for me because I'm not asking it to do anything I couldn't do myself (albeit with more mental effort), which means I am able to immediately spot any issues and ask it to improve or expand on its answer.
The nice thing about generating code too is that you'll know pretty quick if it works or not. By the time its actually working, I've gone through and checked each line and made sure all the tests pass and test the correct things.
You know what's even more over-hyped is GPT-4. Sam Altman recently stated that many people will be disappointed because of all of the rumours that it's AGI.
Yeah, it's a useful scaling experiment given the improvement from GPT-2 to GPT-3.5 (base of ChatGPT), but I'd have to guess the "emergent capability" curve is flattening out at this point.
ChatGPT already does well when essentially regurgitating factually correct information, but the obvious weakness of an LLM is that when combining sources its just combining them linguistically so it produces fine sounding streams of often factually incorrect bullshit.
It's hard to imagine how GPT-4 will be much different unless there have been some fundamental extensions to the model architecture, or changes to the training regime. Presumably it'll do better in "search engine mode" where people are hoping it'll "understand" the subtleties of their questions in selecting what to generate, but hard to see how it'll avoid the "bullshit chained deduction" problem unless they've added a way for it to learn from it's own mistakes.
In terms of perception/disappointment, I guess it depends on individual expectations. If you want GPT-4 to be a better "search engine" than GPT-3, then you may be happy. If you expect GPT-4 to be "more intelligent" than GPT-3, then I'm guessing not so much. Let's see!
I've a hunch it may do well as the basis of an improved Copilot since this seems to be an easier subdomain.
On consideration, maybe "combining them linguistically" is a bit too literal. Of course that is what a language model does, but that ignores that an extremely good language model such as GPT-3+ will have had to have learned a pretty good world model in order to do a good job of next-word prediction, to the extent that regarding this as merely continuation statistics (while true) is a bit too dismissive of what it's actually capable of.
So, perhaps the scale upgrade from GPT-3 to GPT-4 will improve it's world model enough that it does a better job of combining sources using a consistent bias/POV rather then happily combining true and false sources as it currently does.
ChatGPT has it's shortcomings, but I've found myself returning to it over and over again. Then consider what this technology will look like in a few years time and the conclusion is that, imo, the hype is justified.