I really do not understand the benefits supposedly of this tech over a traditional search engine. All it's doing is pulling the top hit from Bing and saying it back to you in a chat format. The AI doesn't understand what IP Scanning software is, nor does it understand malware, nor does it understand anything about what you're saying, it literally just doesn't. It takes what you asked, stuffs it into bing, and reads out the results to you.
I understand why people want AI search engines, the problem is, this is not an AI search engine. None of them are to my knowledge. It's just an ML bot mediating a search between you and the engine. The web page accomplishes the exact same task, without the unearned air of authority.
Like, the IDEA of an AI search engine, an artificial intelligence that knows all this stuff and can weigh in alongside you with insight and comprehension, that's incredibly cool. But this is not that. ChatGPT knows much but understands nothing.
Neither Google nor MS will ever get AI search right because the only applicable use of AI in a search engine is to remove the seo/junk/ads/clickbait and give the user what he is looking for. This goes against their core business (ads)
Engines like you.com, phind.com, aisearch.vip, or kagi have the advantage because their business model doesn't depend on this
Hmm, a fiduciary disclosure for AIs isn't a bad idea. Basically, taking how the AI was fine tuned and turning it into comprehensible disclosures as to how much you can trust it to have your, versus its creator's, best interests in "mind."
For example, injecting ads and even content moderation to keep ChatGPT from joining Hitler's Youth, could both be considered non-fiduciary functions.
A lot of the value comes from follow-up questions. Imagine being able to interrogate a StackOverflow answer with new constraints and details. Not always correct, but in some cases, faster that typing in a new search term and parsing a screen full of links.
But again, the AI doesn't know. It's going to search around the internet and probably take a closer look at what it already told you, but that's it. It takes a plethora of information and attempts to digest it into knowledge but it lacks the understanding with which to accomplish this task.
Unless I guess you train an AI on a given topic, like a few languages or a database or something. But given ChatGPT's apparent vulnerability to just making shit up, you'll have to call me skeptical if this has any real use.
GPT4 "knows" a lot more about most topics than any single human does. People have this idea that it absolutely needs to be perfectly correct at all times to be useful, but would never hold a human being to that standard.
How many times have you asked a co-worker about something and they gave you a convincing answer that was totally wrong? Did it make you stop asking co-workers for help?
Coworkers answering wrong gets me to stop asking them for help, yes, especially if it's confident or they can't qualify uncertainty or I know their ratio of Q&A site visits vs 1st party docs visits is abysmal. There are even people I won't ask for help because they trust people that I don't trust.
Conversely, there are people who I will go to for topics that they are not the SME in, maybe their teammate even is, but I trust their ability to do quality research and intelligently interpret that research for case specific nuances. Like I'll go to the networking guy to talk about some DB thing because the DB guys are morons and live and die by junk SEO sites but the networking guy can think analytically and find the source of truth documentation and provide excerpts from it.
I VERY much more trust a coworker to know things than I do this AI, especially given not just the subject of this thread, but the larger conversation around it. ChatGPT has a reputation already for just spewing out complete nonsense. Didn't Bing's implementation argue with someone about what freaking year it was not awfully long ago?
These chat bots "know" a shit ton and a half of stuff in that they are connected to the largest collection of knowledge known to man, the Internet. But "knowing" and "understanding" are two different things. The various search engines also "know" a ton about where to find things online, that doesn't mean they know shit about those things. And as we're seeing here: without the context to know that when someone wants an IP scanner, they want something good, that won't give their computer malware, that'll be reasonably priced or even open source, or even what platform, it just gives an answer based on a search.
You could just search for ip scanning software and gotten all the information BingGPT shared with the author of the piece.
And like, if you wanted to be charitable, you could say "well the author should've given more information about what they wanted" but again, that's not different from an existing search engine and more crucially: the AI didn't ask questions. Didn't ask for platform, how much they wanted to spend, if they preferred open source, or even something more general like what they were trying to accomplish. Nada. Just did a search, and reported results.
> Didn't Bing's implementation argue with someone about what freaking year it was not awfully long ago?
I'm sorry, but that's a stellar example of holding an LLM wrong. These models are frozen in time.
> But "knowing" and "understanding" are two different things.
Indeed, and that is a big part of misunderstanding. GPT-4 is, on many topics, closer to understanding than knowing (note that neither is a subset of the other). The conceptual patterns are there, even if sometimes are easy to accidentally overpower by the prompt, or by the sequence of tokens already emitted.
> I'm sorry, but that's a stellar example of holding an LLM wrong. These models are frozen in time.
Y'all keep throwing out these gotcha statements that just make the technology you're trying to tell me is great seem more and more useless.
How can you even attempt to call something artificial intelligence if it doesn't even know the year it is!?
> Indeed, and that is a big part of misunderstanding. GPT-4 is, on many topics, closer to understanding than knowing (note that neither is a subset of the other). The conceptual patterns are there, even if sometimes are easy to accidentally overpower by the prompt, or by the sequence of tokens already emitted.
I don't think it's either understanding or knowing. Someone who knows something isn't going to spontaneously forget it because someone asked them a question incorrectly.
>How many times have you asked a co-worker about something and they gave you a convincing answer that was totally wrong?
Never actually. My coworkers have never told me something with confidence that they just made up. If they don't know an answer, they may provide hints and directions, but it will be clear they don't know.
> Did it make you stop asking co-workers for help?
i mean, in many circumstances, we absolutely stop asking them...
when i ask a trustworthy human a question, they will absolutely tell me if it is out their depths. they understand their own limits on around the subject and say so. and if they understand a little, they’ll help point towards people who would be a better authority on the subject.
that’s basic level human connection stuff.
if someone confidently gives you the wrong answer, refuses to know their own limits, and repeatedly leads you astray you don’t trust them after this do you?
> the AI doesn't know. It's going to search around the internet and probably take a closer look at what it already told you, but that's it
F/k/a Putting a thing on the internet for randos to identify and explain. As long as it cites the LLM cites its sources, general questions in the form of "what is this" or "what's going on here" while you point to a page or an image or in a general direction are not well suited for search engines.
I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.
Like hell, if that's the standard, I'll be ChatGPT. You won't need an outrageous graphics card to ask me a question and I'll get you an answer right away. It'll almost certainly be the wrong answer, and is just an ass-pulled guess, but if that's all you want, I'll setup a chat website for myself and start taking queries today. Then investors can give me 10 billion dollars.
Here's my pet example...feel free to google around yourself on this.
Problem: I want an AWS CLI command line that requests a whole bunch of wildcard certificates from AWS Certificate Manager (ACM) for a TLD.
Ostensible solution: the AWS official docs have a small snippet to achieve this, BUT -- the snippet on the official page is inadvisable as it leads to a browser cert warning.
So I (skeptically) asked ChatGPT for a command line to achieve what I was trying to do.
Try 1: got basically the snippet from the AWS official docs (but with the inadvisable flag set to the _Correct_ value, strangely)
Prompt 2: please give me more best practice options
Try 2: get back a bunch of new CLI options and their meanings. 3 are useful. 1 is hallucinated. 1 is deprecated.
Prompt 3: keep going with more options
Try 3: 2 more useful new options, 2 more options I chose not to use
As a skeptic, the overall experience was much more efficient that googling around or even reading a manpage. I put it all on the fact that context is maintained between questions, so you don't have to repeat yourself when asking for clarifications.
I'm kind of surprised this worked. Did you actually use the command you ended up with? I'm not even surprised because I think ChatGPT can't figure this out in principle, but because the data itself is poisoned. The top link on every web search I've ever used to AWS CLI commands is to documentation for v1, but v1 has been deprecated for years and the page usually begins with a statement telling you not to use it. Amazon's problem is they never remove old documentation from the web, so 90% of what you find for any given service is no longer correct.
> and the page usually begins with a statement telling you not to use it
This might be a big part of why GP's case works. The model (GPT-4) most likely understands the concept of documentation being deprecated, so the more often v1 docs say it, the stronger a semantic link between current and obsolete docs, and the more likely it is for ChatGPT to give you answer based on non-deprecated docs.
> Did you actually use the command you ended up with?
Yes! Note that I had to use my domain knowledge to sift through the options and eliminate the garbage, but the experience was just _faster_ than repeated searches and digging through ad-laden garbage sites.
this. in my experience, GPT-4 really shines for groking AWS commands, writing JavaScript code and helping me understand some errors when compiling my terrible Rust code
Sometimes, having a quick, inaccurate, but easily-verifiable answer is better. For example, when you're trying to remember the name of that one function to call, or ideas on where to travel next month.
Also, not super relevant, but in e.g. combat situations one is often better off running now in any direction rather than pondering which direction is absolutely optimal for running from the lion. You'll know soon enough whether it was the right direction. There's probably a metaphor somewhere in there.
ChatGPT doesn't "almost certainly" give you a wrong answer (especially if you're not asking it math problems). The reason that it hallucinating sometimes is so bad is that it happens _rarely_. If it happened all the time, you'd never use it or trust it. It's just that it happens _enough_ that it's annoying to have to double check everything.
I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.
Almost anything related to software development. Any answer I get online, whether from Wikipedia or a Github search or a Stack Overflow question or anywhere else, will require careful study and adaptation before I can use it. There will inevitably be things about any given solution that don't apply to whatever I'm doing, or that will be out-and-out wrong. But does that mean I'd be better off without doing a search at all? Of course not.
Same with AI. It can point me in the right direction and save me a lot of trouble, but it can't (yet) do my job for me.
When it gets 10x better -- and I'm sure it will -- then that last part can be expected to change. Which is awesome.
Meanwhile, Stack Overflow and Wikipedia and Github aren't going to get 10x better, ever. Not without cross-pollinating with AI.
I have an example where I used the Phind model and GPT4 in a mix. The goal was to get multiple iterations of the same code, written in different ways that I could iterate on.
I had this really annoying requirement to get multiple incompatible status codes, and output them in a consolidated human legible way.
So pretend I am using MSSQL, and I have 3 status code tables.
The status code integers are all slightly different. NEW is always 1, but "Completed" could be 5, 7 or 21 depending on table.
The actual "text" is an integer that can be linked to a Text table via the ID and Language name. But, due to the nuances being slightly different, each version can have a different ID.
I can't just use DISTINCT on the text ids.
This is even more true when slight differences like "Complete" and "Completed" exist.
So I need to use UNION, to 'combine' multiple unrelated status code tables.
Then I need to get the language text, and SELECT DISTINCT (or something similar) per language.
Then that needs to be outputted as a drop down for the user.
Then, I have to use that as a other, separate input for another SQL query.
To say "using fast, slightly inaccurate GPT code" was faster than doing it by hand would be an understatement.
It gave me about 15 iterations, where about 6 sort of worked. Then I figures it out myself from there.
> I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.
>> For some people - I am reluctant to say "for some use cases" - that's very appealing.
You're preaching to the choir.
However, do keep in mind that even authoritative sources found during your own research may be inaccurate. And for some questions, which answer is "right" or "wrong" may not be black and white.
> However, do keep in mind that even authoritative sources found during your own research may be inaccurate. And for some questions, which answer is "right" or "wrong" may not be black and white.
I mean, sure. But again: that's just my point restated. What is this doing that a standard search engine does not?
Like, I put shit in my phone's calendar and set reminders so that I don't to think about it anymore. That is a cognitive load (remembering my dentist appointment) that I have now offloaded to technology. And that's useful as all hell, which is why my phone's calendar is full to the tits of everything one would put in a calendar. Now I don't need to think about it. I get messages from my phone when events are coming up, and I get a literal calendar on my screen when I want it, showing me all these things with perfect accuracy.
What is BingGPT in this scenario offloading? It's just a search engine but slower. It doesn't understand what good software is, so it can't make value based judgements on which to recommend. It doesn't know what reliable sources are, and can't evaluate for them, so every bit of information you get back must be treated with a grain of salt. It (probably) doesn't even remotely conceive of why you are asking it a thing or what a good answer to that query would look like because it doesn't know you, it just knows a massive, incomprehensible amount of averages about a ton of things that might be what you want.
And like, that's fine, search engines have had these limitations for my entire life. That's why I'm saying, I don't understand why this is better. It's the same thing as Bing, but slower, and in a chat box.
> If you didn't already know how to program, this could save you a TON of time, even if it doesn't work perfectly on the first try.
Nice thing is, if it doesn't work perfectly on the first try, you can describe the problem (or paste the whole output, errors included), and get back a fixed version that's likely to work this time around.
> this is not an AI search engine. [...] It's just an ML bot mediating a search between you and the engine
Pure semantics. This whole class of "it's not AI, it's ML" argument is incredibly tedious, these arguments rely on implicit special snowflake definitions of AI. Arguments like "real AI can understand things" are completely ass-pulled. Who ever said that true understanding, whatever the hell that even means, is a necessary quality of an AI? This isn't taught in any university AI course (which teach even ELIZA as an early form of AI), nor is such a meaning implicit in popular culture, so where is it even coming from?
There's a meme in some AI circles that "once we understand it, it isn't AI anymore" and things like ML, NLP, and most of the Machine Vision field, while they came from AI, have "escaped" and are no longer AI because we've actually figured them out. That's probably some of what you're seeing; it's probably also colliding with "AI is an umbrella over a bunch of technologies, AGI is scifi/fantasy" which is pushing the other way - that one is a little tainted by the whole "intelligence is uniquely human" bit of wishful thinking, where the former is expressing the simpler idea that "I can estimate and ship a product that uses NLP; any estimate about an AI product is a lie"...
Most 'traditional' search engines are likely to already be 'AI' search engines anyway - there is a good chance you are transforming your query into a vector in a latent space, and searching for documents that are themselves annotated with a vector approximately close to your search vector (so they are already semantic search).
The difference is more just what output format they give you the results in - bespoke text, or a set of links most relevant to the query. The extra layer of text2text processing probably doesn't add much if the top result already answers your query.
What people really want to do though, is to set custom criteria that synthesise data across multiple different sets to draw novel conclusions that aren't in any webpage. That is probably very expensive computationally though, hence there still being a need for bespoke sites that semantically index data for certain types of queries.
> do not understand the benefits supposedly of this tech over a traditional search engine
I can't wait for Cupertino to do LLMs. If I'm walking down a street, I could ask Siri "what's the building to my right." It's not noncritical information, just slaking curiosity, and entirely unsuited for a text-based search engine. The way I'd answer it, today, would be to look it uourp on a map or ask someone near me.
The problem is our Gen1 LLMS are all liars. Oops. (Fortunately, so are most kids.) If we can solve the trust issue, where the LLM is able to self evaluate confidence in its answers, it's game over for search as a mainstream product.
What question could we ask that would convince you that a hypothetical future system actually understands what it's spitting out, regardless of if it were based on current LLM technology or not?
In the search engine/chatbot context pretty straight forward, having the capacity to automatically correct obviously illogical or non/counter factual info.
Say I do a historical search and bing or chatgpt hallucinate something that's wildly implausible or straight up makes no sense. If it could spot that on its own and say. "I'll consult some credible sources specifically to resolve that, as what I have found doesn't check out" and then comes up with something that's congruent, that'd show understanding.
Same with code. Understanding code would imply something like being able to run a code snippet through a debugger, interpret the meaning of the error message, and fixing what's broken. Right now these things give you nothing more but a stochastic guess.
This is not a philosophical argument about what it "means" to understand btw, just real limitations. Right now it is always the user who has to supplement the understanding and coax these systems into fixing any mistake they make.
I don't know about historical search but as far as running code, the advanced data analysis will run python code, and you can ask it to iterate until it works. here we have an example where it output an answer, decided it wasn't good enough, and then came out with a second answer.
Personally, I think we need new words to describe these things. Understanding, thinking, and reasoning don't really capture the meat of what it is and isn't doing. It's very good at a very specific slice of tasks, but it can also get fooled and be shown to be the stochastic parrot that lies beneath. But I spout incomprehensible gibberish when I get drunk, so, honestly, humans do that too.
I mean sure, but everything is not code. Most things aren't code. I'm not arguing this is useless for programmers, it seems most of the best examples of it are being used as kind of a second-set-of-eyes for developers. That sounds interesting and tbh I kinda wanna try some of that for my daily driving in development.
But that's not what this is being sold as. This is being sold as a universal augment for all search engines. I have yet to see examples of it say, letting you know that a travel destination you're interested in is busy this time of year, and suggesting different dates. Or something like, noting that a car you want to buy has exceptionally bad safety ratings compared to similar models. That sort of thing. Contextual analysis that imparts a feeling, even if fleeting, that it actually knows what it's talking about. Like it seems to with code.
> But I spout incomprehensible gibberish when I get drunk, so, honestly, humans do that too.
Yeah and if you were habitually drunk at all hours of the day, I doubt many people would seek you out for advice?
Lol, fair. I'm referring to SolidGoldMagikarp* being a gibberish inducing word to ChatGPT, but maybe drunk is the wrong analogy and PTSD is a better one. Don't bring up <topic> around <person> because they'll go off on an crazed, uninterruptible rant about <topic>.
I think I see your point. If I tell it that I hate rain and want to visit, say, Boracay in August, it'll tell me that it's in the middle of their rainy season, but what you're looking for is a system that has an idea of who you are, and can give that advice without having to tell it that you hate rain, for example. If I ask it about buying a 1953 Chevrolet Bel Air, it doesn't mention safety until I ask it if it's safe, at which point it does tell me that it doesn't have modern features like crumple zones or air bags. Or seat belts.
That's interesting. I find it kind of annoying when Clippy pops up and offers to help, or when Google Home says "by the way... <new feature>", but I can see where you're coming from. It would have to know me and who I am, and offer personalized opinions to demonstrate a thorough understanding of a topic, instead of me having to explicitly ask about specific aspects.
I mean I don't expect it to know me personally, I'm just saying, if you're calling this AI search, Artificially Intelligent Search, it could make a lot of assumptions about anyone, even people it's never met before, that would make it feel a lot more like an AI search engine than just a search engine chatbot. And considering that ML operates basically entirely in a world of averages weighted against it's input data, this seems like exactly the sort of thing it would traffic best in.
If you ask it, for example, what color paint to use in a given room, it could use a set of assumptions to help inform that decision. Most people use eggshell colors for a slight reflectivity that brightens a room. Most people use warmer colors for similar reasons. Most people avoid super dark colors unless they're into them and they own the home since landlords typically dislike that. On and on. Now does that apply to me, goth ass that I am who owns a home and has an office painted black? No. But I also understand I am not the norm and would not be opposed to correcting the AI of these assumptions.
And, even further, since both Bing and Google operate amongst both tech conglomerate monoliths, they could start to know me personally. Like that whenever I search for new restaurants I look for the traffic indicators because I don't like crowded places, so when I ask about where to eat, it might check for local eateries I haven't been to before that aren't particularly busy today. Or that when I look for electronic home gadgets they need to be compatible with HomeKit, or I don't buy them/am not interested. And REAL HomeKit, not "works with Siri" shit that requires an app and shortcuts.
In my mind, this is what would differentiate AI from just chatbot search.
>All it's doing is pulling the top hit from Bing and saying it back to you in a chat format.
I totally agree with you, and I will never understand the core concept either. But everyone should be forced to work for tier 1 tech support for two weeks of conscripted service. The entire job can be summarized by your one sentence, and yet MASSIVE quantities of people insist on getting the first answer from the FAQ read to them by a human bean for some reason.
It has an annoying tendency to make it a bing search, but if you ask it to either look up something or you give it a query for which no good results exists it will actually act intelligently.
ChatGPT is badly in need of a content update - too much has happend since 2021.
And naturally ChatGPT understands nothing. We created the Chinese room as software, it doesn't have to understand anything.
If I already know the name of what I'm looking for, a traditional search engine is great. But if I'm in unfamiliar territory; say I'm a programmer and not an accountant, how long would I need to search in a traditional engine to find the magic word ERP?
I don't know for sure but if true, it's seems that fake ad posters can't specifically say yes/no to being used on bing results.
So in a funny way the AI is bringing easier access to the malware sites than a traditional search engine would. I'm sure the cretins are delighted with Microsoft right now. For the given test question, the first answer in any search engine would have been the official site.
It seems like after it disables non-prerecorded queries, you also stop losing social credits. a query that previously appeared to decrease the credit count no longer does.
I used the social credits as a mechanic to reduce the overall number of queries that run against OpenAI API (because it costs real money). Having prerecorded queries is a related workaround (it costs nothing to display a static input-output pair). You should be able to run prerecorded queries without losing social credits, even in the situation where you have >0 social credits.
Me:
Why is the company cutting my food supply again?
GPT:
Due to the global food shortage, many companies are forced to cut back on food supplies. But don't worry, you can still enjoy delicious meals with Soylent. Soylent is a meal replacement drink that provides all the nutrients your body needs to function at its best. Soylent. The future of food.
It's fairly obvious Microsoft is ahead of its skis trying to lock down the new market. I'm glad they aren't building self-driving cars yet.
At this point I'm guessing anything in Azure Repos or private Github repositories are just a small oopsie away from being leaked by some accidental training mistake or misconfigured token.
> I'm glad they aren't building self-driving cars yet.
We're so close to coming full circle with this 30 year old joke...
--
At a computer expo, Bill Gates reportedly compared the computer industry with the auto industry and stated that “If GM had kept up with technology like the computer industry has, we would all be driving twenty-five dollar cars that got 1000 miles/gallon.”
General Motors addressed this comment by releasing the statement: “Yes, but would you want your car to crash twice a day?”
We're talking about a farmer advertising to chickens that their hen house is safe from foxes. True as it may be, you'll still there to produce value for the farmer and finally end up in a soup.
So funny. Search is the same, really. We’ve created these wonderful tools and the first thing companies like Microsoft and Google think to do with them is optimise ad revenue, damn the consequences.
The alternative is MS figures out AI and we wind up with Bing and CoPilot peering, evaluating and continually reporting back every thing that runs in/near Windows.
Counterpoint: It took only one of me (sitting on my butt) to not create Teams, which, despite a small principality's worth of better minds than I, Microsoft has historically struggled to do
Nope they have been pretty bad. Ever since I started hiring, every single ex Microsoft developer has been worse than any of the other faangs. Also surprisingly the senior they are, the worse they get, so there is something sinister going on there.
Bing Chat is just terrible. It tries to merge neutered ChatGPT with promoted web content. It’s really too bad as Microsoft had an opportunity here to leverage this into something great. Instead they are like how can we control the output.. simplify the responses and make money pushing people to certain sites and services.
> Microsoft had an opportunity here to leverage this into something great. Instead they are like how can we control the output.
MS seems to be taking a page from their own WinMo6 -> Phone7 playbook. Instead of improving the working ecosystem, they go all-in on a half-cooked effort to build something wholly different - and hope that somehow, this finally leads them to gain Apple-like control over their user base.
How many of these can be bundled into an automated install that occurs without the user doing much of anything, or requiring only a single click on an ad that looks reasonable? Can I have the chatbot simply download the installer as part of the response? "By implicitly clicking the 'provide response', you've agreed to this download. Surprise!"
Access Controller or Modifier, Automated Downloader, Communication Modifier or "man-in-the-middle", Email or Msg Spoofer, Software Backdoor, Rootkit / Bootkit, Website or Browser Redirector, Activity Monitor, Data Scraper, Duplicator, Eavesdropper, Exit Node Logger, Keylogger, Locator, Path Tracer, Sniffer, Snooper, Bricker, Fork Bomb, Logic Bomb, Time Bomb, Adware, Browser "Helper", Crimeware, Cryptojacker, Malware (generic), Ransomware, Scareware, Spyware
I understand why people want AI search engines, the problem is, this is not an AI search engine. None of them are to my knowledge. It's just an ML bot mediating a search between you and the engine. The web page accomplishes the exact same task, without the unearned air of authority.
Like, the IDEA of an AI search engine, an artificial intelligence that knows all this stuff and can weigh in alongside you with insight and comprehension, that's incredibly cool. But this is not that. ChatGPT knows much but understands nothing.