This story is really interesting for all the not-specific-to-quora parts. It’s pretty obvious that quora has a strange culture of know-it-alls trying to show off, but this AI product makes it clear that the act of showing off is the product. The answers are garbage and they obviously don’t care. The key action for sale is not finding an answer, but writing it.
What’s really interesting is that Quora product team decided they wanted to juice engagement with fake AI created posts, and real humans are complying to answer them. Questions that make no sense, and no human asked. Quora is proving (again) that people will engage in parasocial behavior with LLMs. We’ve seen AI girlfriends and companions, we’ve seen tutors, and knowledge assistants. This is the opportunity to brag - artificial students (likely cut from the same API as those tutors).
I can only assume that question-answering-users are the real market they advertise to, profitably. This would imply that this strategy is profitable or could be profitable for them. The nice thing about this compared to more personalized LLM companions like AI girlfriends is that the inference costs can be distributed across all users by sharing the questions. I would posit that there exists an opportunity to make a Twitter or Reddit clone that explicitly intermixed bots. Maybe even exclusively bots. People want to talk on the internet and told they’re special. The interaction doesn’t need to be organic.
For proof, look at all the top-level comments on this HN post with no engagement, by users who have repeated the same pattern discussed about quora.
Quora knows that it's about attracting people to write answers, more than people with questions. They've had a number of tactics to get more questions so that people could write answers.
At one point, they paid people to do it -- which went about as well as you would expect. Users complained like crazy and it lasted long, long after it was an obvious failure.
Their idea wasn't completely insane. There are questions that people ask Google that they'd never ask a human. Stupid questions that people will nonetheless answer. And Quora wanted to be the place that Google directed them. Combined with a high reputation from its early days, it was well placed in SEO.
They never figured out how to turn that into money, and they've floundered about. They managed to have a bunch of smart people wanting to show off -- actually giving meaningful answers -- and they squandered it.
>look at all the top-level comments on this HN post with no engagement, by users who have repeated the same pattern discussed about quora.
So responding to a post but not getting responses is the same as responding to slop with a financial incentive? I'm not sure I fully follow.
I only very slightly agree in the fact that I rarely make top level comments unless I feel I have something unique to share. And I know that's not necessarily a common enough sentiment (lest, very few people would make top level comments). But given how I usually check the front page or active, my retort 99% of the time is already posted elsewhere. Half the time it already has responses too. So most of my activity is more responding to other comments to try and start a discussion.
And ultimately that is my goal in a forum: 1) ask about and understand others' experiences and 2) try to share my own experinces where appropriate. I have my share of https://xkcd.com/386/ as well, so I'm not perfect in regards to "hearing myself speak" or falling for the occasional flamebait. But I'd like to think that most of my engagement no forums like this passes a few basic tests:
- does this comment contribute to the conversation?
- do I have a goal in mind with my comment? (e.g. if I'm asking a question, am I asking that user? Do I hope anyone at all chimes in with an answer? If I'm sharing a story, does this story hold potential value and lessons for the community?)
- Is the comment civil? Can I make this comment without involving the person (unless the person is the goal)?
> So responding to a post but not getting responses is the same as responding to slop with a financial incentive? I'm not sure I fully follow
HN doesn’t have financial (?) incentives (but does Quora?). The point was mostly about the quality of content, the type of content, etc. I don’t mean to criticize anyone, I think it’s human behavior.
The root question of my claim is: When you create a (top-level or other) comment, how do you measure if it “contributes to the conversation”?
Most comments probably have zero engagement, while the top few comments drive the conversation. Part of that is the UI driving people towards the top. My claim is that most commenters, just like most quora users, are answering out of their own need to speak. I think the XKCD link describes the behavior well - and my claim is that it also includes sharing personal anecdotes and opinions and similar, not just “correcting misinformation”.
I have my own share of comments with no upvotes and no replies, which I can only assume means no one found it to contribute. I mean no disrespect towards others in similar positions. But does its presence in the database for casual passersby’s mean anything despite that? Despite the goals and intentions, does it matter? Should it just be routed to dev/null to silently fill my desire to speak? Should an LLM just entertain me with artificial conversation for entertainment?
The opportunity is for the site to become the bots, and silo each user to their fake world of LLM posts tuned for them to engage with. It’s a site with of perfect knowledge of bots, which would be a perfect advertisement environment.
I wonder if it would really come down to AI if you did the user research, maybe focused on the “onboarding” process. I remember a time back when you would get a Quora result on a search query and you could click on it be find a few answers with comments on them to help you determine their merits.
This is completely anecdotal, but these days I honestly can’t even find the answer if I click on a Quora result on a search engine. Instead I’m created by “similar” questions and what not in an UI I don’t really understand too well. Now, I was never a Quora user, as in, I never signed up for an account. So I’m probably not in their target audience. With their modern entry point, however, I’ve simply banned their domain from my search results along with sites like Pinterest because they are essential just a wasted of time if I accidentally hit their sites.
Maybe that doesn’t matter. I would have probably never signed up to actually answer things, or even ask, but if anyone who would be potential user is like me. Then they won’t ever get to the “onboarding” process of joining the site, and I’d wager that was more damaging than AI. LLM’s are more akin to the final nail in the coffin for a lot of these sites who have made it so user hostile to join their “communities”. Again, it’s just anecdotal, I’ve done or read no research. But I do think that it’s interesting that we’re now at a point where many people will include “Reddit” in their search queries when looking for answers, and Reddit is the easiest side in the world to join or even read without signing in (at least old.reddit is).
Yeah, I've noticed this too: seeing a Quora question that's exactly what I'm looking for, clicking it, and not actually finding the question which the SERP said was there. At least this used to happen back when I actually got Quora results, I'm not sure if its search ranking has dropped or if Quora results get caught by my banner blindness these days.
There are some sites which will generate a bunch of pages which show up in Google searches but which are really just site searches. You google a thing, and a site pops up with a page called "Articles about the thing", but when you go there you see it's just a site search for "the thing". I wonder if Quora is doing something similar? Maybe the question doesn't actually exist, and the page in the Google search result is just a disguised search page?
For some insane reason, Quora's default view is to show you "answers to related questions".
If you click the dropdown box in the upper left, you can change from "All related" to "Answers". That should turn up the answers that the search engine was seeing.
I can generally give you at least some explanation for why Quora behaves the way it does, but I'm afraid I just toss my hands up at this one.
Quora started out fairly interesting albeit often with answers written by self-evident self-promoting know it alls. That’s mostly a decade in the rear view mirror today it’s filled with people trying to shill, promote, or I don’t know exactly. In any case it’s largely low quality which of course drives any high quality answers even less likely to be contributed.
From my point of view the underlying problem here seems to be that someone in Quora product team feels that the best way to drive engagement is to predict what the user might want to know about and then spam them with it, rather than letting them discover on their own or use their feedback.
I stopped using Quora 8 or 9 years back, because despite how much time I spent curating the feed I was served, the questions always seemed to veer back into the questions on life experiences, `write a short story in 3 lines` or about relationships. This was most probably due to topics like these being a fad in India during the time and often received a lot of answers or upvotes.
Every time I marked an question as something I was not interested in, because of the topic, Quora assumed that I did not like the author's answer and then proceeded to show 5 other questions the author has responded to. I assumed it was because of some bug on their side ignoring my preferences of topics, until I spent a week in Dubai and saw most of the questions in my feed match the topics I marked as interested.
I'm pretty good at simply filtering out low quality results. But Quora is so often so aggressively topping SEO. and I've NEVER gotten a useful answer out of clicking on a result. It's pretty much the best way to explain what a hallucination in an LLM is, before such concepts were being talked about at large. If you just skim it kinda looks right, but the more you focus the more you start to realize how little sense the response makes.
IT finally broke me in 2022 and had me download uBlacklist. To this day it's the only site on there.
I cannot remember a time when quora was not broken, so AI certainly didn't ruin it.
The UI and their login popups have always been horrible enough that I try to avoid them. But in addition to that, most specific questions - the ones that would benefit the most from a nuanced human answer - get answered with generic copypasta. My impression is that Quota tried gamifying the process of answering... And they succeeded, to their detriment. Now people keep answering questions that they have no clue about just to make their presumed importance score go up. It's a bit like those grifters that order a ghost-written book in their name so that they can already claim authority before they learn a new skill.
This is an article [1] from 14 years ago where Arrington characterized Quora with “People say they feel smarter after they use Quora” and “It’s kind of like Wikipedia [2]. This is the long tail of information”
> Arrington characterized Quora with “People say they feel smarter after they use Quora”
But… they still do; as the article says, and it’s completely correct, it is essentially selling _feeling smart_. That’s not the same as _being_ smarter, though. I’m not sure there was ever _really_ a golden age; it wasn’t as bad as it is today on launch, certainly, but it was never great.
> “It’s kind of like Wikipedia […]”
Except without Wikipedia’s editing process, which is relatively good at keeping out incorrect things written by internet knowitalls. It _always_, right from the start, had a big problem with the confidently incorrect.
Reading the author's explanation of how Quora became awful by using AI to generate questions gives me a view into the dystopian future where the deluge of AI-generated books on Amazon pollutes and undercuts the whole concept of "knowledge found in books." Instead, we will have books with the verisimilitude of knowledge because the authors did not have the expertise or ambition to edit out the hallucinations.
The post shows limited understanding of what LLMs are. Ok, so Quora has a bad model that generates "Why doesn't Donald Trump shave his mustache?" type questions. That's Quora's problem.
But in general, a frontier LLMs like the ones trained by Open AI would not make such silly mistakes in their output. Its "fancy autocomplete" is trained on millions of documents that include detailed description of Trump's face and the presence of the word "Trump" in the context window would activate countless weights that relate to "orange hair", "pussy grabbing", "bribes" etc. Each word or sequence of tokens can be understood as a vector in a space of say, 12288 different dimensions, and for each round in the algorithm the entire context is refined so that each of those dimensions would access finer and finer details of the subject matter present in the training data.
So the model really does "know" Trump does not have a mustache. What it lacks (or has very limited ability to do) is rational exploration of the information it holds. It's like an idiot savant with the mental ability of a 4 year old that has somehow memorized the entire information that exists on the internet.
> But in general, a frontier LLMs like the ones trained by Open AI would not make such silly mistakes in their output.
This is beginning to wear a bit thin. Every public application of AI produces obvious nonsense, and this doesn’t appear to be changing. The inevitable defence is “oh, they used GPT3.5 instead of GPT4/oh, they used GPT4, but not the version that was only publicly available for three weeks in 2023/oh, they used precisely the right GPT4 but, when sacrificing the goat while petitioning Roko’s Basilisk for useful output, they used the wrong sort of knife”.
Apparently, literally no-one can use these things properly, at least for publicly visible applications. Either that, or they’re just a bit shit.
Founder of Quora Adam D'Angelo is still on the Open AI board board and the founder of another AI startup called POE. I am sure Quora's QnA data would have been used to train lot of AI models. Now they are just taking out the humans.
But that's literally what it is. The only reason you can have dialog-like interactions with language models is because they have been trained with special "stop tokens" surrounding dialog, so the model can (generally) auto complete something that looks like a reasons, and then the inference engine can stop producing text when the model produces the stop token.
I think understanding that helps me get more out of them. I feel like I am better able to provide information to the model with the expectation that it will need that information to autocomplete the dialog that I want.
Indeed, it's actually worse. You should only use it for stuff where factuality doesn't matter. Verifying that an answer is complete and correct is more work than consulting a reliable source in the first place.
I think it's an accurate assessment and still get some use out of LLMs. Brainstorming various things, tutorials, entertainment. Straight questions with a factual answers really are a poor fit. Just yesterday GPT3.5 told me the Belgian city famous for its mustard is Dijon.
I see LLM's as next-generation search. So much of everything created on the internet is useless, outdated, false or just irrelevant to the thing you want answered. It gets increasingly harder to find what you need to get your job done.
A LLM can extract relevant information from the garbage pile faster than I can sift through various forums and mail threads via Google. The more I use it the more I feel that it's a greater step in information-finding than Google itself was when it launched.
You need a second source of actual truth to verify it, of course, but that's always the case anyway.
For coding it's easy, the code works or it doesn't. Lawyers and such have a harder time when they use a LLM trained on "the internet", but one can imagine a LLM trained mostly on actual case histories and law texts doing much better for example.
The best uses of AI (c. 2024) is indeed when it's just "fancy autocomplete". It's great to help finish up boring, repeitive, but necessary tasks so you can focus on the high level.
But the tech cycle has been hyping it to all heaven (or hell), and claiming it can simply reproduce your imagination on a whim. Maybe in another decade, but that doesn't stop companies from buying in and trying to replace skilled labor ASAP.
There's a qualitative difference between the raw LLM model, which I think it is fair to describe as "fancy autocomplete", and AI-as-actually-deployed, where the LLM has access to function calling behind the curtain.
I still use mixtral for exploratory questions and it’s so good, especially with well-crafted character cards. I think there are three types of people: those who tried it, those who never tried it, and those who only tried GPT-<n> and boring alikes.
Totally agree. Well before AI issues the platform was turning a blind eye to low quality questions and answers, ones obviously written by people with limited experience and judgement, focused mainly on chasing completion targets over correctness.
I found this is a “marketing” technique, that I first encountered on LinkedIn Groups.
A shill asks a specific question, which is then “answered” by a bunch of other shills, recommending some product.
I remember answering a technical “question” on the Swift Language group, in LI. I gave a simple, accurate, answer to what I considered to be an earnest question about a basic dilemma. It was something that I knew all about, and had a lot of experience, dealing with.
My answer was completely ignored, but the asker engaged many of the ridiculous “answers,” and I realized what was happening.
I left that group years ago (along with every other LI group, and, pretty much, LI, in general), and now see this technique being applied everywhere.
It may actually be a technique, taught in school. There's a lot of underhanded behavior in that field, especially in trying to drive talking points and corporate glossaries.
I know that was a joke, but the reality is almost as creepy.
I worked for a corporation that drove marketing terminology to use their glossary, and saw the thinking firsthand. It's not paranoia. In fact, it makes perfect sense, and may not even be that underhanded. They submit huge reams of text, interspersed with their talking points, to any and all publications, news outlets, whatever. Some percentage of that gets published; sometimes, verbatim, and that reinforces the glossary.
The underhanded stuff, is when they start sneaking it into places that should normally not be "infected" by this behavior (like news organizations that accept corporate dogma as news articles, and put their own bylines on corporate fodder).
AI is gonna absolutely supercharge this.
I could already do this with ChatGPT, but that's kind of clunky and primitive. Custom LLMs will make it almost completely invisible and undetectable, and create an enormous tsunami of this junk.
Quora was already ruined and pretty much worthless for a very long time. When a query on google has quora, I hardly ever click on it as I know a) it won’t answer the question most likely b) I walk into this incredibly confusing ux experience.
Engagement optimization. 5% of users might happen to see something that will keep them on the site, but 95% are confused and waste their time. Sites only optimize for engagement these days.
Quora was doomed from the start. When StackOverflow started the founders and the early community put a lot of thought and energy into not becoming one of the niche expert exchanges that already existed at that time. Now, Stackoverflow failed, but not by repeating the mistakes that already had been made. That's not something that could be said about Quora.
Maybe I am too harsh, and measured by its usefulness[1] it is not really a failure. I was involved in the early SO community and I think it is failure compared to what it could have been.
[1] However small that may be, even I would have a hard time to argue that it is useless or irrelevant.
I used to be an active Quora contributor and that platform is not worth saving. It kept getting worse and worse to the point where it's entire existance is just confusing.
It had too many Indians. Not that that's absolutely a bad thing, but it was bad for me because the answers tended to reflect Indian cultural perspectives and they weren't very relevant to me.
On a seperate note, why do people feel the need to exaggerate their Quora profile/education level/accomplishments? I'm not the only one who's noticed how many Quora profiles are filled with ludicrous, self-aggrandizing descriptions, right?
I'm not a 100% sure, but I've got a feeling that having a well-regarded Quora profile became a Thing in India and other parts.
Then people started spamming "answers" to every question they could even tangentially produce some gibberish for.
10 years ago Quora was a place where you could get an actual expert answering questions (a thing they should've done on their own site tbh[0] with references and actual insight. Now it's just like what experts exchange turned into.
Going into the article, I was expecting that Quora used AI to answer questions. Instead it uses AI to ask questions. That's pathetic and fascinating at the same time.
It's pathetic because it shows there aren't enough people asking questions to keep the site alive, which means it's already dead and exists only in a kind of zombie fashion.
But it's fascinating because it begs the question, why are people taking time out of their busy day answering pointless questions devised by a bot that doesn't know anything (and that isn't even interested in the answer)?
Quora is now a content farm where contributors work for free. That's horrible... and, in a very practical way, genius.
What’s really interesting is that Quora product team decided they wanted to juice engagement with fake AI created posts, and real humans are complying to answer them. Questions that make no sense, and no human asked. Quora is proving (again) that people will engage in parasocial behavior with LLMs. We’ve seen AI girlfriends and companions, we’ve seen tutors, and knowledge assistants. This is the opportunity to brag - artificial students (likely cut from the same API as those tutors).
I can only assume that question-answering-users are the real market they advertise to, profitably. This would imply that this strategy is profitable or could be profitable for them. The nice thing about this compared to more personalized LLM companions like AI girlfriends is that the inference costs can be distributed across all users by sharing the questions. I would posit that there exists an opportunity to make a Twitter or Reddit clone that explicitly intermixed bots. Maybe even exclusively bots. People want to talk on the internet and told they’re special. The interaction doesn’t need to be organic.
For proof, look at all the top-level comments on this HN post with no engagement, by users who have repeated the same pattern discussed about quora.
reply