Many people add "Reddit" to their search queries to find authentic product reviews. We fine-tuned a BERT model to extract product mentions from over 4 million Reddit comments and posts with Named Entity Recognition (NER). The result is a list of the most popular products across many subreddits.
No platform (including Reddit) is resistant to fake reviews and spam, but we think it's happening less frequently here for various reasons:
- Redditors and other forum members are more interested in boosting their ego by showing their depth of knowledge on the topic (and correcting others on the topic), whereas corporate websites are more interested in raking profit by displaying (potentially) dishonest information.
- Enthusiasts in subreddits are pretty good at spotting dishonest or fake content, which results in immediate downvotes. The whole karma system helps with trustworthiness.
- Most subs are moderated well and spam gets removed quite quickly
That being said, good fake reviews are technically almost impossible to detect, even with sophisticated network analysis of the reviewer's profile.
Looria is still in Beta and we're working on improving our classification, summarization, and sentiment analysis. Let me know what you think!
I should be happy that you have automated a process that I manually perform frequently, but instead it gives me anxiety.
The set of incentives means the more successful you are, the more you will directly be attacked in the form of bad faith posts. Reddit is not better because it is a better system or because it has better moderation. Reddit is better because there isn't an entire industry (SEO) devoted to corrupting Reddit posts yet. Even google hasn't done a stellar job of winning that arms race, and they have vastly more resources.
You add value by performing the manual labor of finding and "reading" through posts, but the core value is generated by Reddit's abuse prevention team. If you were to be acquired by Reddit, my trust would go to zero. If you took outside investment, my trust would go to zero. My trust is already a little low because wirecutter specifically avoided items they couldn't get affiliate links for and now they are completely corrupted by new ownership. Wirecutter was the last review site that I used to make a purchase decision.
More than anything, I am worried that the dark mirror of this website, the "reddit optimization" industry is probably just as advanced and validated by efforts like this.
Goodhart's Law in action. Any "product quality" metric will be gamed to the point of uselessness as soon as it gains traction. In other words, services like this one can only work as long as nobody knows about them.
The only stable solution is for each person to curate and maintain their own set of sources, so that there are no high-value metrics for marketers to target. Exactly the opposite of what this service is trying to do.
You are obviously correct that the more value this tool has, the more work bad actors will do to take advantage of that, reducing the quality of the result. However, I do think Reddit is more resistant to it than some other platforms due to the large amount of distributed work done by the participants. Assuming that a given subreddit is moderated in such a way that the vast majority of participants are individuals interested in high quality products then the Reddit voting system does help to keep other stuff out.
I hate that it is in my interest for this to never become popular. It's a situation where my faith in the listed products is inversely proportional to the popularity of your site.
If the site becomes influential, brands will be motivated to pollute the source data, ruining both the usefulness of Reddit for reviews and Reddit communities as a whole.
There is also the fact that as you become more influential the value of your influence increases proportionately. The offers for "selling out" just keep growing bigger as you grow more popular. It also becomes easier to convince yourself that you're not really "selling out" if you just take money for this or that. And, like any business, of course you do need to monetize somewhere.
It's just an unfortunate alignment that goes against the core thesis of the (great!) idea.
Aren't reddit reviews already influential, isn't that why the site exists in the first place? If you're right, then it follows that this is already probably happening :)
that was the first thing I thought of. I know the owner means well but if reddit becomes a bigger target that it already is for mass marketers then the value of the reviews there become even less important as marketers dilute the reviews and it becomes a spiral into uselessness. Maybe it could be mitigated by adding things like "commenter history" and give commenters that have been around for a while with more comments a heavier weight that news accounts that seem to only post in one or two subreddits a couple of times and then disappear. Maybe somethings like
weight ~= max(num_months_existencenum_commentscomment_score, 0 account not active in the past year) or somesuch, obviously adjusted.
Looria's ranking algorithm has no effect on what happens to the actual quality of content for Reddit users. Volunteer moderators cannot be expected to keep up with paid bot programmers. Paying objective moderators is essentially impossible, and the whole business model is built on harvesting review data for free anyway
Reminds me of the old fivestar.io - which aggregated Amazon reviews back when they were somewhat reliable. It would bucket them by price category (highest rated products in the $100-$250 category)
Back when searching "best [product] 2014" was actually a way to find good stuff on the internet - fivestar was able to generate those answers on-the-fly. I'd find I'd spend an hour or so researching something like a router, plug "router" into fivestar, and my pick would be at the top of the list. Those days are gone for the same concerns others have raised for your site. fivestar worked until reviews started to get gamed.
This is an awesome website and I see myself using it. I hope it doesn't meet the same fate!
Seems Amazon is your go-to when I drill into a specific product. It's clear to anyone though that an "REI" tent is not going to show up on Amazon (or a "Specialized" bicycle as another example). Maybe you can broaden your outbound link pool and use an algorithm to determine where to send the click.
> We're running sentiment analysis to identify the emotional tone behind the mentions. Handling multiple mentions in a single sentence and filtering out things like questions that shouldn't count as an opinion requires some effort. In addition, a minimum sample size is required to get statistically relevant results.
That's great that you are taking that into account, but I'm not totally sure how to interpret the value bars. So I presume a high value means lots of positive engagement. But how does a lot of negative engagement show up? I think it's valuable to know when there is a lot of chatter about something and it's mostly negative. Or when it's controversial.
Maybe something like a statistical distribution graph or weighted color gradient, so you could tell at a glance the density/quality/depth of discussion, and also the distribution of sentiment.
Agree. And while I don't agree that the T420 is a worthwhile recommendation, the reddit product recommendation groupthink on that item hasn't changed in nearly 10 years.
Funny, the demographics who post/comment on Reddit are just about the last people I want product advice from.
If there was a "salty old mfer posting on a forum in 2010" search I'd be much more inclined to use that. Google used to do ok for finding those but the SEO arms race killed it.
There is an FAQ on each page with a question "What's the time frame for this analysis?", and the answer is posts and comments from within the last two years
As someone who has been doing this for over a decade with Twitter data in a specific niche (https://reviewsignal.com - for web hosting) it's interesting to see this getting so much attention. I played with Reddit data years ago as well but never moved forward with it. The volume really wasn't there for my needs. Why the volume mattered is exactly the reason you are kind of touching on, fake reviews. You need an overwhelmingly large sample size to drown out fake reviews.
I am also curious what sort of spam filtering mechanisms you have in place? Just the spam filters before content ever hit sentiment analysis or relevancy analysis was 98% in my data. I imagine Reddit is better than Twitter, but there is still is going to be spam. What measures do you have in place and do you determine them?
Do you take upvotes into account with weighting reviews? That was/is a concern I had when working with reddit data. I used retweets as a proxy for sorting popularity, but not any other weighting.
I'm definitely interested in this segment as I've been doing it for a long time, if you want to talk please reach out my contact is in my profile.
With no ill assumption and no aggression intended: what is your business model and idea of giving back to the communities you are distilling the value from? And how much did you think about the incentives this would place on spammer and SEO artists to pollute your data source?
The way we use it in our apps is to take a pool of identified experts for a search query, augment it with relevant news from other sites about the topic, sprinkle a little bit of quantitative KPI data over it, calculate various NLP scores, and then shake it well and voila, you get a nice aggregate of socially validated data.
Works well and it reduces the noise significantly.
Would be awesome feature if you could input your reddit handle and it would automatically filter to those subreddits that you subscribe to, rather than the full list.
Love it! Pasting my top level comment here: This looks really neat! Do you define "Products" anywhere? It would be cool to have categories like 'Most discussed cars|phones|computers|movies|websites|etc'. The nerd that I am would love 'Most discussed programming languages on *'
P.S. Are you raising money from angels? Drop me a note if you are. breck7@gmail.com. I really need something like this!
I think it's neat. There's lots of negative feedback on here, which I would encourage you not to get down about. As always on the net, folks are probably far more likely to leave a message if they have some objection or disagree with some aspect, rather than if they just plain like it.
The picture shows it does have armrests though. It would be great if you could have some button there to report incorrect information for each product.
Most discussed makes it sound like you highlight polarizing products, not necessarily just good ones. You may want to highlight your sentiment analysis on the front page. It took me some digging to find it.
A feature request - I’d like to be able to adjust the time frame. Things like basketball shoes move a lot year to year, and it can be hard to find ones from 18 months ago.
This is great, but the sentiment analysis is way off. Take these 'negative' reviews for the Staples Hyken office chair:
> For months i was waiting for the Staples Hyken Mesh chair to drop to a good price. I pulled the trigger at 150ish, received it really fast, and set it up right away. I actually really love the chair, totally worth it. Perfect size for me and i can sit in in for 12+ hours just fine.
> I personally haven't found the thinks to be more comfortable than other chairs, I'd rather go with a staples Hyken. Also, I see these go for much less. This looks like a V1. I sold one like that for $200 in Virginia.
> Hello all, I went on a deep plung into chairs over the weekend to find a good new chair (instead my junky racer/gaming chair that hurt my back and neck. I ended up finding the Staples Hyken and Staples Dexley. I went to store and tried them both out. I really liked the Hyken way more than the dexley....
In the small number of negative reviews I've looked at it seems like around half of them are categorised incorrectly.
GPT-3 does a much better job of sentiment analysis. Parts in italics are output from GPT-3
------------
I am a sentiment measurement bot. I take arbitrary text input at the INPUT: prompt then reply with a SENTIMENT: output that includes a numeric sentiment indicator ranging from -5, negative to 5, positive, a PRODUCT: output that extracts the name of the product being described and a CONFIDENCE: output that estimates the confidence in in the rating, from 0% where there is no confidence, to 100% where there is complete confidence.
INPUT: For months i was waiting for the Staples Hyken Mesh chair to drop to a good price. I pulled the trigger at 150ish, received it really fast, and set it up right away. I actually really love the chair, totally worth it. Perfect size for me and i can sit in in for 12+ hours just fine.
INPUT: I personally haven't found the thinks to be more comfortable than other chairs, I'd rather go with a staples Hyken. Also, I see these go for much less. This looks like a V1. I sold one like that for $200 in Virginia.
SENTIMENT: 3
PRODUCT: Thinks
CONFIDENCE: 60%
INPUT: Hello all, I went on a deep plung into chairs over the weekend to find a good new chair (instead my junky racer/gaming chair that hurt my back and neck. I ended up finding the Staples Hyken and Staples Dexley. I went to store and tried them both out. I really liked the Hyken way more than the dexley.
You can even use it to extract comparisons. Again output from GPT-3 in italics:
---------------------
I am a comparison evaluation bot. I take arbitrary text input at the INPUT: prompt then reply with a COMPARISON: boolean output that indicates if the text is a comparison between products and a BETTER: output with the name of the better product and WORSE: product with the name of the worse product
INPUT: For months i was waiting for the Staples Hyken Mesh chair to drop to a good price. I pulled the trigger at 150ish, received it really fast, and set it up right away. I actually really love the chair, totally worth it. Perfect size for me and i can sit in in for 12+ hours just fine.
COMPARISON: FALSE
INPUT: I personally haven't found the thinks to be more comfortable than other chairs, I'd rather go with a staples Hyken. Also, I see these go for much less. This looks like a V1. I sold one like that for $200 in Virginia.
COMPARISON: TRUE
BETTER: Staples Hyken
WORSE: Think
INPUT: Hello all, I went on a deep plung into chairs over the weekend to find a good new chair (instead my junky racer/gaming chair that hurt my back and neck. I ended up finding the Staples Hyken and Staples Dexley. I went to store and tried them both out. I really liked the Hyken way more than the dexley.
Yep! I’m guessing one of the openly available models would do similarly as well. Obviously more expensive compute-wise, results are pretty good though.
Do not trust product recommendations with a focus on affiliate links to Amazon.
I only checked best Laptops. There wasn't a single Laptop which was not listed on Amazon. Case closed for me.
However, I would really like to build this myself or have an open source alternative. Seems like a good idea to search for product recommendations like this.
Ah, the old "the internet can solve information aggregation, for free, for all" case. From price search engines to review aggregators to services-searchers, the vision of the internet was that it would allow us to find anything, properly annotated and metadata-tagged, at no cost to anyone because information wants to be free. Easy to find the lowest price, from a vendor who has good reviews, for a product that appears to work and do what it promises, with some tips and tricks submitted by current users for a bonus... all just open and available. Those were the days... that never were.
Every price search engine, review aggregator and services-searcher (think Angie's List or other "find a trusted provider" orgs) since the 90s is either gone or just does affiliate links. So, even if you see multiple stores with links (Amazon AND Best Buy, 2 whole options!), it will only be to stores with a payback vig. Some of these sites will also use advertising to help pay the rent, and of course, eventually allow sponsored products to show up higher in the list.
I loved the idealism of the 90s, when we thought the mere presence of the internet would bring these things to life. It's too bad that the only way to make this stuff work is to basically sell out, if only a bit... and sometimes, the whole farm.
But is it always a sign that such a site or info provider can't be trusted? In some cases, sure. But note that Consumer Reports, one of the oldest and most trusted product recommendation orgs in the US (not trusted by all, of course, but by many) charges a subscription fee AND has affiliate links, and they still continue to survive with some sheen of quality (though, again, some think they are buffoons... I know this thanks to the Internet).
You misunderstood, OP isn't complaining that there are amazon affiliate links or that this site is trying to make money.
They're pointing out that only products that are sold on amazon, are being aggregated. So the web app isn't fulfilling the claim of "listing Reddit's favorite products", but instead they are listing "Reddit's favorite products that are sold on amazon".
Which causes such issues that another commenter pointed out, there are no ThinkPad laptop models listed as a favorite product for /r/laptops.
Fair point, but it still highlights the issue that bugs me: the reason why they are amazon only is to pay for the site. If there was a business model that supported this aggregation without pay-for-traffic, I bet we'd see a lot more products from a lot more potential sources.
BTW, not saying that no model can exist (subscriptions, whatever) but that most implementations appear to fail out and leave us with affiliate links.
Many useful recommendations can't be bought at all. My recommended pan is any antique cast iron, my recommended antidepressant is exercise, my recommended boat is don't get a boat.
These options are part of what makes going to Reddit better than a retailer for reviews
These sound like great examples of the noise you get on Reddit and back up OP's approach in my eyes.
If I ask for antidepressant recommendations I want something with an Anatomical Therapeutic Chemical Code of N06, not some glib remark about exercise. It's great exercise can help you feel better, but antidepressant has a very specific meaning.
If I ask for recommendations for a boat, I want something that floats in water, not some glib remark about how boats sink money. Yes boats a very expensive, but I'd ask "should I get a boat" if I was that early in the decision making process
And an "antique cast iron" is a very vague criteria and will lead to a goose chase, where as recommending a basic Lodge that will do the same job for 99% of the population without throwing them in the deep end of learning how to strip and season a 50 year old pan from scratch
I agree. Maybe my comment sounded a bit harsh. Let me rephrase it.
It is totally fine for me, if there are affiliate links. You have to pay the rent somehow. However, if it's the site's only focus (excluding products because there is no affiliate link) and the content is obviously auto generated, I have no use for it.
It would have been nice of the creators to get money for the affiliate links, but allow other products to exist on the platform. I think the revenue would still be more than high enough.
Yeah, I jumped to a conclusion. You make a very valid point about inclusivity of listing all the products recommended (or recommended highly enough), not just the ones that pay a vig.
Looria co-founder here. I want to clarify that we do not exclusively list products on Amazon. It just so happens that most products people buy are on Amazon.
It would be nice if you had an option to never show Amazon links in the results. Due to their commingling, counterfeit problems, and their scummy business practices, many people who would use your site probably don’t want to give Amazon their business. (I’m betting a higher number of your users are power users than other sites.) One of the features I like about kagi.com is that I can exclude sites from search results.
I try to go to the manufacturer’s website when possible. If that’s not possible, I try to go to a site that doesn’t have any other sellers on it. The whole “marketplace” that many websites are becoming is awful because you can no longer find a store you trust and buy from it knowing you’re more likely to get something of quality.
Anyway, I’m sure that doesn’t help you get revenue for your site, but since you asked, I wanted to respond.
Insane that no ThinkPad appears in the best laptops list.
But agree that this is a good idea. I am clueless about monitors and found some nice recommendations there. Hope to see more improvements in the future.
Are thinkpads still worthwhile? The latest versions don’t have as good of Linux or BSD support from a hardware perspective and the older versions are starting to be unattractive from a pricing/performance perspective (because of performance). I know there are upgrade kits for older models but last I checked that requires purchases from China (long shipping times, language barrier in ordering).
Can't speak for the newest ones, but the older ones still hold up with varying success. I love my X220, ran Fedora flawlessly and never had any issues with it. Eventually was forced to switch to something with a bigger screen (I used it exclusively as my personal machine for maybe 2 months), and went with a P50. Really nice machine, but sadly it comes with a (now very outdated) nvidia graphics card, so compatbility can be a bit shaky.
I also tried a T520, which was a solid laptop in principal (basically just a big x220), but in practice I had issues. The only way to get it with an i7 is to get a gpu with it, so I got the i5 and then upgraded to an i7 manually (nice to have a normal socketed cpu), but as it turns out the out-the-box i7 model comes with extra cooling to deal with the heat, which my own upgrade struggled with. After that, my 520 didn't see much more use. Maybe soon I'll downgrade it again so its more usable.
Edit: Think I got a bit distracted there with my own issues. Also tried the X240 and X250, and both are solid laptops as well. I dont use my X240 everyday, maybe once a week, but its been running strong for years now.
The trick I’ve found is to use something like this to get some model numbers - read up on the specs, and then start searching for the model number on Reddit and other places until I find people talking about it, then read as much as I can.
And also search “model number problem” too.
It’s not perfect but it usually works well enough.
It's a good one, but worth also making a collection of alt model numbers. Costco, for example, gets custom models just for it's stores, as does Best Buy in the US. Often the products are the same except for a color, though some include extras either in the box (a sleeve!) or in the product.
Yeah, once you start searching on a model, you often can find details about what that model actually is, and the various forms it appears (manuals, especially repair manuals, can be tremendous for this).
Search engine "item + problem" is where i always start. Of course, you get the 'piece of crap, took three months to get during covid' bs, but i really want to know how bad the item may be, as much as how bad the company may be ("didn't solve my problem").
It's using the past two years of comments, but ThinkPads have seen a severe decline before that. For example, the arrow keys are no longer a different shape from other keys.
Perhaps it's because Thinkpads have their own subreddit so a lot of mentions|questions|comments get siphoned off from a more general subreddit like /r/laptops
Reddit was the last place for online product discussions, that wasn't crawling with affiliate links and astroturfing.
Now that there is a 'data-driven' robot crawling these subs, there is no doubt the volume of spam posts will increase to bump up the numbers on this website's front end.
One day we'll look at affiliate links as one of the 10 major factors that made the web worse.
> Reddit was the last place for online product discussions, that wasn't crawling with affiliate links and astroturfing.
No, it's had rampant astroturfing for years now - you just haven't noticed. Reddit was never going to be a good product recommendation place, because there's no (1) verified purchase mechanism (2) web of trust mechanism or (3) gatekeeping into who could enter - and without those things, there's simply no way to prevent astroturfing/influence operations - and the same applies to HN.
Certain categories seem like they would be impossible to astroturf, or at least the brands in those categories wouldn't know the ROI on doing an astroturfing campaign.
Two randomly cherry picked categories:
https://looria.com/search?category.subcategory=TVs - this is pretty consistent with rtings.com. Samsung is very likely astroturfing, but LG's OLED are still consistently rated high no matter how much astroturfing and is consistent with my personal experience.
The astroturfing was more of the 'submarine' type [0] that took skill and effort for more niche products than the more low effort mass spam & fraud types you see on amazon and such.
I remember talking to an accquantance who had a marketer for their kickstarter-esque product and how they leverage reddit to generate interest for their thing via some astroturfing behavior years ago.
I'm not sure with current models but for a long time LG OLED isn't a good choice if you have a room with bright ambient light. The QN80 or QN90 are the best options since they are the brightest TVs and they also have/had anti glare coatings.
I don't need all that stuff. Reddit is plenty credible as long as you use comments as an exploration point, instead of a "buy" indicator.
For categories like men's shoes, I've gotten more value out of 3-line Reddit comments than I have with 20-minute influencer reviews that are mostly ad-infested infomercials.
I'm somebody that grew up on message boards like SlickDeals and RedFlagDeals so maybe I have a better eye for spotting astroturfing.
Yeah, a lot of value is that in these "recommend a product to me" threads, there are genuine comments. I don't really care that some are from marketers, I get some good suggestions and can go look for myself if they're right for me. OTOH, finding out if a product is good has become almost impossible.
Interesting product the problem with the model is if this ever takes off then the underlying system will be gamed. If it doesn't then its just affiliate advertising in another mechanism putting the reviews of other people to use for the website owner.
It's like cream skimming for funding programs which then leaves the core product worse off. It's extractive in nature.
This system is already being gamed, and it has been for years. You can find comments on Reddit by marketers who describe how they create organic-looking accounts, and then find recommendation threads and promote their brands there.
People know how to write content that looks organic. Some people get caught, but others are fairly skilled at slipping marketing material into ordinary threads. I'm not even talking about the karma-farming bots out there--it's much easier for a bot to farm karma on Reddit than for an unaided person.
The most obnoxious are the obvious ones in subs like /r/oddlysatisfying and /r/blackmagicfuckery where it's some cheapo product with a staged product video from one poster, then the inevitable plant post of "OMG where can I buy this amazing thing!" Followed by some helpful link to a drop shipping store, sometimes not till substantially after the post has fallen off the front page to avoid scrutiny.
I think in niche subreddits are pretty resilient to this kind of gorilla marketing, at least in my experience. Folks who join them are usually pretty vocal when they've used a product and had good or bad experiences with it.
Even in large niche subreddits like /r/coffee, /r/homebrewing, and /r/skincareaddiction which are all over 1m followers, there's pretty good self-moderation and calling out of astroturfing. /r/skincareaddiction might be the most susceptible to it since there are just so many products and skincare is pretty specific to the person, but I still see folks calling out shill posts and comments.
If you are a member of these forums over time you see the same products / etc. get recommended.
For example, the /r/coffee forum always recommends 1-3 particular espresso machines. People are posting hacks where they add arduinos, etc. to control it better. There are github repos to describe all of this. Either its an incredibly elaborate marketing ploy (to sell a couple more $500 espresso machines..) or its organic content.
Also, redditers are incredibly harsh when shilling, etc. is discovered. In the /r/diablo2 forums, a moderator started a Discord for everyone (to play and trade items for Diablo2 Resurrected). He posted an ad (in the Discord) for a site to buy items (frowned upon in /r/diablo2). Redditors did a thorough investigation and kicked this guy out of the sub. So his $100 of profit from that ad lost him a moderator spot and his Discord community.
The risk/reward of sneaky marketing on Reddit is just not there (for many scenarios, I'm sure it still happens sometimes).
> redditers are incredibly harsh when shilling, etc. is discovered.
Ehhh that depends the subreddit, big corporate products as Disney are highly astroturfed and get downvoted in to oblivion if you call them out. That's why you see alternate subreddit as r/saltierthancrait and r/freefolk.
Their astroturfing gets more agresive as they lurk and comments on those subreddits too.
Equally anecdotal--I've seen lots of examples of astroturfing that hasn't been caught. People who run social media marketing accounts have occasionally explained exactly how they create their marketing accounts and how they make marketing posts. The account will have legitimate Reddit activity--basically, because somebody will be using it as a personal account. And when someone asks for recommendations for hiking boots or whatever? "I've been using X for a while now, they seem pretty reliable." Anyone looking at the comment history or post history will see what looks like a normal Reddit account.
How much of the astroturfing is called out? 10%? 90%? Certainly nobody thinks it's 100%...
But at what point is "marketing" just engaging with the community? If you go out and develop a suberb product that some niche reddit community might love, you post it...and they like it... OK, good for you?
What I am saying is, if your product / marketing is sooo good that it passes the smell test of the relevant subreddits..maybe your not shilling / astroturfing but helping?
Similar to how a lot of SEO advise is: "Create good content". Where is the line here?
Just like it's really helpful for a waiter at a restaurant to suggest you get the <product with largest margin that is about to expire> if they, without you knowing, get a cut of the savings for doing so!
The account will have legitimate Reddit activity--basically,
because somebody will be using it as a personal account.
And when someone asks for recommendations for hiking boots
or whatever? "I've been using X for a while now, they seem
pretty reliable."
This would definitely work, but it seems like a large amount of work for a single organic-looking "I've been using X for a while now, they seem pretty reliable" post.
I don't think it's one size fits all. I moderate a niche hobby subreddit and we have been fighting astroturfing for years now. We're over 5,000 subscribers but this has been a problem for the mod team since we were around 1,500 or so.
I actually think if we were larger we'd have more subscribers who even knew what astroturfing is, how to spot it, and why it's important to shut it down.
If it’s posible to recommend a product earnestly, then you must take for granted that it’s possible to recommend a product dishonestly.
There’s no way to tell those two things apart when done well.
Catching the obvious cases only means you’ve caught the obvious cases. Reddit is really naive here. They’ll punish “look what I made” but not “look what I found” . We did it, Reddit! We beat ads!
> Catching the obvious cases only means you’ve caught the obvious cases. Reddit is really naive here. They’ll punish “look what I made” but not “look what I found” . We did it, Reddit! We beat ads!
Also catching the obvious cases can easily create an illusion of competence, which quickly leads to overconfidence.
If it’s posible to recommend a product earnestly,
then you must take for granted that it’s possible
to recommend a product dishonestly.
There’s no way to tell those two things apart when
done well.
This is true for some things, less true others.
For products that are almost purely subjective (books, movies, etc) yeah. There's really no way to spot well-concealed astroturfing.
As things become more objective the astroturfing gets a little bit difficult.
If I'm an astroturfer and I say, "I think brand XYZ leaf blowers are the best" but unless I go into some details that recommendation is not going to carry a lot of weight. And if I falsify objective details, things will start to smell more and more like astroturf to anybody that is actually knowledgeable.
Once products take hold as being "good" according to the groupthink Reddit's strong rewards for parroting groupthink make it almost impossible to go against the grain and get heard to listen to anything negative.
Anyone complaining about M18 tools, Toyota vehicle, Kitchenaid mixers, speed queen washers, etc, etc will find their comment shit upon by people looking for a few quick virtue points for dunking on the dissenter and downvoted to oblivion.
Meh, maybe “shitting on” a piece of tech just isn’t that interesting and your points aren’t as novel as you think?
“Javascript bad amiright guys? :D” isn’t even so welcome here anymore.
Nah, I'm not "this thing sucks" kind of guy. More like urghh it's too late now to speak out about "this is unhealthy, don't you see, it needs fixing, shall we?".
Simplifying issues into kids' fistfight isn't even so welcome anywhere.
Looking at the top 20 hot posts on /r/coffee right now the only one that has to do with expensive equipment is 1 post where someone is asking if the Mazzer Super Jolly that they got from a business is overkill to use at home [1] (it is but if you got it for cheap then might as well use it) and another post about a calibrating a refractometer [2]. The rest are questions about tasting coffee and troubleshooting cheap pieces of equipment (pour over brewers that are all <$40, moka pots, french presses, etc.).
A good example of this is the Reddit post below. The post looks somewhat natural but if you read the text critically it sounds very suspicious. This post had 8k+ upvotes and only a few comments calling it out at the very bottom of the thread that most people would never see.
Makes me wondering how many of those thread replies are part of the same campaign. Some are so insanely praiseworthy of the venture. This also reminds me just how atrocious the reddit UI is.
Reddit in particular with its very strong groupthink/circle jerk loop is really susceptible to this. Marketer's content gets popular/upvoted, then people parrot it, they get upvoted, repeat.
Recently, I successfully shut a 10,000+ karma reddit user on /r/webdev up because he spreads false information on particular html spec (that comment got 10 upvotes, unbelievable).
True. I've been reading about CMS/headless CMS on reddit especially /r/webdev for the last 2 years. Not authentic at all. Even if it is, reddit population seems to be very outsider (can't find word to replace "newbie"), though it also depends on particular sub. /r/asianbeauty is quite authentic but lots of products aren't there, they seem to have discovery problem. Thus, reddit is good for market research where you wanted to read/hear newbie opinion (the mass).
The catch here is that real users also make recommendations. I recommend things on reddit all the time, and I'm not some nefarious marketer.
I've also had a lot of good stuff recommended to me on reddit over the years. If that was by a marketer, then frankly, the best marketing in the world is "hire subject matter experts to go into amateur forums and recommend good solutions for them" because that's what it feels like I'm getting on a lot of good nitty gritty subreddits.
I agree. I'd even go a step further - I'm a mod on r/budgetaudiophile and we're even open to company reps as long as they're from reputable companies and are transparent.
subject matter experts
The trick in a lot of hobbies is that it's so hard to have anybody, even a professional reviewer, stay current with this stuff.
There are a lot of audio products out there. It's hard to say what's good and bad without putting each one through some objective measurements, and for some products (ie, DACs, amps, etc) this requires $50,000+ pieces of equipment and/or hours of time.
The balance between free marketing and access to community-loved brands is an absolute razor's edge to balance. Each sub does it differently and there's no correct solution. One person's astroturfing is another's excitement for an AMA with a company or retailer.
Yeah, I make recommendations on Reddit fairly often. I try to provide enough detail to make it clear that I’ve actually used the product in question, hopefully that helps separate my posts from those of astroturfers.
And self-congratulation of indiehacker networks. Those making $10k-$20k moving to twitter though, tweets like guru now. Those making $40k+ becomes business guys who talk un-related stuff for those who are finding good content and product.
As someone who's struggling on market research right now, the current situation of these site is pretty bad, hard to get real insight.
I stopped reading IndieHackers the moment it was acquired by Stripe. It's nothing more than a marketing tool now, and the quality of the content has shifted accordingly.
Having spent some time on these subs, especially /r/edc and /r/coffee, I could have guessed 8 or 9 out of the top 10 in each.
The more interesting things are the obvious ones that don't get discussed, but form some sort of an "accepted canon"
For example, the v60, despite being out of the top 10 on /r/coffee, it's assumed if you do pourovers, you have at least some form of this classic coffee maker.
Karma-farming bots exist. It's trivial to make bot accounts and farm karma on meme subs until you're over the threshold, and then go and comment in product-related subreddits.
Technical solutions are extremely difficult. A verified purchase mechanism is difficult to do unless you control the purchase platform.
The "easiest" solution is a web of trust, but that has a significant non-technical component by definition.
It does and you can limit posters to in-sub karma. afaik the only way to implement that solution is to write a custom bot though. Reddit's tools for moderation are basically garbage.
The Reddit karma algorithm is already a black box, so there's not much harm in making it more complex, thus:
Measure the general up-votiness of each sub, and scale cross-sub karma based on this value. A +100 post on a sub with 1000 members on a technical topic where people tend to downvote for minor errors is worth way more than a +1000 post on a fast-moving sub that's mostly reposted memes.
It is never as simple as "just" doing something. A lot of people have multiple Reddit accounts because it is a forum for everything. Then next is how to determine that arbitrary karma value, what if a lurker has some valuable experience to share? Besides, these are relatively low barriers to entry. Check @weird-eye-issue anecdote below.
I did this a decade ago on Reddit when I had a website drop selling watches. Made a post on a couple subreddits (men's fashion advice was the best iirc) with a nice wrist shot and in the comments casually mentioned where I bought it. Directly drove hundreds of sales (it was only $80 or so)
This is a valid concern for the whole review space. That's why most people don't trust a single source and look at multiple opinions (4-7 websites on average). One single source will never provide "just enough" research for a product you plan to spend a lot of money on and use for a long time. There is often at least one deal-breaking issue that you will find after reading through different sources.
We belive that a product research engine that provides honest and transparent recommendations from trusted sources is still a massive improvement over the current process.
We're not immune to the "garbage in - garbage out" problem, but we do our best to keep track of and remove sources caught getting paid to write fake reviews. Some categories are more prone to astroturfing than others, and we account for that by restricting the sources. Actively curating the sources is part of our daily work.
Not to mention Google results are so bad now that you'll frequently find the same sponsored blog spam review sites showing up, or worse, ones that just list top bought products from Amazon with affiliate links. 4-7 reviews sites will just get you more of the same unless you're looking for things like Consumer Reports, rtings, etc...
It already is gamed. Almost anyone who has a brand targetted at high class consumers already astrotufs the living crap out of Reddit.
In Reddit's case this is made even more odious by its propensity for group-think and dogpiling so you can't even call it out once the liking for the product has taken hold.
Good use of dog piling / group think. Very little independent thought on reddit once something gets rolling. Though i guess thats human nature in a nutshell.
Borrowing money almost always involves interest but there's a big difference between a government bond and an usury payday loan. Likewise I think that there is a big difference in critical thinking ability between the "reddit" blob of humanity and say, the US senate, subscribers, chemical engineering professionals, people on the Debian mailing list, etc, blobs of humanity. While the lowest common denominator in all groups may be similar I think the mean and medians are in very different places.
Marketing firms are already aware of this and they use what is called "organic" marketing. This means the accounts will post all over the website generically so it seems at a glance to be an actual person. I imagine it can still be detected with some kind of heuristic, but its a classic arms race between spam and detecting spam.
My guess is that such posts can be identified by lack of depth and few or no mention of negative aspects of the product. Even the happiest customer will have a gripe or two, even if it’s minor.
Thank you for this. I wish more people would suggest solutions instead of just being a Debbie Downer constantly and crying about how the system can be gamed therefore it's all futile. We didn't create technologies we have today because people sat on a chair and kept pointing at the flaws without trying to make improvements.
Every one of these "X's favourite products" seems to be an affiliate link farm. This one goes slightly further than most and actually links to the references. If anybody deserves a referral fee though, it's the originator of the suggestion and not the bot scraping them.
So, the DX7 (in synthesizers) gets labeled "Great for beginners". That's quite the mistake. One of the positive reviews says: "For me, the DX7 would be the last choice." Some reviews are actually for "Dexed", which is a VST/plugin that simulates a DX7. A negative review got "...a DX7 off reverb a couple of weeks back… Unfortunately it wasn’t packed too well and I received it in a state much worse than I anticipated", but doesn't mention anything about the instrument itself.
That's really a lot of errors for a quick review. I like the fact that each product is split in more or less relevant topics, but the actual content doesn't give much hope that it will help me choose products I know little about.
Reddit is only one of the many sources I use to determine if something is worth purchasing especially from unknown sources or products that do not have a brand name attached. Just recently I was looking for a wireless card to replace the one I had in my laptop. Reddit not only gave me the the a link to the card that works for my laptop but posters also chimed in on how to install it but also gave me an idea on how it performed.
I think the most important thing that Reddit gives is a shared perspective on any purchase where people can comment on others reviews. You don't get that with Amazon, it's always a mix of 5 stars and 1 star reviews and your left to determine if these 5 star reviews are real or not. It's the same with the 1 star reviews because I feel that other vendors may negatively brigade reviews as well.
In the end information for my purchases are sourced from traditional reviews, Reddit, online forums and of course the sellers site. It's not always prevented me from buying something that doesn't work for me but Reddit has by and large been my source of trusted reviews and has never really let me down.
How many of those user perspectives are real? It seems to be if you are a manufacturer of a certain product that it wouldn't take too much effort to promote your product with a few sock puppets in a subreddit until your product is on the subreddit's shortlist of recommended products. From that point on new users to the community will evangelise your product for you and you can scale back your sockpuppets.
If you manage to get your sockpuppets to be admin of the subreddit it's also not too hard to delete negative comments or reviews about your products or to generally steer the discussion in the direction you want it to take.
I don't trust reddit product reviews anymore than a top 10 SEO site on google with top 10 best <product>.
True, but I always take in information from many sources. As I type this I'm sitting next to my no name bluetooth headset from some Chinese company I have never heard from before and it's worked flawlessly for the last 6 months. I found reviews from Reddit, Amazon, Online forums and even one person who has a blog. I found enough of what I consider actual users experience. Especially on Reddit people will comment and generate back and forth conversations that give me confidence that I'm not wasting money on something that does not suit my needs.
It may not be perfect, but in Reddits case you can see a users comment history as well and if they are active on other subreddits not dedicated to a single product or set of products. Redditors also tend to overshare information about themselves, so, you can really tell who's a user and who's not. I could be wrong but it takes more than cat pictures to fool me.
Even Reddit has a limit to how much it can be trusted. Marketers are savvy to the fact that people take purchasing advice from Reddit. Definitely need to be sure the community you're checking is active and aware enough to find and downvote out the shill posts.
But still it's better than reviews on Amazon and even from trusted professional reviewers. The reason being that with Reddit you can get the perspective of people who have owned the product for a while. Even if all of the reviews on Amazon were real they'd be biased towards the first days of the reviewer having the product. People don't tend to come back 2 years later and post a review.
I have a raspberry pi gathering Reddit comments with an Amazon link in them every hour, then exposing them on Shop By Sub categorized by subreddit, sorted by upvotes.
Same with OP I put my Amazon referral link in there.
It's been decently well-received by Reddit, but I have a hard time getting recurring users. It seems like they just forget about it by the time they need a Reddit recommendation again.
Your site has a lot more subreddits than OPs. Wonder how you did that given there are literally millions(maybe even billions) of comments on popular subreddits. As for getting recurring users you can ask interested users to join a mailing list, you don't seem to have that on your site.
I've just been using a combination of the Reddit API and PushShift, warehousing data in such a way where it makes sense for Shop By Sub since Sept 2020.
Using the PushShift API, I started backwards in time and searched for a bunch of different regexes I thought were important, slowly creeping my way towards the present.
Using the Reddit API, I wrote a script to do a deep dive into specific subreddits, so I could index more popular ones first.
It's mostly just patience to get the amount of data I've got :) (and PushShift can't be overstated, it's very useful).
A mailing list isn't a bad idea. I'll have to figure out a way where the user can see a sign up box, but have it not be intrusive. I've been thinking of adding infinite scrolling to the most recents on the homepage, too, just for folks who are curious as to what weird things Reddit is talking about
Reddit really is great for recommendations: is less impacted by SEO and marketing that most of the internet, upvoting captures the wisdom of the crowd, and comments usually provide context for their recommendation.
HN is similar actually, though much smaller.
On both looria and redditreads I find it interesting how the products recommended reflect the culture of each subreddit.
Just curious - how long have you been running the site and how sustainable of a model you feel it is? Sustainable in the sense that the affiliate or ad revenue it brings in is consistent enough to justify continued maintenance, improvement, etc.
I'm also interested in starting up something similar for a particular niche, and would like to hear first hand accounts of the return on investment for aggregators such as these.
I've been running it about 4 years now? It's very low maintenance (run process_new_comments.py every month and re-deploy). It also doesn't bring in much money: it's definitely a hobby rather than a business.
I don't do any marketing, most traffic comes from people mentioning it on reddit or from being included in "cool internet stuff" newsletters. I have some ideas for marketing, but I also hate marketing and do enough of it for my day job SaaS.
So I just run it because I really enjoy it! I used to have a newsletter but stopped that as it was too stressful having a regular deadline and dealing with email spam filters. Everything else about the site I enjoy: writing occasional blog posts, looking at culture through the lens of books, finding new books. I really like books: I used to own and run a bookstore, and a book price comparison website in the past.
Yeah I've found a few great authors over the last two years due to reddit that I would have never read on my own. Pretty much because they are small or independent and their cover art is bad/not impressive. Crazy how much those things impact buying habits.
The two that come to mind at Blindsight by Peter Watts and Wherever Seeds May Fall/any of the other, 20!!!, First Contact books by Peter Crawdron.
It's a giant problem to solve. We've considered trying to build something but way more curated using industry professionals. For instance, I don't care what some schmohawk on reddit thinks of a kitchen knife but I'd place great weight on a professional chef's choice in knife and what they use in their home kitchen.
You have to disincentivize the feedback loop of product recommendations somehow. Any sort of community based recommendation engine will ultimately result in abuse.
Like others have said, if it does get adoption, results will only become worse and worse over time. Beautiful, fast, intuitive site though!
-- Asian Beauty is hilariously accurate but not good - Cosrx always comes up because it's cheap - however in my experience tiny small amount of Dr. Jart will provide much better results for general purpose skincare - Also Melano CC Pure - people use it for general pimp reduction but a pimple patch like Acropass Trouble Cure is targeted - people like Melano CC because it's cheap - however shouldn't be used like a moisturizer for pimples as I always see in the subreddit --
There are other services that offer this and have been building their graphs of high trust reviews of products for a while. I see some issues with this service’s sentiment analysis - this was already highlighted well by Owen Pearson (https://news.ycombinator.com/item?id=32764846)
Vetted.ai (formerly Lustre) is probably the leader in this space.
This isn't very useful IMO, because Reddit (and subreddits) are echo chambers. I can't comment for every category, but good examples that I am familiar with are Cars, Coffee, and Vacuum Cleaners. Without deep diving each case, I can say that certain product mentions in these categories are basically guaranteed upvotes because those subreddits' primary readers are biased towards particular brands or attracts a particular kind of person.
The love of the MX-5 Miata and the Aeropress are good examples here -- they are both very niche things loved by a very particular group of persons, but I would never recommend either as a general answer to "What car should I buy?" or "What coffee maker is the best?". They are specialized products for specific kinds of people.
Many of the other categories I glanced at look similar.
Also, pianos doesn't make sense, as the Piano subreddit isn't about buying pianos, it is about playing music and learning to play. There aren't even any actual acoustic pianos recommended!
Cool project. I can’t help but wonder if this is more of a list of who’s great at viral marketing rather than favorite products. I’m not doubting that the products on here are good.
> Marketing tricks eventually fail if the product is crap. You can only polish a turd so much.
For a particular product, sure - but that doesn't matter if the company makes enough money off of that product first, and then releases another one and starts the cycle over again. The company's reputation, meanwhile, takes far longer to decline than that of an individual product, especially if information transfer between buyers is low (which seems to be the case for most casual consumer markets, i.e. not business and professional).
I saw the astroturfing as early as 2016. It blows my mind that people aren't more outspoken about the threat.
I even see people defend it "you really think a trillion dollar company is going to be advertising here?"
Yes duh? They are called reputation management companies, and they are contracted for plausible deniability.
I've seen it play out like this- a few well placed top level comments that defend a companies actions. Those are upvoted, anything contrary are downvoted.
You can see this when a thread is massively upvoted to a front page, that's something that a company can't really control. They don't have enough voting power. However inside the thread, people won't spend as much time upvoting and downvoting comments. This is where voting rings make the difference, controlling the narrative in the comments.
The companies with rabid fanbases are those that benefit most. If they can control the narrative, the fans will take the bait and run with it.
Even worse, this same kind of thing is bound to happen to HN (in fact, I bet you that it's been happening for a few years already - HN has a remarkably large audience) unless something changes.
The obvious solutions are (1) implementing a web of trust and (2) changing the site guidelines to forbid product recommendations. Both of these approaches are super disruptive, but ultimately we need something to either (a) reduce the economic incentive for marketers on HN or (b) make it more difficult for them to conduct their influence campaigns.
No, you see Reddit is a magical place! The moderator cabal is great at their job, and the users are all angels. Why would anything be fake on Reddit? /s
I think it's a clever idea but this is the kind of thing that incentivizes companies to manipulate them (like Google and SEO). If everyone knows about the "just add Reddit trick", it will eventually stop working.
I immediately looked up the one of t most shilled products on the parts of the internet I frequent: VPN.
I was presently surprised to see Mullvad number 3 (instead of further down the list).
I noticed that it only gets a B rating due to a decent number of "negative" reviews. However, the majority of these reviews are negative about something else and positive about Mullvad. I think the AI gets confused in cases like that.
This is surprisingly good information, right now. I'm somebody that does intense product research before buying anything, and I went through the first 25 categories on the site that were relevant to me, and in each case the products I had decided on were in the top 3 results. One thing I did note based on that experience/prior knowledge is that Reddit seems to trend towards lower cost products ("value") vs recommending the absolute best product for the category, but the absolute best tends to be at least in the top 5 and usually top 3, but is not number 1.
My only concern with this sort of capability is now that it exists it will encourage gaming sites like Reddit even more than they already are. This just becomes the next generation of blogspam, fake review sites, and SEO gaming to get your product at the top of the list for "best X". A lot of people for awhile have been adding "reddit" or "site:reddit.com" to their searches to filter this, and so people have already been gaming Reddit for awhile, but this just drives it further up the value chain.
Are you allowed to re-post Amazon reviews, while earning Amazon Associates commission? This would appear to fall foul of https://affiliate-program.amazon.com/help/operating/policies..., specifically:
"(t) You will not display or otherwise use any of our customer reviews or star ratings, in part or in whole, on your Site unless you have obtained a link to that customer review or star rating through PA API and you comply with the requirements for PA API described in the License."
The PA IPA T&Cs state "(i) You will include a date/time stamp adjacent to your display of pricing or availability information on your application if you obtain Product Advertising Content from Data Feeds, or if you call PA API or refresh the Product Advertising Content displayed on your application less frequently than hourly.", so it doesn't appear you're using the API.
Really interesting and potentially useful product. And congrats on launching a thing. I'll try to remember this the next time I'm looking to buy something and trying to choose.
I've noticed a couple of product categories that are interesting to me that are missing here. From your message commenting how it's implemented, it seems like it's extremely labor-intensive to add new product categories; may be you should put something in there for users to suggest new categories to you.
Finally, I'll share a painpoint I often run into into when researching a new purchase: I live in Europe, and for many things a lot of reviews are very US-centric. Some products are not available here, and others are at a much higher price that should change recommendations.
I'm not sure how exactly you could tackle this, if there is any way to distinguish between US-centric reviews... just something I have to work around when looking at laptops, power tools, airsoft equipment...
Wow, y'all need to put some affiliate links on there and you've got yourself a revenue stream. I'd consider it appropriate compensation for the value add you've created.
This is a well-made website, but I will concur with others that I've been dreading the day this tool gets created.
IMO the core value of using Reddit for reviews was that it wasn't easily searchable. Sure, there's already some astroturfing that occurs, but not nearly at the scale of every other hit you get via Google. Please don't ruin a good thing!
This is a bit like comparing apples and bananas and hens and toothpicks even within a single category. Take cars for example., Mazda Miata MX5 is #1 discussed item and for comparison sake Tesla is in #16. Love to see Tesla at the rank bottom but what is it that the discussions are comparing, except for some easily gameable metric like comment count.
Recency weighting - is this from 4 years ago or last week?
Sentiment weighting - is this people having issues or denigrating the product by mentioning it
Engagement weighting - are people engaging with this content? Are those posting it shilling? Are the people that engage with that content just brigading?
As others have noted this is very prone to gaming.
Seems like a missed business opportunity for Reddit (or acquisition target!). They're sitting on an enormous trove of conversation data that could be better grok'd and structured (for discovery purposes) and monetized through affiliate links without being intrusive to the user.
If reddit did this I'd shut down the sub I moderate immediately. I don't mind them attempting to place ads on the site or otherwise monetize accounts with dumb shit like NFT avatars but if they start making money off of the free content our community's users are creating I'd turn it off immediately.
Cherry MX browns are the second product for r/mechanicalkeyboards, but I can't say all the talk about them is positive. Most people meme on how bad they are. So in this case popularity in discussion doesn't really correlate to a product you should buy.
This looks really neat! Do you define "Products" anywhere? It would be cool to have categories like 'Most discussed cars|phones|computers|movies|websites|etc'. The nerd that I am would love 'Most discussed programming languages on *'
Just the choice of project categories is interesting. But the site is questionable right out of the shoot. I clicked ebikes. There I see link "Best electric cargo bikes - Consumer Reports". Clicking that link takes me to GearLab not Consumer Reports.
This looks great. I clicked through to some interesting subreddits and the top mentioned products looked like good starting points for research. I wouldn't blindly make any purchasing decisions based on this list, but it's a decent aid.
This is really great. I often add "reddit" to my search terms when looking for info or products. Currently enjoying the r/audiophile list here. Good work!
I wish it would just show at least what the best product was in each category on the main page. Sick and tired of pogo-sticking around to see different categories.
Everything was nice until I encountered a knife in monitors category. It had the brand LG (probably not the tech LG) and it found itself in monitors list.
What board game do you expect to be at the top of a list? Catan is widely popular and thankfully has replace monopoly as the default game for a lot of families. It's not perfect but its easy to get into, has some depth, and is affordable.
> What board game do you expect to be at the top of a list
Personally? Concordia. But more likely: Pandemic, Terraforming Mars, Twilight Imperium, and Terra Mystica. Anything in the BGG top 50. There is substantial overlap between the two communities.
Catan is on this app's "favorites" list for an unexpected reason: it's a well known baseline, with a number of major flaws that people want to see improved upon*. So people complaining about the game leads to this algo declaring it a favorite, which is ... unexpected.
To me, this suggests some opportunities to refine the model with sentiment analysis. Just raw frequency analysis will be subject to this kind of base rate effect.
Also, looking into this more, it's a bit misleading -- the supporting data for catan also comes from the r/Catan subreddit. Like, yea, if you include that biased sample, it's really surprising Gloomhaven still won.
It looks like the sentiment analysis is being done but isn't super reliable. Here's a "negative" Concordia review:
> Concordia - I don't get to play this nearly enough since I really prefer it with 4-5, but the depth-to-complexity ratio just just fantastic. I'm really looking forward to the app implementation so I can get more plays in between actual plays.
This probably affects all products in similar ways but adds noise to the distribution which can affect rankings -- one misjudged call matters far more when the sample size is smaller.
edit: oh and their Wingspan data doesn't include _any_ reddit data, just Amazon reviews. Seems weird to call it a reddit favorite when you have no such evidence, even when you totally should
It's not about what I expect to be at the top, it's that I know this community and know that based on how the game is discussed there, and often mocked, it should be near the bottom.
This site is failing at either sentiment analysis or ranking.
Some people hate Cherry switches. But the complaints are largely subjective. I've tried a few switches that are supposed to be better tactile switches than MX Browns and yet I keep coming back to the MX Browns because I like force curve. They aren't the smoothest switches, but that's really my only issue with them (I'm sure I could lube them to fix that, but that would be far more of a pain in the ass than it's worth).
I was an active Reddit user from 2009-2014 and left partially due to the intense groupthink that was forming. Around the time that I left distinctly remember that there were subreddits popping up that were aimed at finding undercover marketing campaigns from long dormant accounts that would otherwise "seem legit". This was a problem then, and judging by some of the comments here, still remains a pernicious problem.
Second, a lot of these recommendations haven't changed since my time in the early 2010s. Just browsing through a few of the subreddits I notice a lot of the same items from 10+ years ago. This reinforces my belief that the cultish groupthink remains strong on that site.
Third, a lot of the "top products mentioned" aren't necessarily good products but rather "entry level" or "bang for your buck" oriented.
Let me add some context:
Many people add "Reddit" to their search queries to find authentic product reviews. We fine-tuned a BERT model to extract product mentions from over 4 million Reddit comments and posts with Named Entity Recognition (NER). The result is a list of the most popular products across many subreddits.
No platform (including Reddit) is resistant to fake reviews and spam, but we think it's happening less frequently here for various reasons:
- Redditors and other forum members are more interested in boosting their ego by showing their depth of knowledge on the topic (and correcting others on the topic), whereas corporate websites are more interested in raking profit by displaying (potentially) dishonest information.
- Enthusiasts in subreddits are pretty good at spotting dishonest or fake content, which results in immediate downvotes. The whole karma system helps with trustworthiness.
- Most subs are moderated well and spam gets removed quite quickly
That being said, good fake reviews are technically almost impossible to detect, even with sophisticated network analysis of the reviewer's profile.
Looria is still in Beta and we're working on improving our classification, summarization, and sentiment analysis. Let me know what you think!