The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.
I've used DDG for the past ~5 years, and it is typically worse without using a hashbang like !so for technical queries. I guess that is what the web has evolved to-- knowing which mega-site you want to search against rather than discover new sites?
Yes, that is also my observation. They will show content that completely does not contain your query words if that content is just popular enough. The result is that niche technical topics get drowned out by related popular discussions.
Also, I am by now 100% sure that Google has just stopped indexing the long tail. Like if I search for function names of public source code that I downloaded from GitHub, Google won't find it. But of course, it's still on GitHub.
Similarly, Google will sometimes not find a single result for some Windows API function names, despite them being publicly documented on docs.microsoft.com.
(same poster) BTW, amazon.com is in my opinion even more infuriating. I just searched for "Odense Marzipan" (which is a 100+ years old brand serving the royal danish court) and they show me pictures of gamepads made out of chocolate along with a note: Your search "odense marzipan" was automatically translated into "odicht marzipan".
Then searching for "odicht" out of curiousity, they auto-correct it to "olight". So I start with almond-based sugar sweets, follow their auto-correct twice and now I'm staring at headlamps. And even Google has no idea what "odicht" might have been, so I really wonder how Amazon decided to auto-correct from an existing product into a non-word.
Searching for "odense marzipan" including the quotes then works, but it yields the cringe-worthy message:
Your search ""odense marzipan"" was automatically translated into "„odense marzipan“".
(where the only difference between the first and the second thing is that they converted the ascii quotes to up and down sentence quotes)
Amazon also really seems to push what they _want_ to sell over what they have available.
A while back I spent a lot of time looking around for a basic SATA bluray drive on Amazon before finally giving up. All I could find was burners for several hundred dollars when I really just needed a drive to quickly rip a single disc I'd bought. I spent probably an hour scrolling through results and trying all sorts of variations on search strings.
I eventually gave up and punched it in Google to get... kicked back to Amazon to a simple, cheap SATA bluray drive that had been there all along.
>Amazon also really seems to push what they _want_ to sell over what they have available
This is exactly it. Amazon isn't a shopping site, they're a corporation using third party sellers to offload the risk and cost of providing a wide array of goods. They let customers experiment with product offerings, find products that sell using their web site, then cherry pick the most lucrative ones to produce and stock to compete with their own "customers".
I almost always filter for items shipped from and sold by Amazon. The only times I buy from third-party sellers are when the seller is clearly the manufacturer or a large distributor.
I just gave up on Amazon Spain about 2 years ago already. It's just completely impossible to filter and rank stuff properly, just loads of crap you don't want. I heard it's worse in Amazon US.
What I don't get about this companies (because this seems a problem shared between google, YT and Amazon) is when they optimize for clicks or whatever their KPI, what are they thinking is the outcome in the mid term?
I mean, IDK around you, but I'm the prosumer regular folks ask for recommendations. They may be safe for the time being but of course I'm going to contribute for their competitors getting klout.
I don't have an alternative for YT, but people watches me using DDG and whe they ask for recommendations for buying stuff I don't even bother with Amazon.
I'm not the type who pushes his decisions onto others, but I already got asked why I don't use google and amazon.
I just tried using Amazon.com to order something to Hong Kong for Christmas - sweet Jesus I don't know why anyone would ever visit that site more than once. I ended up giving up and just ordering the things with the co.uk version and sending them myself.
They extra postage cost is nothing compared to the insanity of the .com site.
This seems like an opportunity to build a "better" search engine for Amazon and reap the affiliate revenue.
There's got to be a reason no-one (to my knowledge) has done this: They probably forbid affiliates from doing their own indexing and ranking. Does anyone know for sure?
> They probably forbid affiliates from doing their own indexing and ranking. Does anyone know for sure?
They do. I came across a reddit thread where a guy had built a simple php-based search indexer for Amazon and managed to pay for his college through it. After this incident they apparently put changed their developer terms of service!
Amazon's search engine was systematically destroyed from within starting 5 years ago. It's entirely driven by revenue now and not at all by their faux customer obsession tenet.
Just try searching for a high-end appliance and watch all the Chinese knockoffs that will get ranked higher than the actual thing you're looking to buy. Sure, they cost less, therefore you're more likely to buy something therefore such changes tend to win web labs and go into production. But they've completely destroyed the intent of the search doing so.
Maybe I am looking upon the past with rose-colored glasses, but I recall a time when shopping on Amazon was like shopping at a huge Target. A lot of well-known name brands mixed with some no-name stuff. It was a nice relief from the limited selection, daytime hours, lines, and out-of-stock errors at department and big-box stores.
Now shopping on Amazon feels like shopping on eBay or Aliexpress with fast shipping. Everything name-brand could be counterfeit, and everything no-name is a complete gamble.
There is an actual product (https://www.amazon.com/gp/product/B084GKQB9T) with all of my keywords in the title. I found it via a google search because I gave up scrolling through Amazon.
I actively avoid shopping Amazon where possible these days, just because it's such a trashy experience.
Thanks for introducing me to Odense marzipan (I have a job in Aarhus!)
I very much get the feeling that amazon and eBay "Don't like" Denmark – the former redirects amazon.dk to amazon.de saying "we deliver!" and the latter doesn't exist (but it does wholly-own a local alternative). Going to Denmark always makes me aware of how good the local Danish things are, and how inflexible and annoying the "global" options usually are by comparison. Riding roughshod over the language is a good example of this.
Yeah, Amazon search results are often infuriating. Most searches are flooded with similar items that need better filters and some notes on product differentiation. Amazon feels like a bazaar set up inside a Walmart where everyone is yelling at you to buy from them and you don't really know what you're going to get. I know it's a hard problem, but I can often look at the results and imagine some easy UX improvements. Maybe they have higher priorities than helping me buy insulated pants.
I wish shopping for consumer goods was as nice as faceted search on Octopart or Digikey. It seems that even for Amazon-sold products, either the products don't have relevant data listed accurately and consistently, or the faceted search doesn't let you query it.
(And that leaves aside the issue of the product page of a really nice wastebasket I bought years ago being hijacked by a meat slicer.)
Amazon search is horrendous garbage. I was trying to buy replacement ear tips for my earbuds. I knew exactly what to search for. Yet, Amazon search only returned the Large size and none of the others. (Clicking on the large listing only had the large size available. No drop down selection or anything) I go to Google and type in the same search and put Amazon in the search too. Immediately shows up as first result and I was able to buy them. I figured they were out of stock and that’s why I couldn’t get them. Nope - Amazon search is just that bad.
It must be a recent thing as well as I recall being able to get good results from them last year at Xmas when I was searching for some science books for the kids. This year my searches got absolute garbage in return.
Are others getting the same problem? I searched for "Odense Marzipan" on duckduckgo.com, google.com, amazon.com, and amazon.co.uk, and all of them return almond-based sugar sweets in the top links.
> Also, I am by now 100% sure that Google has just stopped indexing the long tail.
Rather weird if true, but I can't really disagree with your observation. It seems like large parts of the web have disappeared in the last five to ten years.
Google do most likely index the sites, but their current algorithm just don't use them, because it as much a promotion algorithm at it is search.
Haha I just tried and guess what? Your comment was the only result.
Ask HN: Has Google search become quantitatively worse?
https://news.ycombinator.com › item
4 hours ago — But of course, it's still on GitHub. ... Search for "github drv8830pico" - it's literally not in any results.
Yep, that's it - and indeed, HN comment got indexed pretty much instantly by google, but for some reason that repo has been up for a few months and just doesn't show up at all(and it's not like google never shows any results from github either, so no idea what's going on - as someone else pointed out, it comes up first on DuckDuckGo just fine)
Actually, that might be because of their antispam filters. They were always fighting websites with countless subdirectories or subdomains, having completely wiped them off their SERPs in the past.
GitHub does fit the "huge website with lots of duplicate content" description very well.
Then again, Twitter seems to be doing fine. But I have a suspicion that they have some sort of agreement (and GitHub doesn't), because I have to regularly "report as spam" (note how that implies Google decides what's spam and what isn't) Twitter emails and they still pop up in the inbox after a while.
Maybe, but then you get all the GitHub issue clone sites and all the Stackoverflow clone sites showing up at the top of the results with the original no where to be seen.
I mean, that's probably true, I just made this repo specifically to help people who are looking to control a DRV8830 chip from the raspberry Pico, and I'm not sure how anyone is supposed to find it if there is no combination of "raspberry pi" and "DRV8830" that would get you to that repo on Google.
Well, ads and information are at direct competition, and we all know who's winning the battle. It seems Bing is better for most searches now, even technical ones because they seem to index more of github and stackoverflow than Google.
Yeah. Absolutely infuriating if you specifically use quotes and you get all sorts of dross not containing what you're looking for. Clearly they have figured out in some way that giving the end user what they want does not maximise income.
Google News too has become flaky. Often does not find stuff you know is there, or finds it one day, but not another. Hrmph.
I'm in a similar boat with the "Google Now" feed on Android. When I say "not interested", it fails to note that as expected, and I see the same news "story" pop up (usually different sites, but occasionally the exact same site!). It's likely related to the fact that it almost NEVER gets the subject/topic correct, so I can never use that option.
There was a time I thought I'd trained it to stop telling me about what celebrity said or wore what, but those are creeping back into my feed.
YouTube has started showing me the same videos over and over again! I keep telling it I don't want to see this content because I've seen it before and it keeps forgetting!
I absolutely hate the Google has changed the semantics of search so that it doesn't really pay attention to your keywords unless you quote them. In Google's mind you're not serious about a keyword until you quote it! Garbage!
Is there a market for a focused, specialized search engine, for example, relatively speaking -- google circa 2010 with all the specialized search operators etc... focused on technical content?
What does the business model look like? Ads (it would be in front of a very valuable audience of technical folks)? Or paid subscriptions (perhaps the community votes which resources get crawled / indexed)
Yes, I actually find Bing is now better for the long tail.
Most of my searches are for really old pages or really long tail stuff and Google just simply doesn't bubble them up, if it has them at all. I keep finding web sites lately from links on other sites and find myself asking "Why the fuck did Google not find this?" .. then I go back to Google and try to find it with keywords from the site, and nothing...
I also recommend switching to Bing. It shows me the results I'm looking for pretty consistently, whereas Google/Duck will sometimes give me Google's home page or the Washington Post's home page instead of anything even resembling the query I'm searching for.
Google definitely changed something about how they index or prioritize GitHub sometime in the last year. I used to frequently use Google to(successfully) find GitHub repos based. Lately, this does not work for me anymore; adding "GitHub" to my search query helps sometimes but I'm forced to search directly in GitHub many times.
You can test this- find a site you think is not indexed, search for the URL of Google, and then look at the cache and the date. I bet it’s been cached more recently than you expect.
The "people also searched for" box is not just useless, but also very much messing with usability for me. Every time I click on a link, go back, then trying to click on the next link, this box shows up and I accidentally click on that (because it shows up with a little delay and an animation).
This filter takes care of that box completely:
www.google.com##.exp-outline
www.google.com##[style="display: block; opacity: 1;"]
www.google.com##[data-hveid]>div:style(height: auto !important)
I cannot stress enough how infuriating it is when a page loads content under my mouse cursor at such a delay that I can manage to point at a link and click on it _before_ the new content is loaded so that I click/tap an unexpected link.
This happens _all_ the time on the Twitter app search bar.
This is infuriating on all UIs, but I haven't seen one that would implement a very obvious solution: if the area around click has changed in 100ms before the event, disregard the click. Either a webpage or a browser could do that.
I'm sure I'm not the first one to think of it, so there must be some reason why it isn't done - does anyone know why?
The browser should do it. Websites often do this intentionally in order to trick users into clicking ads. The only way to stop these abuses is for the browser itself to stop enabling them.
Unfortunately Google would make the Gmail team very grumpy if they did this - Gmail's by far the worst offender in my experience. Every single time I open my email I have to remind myself to wait for a moment before clicking on my newest emails, because most of the time there's going to be a couple ad "emails" popping in and shuffling everything else down a few seconds after all the real emails have loaded.
Google's grumpiness is an excellent measure for how much we need a feature. The grumpier, the better. Other browsers should do it. Firefox, Safari, Brave, everyone else.
Great idea, although from an engineering POV I think this is really hard. Most websites are super JS-heavy and hence, from a browser's POV, change constantly.
Quite a ton of difference between pixels on the screen changing and the underlying DOM changing.
If something changes within ~300ms before your click, the only way you'd be intentionally clicking whatever's there now is if you're amped up playing some online game
This is my normal android experience. Use the share function, it displays a list of apps or chat conversations which actually are relevant, tap the app.... annnnnnnd the list refreshes with a new list and congrats, it buffered the tap before the menu refreshes and opens whatever app is now under your finger which is of course the wrong app. So when I use share I have to wait a few seconds to ensure the menu is refreshed and then pick an app. This behavior is throughout most of the ecosystem where input is valid and buffered before things are rendered to screen. Such a well engineered UX.
bad enough when it’s accidental, more infuriating when it’s done with a profit motive
theverge.com presents a “show comments” button that shifts out of the way to reveal the worthless taboola links, I honestly don’t know how it isn’t considered click fraud
Lazy loading is the absolute bane of usability in web UIs. The fact that this is the default/most common behavior in modern web frameworks seems simply idiotic to me.
This is particularly hilarious and ironic when considering they are really punishing for cumulative layout shift in their Lighthouse tool - to prevent exactly what you are describing from happening! Seems like Google hasn't used lighthouse on their own site!
I'm using DDG for almost a year, and I'd say that DDG is slightly better because I have a bit more control. It's important when you're multilingual (common in Europe) to be able to de-localize results chose language etc.
But it isn't MUCH better. DDG is just slightly better than Google, that has become infuriating.
I use it regularly but it is getting to be unreliable for technical searches. I often have to resort to using google when trying to track down information about programming problems.
> this box shows up and I accidentally click on that (because it shows up with a little delay and an animation)
And somewhere a team of designers and PMs got their bonus for increasing the engagement OKR. Clearly users love the animation and added delay because look at the metrics skyrocket!
Google now also has an extreme recency bias. Most things I search for will give top results for recent SEO optimized blog posts/Youtube videos instead of established authoritative resources.
If you search for something that happens to overlap with a recent movie name, good luck, it'll drown. And I constantly "have" "to" "search" "like" "this" because Google thinks it knows better than me.
While I also am annoyed how much I need to enquoten to refine search, in most cases I wish it upped the bias further. My default search applies the 'Past year' filter because otherwise I get lots of outdated answers.
I've also completely switched over to DDG and I'm seeing similar things that infuriate me about Google. Most of all the fact that ignores my "literal" searches using double quotes. The documentation [0] says
> Results for exact term [...]. If no results are found, we'll try to show related results.
But very regularly it fails to find results I know exist in not so unpopular places.
this always baffles me. If my search terms return zero hits, THAT is helpful and meaningful to know, when they change my search to enable hits, that isn't helpful, because it 99% of the time will not have the information I want, and frequently I won't know that they are ignoring search term, so I'll click through to half a dozen hits before I discover what theyve done. I got to the point that as soo as I clicked a link, the first thing I'd do on the landing page was command-f, and search for the most specific term in my google query. If it wasn't found, I'd instantly hit the back button and do the next hit.
This really got to me about 6 months ago, so I changed all my default searches on all my browsers and mobile to DDG, and haven't looked back. I tried DDG ~5 years ago, and there was no way it could have replaced google for me then, but when I did it 6 months ago, it didn't seem any worse, maybe a little better.
I've thought about it and I think that the really infuriating thing is that somehow the program wrongly assumes that I made a typo, so it's wrong and telling me that I'm wrong. No! It's you!! You're wrong!!!
DDG is heading this wrong direction too. Today I've searched for some ecommerce platform called Comerzzia and it showed me some Comerzia or Comercia or whatever shops near me. It shows maps if it thinks they're related and apparently I can't disable that feature.
> The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries.
The fundamental, unavoidable problem is that the cost of providing high-quality results on the long-tail of possible searches tends to grow faster than the revenues that can be earned from those increasingly rare, obscure, long-tail searches. Any search service seeking to maximize profit, like Google or DDG, ultimately always evolves to perform less and less well on the long tail of possible searches.
The search service we all wish we could have -- a service seeking to maximize the quality of individual searches, no matter how obscure -- may not be feasible as a profit-maximizing business.
The fundamental, unavoidable problem is that the cost of providing high-quality results on the long-tail of possible searches tends to grow faster than the revenues that can be earned from those increasingly rare, obscure, long-tail searches.
I think even two years ago, Google searches had far more depth and yet Google was quite profitable (then the searches were still biased but now stuff is simply gone). Sure, if someone looked at the marginal profitability of every single search result, it would look like what we're seeing. But there was a time when good indexing of stuff that didn't turn a profit by itself was done as a service to attract people to Google and/or to improve the Internet generally. That time has passed, clearly but it was a decision.
I agree. In all likelihood, the decisions were made gradually to improve overall efficiency without losing search volume, but the unintended consequence was to degrade search quality, at first gradually in subtle ways, and then suddenly in very noticeable ways.[a] It's possible no one in the company's executive team has noticed the loss of quality. In fact, they may not think anything's wrong even now.
For what it’s worth, I have been using the new you.com headed by Richard Socher as my search engine for the last 2 weeks. The condensed search results with sections from reddit, wikipedia, stack overflow or arxiv is really great. It’s really suited for technical users.
you.com looks good. I am USA and English centric. I’m annoyed at the futility of google search for ordinary technical topics. Bing, duckduckgo are also more futile as time goes on.
What are the ways to direct more air into the likes of you.com? You.com was an “Show HN” topic 3 weeks ago[1]. The you.com improvement in 3 weeks is noticeable.
This post has been simplified for sentiment parsers...
I hadn't heard of that one before, but that's actually really nice. I really like how they're trying something new with the presentation and organizing the search hits by the source like that!
This helps to find the specialised search engine, if you need one. (it scans duckduckgo for any new !bang operators, in a nightly built, here is the script: https://github.com/mosermichael/duckduckbang )
I suspect, that most of these specialised search engines are powered by elasticsearch. It may be, that elastic is starting to cut into google search, from the low end.
Try searching for any product outside your wheelhouse and it quickly devolves into an undergrad research endeavor.
I can't trust the first or second page results because of SEO. Then every page after quickly veers off topic or just features sites that aren't as good at SEO.
This combined with the fact that programmers have went from highly specific names like "winamp" to just capitalizing a random english word to name their tool makes it very hard to find relevant information.
A lot of sites that just scrape github too end up high in the results. The actual github page isn't even on the first page if anywhere at all. The spam sites do a better job at crawling code than google does.
> The results keep getting "refined" so as to suit the popular 80% of queries
Google is simply maximizing profits by giving users results that would cause either more clicks on ads or show more ads. It's mission is to make money this quarter/year. If you believe any of their Silicon Valley-style new age talking points you probably don't have critical thinking skills.
If their products are getting worse for you perhaps you are not part of a profitable segment for them.
Frankly, I mostly receive political articles and columns when asking for more more objective things.
I remember once searching for how common same-sex relationships among teenagers are in Japan, as some say it is very common, and all I received were political opinion pieces that did not in any way come with the numbers I sought on Google, so I then tried DuckDuckGo and to my amusement what I received with the same query was mostly pornography.
Neither particularly useful, but the contrast in how both prioritize was interesting to me.
What I'm wondering is if the efficiency of the spider itself has dropped. I've often hunted for an old Reddit post I once saw a long time ago and found no hits - maybe I got the wording wrong, but I suspect the real reason is that lower-popularity reddit content is simply not getting indexed at all. Or maybe it's the extreme recency bias others have discussed. Of course, I have no evidence because I can't find the thing that I can't find.
Most of the time they are nothing more than an site:xyz.com equivalent that is just easier to access and you can group them to say something like !dev and search hn, stackoverflow etc in one search without all the blog spam in between.
The engine should automatically detect that I want to be booted to a third-party site? Err, no. That would be infuriating behaviour if it happened without explicit instruction.
Shocking that tight emotional connection is more valuable to most people than throwing spaghetti at the wall.
It’s almost as if the government and big companies have spent a lot of effort understanding human biology, cognitive function and applying what was learned.
While selling the masses on a contrived story keeps them believing there’s a universe of infinite life available to humanity if you just follow these steps…
Things like human colonization of space, and political memes about wealth are taking advantage of the same biological quirks as religion. It’s just now we can quantify the effects rather than wave it off as mysticism.
But the human story is already set on a path of building a bridge to nowhere.
It's not even just dumbing down. It's heavily weighting to selling one or another thing. It's been getting worse for years but it's really degenerated in the last several months.
You can sort of fight it by including a term showing your topic in your search, I think.
I often see this line of thought come up whether topics like this float around and wonder: how can we demonstrate /quantify /prove this. Especially since I'm already partial to agreement.
I've used DDG for the past ~5 years, and it is typically worse without using a hashbang like !so for technical queries. I guess that is what the web has evolved to-- knowing which mega-site you want to search against rather than discover new sites?