Hacker News new | past | comments | ask | show | jobs | submit login

The point of this isn't asking how to apply boolean search operators, it's showing that the largest AI-focused companies in the world absolutely suck at NLP.



Why would you really apply NLP to a search engine though? Generally speaking a weighted keyword search is good enough 95% of the time and requires significantly less resources to perform.


I work specifically in this field with clients, and deliver training on applying NLP to search.

You’d be surprised how effective NLP is for use when identifying query intent, and pulling out modifiers that should apply as metadata filters.

Weighted keyword search works a lot, but it fails hard for many long tail queries (especially in e-commerce and other attribute heavy domains).

IMO there really isn’t a good excuse for these firms to fail at queries like this. The query itself isn’t particularly difficult when using a decent NLP stack and following well known practices.


If it's technically possible then presumably it's a deliberate product choice to not have better search results for "shirt without stripes". And that seems entirely plausible.

Google is already by far the most widely used search engine, so they don't really need to innovate or improve the search product very much in order to attract and retain users. Presumably capturing more advertising spending from the companies paying for ads is a bigger priority.

Microsoft under Satya Nadella has been all about enterprise and cloud, and I doubt Bing is a strategic priority any more, so it's not surprising that they wouldn't put a lot of resources into making it better.

Amazon is a little surprising. You'd think they'd have a lot to gain from making it easier for people to find what they're looking for. But maybe less than perfect search results are deliberate? Maybe it's like how supermarkets put basic items in the back of the store and high-margin impulse buys in the front - so you have to walk past chocolates and chips if you want to buy a carton of milk.

If Amazon is deliberately nerfing search results then maybe Google would stand to benefit from having better shopping-related results - people would get frustrated trying to find a shirt without stripes on Amazon and just use Google instead, letting Google profit from advertising in the process. But maybe people selling shirts aren't willing to pay much for ads, so there isn't much money for Google to make by getting better at finding specific types of shirts.

I dunno if any of these conjectures are anywhere near accurate, but it's interesting to think about.


I feel like that's stretching the definition of NLP though. Technically you are processing natural language but it seems like you've found that doing essentially a keyword match but using certain keywords as more advanced filters rather than just search terms.


Because you want to sell shirts without stripes to customers who want them.


I expect most users genuinely looking for such a product would just search "plain shirt"


That is an inclusive search though. What if the user actually wanted to simply exclude stripes but wanted to see polka dots and paisley?


I think that is probably not a common enough use case to optimize for. Additionally it would be easy for a user in that position to just search "shirt" and ignore the occasional striped one, or to search "polka dot" and "paisley" seperately


That sounds like a lot of repetitive searching. I wonder if we could get a computer to do that for us.


I understand the point, but there are plenty of bigger fish that I would want Amazon and Google to fry before spending their engineers' time on a triviality like this. I just don't think that having to make three queries instead of one in this occasional situation is such a big deal.


You're arguing the search is "good enough". Because we can adapt to the machines. But the company who will not force us to do so will get our business. The company who creates the best digital butler will win. They know this. They try hard. And they still fail at simple stuff when judged by humans.


Another possibility is that maybe it really is "good enough", and they get a bigger advantage by competing with each other on more important aspects of usability than these kind of trivial issues


Assumptions like this is what makes search so terrible for many many companies.

You can't assume that customers would type one thing or another - you need to gather lots of query log data and see what you find. You'd be surprised how much variation there is, but once you do have this data you can then find patterns to cover lots of (but not all) cases.


Of course, they should look at the data before making such an assumption. I am not suggesting otherwise, I'm just making a guess as to why it's been done this way. I suspect they have looked at the data and identified that it is not a common use case.


"Shirts without stripes" was meant to illustrate the point. The general query issue it represents cannot be solved by adding "plain".


Apple probably expected most users genuinely wanting to make a phone call would hold their iphone 4s a certain way. Turns out expectations don't always match reality.


Like I said in another comment, this yields plenty of results that are not in fact plain shirts. Try for yourself on Amazon or Google.


I did try on Amazon before making the comment and it showed me only plain shirts (at least for the first page of results).


From a product perspective, I would say there is a reasonable expectation that a customer will provide that query and expect the results to come back with plain shirts. Anything different is a degraded customer experience. Sure, a technical user will understand which queries to provide better, but 90% of customers won't have that skillset. Its our job as engineers to serve those people, and the queries they provide.

So NLP is totally a thing you want to have in search. Arguably, its the whole point of search as it exists now.


Agreed. I love how I can go to Men's Wearhouse and tell the salesman:

"Shirts"

"Polka dot shirts"

"Floral shirts"

"Wikipedia list of clothing patterns"

"Houndstooth shirts"


Why would you really apply NLP to a search engine though?

If you go to Google's homepage and click the microphone at the end of the search input box you can search by speaking. All it does is convert to speech to text, but it implies you might be able to search in a more "natural language" way.


Depends on your audience, but I imagine many find the answer boxes on Google search pretty useful. Getting the population of a city without having to click any links is probably good for your perceived value. For this you need some NLP tech to extract intent from the query and match it to the right entity in their knowledge graph (in addition to something to help you build the graph in the first place).

Google have a blog post from October last year with some more complex examples of where more sophisticated NLP helps https://www.blog.google/products/search/search-language-unde...


Because there's something called conceptual search that people are dying for to find what they actually want.


I agree, I think it's a better idea to improve the knowledge management (in terms of a relational hypergraph) before trying to create the bridge between the natural/digital knowledge representations. I have a hunch a good knowledge management system would be more amenable to being queried using natural language since relations are probably constructed similarly to the patterns we see in natural language syntax.


Because not everyone thinks in search operator keywords.


Certain query styles which the NLP operators help clarify. Recent personal examples: "is butyl rubber an organic or inorganic compound?" and "which gloves are best for both acetone and nitric acid?"


Because that's how people use it and kind of marketed as such


Perhaps it's considered "good enough" because users balk at performing NLP-style queries when they find they almost never work properly.


5% revenue sales and revenue increase? Why indeed!


all three of these search engines offer "voice assistant" platforms that encourage you to speak to them in natural language, and send your query to the search engine verbatim under a broad set of circumstances.


Every systems sucks at NLP. It is an extremely hard problem and we are nowhere near matching the human cerebral cortex.


Generally, yes. But you can go very far and cover a lot of cases when you stick to a narrow domain and common patterns that customers of your product will typically use.


Yeah the examples would probably do better if they had a front layer model determining the general topic "clothing" then shipping the query off to the clothing and fashion specific model for details.

Lots of focus on a general purpose mono-model, but I think a collection of specialized subsystems is a better representation and would produce better results, faster.


My reaction to Siri is either amazement at how well it worked or disgust in its failure on seemingly simple queries.


Siri has lots of known problems. My biggest gripes are the lack of context and lack of a "discussion" state.

But Siri is a general domain problem, which is really really hard. Siri set the expectation you can ask it anything, and it works terribly and for most questions just gives up and runs a web search.

If you are an e-commerce company though, that's a narrow enough domain, because you know that for most people, they're looking for products to buy or compare. It's not an unbounded Q&A service.


I'd go as far as saying "AI" is only as good as someone has taken the time to program the use case.


Yes, basically 1990s Expert Systems but with even more manual labor.


So, it's really just a series of if statements?


Don't get ahead of yourself: it's SELECT with GROUP BY.


Don't forget

AND keyword LIKE '%searchStr%'


Not that but maybe someone can explain if the technology is "AI" why it still takes tens of thousands of hours to implement a project? Shouldn't we call it AI when the frameworks can be implemented with relative ease? My point is that the level of customization required to achieve AI, in my opinion, doesn't make it very intelligent.


One of the primary methods to join multiple learned models is a decision tree (or combination of decision trees making a decision forest), which can be simplified as a series of if statements/conditionals. So if you join a 'is it a shirt' model with a 'is it striped' model you get two sets of things, and with how big data approaches this it is something you can do quickly. As other people have pointed out the issue here is that the NLP of the actual search is not creating a negation of two sets, it is returning the intersect of sets 'is it a shirt', 'is it striped' and shrugging its shoulders and either intersecting it with 'things with the text without', or throwing up its hands entirely based on the context because it wasn't programmed to do something smarter.


Yes, each of which wraps a "Not Hotdog" app. Turns out that "Not Hotdog" is quite hard already.


Either that or vague network of trial and error generalizations (deep learning/gut learning).


Huh? It's supposed to be the opposite, hence the "I"


The term has been bastardized. AI is now any app that uses if statements, so long as it's obscured to the user. So now "AGI" exists, because apparently the word "general" now means "intelligent", unlike "intelligence".

At some point in the future marketers will learn about AGI, and we'll have to make yet another term, maybe artificial general practical intelligence?


AGI is some AI that isn't specialized in a single problem.

There is nothing on the word "intelligence" to imply it's not specialized.


Right, because when I think of "intelligence" I assume it's specialized and otherwise is dumb as rocks. Naturally. Which is why marketers are really just advertising specialized tools, not agi. Obviously.

I know what agi is; I just find the terms backward.


When you think of intelligence, you think of the human kind. As we create different things that have some of the properties we usually identify as "intelligence", but lack others that were completely correlated until that time, we need to create better words that can express the new situation.

It's perfectly ok to prefer the qualifier to go the other way around, keep intelligence general and change the name of the specialized form. But that's just not how our language evolved.


Or maybe there's just not a lot of AI in their main search product (for whatever reason). They seem to be pretty good in other areas (Translate, Cloud speech-to-text, Alexa etc.)


For the matter of translation they actually aren't pretty good. DeepL outsmarted all of them (at least for the supported languages). Given the resources thrown at this problem by the different companies I would even say the results of Google, MS and such are actually quite disappointing. (And I don't think one could say "but this companies did the upfront work" as the basic ideas are something 60 years old).


It's not as simple as NLP. The shirts them selves have to labelled as having stripes or not. If striped is not a attribute of the shirt, it doesn't matter of the NLP can parse the meaning of striped.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: