Hacker News new | past | comments | ask | show | jobs | submit login

> I don't believe websites can dynamically alter their index to show up for so many typoed search queries.

One way to do it is to have some path through the site that lists relevant data, but also includes links at the bottom to typoed versions of the data. The web crawler will chase those links, and if each page that you retrieve generates more such typo links, you can feed the crawler an infinite stream of typos.

Google will consume quite a large chunk of an infinite stream before it trips any kind of "circuit breaker" to cut off how much crawling they'll do on one site. They have a lot of storage.




Fascinating. Thanks




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: