How would you begin indexing everything? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tudelo on June 26, 2019 \| parent \| context \| favorite \| on: Ask HN: Can we create a new internet where search ... How would you begin indexing everything?

Jaruzel on June 26, 2019 [–]

This is good question. Crawling and storing the pages is the easy part... searching them with a sub 1 second response time is much harder. Which current DB platforms can handle the size of data that Google indexes?

Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact