We could always ask a different question: what would it take for everyone to have a copy of the index? Humans can only produce new text-based content at a linear rate. If storage continues to grow at an exponential rate, eventually it becomes relatively cheap to hold a copy of the index.
Of course those assumptions may not be valid. Content may grow faster than linear. Content may not all be produced by humans. Storage won't grow exponentially forever. But good content probably grows linearly at most, and maybe even slower if old good content is more accessible. Already it's feasible to hold all of the English wikipedia on a phone. Doing the same for Internet content is certainly going to remain non-trivial for a while yet. But sometimes you have to ask the dumb questions...
Or: what would it take for everyone to have part of a copy of the index, according to their capacity / needs / etc., and for them to be able to easily search the parts of the index they don't have?
To pre-empt the "you've just described BitTorrent" comment - only in the vaguest sense; you'd need search functionality on the chunks themselves, and ideally you wouldn't have to copy (or even stream) your peers' search index chunks to search them.
I guess there's a trust / security issue here around "search index poisoning"; to resolve that, you'd probably have to lean on SSL verification and all of its attendant infrastructure for now.
There are 8 billion people. If half of them were awake, and 10 percent of that half are actually typing at 40 words/minute, that would be about 13Gbit/s. I couldn't receive that feed today at home, but my work could. A satellite feed could work today too. And I wasn't really talking about today, but 10-20 years from now. Storage will be a problem for a lot longer than network capacity.
Storage grows geometrically, while network capacity has grown less so. It's possible we will soon have more storage than humans can create content to fill. No problem; automated systems also fill storage!
Of course those assumptions may not be valid. Content may grow faster than linear. Content may not all be produced by humans. Storage won't grow exponentially forever. But good content probably grows linearly at most, and maybe even slower if old good content is more accessible. Already it's feasible to hold all of the English wikipedia on a phone. Doing the same for Internet content is certainly going to remain non-trivial for a while yet. But sometimes you have to ask the dumb questions...