
Ask HN: Is there any (functional) decentralized search projects? - cannedslime
In these days of censorship, corporate monopoly on what is basically  the free flow of knowledge (Google). Why has no one yet come up with a way to distribute the task of indexing the web? Maybe there is, but all I can find on Google about the subject is questionable ICOs.
======
lettergram
It really depends what you mean by decentralized search?

I've written pretty extensively on search[1].. Basically, I've come to the
conclusion that there are opportunities to improve search, and I've even gone
so far as to start a company around it:

[https://projectpiglet.com/](https://projectpiglet.com/)

The problem is it has to be niche to compete with something like Google. For
instance, my project targets financials (and might have info you're interested
in).

What you're mentioning related to decentralized search is difficult.

Distributing something like search would be hard because of the aggregation
factor. Google, my system, DuckDuckGo, etc. all require some sort of
searchable graph (I actually use relations, but more on that later). To search
a graph on a distributed system is fine, but aggregating and redistributing
the information in real-time is very very difficult. Aggregation typically
requires one node being the final source of truth, which then redistributes
that information.

I suppose it'd be possible if the search results could be delayed. Perhaps for
a wikipedia type of project, where it's not the links that change, so much as
the content.

[1] [https://austingwalters.com/is-search-
solved/](https://austingwalters.com/is-search-solved/)

~~~
cannedslime
Maybe you are right on the whole aggregating and redistributing trust thing,
but couldn't that be solved or at least mitigated by having multiple peers in
the network do the same scraping tasks and having multiple peers return at
least hashes for search results?

~~~
lettergram
You could do that, the trick is determining which peer has reviewed which
links. I definitely think it's doable, but in reality there's little benefit
to the developers and it is going to be super hard.

------
guilhas
[https://yacy.net/](https://yacy.net/)

------
sharemywin
not distributed but open:

[http://commoncrawl.org](http://commoncrawl.org)

