
IPFS Project Roadmap - robto
https://github.com/ipfs/roadmap
======
fwip
"2019 Goal: The most used code and binary Package Managers are powered by
IPFS."

That's kind of stupid-ambitious for 2019 when another 2019 goal is "a
production-ready implementation" and IPFS has been around for 3 years already.

This isn't a roadmap, it's a wishlist. And I'm someone who wants to see IPFS
succeed.

~~~
StavrosK
The IPFS client is such an untunable memory hog that I turn it off whenever
I'm not using it (which, of course, defeats the entire purpose). I would be
ecstatic if we had something like the old uTorrent, but for IPFS. A nice UI,
easy configuration, an ultralight implementation. It would be a dream come
true.

~~~
threeme3
Exactly this: such an ultralight, accessible implementation would make it
better suitable to run on embedded devices and mobile phones. And since we
still live in as fairly disconnected world this is probably an area where IPFS
can accelerate.

~~~
hecturchi
I kind of made this [https://github.com/hsanjuan/ipfs-
lite](https://github.com/hsanjuan/ipfs-lite)

May not be ultralight (running a DHT node isn't, but it does remove a lot of
crust).

------
pdxww
This is not a roadmap, but rather a wishlist. There is a fundamental problem
that IPFS needs to solve first. This problem is called an efficient WebRTC-
based DHT. In order to change the web, IPFS needs to become usable in the
browsers. Since the backbone of IPFS is DHT, there need to be an efficient
UDP-based solution for "DHT in the web". Right now this isn't possible and
reasons are not just technical, but political. The IPFS team would need to
convince all major players that enabling this DHT scenario is a good idea.

~~~
nmca
I actually wrote a DHT that operated over WebRTC itself with in-band
signalling for my undergrad thesis, in the application/js layer. Total PITA,
but a ... "good?" learning experience.

~~~
pdxww
How could you possible make such a DHT? We live in the world of NATs,
especially symmetric NATs, where each mobile phone user gets assigned a random
ip:port every time it makes a connection. DHT, on the other hand, needs every
node to have a persistent address that can be contacted any time. In other
words, with the NATs, a DHT node cannot cache a bunch of peers and contact
them later because those peers are no longer available at those addresses, so
every time a node re-joins DHT, it needs to restart the bootstrapping process
from scratch, from those initial hardcoded bootstrap servers. Effectively this
makes this DHT a fully centralized system. WebRTC cannot solve this problem.

~~~
nmca
You have some proportion of power users that are outside NAT and WebRTC (they
run a non-browser executable), their addresses can be shared over the network
and stored locally. They provide STUN on rejoining, but are not hard-coded.

(Overall, yes this whole thing is a vaguely bad idea. But not totally
degenerate / non viable, just not practically worth it.)

~~~
pdxww
If this idea worked in practice, go-ipfs/js-ipfs wouldn't have so severe
problems with connectivity.

------
Ericson2314
If you want to do package managers, your #1 priority should be Nix. Don't do
something more popular where you help less, go with the thing that you can
really provide the killer missing feature.

Nix + IPFS has been tried before, but what was missing is the Nix-side
hierarchical content addressing of large data (not just plans). With the
"intensional store" proposal, this should finally happen. Please push it
along.

Data shouldn't be hashed like Nix's NARs, or IPFS's UnixFS for maximum
interopt. Instead please go with git's model for all it's problems.

Thanks, hope someone can take me up on this because I'm up to my neck with
other open source stuff already.

~~~
sideeffffect
there's been some effort

[https://github.com/NixOS/nix/issues/859](https://github.com/NixOS/nix/issues/859)

~~~
Ericson2314
Right but not connected to the intentional store.

------
fiatjaf
The vision of a IPFS-powered web working is beautiful.

However I would love to see a reference implementation that works at minimum
and not just drains out your computer up to latest resource it may have. If
we're so near the "production-ready" status of the reference implementations
then I think that goal will never be achieved.

~~~
dymk
I see this comment often when IPFS is discussed - but the devil is in the
details when it comes to replacing the underlying tech of "the web" with
something else.

How does an IPFS powered website do dynamic content? User sessions? Is all the
client's session data encoded in the IPFS address itself?

Even if there's no user sessions, but the page content updates, how do you
continuously point clients to fetch the right updated page (e.g. how would you
implement a Hacker News style aggregator that updates every minute)?

IPFS does static content just fine - CAS-es are wonderful for that - but
websites are much more than static content.

~~~
marknadal
For dynamic data, you need to use a CRDT system like GUN (
[https://github.com/amark/gun](https://github.com/amark/gun) ).

For instance, see P2P Reddit ( [http://notabug.io](http://notabug.io) ) which:

\- Running in production with GUN.

\- Handling about 42,000 monthly visitors. (
[https://www.similarweb.com/website/notabug.io](https://www.similarweb.com/website/notabug.io)
)

\- Has done ~1TB in 1 day of decentralized traffic.

You can then configure GUN to save to IPFS as the blob/storage engine (or we
have filesystem, localStorage, IndexedDB, S3, etc. as options).

It is basically like having a P2P Firebase :) with IPFS plugin.

~~~
fwip
It's a shame that so many P2P advocates - or at least the ones using the stuff
built with P2P tech - are very loud alt-right types.

I popped on notabug.io just now, and the chat is full of swastikas, racial
slurs, and the most up-voted posts are primarily anti-gay misinformation.

I see you wrote GUN, which I'm sure was no small feat, and it looks like an
impressive piece of technology. How do you feel about your tech being
primarily used in this way?

~~~
fiatjaf
I just clicked on notabug and didn't see any swastikas or racial slurs. Are
you lying?

~~~
fwip
You think somebody would do that? Just go on the internet and tell lies?

Warning: image contains racial slurs and swastikas:
[https://i.imgur.com/0kdxzVR.png](https://i.imgur.com/0kdxzVR.png)

A little later in the chat, the developer chimes in and talks about some
stuff. Then some people angrily accost them for censoring things. If the Nazis
and the racists aren't censored, I don't want to think about the odious
content that is.

Edit: Here's a thread from... yesterday, where a bunch of users are mad about
the "censorship" of the developer hiding a swastika post from the front-page
(not even deleting it or removing it from whatever their equivalent of a sub-
reddit is):
[https://notabug.io/t/whatever/comments/509b9189ece85515671d3...](https://notabug.io/t/whatever/comments/509b9189ece85515671d3074e71a40da8955cd03/go1dfish-
censors-posts-based-off-political-content)

~~~
fiatjaf
Well, to me these seem like some adolescents trying to be funny and go against
the mainstream. They post swastikas and call racial slurs because they know
they shouldn't be doing that. They aren't really nazis, right? They don't even
know what a nazi is.

You can call this "decentralized reddit" a bad place, as it really is, but you
can't say it's because it's "full of right-wing people". These adolescents are
not "right-wing people".

~~~
viraptor
> They aren't really nazis, right?

At some point... is there a difference? If you find yourself in a group
"ironically" screaming you all support X for long enough, soon you'll find
that some of you actually support X. And that you enabled those people.

~~~
heyitsguay
Well and besides, the actual content on the site seems to skew alt-right
pretty heavily, whether the racial slurs and swastikas are ironic or not.
Taken altogether, it paints a pretty clear picture of a voat-style forum that
will alienate most other people if it stays that way.

~~~
mirimir
They're there because they've been excluded from mainstream sites. It may seem
extreme, but one way to assess how uncensorable something is by its content of
stuff that you hate. Even stuff like child porn that any sane person would
hate. So you just ignore it.

------
techntoke
Would love to see Arch/Alpine Linux repo move to IPFS by default. Would also
like to see better integration with Git, and an SCM platform comparable to
GitHub (or GitLab). That could really get the developer community heavily
involved in the project if it was sponsored by Protocol Labs.

~~~
whyrusleeping
Yeah! That's a thing we're working towards. We're currently looking into apt
and npm, both efforts are coming along pretty well and driving development
towards fixing the bottlenecks preventing it from "just working".

[https://github.com/ipfs-shipyard/npm-on-ipfs](https://github.com/ipfs-
shipyard/npm-on-ipfs) [https://github.com/ipfs-shipyard/apt-on-
ipfs](https://github.com/ipfs-shipyard/apt-on-ipfs)

------
anandology
In addition to apt and npm, I would like to see docker image distribution
powered by IPFS. It really feels stupid to pull images from a central registry
sitting on the other side of the globe when the image is already present in
the next node in your kubernetes cluster.

~~~
justincormack
There is a big issue, that ipfs uses its own hash mechanism (hash of protobuf
of dag), while registries (and most other existing content based distribution
mechanisms) use sha256 hashes of the whole content.

eg see
[https://github.com/ipfs/notes/issues/269](https://github.com/ipfs/notes/issues/269)

So you can't simply interop between the two without some sort of lookup to
convert hash functions. It is hugely frustrating, as basically different
content based distribution mechanisms cant work together.

In theory docker image registries support pluggable hash functions, although
it is not clear to me that the ipfs function is even very well defined outside
its own code. We could start to add a second hash calculation to every
registry operation, but it would be a performance hit which some users would
not like.

(tree based content hashes that allow parallelisation are nice, but the ipfs
one is very ipfs specific and more complex than it needs to be I think).

------
microcolonel
Discovery performance is the biggest issue I see. If I deliberately load the
same file on a couple of peers, it can take hours (or forever) to be able to
find a peer with that file to pin it. It is clumsy and difficult to explicitly
connect to peers (because you can't just try to discover peers at an address,
you need to include the node ID as well), and even if you manage to enter the
right information, you won't necessarily succeed at connecting to the peer the
first time.

------
rob-olmos
It's nice to see #2 for package managers, something I've been thinking about
recently. I haven't look much into this yet, but I wonder if IPNS could
provide a step forward in supply chain protection since package signing isn't
available yet in certain managers/repos or not commonly utilized.

------
0xb100db1ade
I love the idea of IPFS, but I can't think of a use case not covered by
torrents.

Would someone mind enlightening me regarding what sets IPFS apart from
torrents?

~~~
fwip
Main feature is automatic data-sharing between distributions. With torrents,
everything is siloed, and data is only exchanged between peers of that
torrent. IPFS doesn't care /why/ you're getting information or the link you
found it from, just that it can find it by its hash.

Say you distribute "Julie's Webcast Complete Series" and somebody else
distributes "Julie's Webcast - Episode 3, with Russian subtitles," peers and
seeders from both distributions can share data for the shared content.
Similarly, updating a dataset only requires downloading the new data.

This is done automatically, both per-file hashing and (optionally, not sure
the current state) of in-file block hashing.

~~~
genpfault
> Julie's Webcast - Episode 3, with Russian subtitles

> peers and seeders from both distributions can share data for the shared
> content

So does IPFS have "plugins" for different archive/container formats so it can
"see" that the underlying video/audio streams are identical between "Julie's
Webcast - Episode 3.mp4" and "Julie's Webcast - Episode 3, with Russian
subtitles.mkv"?

Otherwise container stream interleaving will play holy hell with any sort of
"dumb" block hashing :(

~~~
fwip
Last I checked it was dumb. Possibly breaking block boundaries based on a
rolling hash.

~~~
sideeffffect
[https://github.com/ipfs/go-ipfs-chunker](https://github.com/ipfs/go-ipfs-
chunker)

> go-ipfs-chunker provides the Splitter interface. IPFS splitters read data
> from a reader an create "chunks". These chunks are used to build the ipfs
> DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses
> to address content.

> The package provides a SizeSplitter which creates chunks of equal size and
> it is used by default in most cases, and a rabin fingerprint chunker. This
> chunker will attempt to split data in a way that the resulting blocks are
> the same when the data has repetitive patterns, thus optimizing the
> resulting DAGs.

I think they should use the rolling hash based chunking by default

[https://github.com/ipfs/go-ipfs-
chunker/issues/13](https://github.com/ipfs/go-ipfs-chunker/issues/13)

------
viksit
One of the biggest challenges with IPFS in my mind is the lack of a story
around how to delete content.

There may be a variety of reasons to delete things,

\- Old packages that you simply don't want to version (think npm or pip)

\- Content that is pirated or proprietary or offensive that needs to be
removed from the system

But in its current avatar, there isn't an easy way for you to delete data from
other people's IPFS hosts in case they choose to host your data. You can
delete it from your own. There are solutions proposed with IPNS and pinning
etc - but they don't really seem feasible to me last I looked around.

This list as @fwip said is great as a wishlist - but I would love to see them
address some of the things needed in making this a much more usable system as
well in this roadmap.

~~~
Risord
Managing universal data removal is not universally solved (or even wanted) on
internet scale. So it sounds weird to demand it from technology which is
trying to solve completely different problem.

~~~
nine_k
A best-effort deletion could be beneficial for any node. It reduces storage
requirements a bit.

A way to mark something as deleted, in a form of another ipfc object, could
serve as soft delete, and also a permission to actually delete hunks of the
object marked as deleted.

This, of course, is not secure deletion, and should not be.

~~~
setr
>A best-effort deletion could be beneficial for any node. It reduces storage
requirements a bit.

As far as I'm aware, any node is free to delete its own data. In fact, isn't
only storing data at the user's explicit request in the first place? It just
can't do anything about what other nodes choose to keep or delete. If you're
referring to a particular node wanting to delete do a best-effort deletion of
data on other nodes, it's not clear to me why node A cares about reducing
storage on nodes B, C and D (if you own those nodes, delete it yourself; if
you don't.. then I don't know what you're up to)

------
woodandsteel
I have a question for the IPFS people. I am a non-techy who really likes the
IPFS idea and wants to see it succeed.

However, whenever this topic comes up here at HN, we get a bunch of people who
say they tried to use it but it was basically unworkable, like too much RAM
usage and various sorts of failures. And rarely does anyone respond by saying
that it is working just fine for them.

So my question to the IPFS people is, when is it going to get really usable? I
am asking for something reasonably specific, like 2 or 3 years, or what? And I
am supposing that would mean a different promise/prediction for each main
different use case. So how about some answers, not just "We are aware of those
problems and are working on them"

~~~
momack2
As you seem to foresee - being "really usable" depends on the use case. The
one we're focused on this year is package managers - and making IPFS work
really well for that use case in particular. There is lots of room for
improvement on performance and usability - setting the package managers goal
gives us really a specific target to focus and deliver on. This won't solve
"all the problems" (there's a lot to solve for package managers alone!) - but
will help us take a big step forward in production readiness and hopefully
knock out a swath of performance issues experienced by everyone.

------
kamfc
What's the difference between DAT and IPFS? I'm trying to understand all these
new technology with a grand aspiration to replace the current infrastructure.

[https://ipfs.io/](https://ipfs.io/)
[https://datproject.org/](https://datproject.org/)
[https://beakerbrowser.com/](https://beakerbrowser.com/)

------
paulsutter
So I can store a file in ipfs by its hash, but there’s no way to link to the
next version of the file. I can only link to older versions?

I’m a giant advocate for decentralized architectures but so far I’ve never
found a use for it that doesn’t rely on a centralized way to find out about
new data

~~~
johntash
I'm not super familiar with ipfs, but I _think_ ipns is supposed to solve that
problem.

> Inter-Planetary Name System (IPNS) is a system for creating and updating
> mutable links to IPFS content. Since objects in IPFS are content-addressed,
> their address changes every time their content does. That’s useful for a
> variety of things, but it makes it hard to get the latest version of
> something.

[https://docs.ipfs.io/guides/concepts/ipns/](https://docs.ipfs.io/guides/concepts/ipns/)

------
nullobject
This is exciting. I would love to make it to the 1st IPFS Camp in June.

------
ezoe
IPFS is a joke. They have name lookup feature but relies on traditional DNS!
What are they thinking?

Also, if the IPFS's idea of working as local server is sound, BitTorrent
DNA(browser plugin, steaming video over BitTorrent) should had been worked.

It seems to me, they suffered NIH syndrome. They tried to reinvent the wheel.
The P2P file transfer protocol over IP has already been covered by BitTorrent.
What we need is a nice front end which use BitTorrent protocol as back end and
offer a illusion of Web site.

~~~
miguelrochefort
The real innovation is making files content-addressable.

~~~
Lx1oG-AWb6h_ZG0
Don’t magnet links solve that for torrents?

------
caprese
I've been considering Swarm distributed file system because of its closeness
with the Ethereum development.

It seems to do the same thing and works already but hardly gets any press.
IPFS and the Protocol Lab's Filecoin sale seemed to generate a lot of
marketing despite it becoming clearer later that Filecoin is for an unrelated
incentivized network.

It is hard understand the pros and cons of choosing to use IPFS over Swarm, or
where they are in comparative development cycle.

I know many decentralized applications that opt for IPFS for their storage
component, and know of the libraries to help different software stacks with
that. But I can't tell if it is right for me, versus the state of Swarm.

~~~
nonsens3
Swarm and IPFS together with Filecoin try to address the same problem -
persistent data storage in a decentralised network.

Swarm is not at all "working already" \- the incentivisation layer for nodes
to store data for other users is not implemented and currently mostly
theoretical and work-in-progress.

IPFS is more mature in comparison to Swarm, but the underlying architecture is
rather different.

~~~
caprese
What is Swarm's intended incentivisation layer and where can I read about
their plans? It seems like all documentation including plans are outdated, and
I was ignored in their gitter chatroom where devs wanted to talk about dev
things and outreach people seemed nonexistent.

I see things being stored on Swarm without incentives, like plain text

~~~
nonsens3
Swarm documentation might not be perfect, but it is not outdated -
[https://swarm-guide.readthedocs.io](https://swarm-guide.readthedocs.io)

I believe the chapters about PSS, Swarm Feeds, ENS, Architecture, among
others, are mostly up-to-date.

You can read about the incentivisation layer at [https://swarm-
gateways.net/bzz:/theswarm.eth/ethersphere/ora...](https://swarm-
gateways.net/bzz:/theswarm.eth/ethersphere/orange-papers/1/sw%5E3.pdf)

Currently incentivisation is not integrated or implemented in Swarm, so a user
has no guarantees about what happens with their uploaded content. If the node
hosting it disconnects from the network, it will be gone. The plans to address
this are through the sw^3 protocols suite and/or erasure coding.

Regarding plain text - it doesn't really matter what bytes you store in Swarm
- encryption is implemented and you can store non-encrypted or encrypted
bytes, this has nothing to do with incentives for persistent storage.

We try to do outreach and answer community questions when possible, but the
team is not big and this is currently done on a best-effort base, we could
definitely improve on that front, I agree.

