
On Dat - unicornporn
https://www.kickscondor.com/on-dat
======
SturgeonsLaw
> The prolonged obsolescence of distributed protocols like Bitcoin and
> torrents means we’re maybe skeptical or jaded about any new protocols

Funny choice of examples, personally I would have pointed to bitcoin and
bittorrent as two examples of distributed protocols with runaway success

~~~
ahje
Yeah, Bittorrent is actually getting fairly old compared to many other
technologies in use today; it's _older than_ Windows XP. :D

~~~
cm2187
A newcommer compared to nntp, which arguably is a kind of p2p network!

~~~
josteink
nntp relied on central servers which clients connected to, so not fully
decentralized.

That the servers did inter-server replication and forwarding doesn’t really
change that.

~~~
cm2187
But the servers are p2p between them, i.e. they propagate the content to the
other servers. If you take down one of them, you can still connect to any
other server to get the same data. Keeping in mind this is 1980s technology,
it looks pretty close to p2p to me.

------
deanclatworthy
I tried beaker when it first featured on HN. My website isn't particularly
heavy. It completely crashed the client trying to upload it. We're talking <
100mb of data. I believe they're still working on the issue [1].

[1]
[https://github.com/beakerbrowser/beaker/issues/952](https://github.com/beakerbrowser/beaker/issues/952)

~~~
pfraze
The next major release of Beaker (0.9) is going to have a new version of Dat
that improves scaling quite a bit. Mafintosh and his team has done some
amazing work using trie indexes for much faster reads across the network.
Beaker's also gotten much better at syncing with folders, which was the issue
in your link.

~~~
deanclatworthy
I believe the issue was with the client, not the protocol :)

------
tannhaeuser
While we're at it, could someone share their experience with Secure
Scuttlebutt vs dat vs ipfs? My interest is to have the simplest thing that
could possibly work to make existing static and moderately dynamic sites
p2p-publishable. From what little I know, ipfs seems too heavyweight,
requires/repurposes git with GPL license, etc. and isn't rooted in HTML/http
concepts, while dat/beaker is at least targetting my use case directly. But
I'm open to get convinced otherwise.

~~~
bennofs
I don't have a lot of experience with these, but I tested each of them a
little bit and know mostly how they work.

I believe that for small sites, the main aspect that differs between them is
how peer discovery/name resolution is done. For this, IPFS has the most
distributed approach (using a DHT and IPNS) but is also the most fragile and
highest latency one. DAT has plans to switch to a DHT in the future
(hyperswarm) but relies on centralised DNS discovery servers + local network
multicast right now. Secure Scuttlebutt takes a more social approach and
organises its network in terms of pubs. There is no central discovery, you can
only find users that you "meet" in a pub.

This means that DAT has the most website-like feel because their solution
provides low latency (only DNS lookup) and is global. I also think that DAT is
much more pragmatic compared to IPFS thus at the moment it is much more stable
and its us ability is better (also because it has more focus on stable, core
components compared to IPFS which is a bigger project trying to do lots of
things at once). I like secure scuttlebutt's idea of focusing on communities
but that is a different approach to how the web works right now.

~~~
kickscondor
> Secure Scuttlebutt takes a more social approach and organises its network in
> terms of pubs. There is no central discovery, you can only find users that
> you "meet" in a pub.

The caveat is: those users can’t meet _you_. I used SSB for a few months,
joining pubs, commenting, trying to participate. No responses. (Not a huge
surprise—you can get lost in the piles of people on Twitter, Reddit, etc.)

However, eventually I discovered that no one could see my contributions unless
they added me—and SSB (or Patchwork, in this case) gave me no way of
advertising my presence. This was pretty self-defeating. So now I don’t have
to just build a ‘presence’ inside the network, I also have to build a
‘presence’ outside the network to announce that I’m somewhere inside the
network. The SSB tools also give you no inkling that this is the case. So just
know to bring friends!

~~~
trickstra
Yes, they consider this a feature, not a bug, though. Their social network is
proudly invite-only, so you need someone to pull you in. Once somebody
subscribes to you, your content will be visible to all their subscribers.

------
AgentME
>IPFS is really cool—but how do I surf it?

If a site has a domain that's set up with a IPFS DNSLink record, then you just
go to its domain name and access it over HTTP(S). But if you have the IPFS
companion extension, then your browser will access the site through IPFS peers
instead of HTTP. So even if the site's webserver is down, you'll still be able
to access the site if any other IPFS peers have its content pinned. You could
then pin the site's content and help host it.

This all would work through a normal web browser, and gracefully degrades to a
plain HTTP connection to a web server for people without the IPFS companion
extension.

If you have a domain name and are able to keep your site content available in
IPFS (on your own machines you keep up, on a pinning service, on a vps, on
some friends' machines who you convince to pin it, etc), then you could have
someone else host a ipfs web gateway for your domain scoped to your domain
such as Cloudflare ([https://www.cloudflare.com/distributed-web-
gateway/](https://www.cloudflare.com/distributed-web-gateway/)). Then you
don't need to keep your own web servers up, and the only maintenance you have
to do for the ipfs web gateway on your domain is to keep your IPFS DNSLink
record correct. Regular users will access your content through Cloudflare (who
fetch it from IPFS and then aggressively cache it), and IPFS companion users
will access it directly through IPFS.

There's also the concept of IPFS links (like
ipfs://QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnKi385KPN/introduction/usage/),
which you can access directly if you have the IPFS companion extension, or you
can access through a public ipfs web gateway (like
[https://ipfs.io/ipfs/QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnK...](https://ipfs.io/ipfs/QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnKi385KPN/introduction/usage/)).
But you probably don't want to give links like either of those to your users
unless you don't care for domain names. I think the IPFS documentation makes a
mistake by emphasizing raw IPFS links so much as opposed to DNSLink records.

~~~
xchaotic
Too complicated for very little gain.

~~~
AgentME
Assuming Cloudflare is being used, then the effort on your side is to run
"ipfs add -r directory_of_your_site_content" on a machine that stays up (or a
pinning service), and then you copy the hash into a DNS record.

The gain is that you don't manage any webservers, and other users can help
host your site content and can keep your site alive after you stop hosting it
on ipfs.

I find the concept of being able to make a site that outlives my ability to
host it (and hopefully outlives me) as long as people find it worth pinning is
super interesting.

------
hobofan
The thing that always rubbed me the wrong way was how much Dat seems to
embrace mutability. That the content behind every URL is mutable is one of the
things I dislike the most about the current internet. A lot of the content and
data I consume on a daily basis is rarely-changing, so it would be fine to be
immutable.

I feel that dragging along all the problems of mutability-by-default that
plague the current internet when rebuilding it in a more decentralized way is
a big mistake. Yes, I know that there are some fringe efforts to enable more
immutability with "hypercore-strong-link" etc., but they look like an
afterthought, that won't be supported by most of the Dat ecosystem if it finds
major adoption.

~~~
ComodoHacker
>A lot of the content and data I consume on a daily basis is rarely-changing,
so it would be fine to be immutable

Would you want to see typos corrected, facts checked and other improvements on
that content?

~~~
iamnothere
You could simply make all content versioned, with a full history available.
Browsers would probably want to show the "latest" content by default, but they
could implement a function to view previous versions or even to highlight
changes. Ideally versions would come with Git-style comments so you could read
the reason for the change.

This would provide an amazing archive for anyone wanting to see, for example,
what was on a "front page" on a certain date. It would also let you see if
someone had altered a specific article for whatever reason.

~~~
kickscondor
This is true of Dat, though. You can jump back to a website’s prior history at
any time, so long as those chunks are still seeded.

~~~
iamnothere
So in theory an archive.org type entity could continue seeding old versions of
websites, allowing you to go back in time, and it could be verified that it
was an older version? If so, that's a good compromise.

~~~
kickscondor
For sure! And the URLs aren't theoretical - you can view an older version of
my own blog by adding '+version' to the end of the URL. Like so:
dat://kickscondor.com+1600/.

Each atomic file change creates a new version.

------
hyperion2010
On the question about how a search engine would work. The Dat team have
thought (and implemented) quite a bit about building performant indexes for
distributed file systems. I've wondered if we relaxed the time requirements
for search whether there are other processes that we could use that would e
more robust. "Who has 10.19.20.13?" seems like it might be a little bit too
low level, but what if it was "Who has the most reliable data about the
publishing industry between 1500 and 1600?" expiration in 30 minutes with an
extension of up to 90 minutes for each preliminary result with more than two
nodes responding.

Could it work? What kinds of use cases could it support? How would a bot that
had a good model of which forums to post things to do in comparison if it had
2 hours, 24 hours, 1 week? What about if we put _lower_ bounds on the amount
of time that had to be spent looking (this seems like a fun challenge for
proof of 'real' work).

While this is at a much higher level than the current dat protocol, it is in a
sense quite similar to the query "Who has
353904391670d2803b34990e37f4d2e96f49351998e162d0e335b16812daf592e0f71470af7bee31f6a1da03744d03bcde659d73a0ebf56fd4a9fc6ef67edf60
that is 5 bytes long?"

~~~
saurik
> "Who has the most reliable data about the publishing industry between 1500
> and 1600?"

So, to verify I understand: the backend of this would be handled by grad
students (working with some kind of 30/90 minute timer alarm), and would
experience distributed consistency failures caused by differences in which
departmental tradition they did their major area exams?

~~~
hyperion2010
Pretty much. Though for that question you might have to go all the way up to
tenured faculty, which could incite a full on Byzantine war over which
particular source was more reliable, however I think at that point it is clear
that we have lucked out and found two really great sources of information
instead of just one and potentially triggered the creation of a third :)

------
tekknolagi
I wrote a small essay about Dat for a course on distributed systems:
[https://bernsteinbear.com/dat-paper/](https://bernsteinbear.com/dat-paper/)

In it I do some analysis of its strengths and weaknesses.

------
llao
Having to stay on a website 24/7 to get new content sounds not very usable or
user-friendly.

~~~
ShorsHammer
Great for engagement metrics though.

~~~
kickscondor
Haha! :) Actually, as long as one peer is connected 24/7, everyone will see
everything. Which seems reasonable.

------
gioscarab
Has anyone looked into PJON? I see a lot of work done in the direction of
decentralization, but is all at a really high level, shouldnt we re-work the
layer 2, 3 and 4 to be compatible with a decentralized structure before
thinking about content ids? See
[https://github.com/gioblu/PJON](https://github.com/gioblu/PJON)

~~~
macawfish
Some of this high level stuff is really important for thinking about stuff
like data structures and "asynchronous ux design". These problems don't
strictly depend on the exact underlying transport or discovery mechanisms,
although there is definitely work to do there as well!

Thanks for sharing this interesting thing I'd never heard of!

~~~
gioscarab
Ciao macawfish absolutely :) the only issue is: until we don't rework the
existing low level standards and we build our own private physical network
infrastructure (each user takes care of his own router and provide others with
connectivity for free without intermediation of any third-party or service
provider) we will not have a truly "decentralized" network.

------
1023bytes
I just tried the Beaker browser and their own main site is currently
unaccessible through it.

~~~
pfraze
Thanks for mentioning it, I just deployed a fix

------
Crinus
If you need a custom browser for this to work then why not get rid of the
entire HTML+JS+CSS+Wasm+whatever else complexity monstrosity and not build
something much simpler based on almost three decades worth of hindsight?

~~~
iamnothere
It should be possible to build a DAT-capable browser for simple types of
content (RST, MD, orgmode, etc); no HTML capability included. If you made it
download media content instead of displaying it in-browser, it might even be
simple enough to implement this without spending too much time chasing down
security issues.

~~~
astrobe_
One could even go down to a slightly modified Gopher protocol. Just let
publishers publish in whatever format they see fit (PDF, docx, html, ...) and
let the users configure the viewer in their browsers.

------
m4r35n357
Why do we need node?

------
_bxg1
This blog has a really cool page layout

------
marknadal
Multi-writer not working, and still requiring hacking workarounds?

This bothers me.

P2P Reddit ( [https://notabug.io/](https://notabug.io/) ) has been doing this
for over a 1+ year.

All content there updates in realtime too, fully decentralized. (WebRTC or
daisy-chained socket relays)

I did not build NAB, but I work on the underlying P2P protocol that powers it
( [https://github.com/amark/gun](https://github.com/amark/gun) ) which
competes with DAT - but I've always recommended DAT to people because I
thought it already had working multi-writer apps on it.

Paul, isn't this already possible? You've shown me demos!

~~~
ddrdrck_
I had a look at [https://notabug.io](https://notabug.io), oh my ... this is a
great reminder that full decentralization may not actually be the most
desirable thing to pursue.

