
Internet Archive, decentralized - justin_
https://dweb.archive.org/
======
marknadal
GUN author here ([https://github.com/amark/gun](https://github.com/amark/gun))
happy to answer any questions.

IA did this integration in 1 week, Mitra is awesome.

Also, decentralized Reddit ([https://notabug.io](https://notabug.io)) was
built in 1 week on us, and pushed 0.5TB P2P traffic on 1st day.

Note: I may not be awake for several hours, and might not be able to reply
until Monday.

~~~
modernerd
GUN looks great! I love the quick-start tutorial too:
[https://gun.eco/think.html](https://gun.eco/think.html)

How suitable is GUN for live multiplayer (non turn-based) web games? (Similar
to [https://airma.sh/.](https://airma.sh/.))

I see one game example at [https://github.com/amark/gun/wiki/Awesome-
GUN](https://github.com/amark/gun/wiki/Awesome-GUN), although it's turn-based.

~~~
marknadal
Thank you!!!

I need to fix the organization of the documentation (and update the docs,
oye!). I'm impressed you found Awesome-GUN.

[https://github.com/amark/gun/blob/master/examples/game/space...](https://github.com/amark/gun/blob/master/examples/game/space.html)
is probably what you were searching for.

What would be better is if I made a blog/tutorial for ^ link. Not a priority
for me, sadly, but maybe it is for somebody out there, that they could help?

Thanks again!

------
ezequiel-garzon
I strongly believe IA or any serious project working on permanent persistence
must provide an option (opt-in if you will) to make the published material
irremovable, à la arXiv [1].

[1] Red parts of
[https://arxiv.org/help/license](https://arxiv.org/help/license)

~~~
quickben
And that is your right. However most people would like to remove slander with
a court order.

~~~
ezequiel-garzon
Pretty sure arXiv would remove slander with a court order as well. It's an
unrelated issue.

~~~
crooked-v
If material can be removed, it's not irremovable.

~~~
mehrdadn
I think it's fair to assume "not-${VERB}able" means "as not-${VERB}able as
legally possible".

~~~
ndr
This gets tricky with things like blockchains.

------
bo1024
What is it? My browser (firefox) sees nothing* even after enabling javascript.

*Except the sentence "The decentralized web is everywhere, but we have to find it." and a Name form that does nothing.

~~~
trumped
the problem is not firefox... it's probably addons or your internet connection
that blocks content

~~~
superkuh
It is also nothing for me in a Firefox fork with almost all add-ons disabled.
Error console says there's a bunch of "class is a reserved identifier" and
syntax errors of missing semicolons (before main() can even run).

My guess it's trying to use some bleeding edge emcascript stuff which borks
the parsing in non-bleeding edge browsers.

~~~
trumped
oh... I forgot, I'm using the beta version of Firefox, that might be why.

~~~
mitraardron
If you run into problems, I'd love to know specific details - this is an
experiment to see what was achievable with Dweb tools like IPFS, WebTorrent,
Gun on browsers, with a big site like the Archive. It certainly pushes the
edge of browsers - and does use the latest ES6 features without much attempt
to support older browsers (unlike the production site at archive.org). We only
currently test in the latest Chrome & Firefox, though it seems to work
(mostly) on current Safari on iPad and iPhone and I heard its working on
Android though I haven't seen it yet.

If you can get on, there is a feedback button, if you can't feel free to go
straight to the form at
[https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2...](https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2wODcL3ro7D6LegPksb86jr5bCJa7Ig/viewform)
or open an issue on [https://github.com/internetarchive/dweb-
archive/issues](https://github.com/internetarchive/dweb-archive/issues) .

~~~
superkuh
> ... and does use the latest ES6 features without much attempt to support
> older browsers (unlike the production site at archive.org)

Except the wayback machine which recently changed it's interface to be
entirely JS dependent and fails on older browsers. And the old interface is no
longer accessible.

------
Kagerjay
I didn't see any articles relating to how it was decentralized. Is internet
archive going for something similar to a blockchain setup, because that would
actually be one of the few cases it would make sense

I am not familiar with this new dweb subdomain, what is unique about it?

Archive has actually lost some of the archives I stored on it, which is weird,
because I ran multitude of backups on it a few years back when the site got
taken offline

~~~
justin_
I wanted to submit an article about this but couldn't find anything either.

My understanding is that this loads content from various protocols (listed at
the top of the page), many of which support replicating data in a
decentralized way. As far as I know, there's no blockchain involved in
anything here yet.

If you browse to the Community Video section and choose a video, you can see
peer information as though downloading through WebTorrent. If I disable
WebTorrent and look at a video, I don't see the peer information and it seems
to fall back to HTTP. Pretty cool! It looks like almost everything is only
seeded by the Internet Archive right now, but hopefully they want to encourage
more people to participate.

~~~
Kagerjay
Hm that's interesting, I imagine the internet archive has lots of
seedboxes/CDNs/datacenters distributed globally. I have no idea how much
information the internet archive is currently backing up though, but its
growing at an increasing rate.

I would love to see a write up of how their infrastructure works though

~~~
toomuchtodo
My understanding is all of their digital content is stored and served from
their location in SF, although it would be awesome if they started
geographically distributing their storage nodes.

[http://archive.org/web/petabox.php](http://archive.org/web/petabox.php)

~~~
sp332
There's been at least a partial copy in Alexandria and Amsterdam for a long
time, and they opened a full replica in Canada last year. At least I think the
Canada one is done, can't find a blog post saying it was finished.
[https://blog.archive.org/2016/12/03/faqs-about-the-
internet-...](https://blog.archive.org/2016/12/03/faqs-about-the-internet-
archive-canada/)

~~~
toomuchtodo
Was not aware; this is excellent news!

------
CuriousSkeptic
When cloudflare screwed up and published everyones secrets there was a
coordinated effort involving, amongst others, archive.org to try and scrub the
internet of those secrets. Are there any mechanism available to allow similar
efforts with this dweb version?

~~~
comboy
There should not be a mechanism for that. China mastered such mechanisms. We
should rather optimize for not having that kind of points of failure.

~~~
alkonaut
I think of the two evils, the possobility of censorship or mistakes being
eternal, I think I prefer the former.

For the same reason I find it comforting that an old fashioned bank transfer
can be corrected if I transfer money and make a mistake writing the account
number. Mutable history is a powerful feature.

~~~
comboy
This feature is an illusion though. Mallory still can save anything that
appears online for a second and so can you.

~~~
alkonaut
Sure. But if I e.g. accidentally uploaded something sensitive to GitHub (that
can’t simply be changed to a new secret), I’d certainly delete it in a hurry,
rather than shrug and say ”oh well It’s on the someone has already copied it
so I’ll leave it”.

~~~
vageli
But in that case, are you saying you _wouldn't_ immediately change the
credential you committed? Sure, the possibility of an adversary forking your
repo after that commit but before your revision is small, but still exists.

Once a secret is exposed to the internet, it should be considered public and
rotated. In this case mutability/immutability is moot though likely there are
applications for other, non-credential secrets that are not so easily rotated
(like your home address or something).

~~~
alkonaut
Yes a changeable credential you just change, but say the medical records of
all staff your entire company or similar.

------
nikisweeting
Aww I was going to add almost exactly the same thing to
[https://github.com/pirate/bookmark-
archiver](https://github.com/pirate/bookmark-archiver) but IA beat me to the
punch! I still hope to add decentralized storage and lookup mechanisms to BA
eventually, but considering one BA's archive outputs is the Internet Archive,
it's less pressing now.

------
krylon
This is really cool! A bit sluggish, but I love the idea!

If you think back to, say, the library of Alexandria, to how much knowledge
has been lost over the ages, it is so important to preserve as much as we can
for future generations.

And building a decentralized foundation for this archive is a big step going
forward, congratulations!

~~~
pixelpoet
I recently went to the Internet Archive in SF, got a tour of their operations
etc. Absolutely amazing place, very forward thinking, and they indeed are very
serious about prevent something like the destruction of the Library of
Alexandria from happening again.

~~~
krylon
There is a (great, IMHO) science fiction novel, "The Mote In God's Eye" by
Larry Niven and Jerry Pournelle that describes humanity's first contact with
another intelligent species.

Due to their biology, this alien species experiences periods of massive
population growth that eventually lead to all-out war and collapse of
civilization. Over uncounted thousands, maybe millions of years, these aliens
have accepted this vicious cycle as kismet, and deal with it by building
"warehouses" filled their most advanced technology to jump start civilization
after the next, inevitable, collapse.

I hope humanity will never have to deal with such a collapse, but I given our
collective tendency towards self-destructive behavior, maybe we should build
such an archive as if it was meant for future cavemen to jump-start them into
a new Anthropocene. Even if that collapse never happens (I am keeping my
fingers crossed!), the resulting tome of knowledge would be a suitable
monument to all the incredible things that humanity has accomplished, as well
as an insurance policy in case we manage to mess up on a monumental scale.

EDIT: A more positive perspective would be The Library from David Brin's
uplift saga, a humongous collection of knowledge acquired by many, _many_
species over millions, if not billions of years.

EDIT: typo

------
nanna
How does this work?

------
krautsourced
Look, feel and content-wise this feels a lot like the internet of the 90s...

~~~
pmoriarty
Unfortunately, it's not 90's enough, as it still requires fucking javascript.

~~~
theoctopus
This is one of the (admittedly few) cases where javascript is actually
required.

~~~
mitraardron
Of course it is, Javascript is what allows us to run significant code on the
browsers without extensions or plugins or downloaded apps/peers, and its
amazing how far its come as a language in the last few years. WebAssembly
would probably work as well, but was far less developed when we started this
project last year.

------
ryanlol
Unsupported mediatype: data

:(

~~~
mitraardron
Yes - we don't yet support some of the more unusual types on the Archive
(including "data" and "software")

------
drc0
on firefox it hangs past http.

~~~
mitraardron
Its still an experiment to see what was possible with a big site, in today's
browsers using some of the emerging tools (IPFS, WebTorrent, GUN). We only
test on absolutely current releases of Chrome & Firefox. If you run into
problems on the site, there is a feedback button, otherwise you can report
directly on
[https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2...](https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2wODcL3ro7D6LegPksb86jr5bCJa7Ig/viewform)
or open an issue on [https://github.com/internetarchive/dweb-
archive/issues](https://github.com/internetarchive/dweb-archive/issues)

------
agumonkey
very nice

