On: ArchiveTeam has finished archiving all goo.gl short links

[do_not_redeem]

Does "all" mean all the URLs publicly known, or did they exhaustively iterate the entire URL namespace?

[jedberg → do_not_redeem]

They iterated the entire URL namespace by having volunteers run a client so they didn't get IP banned.

[Imustaskforhelp → jedberg]

are we sure that the whole entire URL namespace has been mapped?

How would that even function, I mean, did they loop through every single permutation and see the result, or what exactly/ how would that work?

[jedberg → Imustaskforhelp]

> did they loop through every single permutation and see the result, or what exactly/ how would that work?

In short, yes. Since no one can make new links, it's a pre-defined space to search. They just requested every possible key, and recorded the answer, and then uploaded it to a shared database.

[toomuchtodo → Imustaskforhelp]

The pipeline code is available for review of the mechanics of http requests made if you follow the ArchiveTeam wiki links.

[barbazoo → jedberg]

Beautiful. I wish I had seen this and could have helped.

[brokensegue → barbazoo]

they are still archiving other url shorteners https://tracker.archiveteam.org:1338/ you can participate in that

[ccgreg → do_not_redeem]

The goo.gl URLs that are publicly known are already in the Internet Archive and Common Crawl crawls.