I'm pretty fond of using this tool to take trips down memory lane, revisiting lost content I used to enjoy.
Browsing through crawls has this neat side-effect of being able to serendipitously discover things that I missed back in the day just by having everything laid out on the file system.
PSA: There's a lot of holes in most crawls, even for popular stuff. A good way to ensure that you can revisit content later is submitting links to the Wayback Machine with the "Save Page Now" [1] functionality. Some local archivers like ArchiveBox [2] let you automate this. Highly recommended to make a habit of it.
Another convenient way to interact with "Save Page Now" is just to email a bunch of links to the savepagenow address at archive.org. I especially like to copy all the HTML of a page and paste it into a HTML email to get all the links.
There are two things to note, neither of which are well-advertised:
1. The parent comment you're replying to links to the main page for the Wayback Machine, which includes a Save Page Now widget, but Save Page Now actually has a dedicated page <https://web.archive.org/save/>
2. If you have an archive.org account (lets you submit and comment on collections; the library is bigger than just the Wayback Machine) and you visit the Save Page Now page while logged in, you get more options, including the option "Save outlinks"
Yeah, I use that API from the browser, I found the bulk asynchronous zero-download email API more convenient, since for a while, the save API stopped supporting HEAD requests, although it seems to support it again now.
During first wave of the pandemic the leading online grocery company in my country started slot system which opened at random times and people had to spend sleepless nights trying to see if the slot opens.
I wrote a script to automate[1] this and used Pushover for notification. It was used my many and so I guess they might have gained several new customers.
Brenger | Amsterdam, The Netherlands | ONSITE (50/50% office/work from home) | VISA SPONSORSHIP
Come join Brenger to build the transportation platform that solves the surprisingly unsolved problem of shipping bulky goods economically, optimizes the utilization of vans on the road, and reduces CO₂ emissions along the way!
We've got two developer positions open at the moment:
Brenger | QA Engineer | Amsterdam, The Netherlands | ONSITE (100% remote during pandemic, 50% after) | VISA SPONSORSHIP
Come join Brenger as a QA Engineer to build the transportation platform that solves the surprisingly unsolved problem of shipping bulky goods economically, optimizes the utilization of vans on the road, and reduces CO₂ emissions along the way!
Brenger | QA Engineer | Amsterdam, The Netherlands | ONSITE (100% remote during pandemic, 50% after) | VISA SPONSORSHIP
Come join Brenger as a QA Engineer to build the transportation platform that solves the surprisingly unsolved problem of shipping bulky goods economically, optimizes the utilization of vans on the road, and reduces CO₂ emissions along the way!
Brenger | Backend Developer (Python) | Amsterdam, The Netherlands | ONSITE (100% remote during pandemic, 50% after) | VISA SPONSORSHIP
Come join Brenger as a backend developer to build the transportation platform that solves the surprisingly unsolved problem of shipping bulky goods economically, optimizes the utilization of vans on the road, and reduces CO₂ emissions along the way!
With Common Table Expressions you can process the shared data in one set ("temporary table" is a good mental model for it), compute the unique parts in subsequent sets and use the results for the UNION in a single query.
Browsing through crawls has this neat side-effect of being able to serendipitously discover things that I missed back in the day just by having everything laid out on the file system.
PSA: There's a lot of holes in most crawls, even for popular stuff. A good way to ensure that you can revisit content later is submitting links to the Wayback Machine with the "Save Page Now" [1] functionality. Some local archivers like ArchiveBox [2] let you automate this. Highly recommended to make a habit of it.
[1] https://web.archive.org/
[2] https://github.com/ArchiveBox/ArchiveBox