- I didn't want to download just a few images and reuse them everywhere. That makes the design feel a lot less "real" and a lot less interesting. I'm also to lazy to download lots of images manually.
- Some of the images were set with background-image in CSS, what makes it more tedious to find and actually download them.
Yes, all the images used on a site are visible in the browser dev-tools. But you don't really have a preview or an easy way of finding the image you want. And you still need to download them all individually.
Hopefully this is useful for someone else out there.
For those unaware, Firefox has a built-in way to do this which will work even on sites where you need to be logged in.
If you right click on the page and choose "Page Info", there's a Media tab. It can be sorted by the Size column. It includes video and audio files as well.
Wow, didn't know about this feature in firefox. I was thinking about adding support for audio and video as well, but I'm not sure if there is really a need for that.
(using firefox, did disable ublock origin, still no download) -
Love getting the visual of the 75 pics with the basic details on size! Would love some extra info with a click perhaps showing the url it came from. (currently wondering if the 75 pics at 210x158 are just the thumbnails and I actually need a scraper to traverse further to get the bigger size for the next web site..
I've been considering some of the scraping tools out there.. but not sure I want to spend the time learning something only to have it not work the way I need.. see this particular site has a weird (hosting?) issue - where pics will 404 on some albums.. then if you go back and reload the page manually some minutes later - they appear - then you can click to download..
So I really need something to show me urls of pics that were in the html code but 404 - so I can go back and get them.
I will look into the download issue. But i think it might be caused by the actual size you want to extract the images from. Currently the whole download may fail if one image fails to download.
Adding an option to show the actual source of the image, might be a good idea. I will look into this.
About the 404 issue: Currently every image that fails to load in the browser or is smaller than one pixel automatically gets removed. I will add an option later to disable this behaviour.
One tiny UX thing to consider, instead of validating the string is a url, just try to load the site and if it fails, error then. I typed in a valid url without the "http://" part and the validation complained. You need to handle the "wait this isn't a valid site" case anyway, might as well kill two birds with one stone.
Nice catch. I just removed the url validation, so using urls without the http:// part will work now.
Frontend: Vue.js, Buefy and Tailwind.css
Scraper: Puppeteer with puppeteer-cluster to enable concurrency and using a proxy to prevent getting blocked