Show HN: SiteOne Crawler – in-depth website analyzer and exporter (open-source)

[janreges]

https://github.com/janreges/siteone-crawler

Please try out this website analyzer and provide feedback on how to improve the output report so that you love the tool. In the first few sentences of the README you will also find a link to the GUI application if you don't want the CLI. To give you an idea of the report, I am sending a sample HTML report for Apple.com - https://crawler.siteone.io/html/2024-08-19/forever/v-bnb7tu5...

[ablation]

This is actually pretty neat, so thank you. I think the output reports could perhaps benefit from some nicer formatting - font is a little hard on the eye? But it's fast, detailed, and I could see this being useful to a wide variety of people.

[janreges → ablation]

Thank you for your feedback! Our goal is to help improve the quality of websites around the world, so we will continuously implement improvements based on feedback from real users.

I adjusted the font size for a long time, trying to choose one that would be a compromise in readability and not breaking the content on different screens and typical resolutions. In addition, the report is optimized to work well even in a mobile e-mail browser. See Mailer options - https://crawler.siteone.io/configuration/command-line-option....

That is also why, for example, switching the left menu and light/dark mode is implemented purely using CSS, without JS. The mail client in iOS, for example, supports CSS when opening an HTML attachment, but does not execute JS at all.

If you have other ideas for improvement, feel free to submit a GitHub issue with a feature request. Thank you ;)