Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Building a web crawler from scratch in Python (dev.to)
3 points by max0563 on July 28, 2019 | hide | past | favorite | 2 comments


I found this mega helpful in terms of a building walk through. Can you point me to any other python walk through (not the tutorials, like ATBS) that build core things like this method-by-method and explain each method in plain language.


Is it just me or is the service being advertised here(proxyorbit) seem less than ethical or legal? The way it is described and the way terms are worded, it sounds like they are selling access to open proxies that they have found without permission or have even asked permission to use them.

If true, integrating the proxy service into your product like they suggest just seems like a terrible idea for a number of reasons.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: