
Building a web crawler from scratch in Python - max0563
https://dev.to/fprime/how-to-create-a-web-crawler-from-scratch-in-python-2p46
======
thwythwy
I found this mega helpful in terms of a building walk through. Can you point
me to any other python walk through (not the tutorials, like ATBS) that build
core things like this method-by-method and explain each method in plain
language.

------
rednixion
Is it just me or is the service being advertised here(proxyorbit) seem less
than ethical or legal? The way it is described and the way terms are worded,
it sounds like they are selling access to open proxies that they have found
without permission or have even asked permission to use them.

If true, integrating the proxy service into your product like they suggest
just seems like a terrible idea for a number of reasons.

