

Ask HN: textbooks, references for crawl+ingestion architecture - diN0bot

Hi, 
I'm writing a system that ultimately wants to grab lots of data and organize it into a database. I'm currently using a crawl to file, ingest file to database scheme, and I'm just wondering what is some good literature to read so I don't re-invent all the wheels (not just in the task at hand, but also in logging to evaluate performance, etc)<p>Thanks.
======
babyshake
You might want to flesh out your description a bit more, since it's somewhat
generic as is.

Maybe Stackoverflow.com would be a good venue for this type of question.

