

Web crawler - kachen

what is the best refrence for studying basic we crawling?
======
wlrd
If you have an urge to work in python, I would strongly recommend Scrapy. You
could also use Beautiful Soup if you wanted. BS would be a little harder to
use but is very useful.

------
kachen
thank u bros, actually I am planning to make a livescore app. in android and
need online feeds for it.. using API seems limited and I don't know any thing
of web crawling...So suggest what should I do.

------
jafingi
You should read about screen scraping :-) That's the essence of web crawling.

~~~
kachen
thank u bro.. Mr jafingi do u have any good refrence of screen scraping

~~~
jafingi
Well, you'd have to understand the fundamentals of DOM:
[http://www.w3.org/TR/DOM-
Level-2-Core/introduction.html](http://www.w3.org/TR/DOM-
Level-2-Core/introduction.html)

Also learn about XPath, which is often used as a language to selecting nodes
in documents.

Scrapy and BeautifulSoup are popular libraries for scraping in Python.

