

Querying Google/a search engine through my webapp? - schtog

Googling "query searchengine" and the like doesnt give a good answer so i try here.<p>Searchengines generally do a good job of sorting out relevant data/links but they rarely ever take you all the way to the goal.<p>So I have some different webapps I'm working on and I would like to query Google or some other searchengine for a subject and then go through those links and present the relevant information.<p>I could obv write something to do this, but is it allowed?
Or I have to pay for this service at Google?
If so, is there some automated way to do this?
======
simonw
Yahoo! have a great search API: <http://developer.yahoo.com/search/>

Google quietly re-introduced their search API a few months ago under the guise
of the "Ajax Search API that doesn't require JavaScript" - it requires you to
provide a referrer which is a bit weird, but it should work OK:
<http://code.google.com/apis/ajaxsearch/documentation/#fonje>

~~~
schtog
Went with Yahoo's, you are right, it is very easy to use.

Just get a Yahoo-ID and download the API. The following code searches for
Zlatan Ibrahimovic and prints the top 40 results:

    
    
      from yahoo.search.web import WebSearch
     
      srch = WebSearch(app_id="YahooDemo", query = "Zlatan  Ibrahimovic", results = 9)
    
      for res in srch.parse_results():
         print res.Url

~~~
schtog
sorry top 9 results obv.

------
bluelu
Google doesn't allow automated queries. An old soap api key might still work.
(At least it works for the former company I worked for, but I think they also
had a special deal with google)

------
FiReaNG3L
Once upon a time there was a Google API to do that, but its not available
anymore. And im pretty sure scraping their result pages is against the TOS and
will get you banned pretty fast.

------
aneesh
There is a perl module that you can use to search Google:
[http://search.cpan.org/~bstilwell/Net-
Google-1.0.1/lib/Net/G...](http://search.cpan.org/~bstilwell/Net-
Google-1.0.1/lib/Net/Google/Search.pm)

I'd imagine other languages have similar libraries.

------
kp212
I wrote a page scraper for google some time back. Its actually easier to parse
their pages than yahoo's imo. You will get blocked for hours if you intend to
post hundreds of times a minute though.

------
alexstaubo
Seconding simonw's Yahoo recommendation. The API is great and the results are
often as good as Google's. If you scrape Google searches, they _will_
eventually block you.

