
Microsoft Will Soon Start Charging For Its Bing Search API - motti_s
http://techcrunch.com/2012/04/12/microsoft-will-soon-start-charging-for-its-bing-search-api/
======
dewitt
Having been through similar decision making processes myself (with the Google
Custom Search API, the Google Translate API, etc), this is just as likely an
abuse mitigation technique as it is a revenue generation opportunity.

Requiring even a modest at-cost fee for a web service does wonders to
discourage all sorts of misuse, from wanton large-scale data mining, to
blatant repacking and resale, to worse. (Heck, simply requiring a valid credit
card alone helps.)

And sadly no, simply having low quotas for free access doesn't entirely
suffice. If there's material value to be extracted from a free service, you'd
be amazed at the lengths people will go through to create large numbers of
low-volume scrapers. Most of these are obvious and easy to detect and defeat,
but continually doing so adds up in cost, and it takes engineers away from
providing better services to legitimate customers.

In short, most people on the outside don't appreciate just how difficult the
handful of bad guys make it for companies to do something good for the other
99%. So I'm sympathetic to Microsoft here, I really am.

~~~
Aaronontheweb
This. The best way to fight these types spammers and scrapers is through
economics - provide the content at-cost and it no longer becomes cost-
effective for pray-and-spray models.

Imagine what the world would be like if there was a cost-per-unit to sending
email - our inboxes would be a much saner, friendlier place. Fewer marketing
emails, virtually no spam (no longer economical).

------
zaidf
This got me thinking...is it possible for Microsoft to _secretly_ buy out DDG
and leave them the fuck alone except where they need help(ie. access to Bing
index)?

For all the money the Bing unit keeps pouring, I feel buying DDG and leaving
them the heck alone can be a reasonable long-term bet with relatively little
risk.

The challenge would be to keep the founders/team motivated. They could spin it
off as a completely different company on IPO track and give significant
equity. But then what if Google wants to buy them out? And of course, the Bing
team may have a problem with MSFT creating internal competition though that
may just push em to do better.

More likely if DuckDuckGo gets in an acquisition bidding war, I put my money
on Gabe passing acquisition for raising a huge funding round that lets him
take some money off the table.

With all the flaws(enough that I don't use it), it remains a rare search
engine start-up that has its heart in the right place: to actually serve
consumers versus build some technology or team and get acquired(looking at
you, Powerset).

~~~
guimarin
I think you're getting ahead of yourself here. DDG has 0.1% of all search
traffic ( 30m out of 23b queries per month ). They get all of their
'relevancy' from bing, and blekko. The reason you like them is because they
clean up these results, provide peripheral add-ons like user privacy, no ads,
easier syntax for power users, and one-boxes which function more as a
knowledge-base than a search engine. They don't crawl/index the web. or if
they do, we haven't heard anything about it, or seen any different relevancy
rankings from the BOSS api. So they are really a new face to already extant
search engines.

You mention a 'bidding war'. Who would buy them? Any 'features' they provide
can be copied by msft/google if they feel threatened. They don't have their
own backend search technology so partners who might want to do search with
them, ask.com, search.com, etc. have no reason to work with them as opposed to
the BOSS api themselves or even Bing. I'm just as excited as you are about
innovation in search and competitors to google, but DDG needs to be re-
architected on the back-end before these sorts of pronouncements make sense.

~~~
AznHisoka
yep, i completely agree. DDG is built off Bing's platform, so they're screwed
if MSFT ever made their index private. They also don't got a strong brand
outside of hackers who've heard of them.

I must admit they do a good job in the PR area of making themselves seem big
and sexy. But they're really not innovating search, or providing any thick
value in improving search. At least Blekko was trying to create a better
algorithm.

~~~
timr
_"At least Blekko was trying to create a better algorithm."_

Why the past-tense? Blekko is still alive and kicking. (And I hope they find
success, because they're doing amazing technical work. From a purely technical
perspective, they deserve ten times the press that DDG is getting.)

~~~
rehack
I like your defense of Blekko. I am often puzzled why blekko is talked off so
little, compared to DDG, on Hacker News. Nothing against DDG, but blekko is
the only new entrant trying to fight Google heads on. As it does its own
crawling, and Skrenta believed that search can be improved when he started
(apparent from his blog posts).

But I think, the mistake Blekko did perhaps was being too close to a Google
kind of (i.e. traditional) search engine. On this path, it may take them
atleast another 3 years, before people start taking them seriously. It will be
a hard and grueling road.

But the thing, I like about them, is that they did not make the mistake of
cuil, and are being conservative in making promises.

Overall, I suspect, they may be feeling a bit out of sorts, as when they
started out (2007) the social network thing was still in infancy. And now
people are talking about facebook coming into search and so on, which if it
happens, may be a totally different approach to search, than perhaps what
blekko did, which was trying to emulate and out-do Google.

------
Aloisius
Time for people to start supporting something like Common Crawl
<http://commoncrawl.org/> to build their own search engines.

~~~
greglindahl
blekko has been running a crawl+index of several billion pages for 2 years
now, so perhaps I can talk about this a little.

If you want access to a big crawl to grep through it for interesting data,
then Common Crawl is awesome and inexpensive and I don't think you can get
anything like it for the price, unless your query is simple enough to run as a
blekko webgrep (<https://blekko.com/webgrep>).

If you want to build a search engine, Common Crawl isn't so useful. Search
engines want _directed_ crawling of the pages that they think are good.
Crawling is only a small fraction of the total work done in a search engine.
Search engines generally aren't on AWS, because the right configuration of
machine isn't rented by Amazon -- serving queries needs SSDs or more ram and
less cpu than what Amazon offers. So, what Common Crawl offers a search engine
is higher costs and mostly bad data.

~~~
Aloisius
I believe Common Crawl will do a directed crawl if you contact them.

------
rlpb
Are there any implications for DDG here?

~~~
notatoad
i believe that DDG accesses the Bing indices through the Yahoo BOSS API, and
yahoo already pays microsoft.

~~~
eli
That does not seem like a stable long-term arrangement, but what do I know.

------
cannuk
This seems like an odd move. It's not like bing has any traction with
developers at all. Wouldn't charging them make it even harder to gain
traction? I am not familiar with thei API, what does it have over google that
would make me pay for it?

~~~
sycr
Google doesn't have a search API (it was long ago deprecated
<https://developers.google.com/web-search/>). So the API itself is the
advantage over Google in this case.

~~~
buu700
Did you read the link you provided? They actually mentioned right at the top
that they have a newer search API which is recommended (though I'll admit that
I missed it too at first).

<https://developers.google.com/custom-search/v1/overview>

~~~
sycr
Yes, I did :). As mentioned below, it's not a comparable product. Custom
Search is site-specific. It's an API for the "search this page using Google"
forms you sometimes see on blogs and the like. It's not an API for "capital S"
Google Search, while Bing's API really is their Search API.

~~~
buu700
Ahh, gotcha. I thought Custom Search was that old thing they had which let you
create a "customised" Google Search which had your colour scheme of choice and
optional restriction of the results by topic (i.e. basically standard search),
not the equivalent of 'site:x'.

------
caryme
In my experience with the Bing API in the last several months, I've found that
you get what you pay for. Its performance has been inconsistent at best, to
the point which I created the site <http://isthebingapiworking.heroku.com/>.
The web search api frequently _orders of magnitude_ fewer results than the
actual website, a problem making a frequent theme in their developer forums
<http://www.bing.com/community/developer/f/12254.aspx>.

By charging for their search API, I would just say that Microsoft is beginning
to take their API seriously. It seems pretty clear that minimal resources, if
any, were dedicated to the free version.

I'm bummed, because I've found relative success using their news search api
(particularly for the article aggregation component of
<http://www.congressionalprimaries.org/>), and now we'll have to look into
alternatives, but if this means actually providing a decent product, I think
this is a good move for Microsoft.

------
guimarin
No surprise there. I wonder if all the faux search engines will have to start
either crawling/indexing, or transition to Knowledge Engines ( I'm looking at
you DDG ). Curious to see if this sparks people to start more search
companies.

------
guard-of-terra
This is important because some leading local search engines (seznam, naver
(?), baidu) use Bing for their global index.

------
ww520
Actually the search API would be interesting for domain specific search. You
can use the API to create a site to present result specific to MP3, for
example, formatting the result with the MP3 attributes.

------
RyanMcGreal
Probably not a bad idea on balance. They're not going to capture market share
in any case, so this may be the only way they can recover some revenue.

------
harryf
This could be a good thing. There's a niche for web search as a service plus
paying customers might help steer Microsoft to better things with Bing

------
zmonkeyz
Maybe Megadeath can make a "Bing sells, but who's buying?" album.

------
denzil_correa
Yeah! I will use it.

------
loverobots
Why Microsoft? Do you really need that $2 per 1000 or whatever when you could
gain adoption?

