
Duck Duck Go written in Perl - fogus
https://duckduckgo.com/faq.html
======
deno
How can he use Bing as a backend? I read their API's terms of service and it
forbides you from changing search results in any way.

“(c) modify, filter, obscure, or replace the text, images, or other content of
Bing results, including by changing the order in which Bing results appear
(but this limitation will not apply to Bing results of type "Web"),
intermixing Bing results with search results from other sources, or
intermixing with Bing results any other content so that the other content
appears to be part of Bing results;”

I was quite interested in using Bing API myself but that does seem too
restrictive. I hope you can prove me wrong!

~~~
gorm
> but this limitation will not apply to Bing results of type "Web")

Isn't this what he is using?

~~~
irq11
That parenthetical only applies to the clause about result ordering. if you
read the whole TOS, it's quite clear that Bing doesn't want you to do what DDG
appears to be doing. In particular, mixing results from different services is
a no-no in most search APIs.

------
draegtun
And here are the HN comments on that Duck Duck Go architecture post:
<http://news.ycombinator.com/item?id=525048>

------
jackowayed
The perl code isn't a full search engine though. He uses Yahoo! Build Your Own
Search Service, so that perl code is just taking the results from there and
throwing results out, reordering results, getting the zero-click info, etc.
Yahoo does all the crawling.

~~~
sp332
Searching is not the same as indexing. DDG will give different results from
Bing search, even though they use the same index.

Edit: DDG does some crawling in addition to using the Bing index.

~~~
Keyframe
I wonder if DDG could benefit from <http://www.80legs.com/> ?

------
gojomo
From some early discussion (and their current job listing), search engine
startup Blekko also has a significant Perl component:

<http://blekko.com/jobs.html>

~~~
draegtun
Probably the biggest profile Perl (and Catalyst) public app in last few years
would be the BBC iPlayer:
[http://www.bbc.co.uk/blogs/bbcinternet/2008/12/iplayer_day_p...](http://www.bbc.co.uk/blogs/bbcinternet/2008/12/iplayer_day_performance_tricks.html)

~~~
dragonquest
Actually, YouPorn is powered by Perl and Catalyst and regularly makes the top
25 domains. Not sure if it beats the BBC main site though.

~~~
draegtun
Yep! <http://news.ycombinator.com/item?id=1501063> :)

------
zaphar
That's cool. I've used DDG a few times in the past and generally like so far.
I had no idea it was done in perl though. It's nice to see new perl apps
getting some press.

I wonder if they use any of the newer CPAN libs like Moose and family in the
architecture?

~~~
sdesol
As for Moose, he (DDG author) answered this here

[http://www.reddit.com/r/IAmA/comments/bbqw7/i_am_the_founder...](http://www.reddit.com/r/IAmA/comments/bbqw7/i_am_the_founder_of_a_search_engine_duck_duck_go/)

Having written an indexer in Perl for my current startup product, I really
can't see the need for using something like Moose. I'm biased though, as I'm
not a big fan of OO programming. I gave it a try in Perl years ago but it
really sucked. OO programming that is, but I guess Moose was designed to fix
this, but it's a little too late for me. I was taught C in school and I've
learned to live without OO.

~~~
zaphar
I can't say I disagree with him on the dislike for OO. However if you are
going to do OO in perl Moose is the way to do it.

~~~
mhd
If you agree with the design decisions. It's rather expressive, but not
exactly a minimalistic approach to OO.

Did they ever improve the rather steep performance penalty? (esp. startup)

~~~
jrockway
Runtime is fast, startup is not as fast.

I am not sure why anyone cares about startup time. I start my apps about once
a month. 1 second instead of .1 second doesn't really matter. It's like
saying, "C++ has an unacceptable performance penalty" because you have to
compile your code before you use it. Yeah, you do. You can pay a million times
at runtime or you can pay once at compile time.

For desktop apps, start them when you log in and connect to them via
App::Persistent.

~~~
__david__
It's certainly worth thinking about if the app is a command line app. No one
wants "ls" to have a 5 second startup time.

~~~
jrockway
I agree. Please contribute your fix to Moose.

~~~
natep
I believe he was responding to your "I am not sure why anyone cares about
startup time." While you may have been referring specifically to Moose, it's
possible to interpret that as a blanket statement for all programs and
languages, even in context.

------
sandee
Thanks. Good Post. This is great example that good application and complex
systems can be written in perl as well.

On a side note, i saw the previous post on DDG, where discussion was about how
DDG does not store any private information. It makes sense in the context,
that since he leverages upon other search engines , his cost of running is
low. So he can afford to ignore user information otherwise used for commercial
purpose by others.

~~~
loewenskind
>This is great example that good application and complex systems can be
written in perl as well.

You could write "good application and complex systems" in machine language if
you work hard enough. What are you trying to say?

------
mkramlich
i had pictured something elegant like Lisp. but no it was just hacked together
in Perl? Somebody needs to link to the famous XKCD strip. :)

~~~
piaresquared
You mean this one? :)

<http://xkcd.com/224/>

~~~
alexyim
Interestingly, a search for "ostensibly xkcd" shows the comic in the first
link:

[https://duckduckgo.com/?q=ostensibly+xkcd&v=](https://duckduckgo.com/?q=ostensibly+xkcd&v=)

