

Search Goes Real-Time With Scoopler (YC S08) - rokhayakebe
http://www.techcrunch.com/2009/05/08/search-goes-real-time-with-scoopler-twitter-dominates-results/

======
pclark
Cool. But until there are _a lot_ of real time services their value is very
limited.

~~~
dilanj
Going by the current trend, there will only be more real time services in the
future. It pays to be early in the game :)

~~~
axod
Yeah. Soon I can imagine a time when we could talk to each other online in
_real_ time. Like twitter but without a delay. But maybe that's a couple of
years off.

~~~
pclark
have you seen mibbit.com?

~~~
sachinag
Or some sort of instantaneous messaging or how about a telephonic device that
transmits voice over copper lines (or even wirelessly) or perhaps meeting
someone for a malted beverage at a public house of some sort.

~~~
axod
>> "how about a telephonic device that transmits voice over copper lines"

In real time? You're crazy.

~~~
jimbokun
This is a fun game to play, actually, any time a new technology is introduced.

Imagine the new technology was invented first, and the old technology is
actually the new challenger. What are the advantages and benefits of the old
technology you can promote over the new one?

~~~
jsrn
this is a tradition in the Emacs community: for each release, they publish
"antinews", e.g.:

    
    
        > For those users who live backwards in time,
        > here is information about downgrading to 
        > Emacs version 21.4. We hope you will enjoy
        > the greater simplicity that results from the
        > absence of many Emacs 22.1 features.
        > [...]

[http://www.gnu.org/software/emacs/manual/html_node/emacs/Ant...](http://www.gnu.org/software/emacs/manual/html_node/emacs/Antinews.html)

------
vaksel
isn't Google pretty close to real time anyways? I know I tend to find things
on Google that I just posted 5-6 minutes ago.

~~~
ajasver
Yes Google is pretty fast for newly published static content. However, it's
difficult to use for events happening right now. From the feedback we've got,
people like real-time results because its engaging and effortless, a little
like watching TV.

------
hamai
" _One element that’s really nice, is the Peek feature, which allows you to
take a look at pages being linked to in an overlay on top of the results, so
you don’t have to visit the actual page._ "

Real-Time news for the writer: you actually visit the page. It's an iframe...

In the old days, there was something called _target attribute_. Neat when the
target is _not_ iframe friendly.

~~~
snprbob86
This feature is really curious to me... why would I want to peek when I have a
back button on my mouse?

------
ikayak
Good execution on the trending and the real power here is the deep search on
live content. Anybody who does search right has value.

~~~
ajasver
Thanks! We're working on adding more data sources to provide broader results.

------
anuraggoel
What is Scoopler built with?

~~~
ajasver
Scoopler is written in Python. We use Cloudant (YC s08) for storing our data.
I'm planning to write a blog post about this on soon on our Posterous.

~~~
kyro
Cool. I'll take a note of that using Snipd, upload it to Scribd, then post it
to Reddit, then tell my friends to upvote me using Heysan or maybe Chatterous.
It depends on whether I'll have enough time, since I'm always playing on
DraftMix.

Anyway, good luck with Scoopler. :P

~~~
ajasver
Tu Chez!

~~~
bdr
<http://en.wiktionary.org/wiki/touché>

------
okeumeni
I have one question, are they consuming other companies search API, let’s say
Twitter or are they crawling and aggregate their own data? Either way it’s a
great job!

~~~
ajasver
We actually crawl and index all the data ourselves. That's how we identify the
most popular content.

~~~
hachiya
Is this truly crawling the sites, or querying each site's search service?

e.g. for Twitter, are you crawling all the Twitter users' URLs (seems
difficult to do - how do you find new accounts' URLs?), or are you fetching
results from search.twitter.com ?

~~~
ajasver
We take the raw data streams (e.g. a data mining feed from Twitter) and
process them, ourselves.

------
BenS
This is really cool - congrats. I'd really like to see google-trend-like
features but on much shorter time scales. Even a weekly-zeitgeist tied to news
stories instead of having to wait for the end of year.

~~~
ajasver
For the most recent trends you can check out the hot topics. We also want to
do news updates regularly - follow @scoopler and you'll get updates soon!

------
datums
Nice start. I was expecting more from the twitter feed. It's an rss feed from
a query. Maybe a better way is to group similar real-time results ? v2

------
edawerd
awesome job dude! It would be awesome to have some sort of real-time alert
feature, too!

~~~
ajasver
It's in the pipeline! Thanks.

