

Show HN: A search engine based on social signals - jwblackwell
http://buzzsumo.com/
We built a search engine that shows you the most engaging stories&#x2F;topics being shared across Twitter, Facebook, Linkedin, and Google+. We crawled over 15 million articles the past 3 months, retrieved the total number of Facebook likes, tweets, Google+’s etc and built a search index around it.<p>Here&#x27;s what our infrastructure looks like:<p>Rails&#x2F;Redis: We use the Sidekiq gem as a message queue. We have hundreds of workers that do the crawling, data mining, and number crunching.<p>ElasticSearch: We built the search index using ElasticSearch, with the data imported from our Postgres database.<p>AngularJS: Front-end is built using AngularJS and Laravel 4 (PHP), which sends the requests to our Rails server in the backend.
======
Kiro
Since OP is dead:

jwblackwell 48 minutes ago | link [dead]

We built a search engine that shows you the most engaging stories/topics being
shared across Twitter, Facebook, Linkedin, and Google+. We crawled over 15
million articles the past 3 months, retrieved the total number of Facebook
likes, tweets, Google+’s etc and built a search index around it. Here's what
our infrastructure looks like: Rails/Redis: We use the Sidekiq gem as a
message queue. We have hundreds of workers that do the crawling, data mining,
and number crunching. ElasticSearch: We built the search index using
ElasticSearch, with the data imported from our Postgres database. AngularJS:
Front-end is built using AngularJS and Laravel 4 (PHP), which sends the
requests to our Rails server in the backend.

~~~
jackmaney
I don't get it....why would it be dead? Did the OP delete that post, or was it
just heavily downvoted? And if the latter, why?!

~~~
jwblackwell
It's way down at the bottom! Obviously no one upvoted :(

[https://news.ycombinator.com/item?id=6437715](https://news.ycombinator.com/item?id=6437715)

~~~
Kiro
No, it's killed so no-one can upvote or reply. Not sure why though but maybe
you were flagged as a spammer.

------
kixxauth
Nice.

The speed is really impressive, at least for the queries I tried. There are
still a lot of queries with no results, and you might want to think of
something clever to put there in those cases to keep the user turned onto your
brand. When there are results, they seem quite relevant and useful.

The "Show me how it works" tour is great too.

I think you just need to work on your value proposition on the landing page.
Don't be afraid to sell something. Put together few paid packages as soon as
possible. Don't worry if it's not perfect.

~~~
toumhi
Yup. Agreed with everything.

Tried "customer success" for example and the results were not valuable. Tried
"startups" and got lots of results, but that might be too broad of a topic.

Technically I'd say this is pretty good, I agree that OP needs to work on
value proposition and to sell: that's something for content marketers, but try
to dive more into the problem you're solving. Be specific. "Social insights"
seems too broad to me.

For example, the problem could be not knowing which content to write, wanting
to keep tabs on what your competiton writes about, spending too much time
researching all this...

You're already writing about benefits("discover what people are sharing..."),
I'd try to connect that with how your app is better than current existing
methods used by content marketers to create their content strategy.

Congrats with the launch !

------
jwblackwell
We built a search engine that shows you the most engaging stories/topics being
shared across Twitter, Facebook, Linkedin, and Google+. We crawled over 15
million articles the past 3 months, retrieved the total number of Facebook
likes, tweets, Google+’s etc and built a search index around it.

Here's what our infrastructure looks like:

Rails/Redis: We use the Sidekiq gem as a message queue. We have hundreds of
workers that do the crawling, data mining, and number crunching.

ElasticSearch: We built the search index using ElasticSearch, with the data
imported from our Postgres database.

AngularJS: Front-end is built using AngularJS and Laravel 4 (PHP), which sends
the requests to our Rails server in the backend.

------
AznHisoka
Possible feature to add:

1) Ability to search for influencers that shared an article related to a
specific topic. For instance, I want a way to search for SEO influencers who
shared something related to Breaking Bad b/c I'm making an article about how
SEO is related to Breaking Bad, etc.

~~~
jwblackwell
Great idea! We're currently working on adding more advanced search
operators/an advanced search option so you can do stuff like this.

------
colmtuite
Seems like a pretty smart idea. Good work. I'm impressed by the speed. I like
the tour too. I think the UI is very text-heavy right now, it might benefit
from swapping some text for icons.

I'd like to see more of a pitch by way of how this app can help me and why I
should be using it. Other than that, good job. Hope it does well.

------
Kiro
I really like this!

A small question regarding the tech stack: how is the relationship AngularJS-
Laravel-Rails working? What are you using Laravel for?

~~~
jwblackwell
We're two developers, to get to this point Henley built a crawler and search
index around the data. This is then available via a private API. I (James)
built the web app using Angular and Laravel.

Laravel is currently just used for user auth and as a proxy to the api. Nearly
all the front end logic is handled by AngularJS (it's a single page app).

------
bapi
Really good to know in few minutes what to know about a topic from social
media perspective. I will not miss things of these social media I don't read
but I have to know what they are talking about, and you search engine
aggreator is good for this sum up.

------
vdaniuk
Yeah, quite interesting project. As someone who is interested in marketing and
social media, I find value in the results of buzzsumo. I guess thats because
posts on these topics tend to get a lot of shares and there are many signals
to feed the algorithm.

------
skram
This is really interesting but I worry about the share of the net you all have
crawled/are actively gathering stats on. Perhaps you need a way to request the
crawling of a site or keyword?

------
abemassry
This is awesome, we made a realtime news like this
[http://mashrd.com](http://mashrd.com) would love to work together.

------
sixQuarks
Great execution. My searches were actually very useful.

------
jackmaney
Very interesting! I got a timeout when trying to find sharers, but I suspect
that's due to the fact that you've hit the front page of HN. ;)

------
hnriot
Google Chrome 30.0.1599.47 (Official Build 223717) beta OS Mac OS X

All I get are angular javascript exceptions, no results, blank white page.

~~~
jwblackwell
Please do a hard refresh (ctrl-r) and let me know if this doesn't fix it
(email in profile and on site). I had to push some bug fixes recently and you
might have some old cache files causing problems.

------
parth21shah
Awesome tool and the couple of queries I made has pretty much accurate results

------
dougk7
This is nice. Feature request: Add a safe-search setting.

------
patdennis
This is extremely useful, I'm very impressed.

~~~
jwblackwell
Thanks! Don't forget to sign up to the beta if you haven't already. It's free
and you can use the app unrestricted!

------
dancecodes
i think its will explosive trend to internet

success to you

------
dancecodes
looks nice

my test about gentoo - passed!

~~~
dancecodes
suggestion:

please add your engine to [http://mycroftproject.com/search-
engines.html?name=buzzsumo](http://mycroftproject.com/search-
engines.html?name=buzzsumo)

