
Show HN: Real Time Black Friday Dashboard – using Perl, Postgres, Geckoboard - netvarun
https://thanksgiving.semantics3.com/
======
kjstevo
I like the layout and information overview. However, I'm disappointed clicking
on items doesn't give me more information or send me to the relevant external
page. It looks good and runs well, I would just like a little more
interactivity.

------
Surio
Like the others, I too find the layout and info overview is neat. Good use-
case demo of real-time information scraping of the sale.

Also, like a few others, the "Perl" in your title is very interesting for me.
As a Perl old timer working with a team that hasn't dabbled in any Perl at
all, this would be a wonderful, wonderful hands-on demo of what Perl can and
does do.

Is there a detailed technical write up on your blog, on how you did this in
Perl? Is the code open-source, so I can show them with examples...etc?

------
clscott
Any more details about how it was built?

~~~
vinothgopi
Geckoboard part was relatively easy - they have a nice interface to design
beautiful looking dashboards.

Main magic was in the data pipeline. We tagged products that are Black Friday
specific as seen on the merchant specific "Black Friday" category pages and
other aggregators. These sites' pages were checked and the products' prices
updated more frequently than our regular pipeline.

Everything ran as part of the Semantics3 infrastructure but at the last step
we piped the updated information of the Black Friday specific products to a
separate Postgres DB as well and that allowed us to run fancy queries that
could answer almost any kind of questions we have about the data.

~~~
themckman
I'm interested in knowing more about the Perl component. What can you say
about that?

~~~
vinothgopi
Our entire data processing pipeline is actually written in perl (that is
several 10s of thousands of lines of code). So is the subcomponent that
actually pipes the data to Geckoboard.

