
Show HN: Semantics3 – API for Products and Prices - netvarun
https://www.semantics3.com/
======
coderdude
Here are two real problems you can tackle for online retailers:

Retailers can often get product data directly from the manufacturer. This is
true 100% of the time when the retailer is drop-shipping. The problem isn't
really obtaining this data but the canned data you get is pretty worthless.
Thousands of online retailers were hit hard by the Panda update (some time
ago) because they all used the exact same descriptions provided by the
manufacturers. So a lot of time was spent on custom descriptions and "romance
copy." If you can find a way to offer unique product descriptions or at the
very least somewhat unique then you could save these companies money.

Another problem online retailers face is categorization. Not on their own
sites but with mapping their items to the various taxonomies employed by
Amazon, Buy.com, Shop.com, PriceGrabber, and so on forever and ever. Find a
way to provide the mappings to each of these company's taxonomies and people
will pay you. The alternative is doing it all by hand, using a script to cover
most of the ground and doing the rest by hand, or by outsourcing it. If I had
a service at my finger tips that could have done all that for me when I needed
it, I'd have happily recommended it to the boss.

You should charge for these services, of course.

~~~
davemel37
Channel Advisor solves your second problem. The first problem can be solved
with some creative mashups of data, images and videos... plus all you really
need is 2-3 sentences to be considered unique content.

~~~
coderdude
Just checked out their marketplace material[1] and found this:

> Category-Specific Product Mapping Template: Create category-specific product
> templates that use common attributes and data manipulation technology to map
> your product information to the marketplace’s specifications.

Very cool! Thanks for the tip.

[1] [http://go.channeladvisor.com/rs/channeladvisor/images/us-
ds-...](http://go.channeladvisor.com/rs/channeladvisor/images/us-ds-
marketplaces-datasheet.pdf)

------
g_h
Having historical pricing data seems particularly interesting. I've thought
about building a fashion site that predicts pricing trends so you can predict
when to buy items on sale, and it seems like this could serve as the pricing
infrastructure.

I wonder how I can tell what product sources are available, and whether new
ones appear? I.e. is this only going to cover amazon.com, or does it know
about nordstrom.com too?

~~~
netvarun
That's a great idea. Pricing trends analysis could be done through the
historical data we provide across the different merchants.

We do provide the affiliate purchase links to the products, available under
the 'offers' field. Currently you can only figure out when new merchants
appear by polling the product ('pull' mechanism). One idea which we have been
toying around has been a 'push' mechanism where you can subscribe to a
particular product through our API and we would notify if there has been any
change (price change or new merchant selling it or product has been
discontinued). Drop me a mail at varun <at> semantics3.com if you want more
details or wish to bounce off ideas.

------
dfgonzalez
So cool, a few questions:

1\. The prices info, I assume is for US only, right? 2\. Are you only
analysing new products or also the used ones?

I created a similar (but very, very modest) proof of concept to track
mercadolibre's prices (<http://numok.com/products/view/samsung-t24a550/9>),
however it seems to be unusable without a human verifying each listing, as you
state in your blog:

> This isn’t the highest price that we’ve recorded for a product though. Turns
> out this Samsung TV was priced at $1,000,000,000,000.00 ($1 trillion) in
> early November last year. A dozen sales of this would have gone a long way
> towards offsetting the American national debt!

3\. Are you doing this validation in some way or unreal prices should be
expected by using your API?

As stated before, great pricing! Although I'm not sure how does the limit of
products work for the two initial account types (Up to 10,000).

Overall, I'm glad to have this API, thank you!

~~~
govind201
1\. Right now, we're focusing on the US. But we've made room for expanding
internationally (the "currency" and "geo" fields are in place with this in
mind).

2\. We're analyzing used and refurbished products as well. Each offer is
tagged with a "condition" field that conveys this.

3\. The question of whether a price of a product is right or wrong is, we
realized with time, subjective. Yes, $1,000,000,000,000.00 is very unlikely,
but where does one drawn the line? Hence, we don't mark something as bad data
and remove it from the database at the data layer. But we do handle this
problem at the search layer - we internally rank products based on factors
such as their (estimated) genuineness, popularity and so on. For the user,
what this means is that when you query the API, only the most relevant
products will be returned. The ranking system is constantly learning, so the
vision is that it'll get better with time and data.

Thanks for your words about the pricing. Each API query returns upto 10
products; the free plan provides 1000 API queries a day. So you could retrieve
upto 10000 products each day. Hope that clarifies. Glad you find the API
useful - I'd love to know more about how you plan to use it!

------
blart
I have done some scraping of the amazon.com previously, but they are pretty
good at detecting bot's and shutting them down, how did you get around this
problem when scraping millions of pages?

~~~
netvarun
Some great advice here on crawling at scale, which has inspired our crawlers a
lot : <http://news.ycombinator.com/item?id=4367933>

Basically it boils down to three things: 1\. If the site is slow,crawl
slooowly. 2\. If you see non-200 http error codes, stop! 3\. Obey robots.txt
and speed restrictions.

------
johncoogan
This is very cool, tons of potential for both fun and meaningful use here, but
I do have a bone to pick.

The variety of sources seems extremely minimal and the majority of the data
appears to be coming from Amazon.

When I sampled the data (pulling products from each topic), less than 1% of
the products had data from a source other than Amazon.

Even so, very fun and interesting service. Look forward to playing around
more.

------
MrAlmostWrong
"You can use either your PayPal account or your credit card to make your
payment. All payments are handled through PayPal and are completely secure."

Very curious to know why you would choose PayPal as your payment processor
considering the alternatives out there.

~~~
netvarun
We were originally based out of Singapore and Paypal was the best option we
had. If we had the option of Stripe then, we certainly would have gone with
them :)

~~~
MrAlmostWrong
Makes sense. Thanks for the reply.

------
es3754
Looks great, nice work! Signed up for a free account to try it out. The
pricing on the Large Booster Pack (150K calls for $159) doesn't seem correct
given the pricing/value on the Small and Medium packs...

~~~
govind201
Thanks for pointing that out. We're fixing it now!

------
jliptzin
How does this work? Is there a database of the most popular online stores and
you're constantly scraping (or GETing) product data?

~~~
netvarun
We aggregate data from a variety of sources (crawling, data dumps, rss feeds,
and in some cases even manual curation) after which we integrate them into our
data pipeline. We update them using a power law distribution, where the top 1%
of best selling products (based on our internal ranking system) is updated
hourly, the next 3% updated every two hours, etc.. The whole index is
refreshed at the end of each month.

~~~
jliptzin
Very cool. Thanks for the explanation.

------
duqee
I really like this product and would like to use it in the UK can pricing be
set to GBP? Or is it just USD pricing?

~~~
netvarun
As of now we are only indexing US prices - but we plan to expand out to the UK
and Germany next. Drop me a note at varun <at> semantics3.com, I would love to
chat with you!

------
thejosh
Love the free plan, and the cheap pricing option for people who want to run
this on a small scale.

~~~
govind201
The guideline behind the free plan was to provide enough calls and
functionality for any developer to build, launch and maintain a moderately
sized app. Shout-out aghi from Mashape for helping us with the pricing.

------
seivan
Looks like a well built product. Nice to see this from Singapore!

------
hayksaakian
how does this do books?

If I provide a random book barcode, it's usually an ISBN (10 or 13 digits)

~~~
govind201
We don't have books yet. That's been a common request, so we'll be launching
with books by the end of the month. If you'd like, I can notify you as soon as
we do.

~~~
chromedude
Books would be great - could you notify me too? srmorrisonjit at google's
email

~~~
SachinWordsmith
Hi,

Please check out the Book Prices API from DataWeave here:
[http://www.dataweave.in/apis/dataset-Book-Price-Search-By-
IS...](http://www.dataweave.in/apis/dataset-Book-Price-Search-By-ISBN--
19.html) (Full Disclosure: I am an employee of DataWeave). Currently, we are
serving data from Indian eCommerce stores, but expansion to other geographies
is in the pipeline. You can search by ISBN as well as many other fields.

------
rorrr
Needs more examples with charts to show off the historical data, patterns,
etc.

~~~
govind201
Noted. I'm curious about the motivating factor though - would you like the
graphs to validate the depth of the data, get you excited with potential
possibilities? Or something else?

