
Palantir acquires Kimono - davidbarker
https://www.kimonolabs.com/
======
drglitch
"Why?

As a team, we’re very proud of the product we built, now used by over 125k
developers, data scientists and businesses" ... from FAQ: "service will shut
down Feb 29, 2016"

In fact, they are so proud, they're giving all but ~2 weeks notice to said
125K developers, data scientists, and businesses to migrate off the platform?
That is an abysmally small amount of time and if I was a paying customer, i'd
be furious.

On the other hand, congrats to Kimono team!

~~~
kftaylor
They never even tried to charge me money. I would have paid ... happily

~~~
robertp
me too

~~~
LS1894
me three... I even offered

------
misterrobot
The worst part about Palantir is their ability to masquerade as this hip
startup, as though working for them is as innocent as working for Imgur or
Twitter or whatever. They recruit the hell out of MIT students, and it sucks
seeing my friends interview for this incredibly shitty company that has openly
said it won't go public since that would make running their business "very
difficult" (hm, because you're doing reprehensible work perhaps?).

I wish people picked their jobs to match their ethics instead of the other way
around. It makes me sad when I hear about my friends going to intern for this
place.

~~~
whorleater
Interning for Palantir is a pretty big resume boost for college students, and
they have one of the highest paying internships in the valley. There's some
serious benefits to going to Palantir.

~~~
misterrobot
If you have the ability to intern at Palantir you have the ability to intern
at any number of more responsible companies. Palantir has a bigger booth at
the career fair, but I'd bet working at Apple or Uber is just as much of a
resume boost at much less of a societal cost. (source: am a college student)

~~~
whorleater
Interning at Apple or Uber has much more stiff competition though. The barrier
to entry for Palantir is remarkably low considering their pay, it's not
surprising that college students apply for Palantir. (Source: am college
senior)

~~~
ZanyProgrammer
Will they hire you if you smoke pot? I hear college students sometimes do
that.

~~~
jsprogrammer
Thiel has an entire cannabis investment club.

Just rail on the hypocrisy if confronted.

------
kumarski
Relevant Scraping/Crawling Lead Mining Tools:

[http://grabby.io](http://grabby.io)

[http://fullcontact.com](http://fullcontact.com)

[http://emailhunter.co](http://emailhunter.co)

[http://clearbit.com](http://clearbit.com)

[http://toofr.com](http://toofr.com)

[http://import.io](http://import.io)

[http://kimonolabs.com](http://kimonolabs.com)

[http://apifier.com](http://apifier.com) (favs)

[http://elink.club](http://elink.club)

[http://www.eliteproxyswitcher.com/](http://www.eliteproxyswitcher.com/) \- ;)

~~~
ddebernardy
Three more:

[https://github.com/scrapinghub/portia/](https://github.com/scrapinghub/portia/)
(our open source equivalent of Kimonolabs)

[http://scrapy.org](http://scrapy.org) (allows to scrape more complex sites;
also OSS)

[http://scrapinghub.com](http://scrapinghub.com) (for a cloud-based platform
and a smart proxy rotator)

(Disclaimer: working there.)

~~~
kumarski
scrapinghub is pricy, isn't it?

~~~
ddebernardy
Pricey as in we've a free plan? :-)

More seriously though, it's basically a bargain. You get:

\- Access to the largest team of Scrapy, Portia, and Frontera experts around
at your fingertip.

\- AWS functionality for actually less than what it would cost you to set
things up yourself (think VBR IP over ATM).

\- Smart proxy rotating tech at your fingertip with tens of thousands of IP
addresses spread across a slew of different IP networks.

\- Built-in integrations with a growing list of third parties.

The list could go on and on... It's definitely not pricey. :-)

------
tsergiu
Full disclosure: I'm one of the co-founders of ParseHub.

For folks looking for an alternative, check out
[http://www.parsehub.com](http://www.parsehub.com)

ParseHub is especially good at dealing with dynamic websites and
multidimensional data (arbitrary relationships instead of just rows and
columns).

Our team would be happy to help you migrate from Kimono. We're bootstrapped
and profitable, so we don't have pressure to sell when an offer comes along.
We also pledged to release the back-end code under a liberal open source
license if we were ever in a position where we could no longer provide
service.

~~~
dburn
Parsehub is easily the best hosted craping service on the market. We
investigated everybody from kimono to import.io and if you want it done right
they are the only option. Furthermore the guarantee of an open source version
should they ever decide to close is extremely reassuring.

------
minimaxir
As someone who used Kimono a few times before the closure...this announcement
does not surprise me at all.

The target demographic was developers who were too lazy to write simple
scrapers for a given predefined use case. However, scraping most modern
websites is not simple, and requires a lot of code to ensure the correct HTML
elements are scraped and the data is transformed correctly...which defeats the
purpose of using Kimono.

Kimono joining Palantir was likely an acquihire, especially since the core
service was permissively free.

~~~
michaelmior
"too lazy" seems like an unfair characterization. If a service lets you do
what you want faster, why not use it? I'm not familiar with Kimono, but if you
can just rip it out and replace it with your own code later, why not?

~~~
x1024
It wasn't really faster. The OP actually touched on that point, but let me
repeat it without ambiguity:

An actual parser for the complex use-cases requires some custom code and isn't
covered by what Kimono does.

What Kimono does can trivially be achieved by writing 15 lines of python
(after importing httplib2 + BeautifulSoup). Which is even reusable.

The only real use-case they had was the visual interface, so that "non-
programmers can do it. No code required, etc. etc." Heh. As if that ever did,
or ever could work.

~~~
derefr
We, as programmers, are called in to solve the "complex use-cases." So we tend
to divide the world into "things that need a novel program written for them"
and "trivial problems."

You know what else has a visual interface? Excel. It works great! For "trivial
problems." It gets stretched all the time into places where programming would
have been faster, but _for_ the "trivial problems", just being able to have a
domain-expert non-programmer use Excel to implement the solution directly, is
a lot faster _and_ simpler than calling in a programmer and attempting to
fully specify the problem to them.

~~~
x1024
I'll be the first to agree that Excel "actually works" and "solves a real
problem". But if you had actually used Kimono, you'd know that it was one of
those tools that was just finicky enough to be unusable by "regular people".
And just weak enough to not be productive for programmers.

------
marvel_boy
Man, this is disappointing. No way to trust any cloud offering, more sooner
than later the service is discontinued.

~~~
davidbarker
I was just thinking the same. I rely on Kimono for a few of my sites, so I'm
looking around at what I could use instead. The first I thought of is
[http://import.io/](http://import.io/),
[http://cloudscrape.com](http://cloudscrape.com), or
[https://phantomjscloud.com/site/index.html](https://phantomjscloud.com/site/index.html),
but I fear the same might happen to whichever I pick.

Does anyone have any suggestions on how to run something similar on my own
server?

~~~
rurai
You can use portia[1]. You can try out a hosted version at scrapinghub[2] or
you can host your own. It's a web based spider creator like kimonolabs but the
spiders it creates are just scrapy spiders. If you need to do something that
the UI can't handle it's not a problem, you can extend it with a little
python.

[1]
[https://github.com/scrapinghub/portia](https://github.com/scrapinghub/portia)
[2] [https://scrapinghub.com](https://scrapinghub.com)

------
fapjacks
"access to the world’s most important data problems"

I'm not sure how to put this, but I've used Palantir's products in a couple
different warzones. I can't imagine they're not pointing those tools at
everyday Americans. In fact I'm certain of it (just read FedBizOpps postings
from the DHS and DoD). I'm sure most of the people working there have no idea,
but I'm also sure there are some truly bad people working for that company.
Man, we really need to regulate this a lot better.

~~~
trhway
> I can't imagine they're not pointing those tools at everyday Americans.

[http://www.dhs.gov/state-and-major-urban-area-fusion-
centers](http://www.dhs.gov/state-and-major-urban-area-fusion-centers)

Powered by Palantir, basically it is real Big Brother brain network where all
the information like lic. plate readers, video cam feeds, cell phone data,
Internet data and everything else is "fused" together. One of the immediate
resulting products of it is generation of Suspicious Activity Reports. Hoover
(and Beria) would die of envy.

------
nreece
Disappointing to see just 2 weeks notice for their existing users.

* _shameless plug_ * For longer than most such tools, we've been offering what our customers call the "easiest scraper of 'em all": [https://feedity.com](https://feedity.com) Try it out and see if it works for your needs.

------
danso
From the FAQ:

[https://www.kimonolabs.com/faq#faq-303](https://www.kimonolabs.com/faq#faq-303)

> _Will Palantir (or anyone else) have access to my personal data?_

 _No. All of your user data including names, email addresses, passwords and
APIs will not be shared with anyone and will be securely purged from our
servers on March 31, 2016._

Uh...but what about the scraped data? I don't think I've used Kimono since it
was first announced on HN as a side project...but I'm assuming that you got to
save scraped data semi-privately with Kimono? Is that considered "user data"?
Kind of a vague term, and one I usually associate with data of the personal
information variety. I'm sure someone at Palantir can find some interesting
meta-insights from analyzing the data collected by the kind of people (data
scientists, etc) who would use Kimono heavily.

------
p4wnc6
Palantir is one of those organizations that makes me throw up a little in the
back of my throat whenever I hear about them. They are sort of like the
Nickelback of companies.

~~~
orng
I'm no fan of Nickleback's but I don't think they deserve to be compared to
Palantir. If I was a Kimono dev I would certainly be leaving now.

~~~
smegel
Is it because what they do (helping governments snoop on social media) or are
they actually a bad place to work?

~~~
orng
For morality reasons I would never be comfortable working there. The workplace
could be heaven on earth and pay really well but there are simply things that
are more important.

~~~
p4wnc6
The company culture is quite shitty as well.

------
equivalentideas
You can host scrapers you need to migrate on morph.io (
[https://morph.io](https://morph.io) ) a free platform for hosting and running
scrapers, storing your data and providing access to it in a range of formats
(JSON, CSV, SQLite). It’s free, open source, supports a bunch of languages and
is run by an Australian non-profit, where I happen to work ;-).

Here's a tutorial all about running a scraper on morph.io
[https://www.openaustraliafoundation.org.au/2015/10/13/ruby-w...](https://www.openaustraliafoundation.org.au/2015/10/13/ruby-
web-scraping-tutorial-on-morph-io-part-1/)

------
sergiotapia
It's gross they're giving their customers 2 weeks to "GTFO".

------
svarrall
Disappointing. Was my goto service as it was a joy to use and always seemed to
work. Wish they would follow the Parse lead and release something open source.
Desktop doesn't really cut it for API use...

------
austinhutch
I loved Kimono! It had the best interface and most flexibility for this type
of service. A bittersweet congratulations to the team.

------
jpatokal
Evaluated Kimono for a tool I was building, decided to roll my own with Scrapy
instead. _So_ happy right now that I did...

------
foresight
I'm always disappointed when I read of Palantir acquihires.

Technically talented employees receive a pittance and are sold into the
Palantir machine, a morally unscrupulous high-tech consulting shop that takes
in talented engineers and churns out one-off software black boxes designed to
lock in customers and extract a long-term tax.

------
solveforall
I had a hard time using this product. I was trying to use another developer's
APIs but it didn't seem possible. The one positive thing about the
announcement, from the FAQ:

Will Palantir (or anyone else) have access to my personal data? No. All of
your user data including names, email addresses, passwords and APIs will not
be shared with anyone and will be securely purged from our servers on March
31, 2016.

[https://www.kimonolabs.com/faq](https://www.kimonolabs.com/faq)

------
tn13
Their "desktop tool" is complete rubbish. I have to manually click on each
source to crawl it and then manually click a few buttons to save the response.

Thanks for nothing guys!

------
tn13
Two months is a short time but I had sort of expected this given how bad their
service of performing last few days. That is why I had started moving to other
services.

~~~
tobltobs
Its only two weeks, the equivalent of a GFY to their ex-customers.

------
MeGer
ParseHub is much better at getting websites from shitty, complicated sites.
Kimono was great for something simple, but ParseHub is a more powerful and
flexible web scraper. [https://www.parsehub.com/](https://www.parsehub.com/)

------
gingerlime
Sad to see Kimono close down. I enjoyed using it for a small project, and was
waiting to see a paid version with private APIs, but it was always "coming
soon"...

I don't quite get why startups are not trying to generate revenue from day
one.

------
tn13
These people should be totally ashamed for the tool they have given. It is
just nodejs wrapped inside a desktop app. They could have very well made it
open sourced and offered to community.

------
rmac
Kimono had a beautiful chrome extension which allowed me to easily extract
things quickly without needing to write any selectors / scraping. Much love to
this team; best of luck!

------
jd2025
Anyone offering a GUI solution that can export to an RSS feed like Kimono did?
I know RSS is old but I have no clue how to display the other formats in
wordpress. Thanks!

------
nat
As usual with acquisitions, they have stripped any information about what
their product actually is (was?) from their site.

------
robertp
ohhhhh man that stinks. I love Kimono and have been using for 6+ months.
Hopefully it works out okay for the Kimono team.

There is not a download function on the desktop page, what is the direct
download link?
[https://www.kimonolabs.com/desktop](https://www.kimonolabs.com/desktop)

------
uberneo
few other options using NodeJs - [http://blog.webkid.io/nodejs-scraping-
libraries/](http://blog.webkid.io/nodejs-scraping-libraries/)

------
yueq
This is just an acquihire.

------
uberneo
Looks like we cannot schedule an API in the desktop version

------
oneloop
We can conclude from this that they weren't doing very well, right? I mean,
you don't accept an acquihire if you're doing great.

~~~
orliesaurus
Its a two edged sword, sometimes you're doing great - but the fact they're
shutting down means they're not doing great but have a team of experts in
parsing that will benefit Palantir since they're likely into analysing surface
and deep web data and just data in general :)

basically its like hiring a bunch of regex gods because you have regex
problems on large scale

------
jorgecurio
I wonder what it's going to look like for import.io as well, their business
model is identical to kimono it seems.

edit: why the downvotes seriously.

~~~
orliesaurus
Here's an upvote, because I was thinking exactly the same as someone who has
used most of these "scraping" tools.

------
tn13
To all those badmouthing Palantir. I think they are just a hogwash. They are
just pretending to be something that Fox Maulder would investigate I dont
think they are. They are just helping the government spend more money on
problems that are not worth solving and create new problems for citizens.

The place is likely to be very shitty to work for but by suggesting that they
might be doing something that is James Bond equivalent in software world might
make it sound too cool for some developers.

