
NYPD is canceling its Palantir contract - tefo-mohapi
https://www.buzzfeed.com/williamalden/theres-a-fight-brewing-between-the-nypd-and-silicon-valley?utm_term=.upg46gXexp#.okrN9VDzJa
======
svendisnigh
Palantir has an outdated software stack (Java/Swing). Their genius lied in
creating "mythical image" and cleverly overselling it to the less proficient
government agencies (apparently NSA was smart enough to see through the ruse).
And then in turn sell the "spooky/mysterious" image to bunch of naive
Stanford/Berkeley undergrads and convince them to work for peanuts.

Sadly in 2017, their tech stack is falling apart, they have no meaningful
ML/AI strategy, the agencies/corporation cannot be fooled any longer, and
employees have suddenly realized that the stock (without an IPO in sight)
might be worthless and have begun leaving.

Finally the political environment along association with Thiel isn't helping
them make any new friends.

Having known several people who work or were hired at Palantir, I can assure
you it probably has the worst overconfident brogrammer culture. So much so
that I truly hope no US government agency is using them for critical functions
because its nothing more than hot air.

~~~
foolfoolz
I interviewed there a couple years ago. the guy forgot to call me for the
phone screen. I emailed the recruiter and they said sorry they can reschedule
for 7pm that night when he got home.

on the call (which started at 7:15) he talked 90% of the time. he would ask me
two or three questions at a time, and when I would get a chance to answer one
he would jump in and answer the rest of them for me.

I was rejected for lack of technical skills

~~~
linkregister
I interviewed there. It was extremely challenging, but somehow I passed it. I
had a very positive impression of the company and its culture. Recruiters and
interviewers were super-responsive.

It was in the early days, so it was a very startup-y attitude, "we're so
smart" sort of thing. The engineers _were_ smart, from Stanford and CMU, so it
was partially justified. I ended up not taking the offer. Not for a
particularly good reason, it just didn't fit with life circumstances.

Looking back, it would have been a bad idea. I didn't understand the product,
which is pretty essential when joining a company. The stock was granted as
options, so without an IPO it is illiquid. The salaries were all low, but at
the time someone could have lived in Palo Alto on them.

It turns out they thought they could tackle this problem of disparate data
silos and change the world while respecting privacy. They weren't able to
automate away the expensive process of having Forward Deployed Engineers do
data integration work.

~~~
ethbro
_> They weren't able to automate away the expensive process of having Forward
Deployed Engineers do data integration work._

Enterprise in a nutshell.

And honestly, not what the customers want. If they're hiring you in the first
place, they generally suck at tech. So they'd much rather have "Stanford grads
on site every day" as proof of how they're spending money.

So it's a non-starter for Palantir's management when they're looking at
existing customers and their desires, and the continuing trickle of naive tier
1 undergrads. Just keep the company cool enough to attract the crop under
Google, Amazon, & FB...

And if the customers don't want 100% software data integration, then why spend
a _lot_ of money to automate 85% of it?

------
bane
Conjecture? Palantir is known to be rabidly expensive. The available
information on the internet and from people I know who've encountered the
technology directly describe their "product" as a mix of a very outdated tech
stack and expensive integration services with expensive on-site support [1]

My guess is that the NYPD did an analysis of the life-cycle cost of the system
over another contract period, and probably determined they could do something
cheaper some other way. Since they've built another system based on a
competitor's technology (with some special home rolled glue sauce) they're
obviously getting _some_ benefit from the basic approach, but given a choice
of close enough tech and more badges or the status-quo they've decided to move
forward.

1 - I've basically heard their software described as a big graph backend with
a document store and an ancient Java client for mucking around with the graph.
Most of their tech has been subsumed by very free and highly scalable
technologies.

The most recent video shows a just normal run-of-the-mill web app any decent
team of 3 or 4 good web devs could crank out in a few weeks.

[https://www.youtube.com/watch?v=dFF4gN56Bhg](https://www.youtube.com/watch?v=dFF4gN56Bhg)

They actually don't have any video up of anything recent that actually really
shows their core product. The video here shows a java client
[https://www.youtube.com/watch?v=5OYy_UtINo4](https://www.youtube.com/watch?v=5OYy_UtINo4)
but it's from 2015.

Kind of a shame considering how much money and talent they've vacuumed out of
the market. If I'm right, it's amazing for IBM to actually come in at a lower
price than some competitor.

~~~
mc32
I have one question, will the new NYPD in-house developers be union or not?

It'd be interesting if they were union just to see how that works, for better
or for worse.

~~~
paulddraper
I wasn't aware of any unions with software engineers. Are you?

~~~
mc32
I also am not aware, but given this is the NYPD, isn't there an opportunity
for the developers to become unionized?

And I wonder, the software developers for the larger trade unions, do they get
under the union umbrella or do they get the short shrift form the unions they
would work for?

------
krona
Everyone is focusing on Palantir. As someone who knows how much IBM has thrown
at this over the past 2 years, it's really IBM and their historically strong
relationship with the NYPD that is the real (but far more mundane) story.

Ultimately, if you piss off the IBM execs enough (which is what Palantir is
exceptionally good at), then IBM will mobilize a significant chunk of the
company to obliterate anything in its wake. Even if that means making a huge
loss.

They then try to use that win as a template for future success. Unfortunately
IBM is incapable at that since the harsh reality is its software division is
just a bunch of incoherent acquisitions who can't generally collaborate.

~~~
x3c
What's the beef between IBM & Palantir?

~~~
ryan-c
IBM is competing with Palantir via a product called Analyst's Notebook, which
they bought from a company called i2 a while back. Prior to the buyout, i2
sued Palantir over how Palantir's "import from Analyst's Notebook"
functionality had been implemented.

It's entirely possible that there are other things going on between Palantir
and IBM, but I'm not aware of any other specific squabbles.

Disclosure: I worked at Palantir from mid 2009 to early 2014 and still own
some stock.

~~~
bane
To be clear, the suit that i2 filed alleged that Palantir stole trade secrets
and requested that the suit move forward under RICO law (an anti-Racketeering
law meant to fight organized crime). The judge agreed and Palantir settled
almost immediately after. i2 then sold to IBM almost immediately after that.

The employees on Palantir's side who were involved also kept their jobs. That
sends a really powerful message about who they are.

[https://www.scribd.com/doc/36371667/i2-v-palantir-080910](https://www.scribd.com/doc/36371667/i2-v-palantir-080910)

------
osrec
I remember a friend of mine interviewing there. He was flown from London to SF
for the interview. He got through the technical questions fairly easily. At
the end, he had an interview with one of the co-founders, who asked him "As an
FDE, what do you think your role is?". My friend totally BS'd his way through
the response, thinking he's messed up the interview. The founder however ate
up everything he said, and added his own flavor of fluff to the conversation.
That's when he realised that Palantir was not really selling software, as they
originally made out, but consulting, much like Accenture or McKinsey. Needless
to say, he was truly disappointed by the experience - Palantir was quite
simply not what it was made out to be.

~~~
ng12
To be fair that's how the game is played. You can't get in the door at a
government agency or Fortune 500 company without offering support staff.

------
praulv
I can't quite place them. On the one hand I don't see what their product
offers that existing BI dashboard tools such as Tableau can't deal with.

On the other hand, 2 of my former, incredibly smart, non-douchey, genuine,
down to earth colleagues (both non-white male if it matters) work for them so
I'd question the brogrammer stereotype as I doubt they'd stick around if they
were feeling out of place.

------
kirillzubovsky
Interesting the aim at Palantir's supposedly bad Java stack. It might be
uncool, and even bad from the programmer's POV, but many government agencies
have regulations that actually prevent them from using "cool" technology. I
know of orgs settling gov-nt and medical software, for example, and they use
Java and .Net because those things are easy to sell. Doing anything in Ruby
requires jumping through a thousand additional hoops. EOD, the language for
them is just best business decision.

------
jarym
When was the last time you heard a set of patched together IBM software being
'intuitive', let alone 'more intuitive' than Palantir's built-for-purpose
product?

I'm experienced with both IBM software and Palantir and I am truly surprised
by that bit in the article - imo IBM haven't put out anything intuitive since
OS/2!

------
throwaway201707
Have any of the big Palantir critics on here actually _used_ their software
for anything? Because compared to the usual fare of craptastic trashware that
passes for "software" at NYPD, I can assure you that Palantir is excellent.
Even the old Java/Swing installation still being used there is a godsend
(though Palantir has moved on to a more modern UI in their current versions).
It is (or was, I suppose) the most flexible investigative platform in that
place.

The real issue at hand is internal politics. Tech at NYPD is run by a lawyer
with little law experience and no technology chops who attained the position
due to nepotism (daddy is rich and a huge contributor to the NYC police
foundation). Coupled with brass who have zero understanding of technology and
an aggressive IBM sales team, you get the result outlined in this news
article.

------
tyingq
NYPD gets their source data, but not the algorithm. Hard to say if that's
unfair without knowing the nature of the contract.

It's certainly not unusual to sell analysis without selling the "exactly how".

~~~
kirillzubovsky
Definitely makes sense, especially if NYPD contracted with some firm to
replace Palantir, and now wants the algorithm because that's the hard part to
copy. Obviously the methods by which they are able to process information is
the reason governments pay them big bucks.

~~~
joering2
No its not the algorithm, its an untamed firehouse access to Facebook data,
all of it - not only what you see upfront, photos lead, or what marketers can
target. It goes much much deeper than that.

Its a well-known secret that only Palantir has this type of contract signed
with Facebook. A funny insiders anecdote is that some people joke about how
this agreement is being signed: left side: By Board Member of Facebook: Peter
Thiel. right side: By Board Member of Palantir: Peter Thiel.

This ability to query Facebook data anyway they want to is what gives Palantir
competitive age. Tech stack is irrelevant; you might as well run it from a MS
Access 2005, and you would still find clients, because they don't buy
technology, but the result of crunching of data (well maybe something that
hold more rows though)

Those who knows beginning of Palantir from a cocktail napkin times know it was
a perfect asnwer to a question how to use FB data and make extra extra money.
They's how Palantir was born.

Anyways, what most likely happened is that NYPD got enough of solid
information on this and probably is questioning legality of pulling data from
FB by Palantir without violating whole set of rules, laws, and US Constitution
as well; hence they decided to stop before they may become part of a bigger
lawsuit.

~~~
aaaaaabbbbbbb
This is entirely fiction.

~~~
joering2
Of course. A guy with karma score 1 would know. And NSA spying if entirely
fiction too!

~~~
matt_kantor
Got any evidence?

------
nl
With no technical details it's hard to know what is going on. "Analysis" can
mean many things, and sometimes one person's open analysis data is another's
proprietary visualisation.

~~~
bra-ket
most likely this has nothing to do with tech, simply IBM sales guys are better
at NYPD politics

~~~
tyingq
The article says NYPD did the development.

 _" group of IBM products tied together with NYPD-created software"_

~~~
Spooky23
Smells like Watson came to town.

~~~
nl
This is Analyst's Notebook, plus other things (some of them maybe branded as
Watson)

------
tp3z4u
Wouldn't it be hilarious if their secret sauce analysis turns out to be a
secret backchannel to the CIA/NSA and not some fancy algorithm.

~~~
jacquesm
You'd hope the NYPD knows how to set up a firewall and how to detect data
exfiltration at that scale.

~~~
tp3z4u
By backchannel I meant via people. As in the Palantir analyst calls up their
friends at the CIA/NSA for help. CIA/NSA data collection remains secret,
Palantir justify their expense, and NYPD find the guy. The problem is that you
can't reproduce your results. The other problem is the erosion of civil
liberties - but I'm only speculating with a thought experiment.

------
mola
Had a really bad experience at a palantir recruiting event. First, i got an
unsolicited email all black and mysterious. Annoying. When I got there I felt
nauseous. They actually employed a bunch of good looking women to talk with us
techies. So demeaning for all parties involved. I drank their cocktails and
ran away.

~~~
jKoTc
Have you ever considered that good-looking women can _also_ be techies?

------
CRUDmeariver
I was talking to someone recently that had worked there and they said the
first step for integrating with any client is to convert their data into a
proprietary format called PXML, which I assume is XML with the added feature
of vendor lock-in.

------
jimjimjim
i have no love for palantir and java makes me slightly sick but a lot of
comments here are of the type "they use java, that's ancient, blah blah".

although they are in sv they are NOT a "start-up". They are an old-school
government/3-letter-acronym IT supply/service company, think IBM and SAP.

In this environment, there is a good chance their customers may be using ie6.
java is what their market expects not what-has-the-cat-dragged-in-now-js
framework.

------
kelukelugames
How does buzzfeed get the scoop on this? I didn't know they did tech
reporting.

~~~
jitl
The author of this article (William Alden) writes consistently negative
headlines about two companies: Palantir and Uber. That’s his beat at Buzzfeed.

[https://www.buzzfeed.com/williamalden](https://www.buzzfeed.com/williamalden)

~~~
avs733
Are you saying he is unfairly negative or that there are so many such stories
that it is a full time job?

~~~
tristanj
Neither, it's a feedback loop. By writing negative stories, when company
insiders have a scoop and they want to leak it, they know they can contact Mr.
Alden because they've seen his previous articles on the company and his track
record of not revealing leakers. Mr. Alden then writes an article using such
leaked information, which generates buzz within the company (employees share
these articles among themselves), resulting in more insiders who know about
Mr. Alden and what he does. This encourages more people to share with Mr.
Alden.

A similar situation happened with WSJ's Christopher Weaver and Theranos, after
his initial few articles, several Theranos investors contacted him with more
information, which lead to more articles published, and further attracted
people who had company information to share.

~~~
trimbo
> WSJ's Christopher Weaver and Theranos

FWIW, I think it was primarily John Carreyrou. At least that's who Jesse
Draper blames

[https://twitter.com/JesseDraper/status/799108097675378688](https://twitter.com/JesseDraper/status/799108097675378688)

~~~
tristanj
You're right, thanks for catching that. My google-fu failed me, apologies.

------
jacquesm
At least the software ran on premises of the NYPD. There are many deals with
governments where data gets pumped back-and-forth with external parties doing
the analysis on _their_ infrastructure.

~~~
nl
Palantir generally runs on-prem because in many environments it is in a SCIF.

~~~
jacquesm
As it should be. But I know for a fact that this is not always the case.

------
kendallpark
> The department has created a new system to replace Palantir.

> The new system, named Cobalt, is a group of IBM products tied together with
> NYPD-created software.

You really have to wonder about the quality of this internal product. From
what I've gleaned about Palantir over the years, they supposedly have one of
the hardest dev interview processes. People that work there are on top of
their game.

My gut feel is that NYPD does not have as robust of an internal dev situation.
Or perhaps they shopped it out to contractors? Either way... skeptical that
the system is better. But who knows! It's possible the subset of features a
police department actually needs from Palantir might be much smaller than the
feature set they're paying for (and not too hard to construct themselves).
Whatever is going on, it's clearly a cost-benefit issue.

Although, if they're fighting for Palantir's analysis, it seems they _need_
something that Palantir does that they can't do themselves from the data.
There really isn't much one can conclude from this article. Only speculation.

EDIT: From responses on this thread it seems that the "hardcore super secret
analysis" image Palantir promotes might be more bark than bite. What if NYPD
actually did create something better? Intriguing.

~~~
Maxious
IBM acquired a company called i2 in 2011 that already was a competitor to
Palantir for defence/law enforcement analysis
[https://www-03.ibm.com/press/us/en/pressrelease/35255.wss](https://www-03.ibm.com/press/us/en/pressrelease/35255.wss)

Even if you don't have the roughest and toughest devs in the market, surely
spending more time with NYPD than the Palantir FDEs (are engineers part of the
skeleton staff mentioned in the article?) would get a more valuable result?

~~~
krona
Actually i2 was 20 years old when it was acquired, and it was the dominant law
enforcement investigative analysis product in the US at the time. Palantir
were the hotshots.

Of particular note was when i2 Group took Palantir to court for a conspiracy
to misappropriate its software and trade secrets. They settled out of court
(for a huge sum of money.) IBM swooped in soon after.

------
will4274
The NYPD, which can't even successful migrate off of WinXP is going to build
an in house product tying together a bunch of IBM products??

------
eeZah7Ux
A hundred comments criticizing the software stack and nobody discussing the
ethical concerns around the product.

Perhaps the discussion around rugged individualism in the current thread about
poverty are accurate.

------
fensipens
> Big data helped New York's cops bust Bobby Shmurda

So glad Palantir helped getting _Bobby Shmurda_ (best known for _Shmoney
Dance_ ) off the streets.

