
Do you check HTTPS certificates in your API clients?   - va1en0k
http://unfoldthat.com/2012/07/30/does-https-to-api-make-any-sense.html
======
tav
When you validate server certificates from HTTPS clients, please be sure to
use the right set of root certs. Mozilla maintain a decent list of these [1],
but it's not in the PEM format that most HTTPS client libraries expect, e.g.
Python's ssl.wrap_socket(sock, ca_certs="certs.pem").

Mozilla's list also includes _distrusted_ certificates, so you need to be
careful to leave them out when generating the PEM-encoded format. In fact, I'd
strongly recommend using Adam Langley's excellent extract-nss-root-certs tool
[2] which takes care of the subtle details for you.

And, if you are willing to trust me, you can download my pre-generated PEM-
encoded cacerts file from a month or so ago [3].

[1]
[https://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw...](https://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1)

[2] <https://github.com/agl/extract-nss-root-certs>

[3]
[https://github.com/downloads/tav/ampify/distfile.cacerts-201...](https://github.com/downloads/tav/ampify/distfile.cacerts-2012.06.28.tar.bz2)

~~~
va1en0k
I'd actually recommend just get the exact certificates you're going to work
with. For example, if I'm using GitHub's API, it makes sense not to check all
the chain, but just keep the GitHub's original cert.

~~~
tav
Disclaimer: I don't particularly agree with the Certificate Authority
mechanism that we currently use with TLS.

However, given that it's what we currently have, I'd strongly advice taking
advantage of the security that it provides. Requiring API client library
authors to ship certs will make for poor security. Not only do certificates
expire, they also get compromised.

It would be easy to conduct MITM attacks using revoked certs and API client
library users would be none-the-wiser. Instead, it should be the
responsibility of HTTPS client libraries to use the latest cacerts data and
support features like OCSP [1] for validating certificate revocations, etc.

[1]
[http://en.wikipedia.org/wiki/Online_Certificate_Status_Proto...](http://en.wikipedia.org/wiki/Online_Certificate_Status_Protocol)

~~~
tptacek
Taking a validated known-good cert and, from that point on, simply verifying
that the cert remains bit-for-bit identical is called "certificate pinning",
and while it does create problems with revocations, your service is going to
break as soon as Github revokes their certificate anyways, so it's not like
you won't notice.

------
delinka
In this age of high-level languages, why do I still have to worry about this?
I don't mean 'security' I mean 'managing certificates.' My local framework/API
should complain if I don't have a trusted root and should then make it dead
simple to provide that root.

~~~
rfugger
Python does have some certificate-checking machinery:

[http://docs.python.org/library/ssl.html#functions-
constants-...](http://docs.python.org/library/ssl.html#functions-constants-
and-exceptions)

<http://docs.python.org/library/ssl.html#ssl-certificates>

However, this isn't exposed in the higher-level httplib.HTTPSConnection class
for some reason. I'd bet it's not too hard to write your own subclass to
handle it though.

~~~
natrius
If you're using Python for HTTP requests, you should be using Requests, which
checks SSL certificates by default (as far as I can tell).

<http://docs.python-requests.org/en/latest/>

~~~
voltagex_
Yeah, I posted this below. It's mentioned in the advanced section. Why
checking your SSL certs is an advanced topic I don't know.

------
nl
Java uses a keystore (actually - being Java - you can set a property to tell
it which keystore to use).

Importing a new certificate is documented here:
[http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Security10.htm...](http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/Security10.html)

The verification of the certificate is delegated to a TrustManager[1].
Certificate paths can be checked[2] and the TrustManager is flexible enough to
support a large variety of verification scenarios[3].

[1]
[http://docs.oracle.com/javase/1.4.2/docs/api/javax/net/ssl/T...](http://docs.oracle.com/javase/1.4.2/docs/api/javax/net/ssl/TrustManager.html)

[2]
[http://www.exampledepot.com/egs/java.security.cert/ValidCert...](http://www.exampledepot.com/egs/java.security.cert/ValidCertPath.html)

[3] [http://stackoverflow.com/questions/11681474/how-can-i-
trust-...](http://stackoverflow.com/questions/11681474/how-can-i-trust-a-
specific-self-sign-certificates-not-trust-all?lq=1)

------
josephcooney
If you want to check any 3rd party apps or frameworks that you use you can set
up a proxy like Fiddler, configure it to do a MITM attack on you, and see if
the client/API blows up with an error or just keeps on truckin'.

~~~
va1en0k
better than /etc/hosts approach, yeah

------
pc
Getting this right was a big pain when we were implementing the various Stripe
client libraries. We had to resort to hacks even in mainstream languages.

~~~
va1en0k
Well, having to use hacks for that is just plain wrong. I think we can
consider this as bugs in these mainstream languages. Probably you could file
them?

------
celere
Python: Just check the certificates with Python Requests (or take one of 20
lines urllib solutions):

>>> requests.get('<https://exmaple.com>, verify=True)
requests.exceptions.SSLError: hostname 'exmaple.com' doesn't match either of
'*.exmaple.org', 'exmaple.org'

------
voltagex_
The requests library in Python will check your certs.

docs.python-requests.org/en/latest/user/advanced/#advanced

~~~
polychrome
This will take you straight there: [http://docs.python-
requests.org/en/latest/user/advanced/#ssl...](http://docs.python-
requests.org/en/latest/user/advanced/#ssl-cert-verification)

~~~
voltagex_
Thanks. Did I miss the TOC or was that a hidden anchor?

~~~
forsaken
Most sphinx documentation has a permalink if you hover over the title text. It
will show up in this instance to the right.

~~~
polychrome
Ahh. I couldn't figure that out to be honest, so I just viewed the source.

------
javert
So presumably, decent high-level languages (like Python) do use an encrypted
connection - they just don't ensure the right person is on the other end of
that connection?

~~~
va1en0k
yep

however, encrypted connection without this checks allows, for example, Man-In-
The-Middle attack

------
__alexs
To make it worse in many languages there not built in OCSP or CRL facilities
to go with their standard TLS wrappers. e.g. The best you get in Python is
checking against a CA list. So even if you do go to the trouble to turn on CA
verification yourself you still accept known bad certificates.

------
polychrome
Great Article! Looks like as of Curl 7.10 PHP now checks this by default:
CURLOPT_SSL_VERIFYPEER CURLOPT_SSL_VERIFYHOST

<http://php.net/manual/en/function.curl-setopt.php>

------
Spooky23
The problem with doing this is that many/most enterprise IT shops use security
devices to proxy SSL traffic, altering the cert that the client sees.

So unless you really understand you app's deployment scenario well, you should
proceed with caution. Also note that 2% of enterprise IT people understand SSL
at all, (and the devices that do the MITM part are usually controlled by the
security dept) so troubleshooting will be close to impossible.

~~~
tptacek
Those proxies publish their fake CA=YES certs, so you can just add them to
your root and everything will validate.

------
SlipperySlope
Java HTTPS server code allows the developer to configure the X.509 trust
manager. My API uses Java for both client and server and transmits JSON
messages over secure web sockets which is built over HTTPS. Works great. I
authenticate on both client and server, and am just now developing an iOS
Objective C client that runs wss.

------
polychrome
Node.js seems to omit using a certificate as well. From the documentation:

* cert: Public x509 certificate to use. Default null.

[1]
[http://nodejs.org/docs/latest/api/https.html#https_https_req...](http://nodejs.org/docs/latest/api/https.html#https_https_request_options_callback)

------
jspaur
i know in the the world of .NET (atleast on Windows) it'll automatically check
using the cert manager. Anyone know how this might work on platforms such as
Java? I'd assume the local VM would need to have some OS specific plumbing in
place.

~~~
va1en0k
many servers are deployed using Linux which doesn't have any cert manager (see
citation in the blog)

~~~
jspaur
what about with platforms such as Android? iOS? etc.

~~~
salgernon
iOS and OSX both use the system store represented by the "system" keychain.
(On the desktop, individual users can also have keychain a with trusted
roots.). Apple keeps their root store up to date via software updates and
automatic OCSP checks.

NSURLConnection, the higher level resource API, will by default require a
valid certificate chain, but provides for explicitly allowing an insecure
connection as part of its authentication callbacks. (This is an improvement
over previous versions (leopard and before) where you had to explicitly
specify hostnames that should be considered safe.)

So, certificate validation is treated, at the API level, like any other sort
of authentication challenge.

You can also provide a client certificate using the same mechanism, if
requested by the server.

Using the lower level CFHTTP stream API, you can only fail the connection and
re attempt it after disabling cert checking.

------
st3fan
Validating certificates is a good thing and everybody should do it.

That said ... it really only tells you that a certificate is 'sound'. It by no
means tells you with 100% confidence that you are talking to the right party.

SSL/TLS is still pretty fragile.

------
stanleydrew
The twilio-ruby library does this by packaging a set of root ca certs. There
is a configuration option to override this with your own set when you
construct the client.

------
nivloc
This is default behaviour in Ruby, and as mentioned in the article, one can
set up a custom store.

With all the languages that support this by default, I'm surprised Python
doesn't.

------
jokull
NGiNX reverse proxy can take care of this and does so securely, am I right?

~~~
rfugger
No, nginx is on the server side. The article is about client-side.

~~~
va1en0k
You can use nginx as a proxy on your side for any APIs. Just proxy_pass it to
the API, and listen only on the local side. It can provide caching and other
stuff that way.

~~~
obtu
Yeah, but that's proxy, not reverse proxy. It's possible to do something
fiddler-like using apache or nginx to mitm oneself and dump everything that
goes through.

