Hacker Newsnew | comments | ask | jobs | submitlogin
Secure Your REST API (stormpath.com)
109 points by chunsaker 368 days ago | comments


tptacek 368 days ago | link

You should probably disregard this advice. Instead:

[Late addition:

* Do not use passwords as API authentication. The user of an API is a computer program, not a human. Issue single-purpose random credentials for API access.]

* Make sure that your API is accessible only over HTTPS; test the API endpoint to ensure requests aren't honored over unencrypted HTTP.

* Use the simplest API authentication mechanism that (a) works and (b) you understand. You should probably degrade the capabilities of your API before you try to adopt an authentication system that you don't completely, fully, absolutely grok.

This means, if you don't need delegation right away (for instance, if you don't have 3rd party applications making calls to your API on behalf of your users), don't bother with anything like OAuth.

Contrary to the implications in this article, HTTP Basic Auth is not less secure than OAuth if your endpoints all require HTTPS. If you allow HTTP calls to an API, you have bigger problems than your auth token scheme.

Semantic mismatches between API endpoints and web UI endpoints (ie, semantic differences between API auth and cookie-based authentication) are a classic, pervasive, hard- to- eradicate source of serious security flaws. Don't make things any more complicated than they need to be.

[Late edit: I didn't think I needed to make the first point, but I clearly did need to. Sorry.]

-----

MattRogish 368 days ago | link

So let's say I have an iOS or Android app that needs to talk to my REST API. I have my user login in the app, which does Basic Auth over HTTPS.

Once they're authenticated, I generate a token and send it to them. They store it, and use that token from here on out (HTTPS only).

Perhaps I expire it after some time (hours or days, if I don't want my user to have to login all the time).

That's it? :D Seems like I'm missing something...

-----

krichman 367 days ago | link

That sounds good. Think about how, e.g., Gmail lets you persist sessions on each device. They're doing a password auth over HTTPS and leaving a cookie with some session token.

-----

chacham15 367 days ago | link

That sounds very close to OAuth 2.

-----

tel 368 days ago | link

> Semantic mismatches between API endpoints and web UI endpoints (ie, semantic differences between API auth and cookie-based authentication) are a classic, pervasive, hard- to- eradicate source of serious security flaws. Don't make things any more complicated than they need to be.

Could you explain that one in more detail? I don't think I understand it.

-----

tptacek 368 days ago | link

You have one code path that authenticates web users through forms and sets cookies.

You have another code path that authenticates API callers.

You have code scattered through your whole application that makes authorization checks based on which user you're authenticated on.

Inconsistencies between the first two code paths often break that code.

-----

bradleyjg 368 days ago | link

The one true solution is client authenticated TLS. For the life of me I can't understand why 25 years later certificate management is still such a mess. 

We don't need some perfect universally recognized root trust system to get started. Why doesn't the sign up process for authenticated API access routinely include the issuance of a certificate signed by the API owner?

Unlike for interactive users there's no expectation that a customer will be accessing an API from some random computer where he might not have access to his certificate store. 

-----

tptacek 368 days ago | link

The user interface in browsers for client certificates is appalling.

The API support in client SSL libraries for managing multiple certificates, for applications with multiple API affiliations, is shaky.

I like TLS client authentication a lot, but it's hard to make it work.

-----

bradleyjg 368 days ago | link

I agree it's awful to use, but what I don't understand is why no one has bothered to improve the tools.

It's one of those areas where the underlying tool (either a library like openssl or NSS, or an OS feature like SSPI) could do the hard work in one place and make it simple for downstream libraries to wrap the functionality.

In other, somewhat analogous, domains that happened, but for whatever reason not in this case.

-----

chacham15 367 days ago | link

Look at the OpenSSL libraries for php regarding certificates. Its very close to unusable, you cannot get or set some pretty basic information like SANs.

-----

carbocation 367 days ago | link

Do you think there would be value in accepting HTTP requests and then automatically and immediately expiring every credential that ever gets passed over plain HTTP?

-----

tptacek 367 days ago | link

With an API, I think you're better off just not accepting HTTP connections to begin with. This is library design, so, principle of least astonishment. Break hard and fast.

-----

veesahni 368 days ago | link

If you don't need delegation, there are a number of simple but secure methods: HTTPS + token via Basic auth seems quite popular.

What simple but secure methods are there where delegation is required? Is there something simpler than OAuth or AWS style signed HMAC?

-----

steveklabnik 368 days ago | link

Wow, I don't think I've ever seen tptacek gray in a thread about security. I don't know why, you are absolutely right.

One neat variant on this is how GitHub uses HTTP Basic auth to give you an OAuth token... http://developer.github.com/v3/#authentication

-----

redblacktree 368 days ago | link

Even if you use HTTPS, aren't you vulnerable to lazy devs who put the password in the URL?

If I use https://username:password@example.com/, doesn't that URL show up in server logs all over the internet?

-----

tptacek 368 days ago | link

Don't ever use passwords for API authentication. It's an API, not a browser. The users of an API are other programs, not people. Issue single-purpose random credentials.

I didn't think this was something I had to point out about API authentication, but apparently it is.

-----

CodeMage 368 days ago | link

I have a stupid question: how do you do this with HTTP Basic auth?

-----

tptacek 368 days ago | link

Generate random long passwords.

-----

redblacktree 367 days ago | link

By "don't ever use passwords" you mean, "don't let users set their own passwords," right?

Obviously, you're still using a password if you use HTTP Basic Auth.

-----

lhazlewood 368 days ago | link

You shouldn't ever use username/password pairs for API authentication. If the user ever changed their password, then their API calls would immediately fail!

This is one of many benefits of using multiple (revokable) API Keys.

-----

jimktrains2 368 days ago | link

No. The UA strips those and puts them in an Auth header.

-----

lhazlewood 368 days ago | link

Depends on the UA (curl?)

-----

jes5199 367 days ago | link

Even curl does the right thing for basic auth

-----

jimktrains2 367 days ago | link

OK, so it'll either add an auth header or not know what to do and refuse to do anything. Either way the password is not sent in the request url

-----

jimktrains2 367 days ago | link

Actually, I mused have screwed up last night. The version of curl I have does support it and does put it into the Auth header.

  % curl --trace-ascii /dev/stdout http://jimktrains:password@news.ycombinator.com
  == Info: About to connect() to news.ycombinator.com port 80 (#0)
  == Info:   Trying 184.172.10.74... == Info: connected
  == Info: Server auth using Basic with user 'jimktrains'
  => Send header, 223 bytes (0xdf)
  0000: GET / HTTP/1.1
  0010: Authorization: Basic amlta3RyYWluczpwYXNzd29yZA==
  .......
Some UAs however, do just drop it. Like IE (http://support.microsoft.com/kb/834489)

-----

olivier1664 367 days ago | link

Please correct me if I'm wrong. HTTPS will encrypt the URL, but the DNS lookup is in clear. So the "username:password@example.com" part will be sniffable/loggable.

http://stackoverflow.com/questions/499591/are-https-urls-enc...

-----

jewel 367 days ago | link

That's incorrect. The URI library will parse the URL into a hostname, username, and password. Only the hostname is sent to the DNS server.

The username and password are sent in the "Authorization" HTTP header, which will be encrypted.

-----

dllthomas 368 days ago | link

Shouldn't HTTP Digest still be preferred, so your server doesn't need to see the passwords?

-----

tptacek 368 days ago | link

That's not how HTTP Digest auth works, and you should never, ever be using "passwords" for API authentication.

-----

dllthomas 368 days ago | link

Are you saying the server sees the password in HTTP Digest authentication?

-----

jimktrains2 368 days ago | link

How else would the server know what to check your response against? If you don't want to give the server a password, try SRP.

-----

tptacek 368 days ago | link

... but please don't use SRP for your API authentication.

-----

dllthomas 368 days ago | link

Well, in this case (having gone back and actually checked the docs rather than relying on recollection roughly a decade old) you could store MD5(username:realm:password) rather than password, though that doesn't buy you all that much in a case like this where making the password a long, random string that's not reused is not only good practice but also easy and so likely to be general practice.

-----

tptacek 368 days ago | link

Stipulating all the other points you've made, exactly what is the purpose of hiding from a server a long random string generated by that server and useful only to that server?

-----

jimktrains2 367 days ago | link

Preventing someone else from getting that and using it on the server, especially if the service can rack up charges.

-----

dllthomas 367 days ago | link

If you have the hash, you can log into the site (just not using the typical libraries), so it really doesn't add a lot of security. That's why I was wrong :-P

-----

dllthomas 368 days ago | link

Yeah, I got there, just was confused about the digest thing.

-----

healsjnr1 368 days ago | link

I also wouldn't put too much value in a security article served on page using mixed HTTP and HTTPS. This shows a distinct lack of regard for and understanding of security.

-----

mcavage 368 days ago | link

I expected something very different from this article based on the headline. Specifically, I completely disagree with the "leave it to industry standards" approach, as that doesn't help people understand what they should do, and more importantly, why.

Request/response protocols (well, many things) really break down into 5 top-level categories (some sources will say the 6th is Audit):

- Authentication

- Authorization

- Integrity

- Confidentiality

- Non-repudiation

It's a far more interesting exercise to walk through what you would get from each solution. Basic-Auth over TLS, actually gets you quite a ways towards that goal (specifically, C-I-A (authentication)). Where that, and notably HMAC, fall over is non-repudiation because they're based on a shared-secret model; admittedly HMAC keys are better than passwords because you're not sending the secret on every request, but asymmetric crypto is preferred. Authorization that the server system does is really out of scope in all of these protocols, so Basic-Auth over TLS doesn't really impact that one. It can be as simple as "caller = owner," or as full-featured as a security policy language [1] (full disclosure: I am the original author of [1]).

OAuth really doesn't differ that much besides specifically solving the delegated access and website SSO problem(s); but IMO it does so with an overly baroque protocol that has too many parts. The "long pole" of setting up such a system (that is, allowing 3rd party sites to act on behalf of my site's users) isn't the specifics of what my REST api looks like, but really it's all the "governance" of user decisions, and more-over, key management (in all these cases, key management is generally the hardest or almost hardest problem).

While it can be debated whether it was right or wrong, we (Joyent) released an open-source spec to solve straight up authentication of REST requests using SSH keys [2]. At the end of the day, the user signs the Date header of requests with their private key (which by definition the server has never seen), and all requests must be over TLS. Disregarding Authorization, this scheme gives you Confidentiality (TLS), Integrity (TLS), Authentication (Signature), Non-Repudiation (asymmetric signature), and adds a "poor man's nonce," assuming you disallow requests where the clock skew of the date header is too large. And lastly, SSH solves a lot of key management problems for humans. Note: I didn't drop that reference to advocate for our specification here, but rather the security process you should think about when evaluating whether a protocol is secure.

[1] http://docs.aws.amazon.com/IAM/latest/UserGuide/policy-refer...

[2] https://github.com/joyent/node-http-signature/blob/master/ht...

PS

Mutual-Auth SSL/TLS is a royal PITA, and is basically guaranteed to cause you grief. The client compatibility matrix might as well be considered an NP hard problem to assure yourself coverage, and failure modes of the different browsers/SDKs all differ. As a REST API should have maximum accessibility to clients (i.e., don't wed yourself to any one language/sdk), this is pretty much a non-starter.

* edited: copy/paste formatting

-----

bradleyjg 368 days ago | link

> Mutual-Auth SSL/TLS is a royal PITA, and is basically guaranteed to cause you grief. The client compatibility matrix might as well be considered an NP hard problem to assure yourself coverage, and failure modes of the different browsers/SDKs all differ. As a REST API should have maximum accessibility to clients (i.e., don't wed yourself to any one language/sdk), this is pretty much a non-starter.

Given the inconsistent support for mutual-auth TLS, which along with its direct ancestors has existed for a decade and a half, what makes you think they'll be widespread and correct support for SAS?

-----

mcavage 368 days ago | link

If by SAS you mean SSH, then really it's that there's a much smaller client surface area (really, openssh), and the fallback is always "use a PKCS#8 private key", which is much more uniform than SSL. SSL mixes in x509, and protocol, and UX.

That said, you don't have to agree with it. My macro point was really assessing these things against basic security threat modeling, not whether or not you agree with our choices of using SSH.

-----

bradleyjg 368 days ago | link

Well it's more than SSH. The client library needs to to implement the headers and do the date signing. Sure it's easier than TLS, but the libraries suffers more from a lack of dedicated interest than insane complexity I would think.

But I do agree with your basic point that too many people stop at authentication, instead of considering the full range of concerns.

-----

wereHamster 368 days ago | link

Please stop using the term UUID when you mean 'random alphanumeric string'. Because UUIDs have a standardized format (it's not just a random string): http://en.wikipedia.org/wiki/Universally_unique_identifier

-----

juiceandjuice 368 days ago | link

UUID v4 is, in fact, random and "alphanumeric" in the sense that it's hex.

That being said, I have a few other issues with their wording as well. They should just say "we have a custom HMAC-based authentication scheme for our REST API". Also, it took me about 3 days to realize HMAC over SSL/TLS is about as secure and easy as you can get for most any language -- If you can send HTTP requests, you can probably do HMAC. You can add further safety by making expiring private keys for HMAC and other things, although my use cases are based on long running (weeklong+) batch computations, and not end users. (i.e. initial distribution of an expiring private key for HMAC over SSL, reauthentication schemes, etc...

-----

welder 368 days ago | link

It's not just random, it's a standard requiring some hex digits to be non-random (4, 8, 9, a, b). Otherwise it won't validate as a UUID4.

-----

lhazlewood 368 days ago | link

A UUID is first and foremost a 128 bit number, irrespective of its text encoding.

Its 'canonical' form uses HEX-only encoding. A 'Url62' can be another encoding. 'Url62' wouldn't be a canonical encoding, but it's still a 128 bit UUID number.

-----

jrochkind1 368 days ago | link

> Best practices say to encrypt your passwords in the database to limit a potential data breach. This increases overhead for each request when authenticating a user. Unique API keys authentication skips the hashing step and therefore speeds up your calls.

Wait, why can you can skip the hashing step and still be secure? Because hashing is only neccesary if you call it a 'password', but not if you use it in the same way but call it an 'api key' instead?

I guess it depends on the purpose for hashing. If it's just about 'data breaches', then maybe it doesn't matter if your api keys get out... because they at least won't grant access to any _other_ systems, since they weren't manually entered by users and re-used accross systems. Is that what you're thinking?

But don't you still want to avoid a data breach like this for your _own_ service?

And, I think, isn't the other reason hashed passwords are used, to make it harder to do a brute force attack? Ie, it's quite intentional and deliberate that it increases request overhead. And doesn't this still apply to an api key, possibly even MORE so when you have a single api key instead of a username/pass combo?

-----

tptacek 368 days ago | link

If you lose the database for your own service, it does not matter how you authenticate to that service. Attackers have already fatally bypassed your credential system. At the same time, API credentials should by definition be single-use.

-----

danielweber 368 days ago | link

> At the same time, API credentials should by definition be single-use.

Could you spell this one out a little bit more? Do you mean only a single session should be able to use an API credential?

-----

tptacek 368 days ago | link

I mean the credential should only be relevant to the service, never shared across multiple services, because the API generates it for you.

-----

lhazlewood 368 days ago | link

HMAC authentication requires both the client and the server to have a shared secret (or more likely a derived key based on a shared secret). The secret cannot be saved as a one-way hash (as might be common for a password). So you couldn't use BCrypt or SCrypt to hash the shared (or derived) secret since the server would never be able to acquire the value to calculate the same digest.

You can still encrypt the secret, e.g. using AES 256 bit encryption with secure random Initialization Vectors and rolling keys. This too is not easily 'brute forceable', but is very fast to decrypt compared to a BCrypt comparison (key storage should also be in a separate location than the main data store).

-----

alinajaf 368 days ago | link

Does anyone know of a good reason not to use TLS with client-side certificates (and you as the CA) for API authentication?

-----

jimktrains2 368 days ago | link

It can become a pain to manage from the server side (issuing new ones &c), but technically they're pretty nice. The company I used to work for used them for their api.

-----

epochwolf 368 days ago | link

Yeah, limited library support for client certs.

-----

dllthomas 368 days ago | link

Let's fix that...

-----

pplante 368 days ago | link

In my humble opinion, security through obscurity via UUIDS rather than sequential integers is not great advice. It merely masks the real problem.

-----

dustismo 368 days ago | link

This isn't an instance of "security through obscurity". This is more akin to suggesting people use non dictionary passwords.

http://en.wikipedia.org/wiki/Security_through_obscurity

-----

danielweber 368 days ago | link

Obscurity can be a fine part of being secure. You have to think carefully about what happens when it fails, and realize that automated bots will tend to find things humans would consider obscure, but don't let the usually-good heuristic become a straightjacket.

-----

dllthomas 368 days ago | link

Obscurity of the secret is a necessary part of being secure :-P

Security-through-obscurity refers to the notion that your algorithms are a meaningful part of the key.

-----

knkella 368 days ago | link

What are your thoughts on Amazon like security scheme? As far as there are no third party apps involved, I think OAuth is an overkill.

What I mean by Amazon like securtiy is described in this article http://www.thebuzzmedia.com/designing-a-secure-rest-api-with...

-----

lhazlewood 368 days ago | link

Stormpath's custom scheme is very similar to Amazon's.

But per the blog article, you'd only want to do this if you are willing to support client libraries/sdks that implement it as well. No one wants to spend the time to implement non-standard custom HMAC algorithms.

-----

knkella 368 days ago | link

We tried this with Pagify http://pagify.io And I think supporting client libraries was not an issue, since we had to provide SDK one way or the other.

-----

lhazlewood 368 days ago | link

Totally agree. The key here is that you're doing the work to implement the algorithms, not your customers. If they had to do it, they probably just wouldn't use it.

-----

yoava 368 days ago | link

Thanks for the pointer, I was looking for a sane explanation of the Amazon security and signature algorithm.

I have to disagree with the other comments here regarding the client library. I think that given the precedence of Amazon API, given that people understand how to sign APIs like amazon, this method will be accepted even without a client library.

Being a Java / Scala dev, I prefer that an API provider allow me to select the HTTP client libraries to use and prevent from forcing me to use a specific library & version via SDK transitive dependencies.

-----

cicloid 368 days ago | link

Didn't oAuth2 support signed requests via MAC token on request?

-----

dantiberian 368 days ago | link

How does using bare api keys over TLS compare to these suggestions? It's less secure but is it still a recommended option?

-----

tptacek 368 days ago | link

The advantage to OAuth over basic-auth is that you can delegate the credentials, which is to say you can set up your system so that users can give a limited-use credential to a 3rd party application to perform API calls on their behalf.

Some very large services have abused OAuth to "delegate" credentials to mobile devices, which has set up the expectation among developers that OAuth is the "sophisticated" way of doing all-around credential management of any sort. Not so. If you don't have delegation to third parties, don't use OAuth.

Most applications do not need delegation.

If you need delegation, there are simpler ways to do it than OAuth that won't meaningfully sacrifice the security of your controls. At the same time, using an OAuth solution you don't fully understand (for instance, using OAuth through a high-level library that hides the details from you) can damage the integrity of your whole application by creating new classes of mistakes for you to make.

-----

nicwolff 368 days ago | link

Shoots down Basic auth without SSL, without mentioning Digest auth, weird.

-----

tptacek 368 days ago | link

I'm not sure I've ever seen a scenario where digest authentication was a win.

-----

chunsaker 368 days ago | link

That's a good suggestion - will see if we can add a para about it. We use digest authentication, fwiw.

-----

lhazlewood 368 days ago | link

There are many types of digest authentication - OAuth1.0a and Amazon's and Stormpath's custom schemes are examples. Browser-specific digest authentication wasn't covered however since the article was about REST APIs and most REST clients are not browsers.

-----

bct 368 days ago | link

I'm pretty sure he means RFC 2617 Digest authentication. There's nothing browser-specific about it.

-----

lhazlewood 368 days ago | link

I gathered as much. But in practice, how often do you see RFC 2617 Digest authc used in non-browser scenarios? (I'm genuinely curious. I haven't seen it used much at all outside of web browsers, so I'm curious what others may have come across).

-----

bct 368 days ago | link

I've written Atom Publishing Protocol servers that use it. It's not badly-suited for non-browser tasks (although yes, SSL and Basic is much simpler - if you don't mind paying for the certificate). It's unusual, but it's pretty unusual to use it (or Basic) for web browsers these days, too.

-----




Lists | RSS | Bookmarklet | Guidelines | FAQ | DMCA | News News | Feature Requests | Bugs | Y Combinator | Apply | Library

Search: