
Mailgun Down – SSL cert is expired - tarr11
http://status.mailgun.com/
======
twakefield
Ugh. Should be fixed now. Sorry to anyone that experienced disruptions due to
this stupid error.

------
clintavo
We've had issues previously with Rackspace and SSL certs expiring (webmail api
certs, not mailgun previously). My app has been "broken" twice in the past due
to expired certs, to the point that we now monitor all SSL certs used on third
party api's that we rely on. (As well as monitoring our own certs of course).

We've got our eye on Rackspace's webmail reseller api's SSL cert, which now
has less than 14 days till it expires:
[https://admin.webmail.us/excedentsoap/excedentsoap.wsdl](https://admin.webmail.us/excedentsoap/excedentsoap.wsdl)

~~~
justizin
Pretty well unacceptable for a major vendor of SSL certs. Wouldn't you hope
that Rackspace could help keep you out of this trouble if you bought a cert
from them? I wouldn't expect it, since they can't keep themselves out of this
sort of trouble.

~~~
clintavo
To clarify. We don't buy SSL certs from Rackspace. I was referring to
Rackspace allowing their OWN certs to expire, breaking api calls made to those
endpoints. The URL I linked to was a webmail.us URL which is a Rackspaced-
owned domain.

Edit: fixed grammar.

------
dasil003
This is probably a good time to piggyback an Ask HN: What are the best SSL
providers these days with regards to price/hassle vs device/browser coverage?

I'm ashamed to say my company is still on VeriSign (ie. Network Solutions)
simply out of inertia because by the time we realize the cert is near expiring
there's no time to evaluate and switch providers.

~~~
moonboots
For noncommercial websites,
[https://www.startssl.com/](https://www.startssl.com/) offers free
certificates.

For commercial websites, the cheapest certificates I've seen are PositiveSSL
certificates resold by gogetssl [1]. It's $4.55 for one year or $17.25 for 5
years.

Disclaimer: I have no affiliation with either site. I've never used gogetssl,
but I will probably give them a shot the next time I need a certificate.

[1] [https://www.gogetssl.com/domain-validation/comodo-
positive-s...](https://www.gogetssl.com/domain-validation/comodo-positive-
ssl/)

~~~
tow21
StartSSL has a good deal for commercial websites too - for $120/year you get
unlimited certs, including wildcard.

Only downside is they do proper identity checking which is a bit painful to go
through (they will need to see official company documents, verify ID by phone
etc) - but worth it in the long run IMO.

~~~
orthecreedence
Yeah and sometimes they just drop the ball on the ID check. I didn't have a
bill ready that has my physical address on it (none of my bills print this
info) so they said they'd send me a piece of mail with a code in it. Never
showed up, all attempts to contact them were ignored. Ended up just getting an
$8 cert somewhere.

I will say that once you do get into StartSSL (which I have through a previous
company), it's nice to be able to create/sign unlimited certs.

------
spullara
I'm surprised there aren't companies that look at the expiration dates of SSL
certificates and try and get them to use their service to get a new one, like
domain people do. Seems like it could also be a feature of New Relic / Pingdom
/ etc.

~~~
ntoshev
We have this at our website monitoring and recovery service
[https://t1mr.com](https://t1mr.com) and it's free.

~~~
DenisM
Would you consider adding a check for particular string on page? That would be
handy.

Also, if you check SSL cert validity, you should totally advertise that! I
remember I've been paying for one such service where they would monitor SSL
url, but would not alert you about SSL cert expiration. I found out the hard
way.

~~~
ntoshev
Sure, checking for a string on the web page is also in the basic
functionality. We also ping your site every minute, so you'll know about
problems immediately.

The advanced (paid) functionality is a restart of your vps or other automatic
ssh action trying to recover it when it's down.

------
dchuk
Had an expired SSL cert bite me in the ass yesterday actually. They auto renew
on the billing side, but if you have a year SSL cert, you have to generate a
new one each time. So I saw that the billing renewed, didn't think anything of
it, then was reminded of the manual generation step by a bit fat warning on my
app. Yay.

------
tarr11
Oddly, their uptime charts still show 100%

~~~
pionar
well, technically, their system was up, the certificates were just expired.

~~~
crystaln
Anything that makes a system unusable is downtime...

"Technically our site was up, just the database was down..."

~~~
ChikkaChiChi
Right, but if their uptime monitor is internal, and they are resolving from an
internal address, nothing is wrong.

~~~
garindra
Yeah, but if your status page is made to be seen by external people, the right
thing to do would be to monitor it just like if you were using it externally.
There are whole slew of problems that wouldn't be exposed correctly if your
monitoring point is internal -- like this exact problem, DNS, etc.

------
breischl
So, more or less bad than when Azure let their SSL cert expire last year?

[http://www.zdnet.com/windows-azure-storage-issue-expired-
htt...](http://www.zdnet.com/windows-azure-storage-issue-expired-https-
certificate-possibly-at-fault-7000011705/)

~~~
adventured
My favorite is when Passport.com expired, taking down Hotmail, and a good
samaritan renewed it for Microsoft.

[http://news.cnet.com/2100-1023-234907.html](http://news.cnet.com/2100-1023-234907.html)

~~~
breischl
I hadn't heard of that episode. That's hilarious - particularly that it was a
Linux consultant.

------
jakejake
Expiring certs and domain names kinda haunt me. It's such a silly task and for
most of us we only have to deal with every year or two. Yet if you forget,
your site goes completely offline! All the work spent scaling and automating
the site and one stupid renewal can undo it all.

I have all manner of alerts for these things, but still I worry about it and
check from time to time to make sure nothing is expiring soon.

------
LogicX
I use [http://sslcertcheck.com](http://sslcertcheck.com) to monitor these
situations for me. You can do so for free, real simple service. Just gets
tiring to deal it's these sorts of issues, and great to get a bigger warning
before the service goes down as a result. Full disclosure: I'm friends with
the owner.

------
markolschesky
Heh. I wrote 3 calendar reminders 60, 30 and 15 days out for our engineering
team on Friday reminding us to do this when the time comes. Now, I will
reserve the right to feel smug.

~~~
eddieroger
Yeah, I mean, if you're responsible for an SSL cert, why isn't this common
practice. I'm going to join you in smugness, because I did this as well on a
web app I was partially responsible for. It's super easy, and if your system
is at all critical (mine wasn't even, really - it was an internal-only wiki
with a self-signed cert from a different internal team), then there's no
excuse not to.

~~~
markolschesky
There was a bit of jest in the smugness, but in my case it was because I
inherited the SSL responsibilities from an outgoing engineer and randomly
thought to myself "I wonder when these servers certs expire?". It's the type
of thing that multiple people need to keep an eye out for, not just one person
in the event of inevitable turnover. For something that can entirely take down
your site/service for an indeterminate period of time that is entirely out of
your control, it's a necessary evil.

------
jvehent
protip: nagios.

~~~
WestCoastJustin
More specifically, you can use the nagios check_ssl_cert plugin [1], which
checks if the server is running and delivers a valid certificate. We use this
plugin extensively and it takes the surprise out of an expired ssl cert. You
will get multiple heads up, say 90, 60, and 30 days out (it's configurable),
that your cert is going to expire. Seems like a no brainier.

[1] [http://exchange.nagios.org/directory/Plugins/Network-
Protoco...](http://exchange.nagios.org/directory/Plugins/Network-
Protocols/HTTP/check_ssl_cert/details)

~~~
Dobbs
This feature is built into the standard check_http with the right flags.

Specifically: `/usr/lib/nagios/plugins/check_http -H <ip/hostname> -C 60`

------
spo81rty
There is a free service called [http://certalert.me](http://certalert.me) that
can warn you about expiring as certificates. Check it out!

------
bliti
I wonder what is the economic impact of this error.

