
Elevated error rates on applications and deployments - jorrizza
https://status.heroku.com/incidents/941
======
stpe
I'm currently at the Nordic.js conference
[http://nordicjs.com/](http://nordicjs.com/) (which... is down right now).
Suddenly half the audience started to looking at mobile phones, unpacking
laptops.

Poor timing for the current speaker...

~~~
stpe
At least our apps are now up and running again after being down.

Finally I can start focusing on the talks instead...

------
aquilligan
Down for almost an hour now. Haven't seen anything of this scale happen in 18
months or so of running our apps on heroku. Looking forward to the postmortem
from Heroku (and hopefully my app being up again).

~~~
spleen
Do they really release postmortems? I've seen them mentioning postmortems in
incident statuses, but I've never actually read one.

~~~
aquilligan
[https://blog.heroku.com/tuesday_postmortem](https://blog.heroku.com/tuesday_postmortem)

~~~
spleen
October 27, 2010. Really? :)

------
nateguchi
It's more than some apps... all of ours (~70) are down and deliveroo looks
like it's also having issues

~~~
ablation
Yep, people are melting down on Twitter about not being able to order food.
And on the day that Amazon launched Amazon Restaurants in London with one hour
delivery too...

------
babgyy
What options would you consider if heroku was down permanently ?

~~~
wc-
If you are just using heroku to run a process it is pretty easy to rebuild on
AWS. Here is one fairly solid guide:
[http://blog.clearbit.com/ec2-heroku/](http://blog.clearbit.com/ec2-heroku/)

Depending on what language you are using google's app engine could be a quick
replacement.

[http://rogerstringer.com/2015/05/13/make-your-own-
heroku/](http://rogerstringer.com/2015/05/13/make-your-own-heroku/)

[https://github.com/buttfoundry-community/bosh-
buttfoundry/bl...](https://github.com/buttfoundry-community/bosh-
buttfoundry/blob/master/tutorials/build-your-own-heroku-with-buttfoundry.md)

Another HN thread:
[https://news.ycombinator.com/item?id=4183634](https://news.ycombinator.com/item?id=4183634)

I'm not a big fan of docker for a few irrelevant reasons but using it w/
heroku makes it very easy to migrate to other container hosts.

edit: the founder of Flynn replied to parent as well, that is also a great
alternative that I forgot to include.

~~~
erkkie
Please do not trivialize the aspects of running a server on AWS (or a highly-
available cluster) just for your own single application. Installing is the
easy part, the day to day maintenance, reacting to issues, the 3am alerts (you
set up alerting, right?) is what's costly.

~~~
wc-
I guess I was responding to the parent with ideas and links to use 'in case of
emergency'. It isn't trivial, but some of these tools in conjunction with AWS
make it _possible_ to do alone, which I think is pretty cool. Alerting?
Cloudwatch + PagerDuty gets me somewhat there.

------
fphilipe
It took them 15 minutes to update their status from the time of our last
received request.

~~~
sleepyhead
Just like with previous incidents. I understand operating Heroku is not
trivial but with 15 minutes after my monitoring systems (Pingdom, New Relic
and Logentries) notifies me they need to improve this.

~~~
gryzzly
also not the cheapest service and no direct support line…

~~~
sleepyhead
What are you comparing to cost wise? Please do include costs for setting up
and maintenance in that comparison.

As for support I have always found Heroku to respond very quickly. And they
_do_ have direct support.
[https://www.heroku.com/support](https://www.heroku.com/support)

~~~
gryzzly
Perhaps it’s suitable for the US but in EU we are seeing this for the most of
the working day:

"Heroku Support is currently closed. Normal support hours are 9 AM to 9 PM
Eastern time. Our goal is to respond to your inquiry within 1 business day.
Please note that it may take longer depending on current volume. Running a
critical app?

Heroku's Premium Support offering includes a 24×7 Response SLA and more.
Please contact us for further details."

When it is Heroku’s fault (couldn’t access my account with cryptic error
message) it is very annoying to be suggested to use paid support, for a
product that is already costs more than alternatives.

~~~
sleepyhead
If it's a Heroku error then they obviously have staff monitoring servers 24x7.
Usually I get very quick responses from Heroku Support at all times during the
day, regardless of the type of inquiry - either basic questions or platform
problems. Obviously they would filter incoming requests. If you pay for
support you get a quick er response. If you have a question without paying you
might have to wait a bit longer during off-peak (US) hours. But if it's a
platform error I'm pretty sure they would filter that out and respond quickly.

And what are you comparing to here? Are you getting instant 24x7 free support
from any other vendors? It's quite normal to pay for support in this industry.

------
gryzzly
What is a good alternative for static single-page apps? What are downsides of
S3?

~~~
colinbartlett
I host all my APIs on Heroku and all my static front ends on S3 with
CloudFront on top. No downsides in my mind. Free and instant TLS certs with
CloudFront, too.

~~~
witty_username
And you can even get a free domain name through
[http://www.freenom.com/](http://www.freenom.com/)

Disclaimer: not affiliated

------
guy_c
Uptime robot is reporting my sites went down at 12:55 UTC, so been down for
35mins

------
pmukerji
A follow-up report has been included on the incident page:
[https://status.heroku.com/incidents/941](https://status.heroku.com/incidents/941)

------
cocoflunchy
It's been down for more than half an hour now... don't understand how it took
them so long to update their status page.

------
pluma
I'm not sure why, but I was under the impression that Heroku had been shut
down months ago.

I may have been thinking of Nodejitsu (which has been acquired by GoDaddy).
Did Heroku change owners at some point or am I imagining things?

~~~
0xADADA
Heroku was acquired by SalesForce, and like everything Salesforce touches,
it'll slowly descend into a steaming pile of

~~~
watson
So far it seems that Salesforce is leaving them alone to run their own
business

~~~
mpeg
They're one of the only acquisitions that have been kept running pretty much
the same way.

There was no integration with force.com or anything like that. I don't think
they're even colocated in SFDC datacenters, I think they still run on AWS

------
danielstocks
I got my first SMS alert notification from Pingdom about 55 minutes ago now.
One or two requests managed to reach through the the application logs in the
last 10 minutes, but still mostly down.

~~~
danielstocks
Seems to be resolved now...

------
dbuxton
Seems to be coming back up now, albeit a bit intermittent.

------
fredrivett
And we're back.

~~~
fredrivett
ish.

Deliveroo's error message is on point.

[https://deliveroo.co.uk/](https://deliveroo.co.uk/)

~~~
zephod
What was it?

~~~
calyhre
Nginx default error page, this one:
[http://i.imgur.com/IS9toHS.png](http://i.imgur.com/IS9toHS.png)

------
nateguchi
Back Online!

------
scrown
Up now

------
benmmurphy
This is kind of offtopic but what is the attraction of using Heroku over using
EC2 directly now. I remember back in the day when EC2 didn't have RDS and
Elasticbeanstalk Heroku was an attractive option because you could deploy and
scale without needing to do any kind of system administration.

But now EC2 is offering managed databases through RDS and elastic bean stalk
gives you git style deployment similar to Heroku I don't see what Heroku is
offering other than another point of failure and another set of security
problems. It looks like Heroku uses linux containers for isolation. So not
only do you have to worry about someone attacking the underlying EC2 VMs
Heroku uses but you have to worry about the tenants collocated on your Heroku
VM attacking you through the linux kernel as well.

~~~
stevedomin
There is much more to Heroku than managed database and git-based deployments.
For instance, recently they have seriously improved the CD experience with
Pipelines, Review apps and GitHub sync. They also have pretty seamless
integrations with hundreds of providers (email, logging, search, monitoring,
...) in the form of add-ons. You get some basic metrics on your application.

And if security is a concern, or/and you want your dynos to talk to each
other, they now have Private Spaces ([https://www.heroku.com/private-
spaces](https://www.heroku.com/private-spaces)).

Simply said they are taking the "managed services" experience to a whole new
level imo.

Plus, you may not care about that, but they actually have a UI/UX team that
can design beautiful AND usable dashboards.

(Note: I don't work at Heroku, just a happy user)

~~~
abhishivsaxena
True, and that's why I stuck to heroku for a long time.

But, in the end I realised I can replicate most - if not all - of it with
docker cloud and github. Most CI services support github anyways, so that's
also covered.

And it's significantly cheaper, for my case we have 90% price reduction -
paying only for instances + docker cloud.

Not connected with any service.

