
Ask HN: Is there a way to protect your app if heroku goes down? - buthow
How do people build in redundancy&#x2F;failsafe to an app being served on Heroku in case of an outage. I&#x27;ve had a good look around and I can&#x27;t see a simple solution.<p>The main problem I&#x27;m coming across is the app I&#x27;m working on is built upon another platform that is using webhooks to communicate to the app. So all of the requests from the platform need to go to a static address (which means I can&#x27;t redirect traffic to a &#x27;cloned&#x27; instance of the app directly).<p>I&#x27;ve thought of building a switching proxy, having the platform point at the proxy address and have a task &#x27;ping&#x27; heroku to see if it is up, and if not, change the endpoint of the proxy to send the requests to a cloned backup server. But building my own switching proxy seems risky in itself!<p>Does anyone know of a solution for this? What do you do?<p>Any help is greatly appreciated!
======
stephenr
How many levels of turtles do you want?

Multiple DNS entries for the hostname (i.e. multiple A/AAAA records) - most
clients should re-try if one fails to load at all, but its not guaranteed
behaviour, particularly if its not a browser.

Each A/AAAA record points to a virtual IP that is used by e.g. keepalived as a
'floating' IP between 2+ servers in that specific DC. Each A/AAAA record
should point to a set of servers in a different DC, and potentially different
provider/host too.

Each server that has this floating IP is a HAProxy node, which forwards
requests to the various backend instances - normally this would balance the
load, in case of issues it would become failover.

~~~
buthow
Thanks for you answer, you have opened my eyes! I didn't think to use the
domain records. 2 levels would be enough considering I didn't know you could
do more than 1 level.

Do you know a good starting resource to set up any of those methods?

~~~
stephenr
So keep in mind, the multiple "levels" are about having redundancy.

So if you make something singular instead of multiple, it becomes a single
point of failure.

The dns part is easy but the most variable as it relies on client behaviour.

haproxy/similar to balance between eg two servers is simple enough, but if you
want haproxy itself fault tolerant you need stuff like vrrp using keepalived
or similar. Thats less easy and is dependent on your hosting provider.

Of course you can also find "hosted for you" resources too. Many bigger
"modern" hosting companies (ie rented virtual servers and hosted services
rather than shared hosting) have some kind of "load balancer" sevice. Eg
aws/google/azure will all have an offering of some kind, as do the likes of
Linode, Digital Ocean etc.

The last two also offer "floating ip" (essential for vrrp - running your own
load haproxy cluster)

I havent checked explicitly but I'd imagine at least some of those hosted
loadbalacers only work with their local networks.

I hope this is clear, happy to discuss it further directly if you wish.

