
AWS Application Load Balancer - rjsamson
https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/
======
encoderer
We plan to do a blog post about this at some point, but we had the pleasure of
seeing exactly how elastic the elb is when we switched Cronitor from linode to
aws in February 2015. Requisite backstory: Our api traffic comes from jobs,
daemons, etc, which tend to create huge hot spots at tops of each minute,
quarter hour, hour and midnight of popular tz offsets like UTC, us eastern,
etc. There is an emergent behavior to stacking these up and we hit peak
traffic many many times our resting baseline. At the time, our median ping
traffic was around 8 requests per second, with peaks around 25x that.

What's unfortunate is that in the first day after setting up the elb we didn't
have problems, but soon after we started getting reports of intermittent
downtime. On our end our metrics looked clean. The elb queue never backed up
seriously according to cloud watch. But when we started running our own
healthchecks against the elb we saw what our customers had been reporting: in
the crush of traffic at the top of the hour connections to the elb were
rejected despite the metrics never indicating a problem.

Once we saw the problem ourselves it seemed easy to understand. Amazon is
provisioning that load balancer _elastically_ and our traffic was more power
law than normal distribution. We didn't have high enough baseline traffic to
earn enough resources to service peak load. So, cautionary tale of dont just
trust the instruments in the tin when it comes to cloud iaas -- you need your
own. It's understandable that we ran into a product limitation, but
unfortunate that we were not given enough visibility to see the obvious
problem without our own testing rig.

~~~
bgentry
I was coming here to ask whether pre-warming is still an issue with the ALB
service. Maybe jeffbarr can comment on whether that's changed?

GCE's load balancer does not use independent VM instances for each load
balancer, instead balancing at the network level. So you can instantly scale
from 0 to 1M req/s with no issues at all.

~~~
bashcoder
You can request pre-warming for additional ELB capacity, when you know far
enough in advance that you will have a spike. AWS customer service responds by
asking 10 clarifying questions via email. The thing is, we can't look under
the hood to see currently provisioned and utilized ELB capacity, so we just
have to trust that AWS engineers will properly allocate resources according to
the answers to those questions. IMO, it's a rather cumbersome process that
would be better implemented as a form.

~~~
jdc0589
its cumbersome, and not remotely useful in this specific situation from OP
anyway.

~~~
Johnny555
It's not clear why it's not useful? The OP said _hot spots at tops of each
minute, quarter hour, hour and midnight of popular tz offsets like UTC, us
eastern, etc._ , so wouldn't he just tell Amazon "Hey, we get xx
requests/second at our peak, so we'd like the ELB ready scaled to handle that
load"?

------
ihsw
Can we agree on the terminology for Application Load Balancer and Elastic Load
Balancer?

* ALB: Application Load Balancer

* ELB: Elastic Load Balancer

I have seen Application Elastic Load Balancer/AELB, Classic Load Balancer/CLB,
Elastic Load Balancer (Classic)/ELBC, Elastic Load Balancer
(Application)/ELBA.

In any event, I think it is great that AWS is bringing WebSockets and HTTP/2
to the forefront of web technology.

~~~
mdani
In my opinion, it is HTTP and WS LB rather than Application LB as it supports
just two protocols. In contrast, if you look at F5 load balancer, it can look
at LDAP packets or Diameter packets and do a L7 load bakancing. So ALB seems
misleading terminology to me as HTTP and WS != All L7 applications

~~~
awj
Judging by how Amazon has handled other things, I wouldn't stick too hard to
that conclusion. They tend to focus first on something generally useful and
easy then come back to fill in more difficult/less popular options over time.

------
tobz
The real question: does this provide a faster elasticity component than ELBs?

At a previous employer, we punted on ever using ELBs at the edge because our
traffic was just too unpredictable.

Combining together all of the internet rumors, I've been led to believe that
ELBs were/are custom software running on simple EC2 instances in an ASG or
something, hence being relatively slow to respond to traffic spikes.

Given that ALBs are metered, it seems like this suggests shared infrastructure
(binpacking peoples ALBs onto beefy machines) which makes me wonder if that is
how it actually works now, because it would seem the region/AZ-level
elasticity of ALBs could actually help the elasticity of a single ALB.

If you don't have to spin up a brand new machine, but simply configure another
to start helping out, or spin up a container on another which launches faster
than an EC2 instance... that'd be clutch.

Deep thoughts?

------
0xmohit
AWS still doesn't support IPv6. Good to see them talking about HTTP/2.

Waiting for AWS to embrace IPv6.

~~~
jeffbarr
That's my post, and my misunderstanding. I'll clean it up now, thanks!

~~~
0xmohit
BTW, is there any place to log feature requests?

~~~
johns
Become a big enough customer

~~~
bashtoni
I run an AWS partner who deals with many large ($millions a year each but not
tens of millions each) aws customers.

It seems like the number of customers asking for a feature is more important
than the size of those customers in my experience.

------
boundlessdreamz
So this is pretty much the same as Google HTTP load balancing
[https://cloud.google.com/compute/docs/load-
balancing/http/](https://cloud.google.com/compute/docs/load-balancing/http/)
\+ websocket & http2?

~~~
manigandham
Google's load balancer does do HTTP/2\. It doesnt have native websocket
protocol support (you have to use the network LB for that traffic) but it does
provide cross-region balancing.

~~~
thesandlord
Websockets over SSL is in beta: [https://cloud.google.com/compute/docs/load-
balancing/tcp-ssl...](https://cloud.google.com/compute/docs/load-
balancing/tcp-ssl/)

~~~
manigandham
Yes, saw that, but it requires setting up a whole new LB just for websockets.
It would be nice if it was just a natively supported protocol on the existing
HTTPS LBs, is there a reason why that cant be done?

------
fred256
+1 for CloudFormation support on launch day. +1 for support for ECS services
with dynamic ports (finally!) -1 for no CloudFormation support for ECS

(To configure an ECS service to use an ALB, you need to set a Target Group ARN
in the ECS service, which is not exposed by CloudFormation)

~~~
chris47493
Looks like you can set the Target Group ARN in the LoadBalancers property
group now
[http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid...](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-
properties-ecs-service-loadbalancers.html)

------
cheald
Exciting! Disappointing that you can't route based on hostname yet, though.
I've got 5 ELBs set up to route to different microservices for one app, and
because we couldn't do path-based routing before, that's all segmented by
hostname. As soon as ALB supports hostname routing, I can collapse those all
into a single LB.

~~~
snug
The "Hostname" is in the HTTP Header, "Host." The article states that you can
route based on Layer 7 headers, so this shouldn't be an issue.

~~~
cheald
Yeah, but they only allow you to route via path matching right now. The ALB
has access to the headers, but configuring it based on arbitrary headers isn't
exposed to end users yet.

I excitedly set up an ALB as soon as I read the post, because I've needed it,
only to find that support for what I want isn't available to me yet!

------
agwa
> 25 connections/second with a 2 KB certificate, 3,000 active connections, and
> 2.22 Mbps of data transfer or

>5 connections/second with a 4 KB certificate, 3,000 active connective, and
2.22 Mbps of data transfer.

"2KB certificate" and "4KB certificate"? Is this supposed to read "2048 bit
RSA" and "4096 bit RSA"?

~~~
creshal
Most likely, yes.

------
indale
This looks pretty sweet. The next big thing for api versioning would be header
instead of url based routing, looking forward to 'give you access to other
routing methods'.

~~~
mattlong
Already has it! "An Application Load Balancer has access to HTTP headers and
allows you to route requests to different backend services accordingly."

Edit: On a second read, it's less clear if header based routing is actually
available yet...

~~~
cheald
It only allows you to route via path matching right now.

"each Application Load Balancer allows you to define up to 10 URL-based rules
to route requests to target groups. Over time, we plan to give you access to
other routing methods."

The ALB clearly has technical access to the headers, but use of them isn't
exposed to users yet.

------
rjsamson
They finally added support for websockets! Really looking forward to giving
this a try with Phoenix.

~~~
bas
I just did a little dance at my desk.

~~~
pgtruesdell
Same here, can't wait to start using WS and HTTP/2 without extra work
involved.

------
daigoba66
These new features are cool... but they still pale in comparison to something
like HAProxy.

I guess the tradeoff is that with ELB/ALB, like most PaaS, you don't have to
"manage" your load balancer hosts. And it's probably cheaper than running an
HAProxy cluster on EC2.

But for the power you get with HAProxy, is it worth it?

Does anyone have experience running HAProxy on EC2 at large scale?

~~~
jdub
ELB _is_ HAproxy. :-) Sure, you get a lot of flexibility configuring HAproxy
yourself, but you also have to run it yourself. 90% of the time it's easier to
just use ELB (plus it has some direct integration with other services, like
IAM-stored server certs/keys, ASG, etc).

I have swapped out ELB for HAproxy and/or nginx on a couple of occasions. If
you know your load and feature requirements intimately, you might be able to
do a better job. But it's work.

------
erikcw
I'm curious if this will Convox to route to multiple services with just a
single ALB instead of the historical default of 1 ELB per service. Would be a
real cost savings for a micro-services architecture.

~~~
bgentry
It should allow Convox to do that, yes. Although it doesn't appear (from the
screenshots) that you can route by Host header, so you'd have to put all your
services on the same hostname with different path prefixes to make it work.

~~~
ddollar
We've got some changes coming along these lines. Looks like we might also be
able to use CloudFront for hostname routing.

------
avitzurel
This is very good. Recently my workflow has been ELB -> NGINX -> Cluster.

Nginx was a cluster of machines that did routing based on rules into the ec2
machines. Now that the AELB has some of those capabilities it's time to
evaluate it.

------
archgrove
Any love for Elastic Beanstalk with these? They seem well matched. Though EB
always feels a bit of a red-headed stepchild in the AWS portfolio.

~~~
bpicolo
I love elastic beanstalk (minus it's mediocre docs). Agreed here, does elastic
beanstalk support setup of new-style elbs (without me doing extensive
customization)? When will it if not?

~~~
jeffbarr
Elastic Beanstalk supports ALB.

~~~
jeffbarr
Small correction to my previous answer, Elastic Beanstalk will support ALB in
the very near future.

~~~
perfmode
Great. I'm looking forward to the day when I can retire my hacky workarounds
to facilitate websockets on Elastic Beanstalk.

~~~
renaudg
Out of curiosity, what are these workarounds ?

~~~
perfmode
TCP load balancing with a modified nginx config that forwards the websocket
upgrade header

------
dblooman
It seems that routing is done in the following way /API/* goes to applications
and expects :8080/api/ rather than the root. Would be nice to have the option
to direct traffic to just :8080.

~~~
brianwawok
What client webserver can't handle this though?

------
axelfontaine
It looks like the big missing piece is auto-scaling groups as target groups...

~~~
azylman
It looks like it supports ECS services as target groups, and ECS services can
do auto-scaling.

~~~
M2Ys4U
Not much help if your services aren't containerised though

------
sturgill
This sentence sums up one of my main reasons for appreciating AWS:

The hourly rate for the use of an Application Load Balancer is 10% lower than
the cost of a Classic Load Balancer.

They frequently introduce new features while cutting costs.

~~~
nivertech
no, they replaced bandwidth costs with new pricing component $0.008 per LCU-
hour[1]. If you have 1M idle websocket connections you will pay 100 times more
for ALB vs ELB (i.e. $2K/mo vs $18/mo).

Good thing ELB is still here, so you can choose between them depending on your
workload.

[1] LCU - Load Balancer Capacity Units

[https://aws.amazon.com/elasticloadbalancing/applicationloadb...](https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/pricing/)

~~~
sturgill
I don't believe classic ELB supports websockets[1] making this a tenuous
comparison. There might be workarounds that I'm not aware of (our production
network isn't currently on AWS so I'm a couple of years behind in my day-to-
day experience with them).

That said, I don't dispute that there might be use cases where classic ELB is
a better option. And I'm glad it's still available (as opposed to ALB
replacing classic).

[1][https://aws.amazon.com/elasticloadbalancing/classicloadbalan...](https://aws.amazon.com/elasticloadbalancing/classicloadbalancer/faqs/)

~~~
nivertech
it supports any TCP-based protocol, like websockets or MQTT, similar to
Network LB on GCE.

------
shawn-butler
Anybody know whether the new ALB handles a client TLS (SSL) when operating in
http mode?

I was trying secure an API Gateway backend using a client certificate but
found ELB doesn't currently support client side certificates when operating in
http mode.

There was this complicated Lambda proxy workaround solution but I gave up
halfway through...

[https://aws.amazon.com/blogs/compute/using-api-gateway-
with-...](https://aws.amazon.com/blogs/compute/using-api-gateway-with-vpc-
endpoints-via-aws-lambda/)

------
kookster
As a heavy ECS user, all I can say is thank you, finally!

~~~
leetrout
What are you using for configuration / orchestration of both the container
servers and the tasks? CF? Terraform?

~~~
nzoschke
Take a look at Convox as a fast way to bootstrap managing instances, ECS, and
load balancers via CloudFormation.

We have ALB working already.

[https://github.com/convox/rack/pull/1045](https://github.com/convox/rack/pull/1045)

Disclaimer: I work on Convox.

------
renaudg
I'm the process of containerizing an app that includes a Websockets service,
and given ECS / ELB limitations we'd just decided to go for Kubernetes as the
orchestration layer.

This ALB announcement + the nicer ECS integration could tip the balance
though.

Any thoughts on how likely it is that Kubernetes can/will take advantage of
ALBs (as Ingress objects I suppose) soon ?

------
nodesocket
Do ALBs support more than a single SSL certificate?

~~~
pat2man
Like via SNI? No mention of it and the screenshots make it seem unlikely.

~~~
pgib
The Amazon Certificate Manager uses SNI, and you can request certificates with
multiple hosts and even wildcard domains. I would imagine if you upload your
own multi-domain certificate that it would work in the same way, but I have
never tested that.

~~~
imperalix
I think you mean SAN instead of SNI. SNI is like host headers for TLS
connections, while SAN on certs allow you have to very valid for multiple
names.

------
manishsharan
This is definitely nicer than having to create subdomains for microservices
and mapping each subdomain url to its own Elastic Loaad Balancer + Elastic
Beanstalk instance. But I have already gone down this path so I am unlikely to
use AWS Application Load balancer. I wish I had this option a year ago.

------
nailer
Nice haproxy / nginx alternative. It's got http2 support though which puts it
ahead of haproxy.

~~~
KenCochrane
I wouldn't consider this a full haproxy/nginx replacement just yet. It doesn't
support host based routing, so you would need a different ALB for each host in
order to get the same thing you would get with haproxy/nginx.

------
DonFizachi
Any idea if sticky TCP sessions will be supported on ELB/ALB any time soon?

------
amasad
I wonder if they fixed the routing algorithm for TCP connections. It's round-
robin on ELB, which is performs terribly for long lasting connections.

------
nodesocket
So what would be a use case for using ELBs now? Seems like ALBs do everything
ELBs do, but with websocket and HTTP/2 support.

~~~
kronin
If your security requirements are such that AWS can't terminate TLS would be
one reason.

~~~
caleblloyd
Also if you're load balancing any other application protocol other than
HTTP/HTTPS. (e.g. SMTP load balancer)

------
joneholland
Disappointing. I was hoping they were launching a service discovery stack to
compliment ECS.

------
bradavogel
Does anyone know if it (finally) supports sticky websocket sessions?

~~~
zob_cloud
Yes they do.

For a single connection the websocket will always go to the same back end
regardless of sticky sessions being enabled.

If stickyness is enabled, and the same client creates a new websocket, it will
go to the same back end as previous connections.

~~~
iEchoic
This isn't working for me, I'm not getting an upgrade header back from the
server, it just stalls. Hm.

Did you have to do anything special to get this to work?

------
merb
Virtual Host Load Balancer would be great.

------
NeckBeardPrince
Any idea if it's HIPPA compliant?

