
Amazon Aurora Serverless - tiagobraw
https://aws.amazon.com/rds/aurora/serverless/
======
talawahdotnet
I feel like the choice of the name "serverless" is unnecessarily
confusing/controversial, but I like the concept.

The ability to scale down to zero could certainly be useful for automated
testing use cases.

As with anything on AWS it will take some testing to discover all the quirks
and caveats, but I like the general direction this is headed.

~~~
mrep
Initially, I was responding to agree with you. However, my first renaming
thought was for autoscaling aurora which I realized was incorrect because you
still have to deal with things like server security updates.

Serverless sounds like the better term to me still because it sounds exactly
like what it describes: you never have to deal with a server. Although, I
would actually add auto scaling to the name (serverless autoscaling aurora) so
users know they don't have to deal with servers nor autoscaling.

What do you guys think?

~~~
wmf
You don't really "deal with" servers in regular Aurora, but you do pay for
them. So AFAIK regular Aurora is billed by the hour while "serverless" Aurora
is by the hour but only during hours when it's being used. And serverless
Lambda is billed per request, so it seems like AWS isn't even using a
consistent definition.

~~~
mrep
Don't you? It has been a while since I set one up but I'm pretty sure I had to
set the versions and I think you have to launch 2 servers because they have to
be part of a "cluster". Shit, I remember I was trying to delete them because I
was just experimenting with it to compare with our RDS instances but I
couldn't figure out how to delete them (aurora instance was slave to our
normal RDS instance and the console kept complaining about something when I
tried to delete them after my testing was done. My poor team is probably still
paying for those instances)

------
pritambarhate
This really is a new paradigm shift for relational databases. In general, on
the DB side, you are not able to scale horizontally like application tier. So
for the fear of degradation of service, most of us end up overprovisioning on
the DB side. With Serverless, auto-scaling Aurora, I think things will change
for the DB tier too.

The Serverless section in Aurora FAQ [1] is also worth reading. The main
gotcha I think is:

>> Q: Why isn't my Aurora Serverless DB Cluster automatically scaling?

>> Once a scaling operation is initiated, Aurora Serverless attempts to find a
scaling point, which is is a point in time at which the database can safely
complete scaling. Aurora Serverless might not be able to find a scaling point
if you have long-running queries or transactions in progress, or temporary
tables or table locks in use.

So I think one will need to focus on quick OLTP type queries.

Still, I am quite excited to see how this shapes up.

[1]:
[https://aws.amazon.com/rds/aurora/faqs/](https://aws.amazon.com/rds/aurora/faqs/)

~~~
stephenr
The 'main gotcha' seems like the common thing people always gloss over when
using IaaS.

It solves already solvable problems (albeit with more specialised
tools/people) with a 'magic' solution that works until it doesn't, and then
requires even more specialised people to _maybe_ solve it.

But that's the whole profit model for IaaS: offer 'magic' solutions that are
worse than what a qualified ops team will build, for the same money, to small
companies who dont actually need the bells and whistles, and hope that by the
time they do need the bells and whistles they're locked into your platform too
much to change when they inevitably realise they made a huge mistake.

~~~
braythwayt
Well, there is a general thing where small companies build a thing, and then
they scale to the point where the thing doesn't work any more, and they have
to make some changes.

History has taught me that accepting the inevitability of change is far better
than trying to build a thing with so many features that it need never be
changed.

In this particular case, I find it hard to believe that anybody will be
"locked in" to a serverless SQL database in the same sense that, say, being a
game company on top of iOS "locks you into the Apple Ecosystem," or writing
your app in Rails "locks you into Ruby."

~~~
stephenr
> I find it hard to believe that anybody will be "locked in" to a serverless
> SQL database

I didn't say that did I? AWS/GCP/Azure/etc as a whole, are platforms that
companies become locked-in to, both technically and in terms of mindset.

------
joecot
This is extremely frustrating. One of the biggest use cases for wanting to use
Aurora serverless is connecting to MySQL using lambda functions -- Function as
a Service apps that can still use rdbms. The problem is that RDS endpoints
only exist inside vpcs, and cold starting a lambda function with a vpc network
interface can take 10+ seconds, making it useless for an API. I had hoped
that, given they were making a serverless MySQL service, they'd make sure it
actually plays well with lambda. Nope. Same problems regular RDS has. No
better method of securing the connection beyond vpc firewall rules. Amazon,
it's been a problem since lambda was launched and it hasn't been fixed. Either
fix lambda vpc cold start times, or provide a better way to connect lambda to
RDS. Just burying the problem only pisses off your customers when they try to
buy into your hype.

And since it always comes up:

* Yes, there is plenty of reason to want to use MySQL with lambda. Wanting to run software on FaaS does not mean wanting to abandon rdbms. For a small app, dynamodb is overkill; for a small app that turns into a large app, dynamodb is a money pit.

* No, adding scheduled heartbeat requests to the lambda functions so they never have to cold start is not a real, long term, scalable solution. It's a hack, it doesn't solve the problem if your app actually scales up, and infrastructure shouldn't depend on horrible hacks to function correctly.

~~~
dpeck
I was disappointed to see this as well. I have just recently started to learn
a bit more about lambda and the rds/vpc thing has been a bit of a sticking
point. I had thought aurora-serverless would fix the vpc issue, I appreciate
the autoscaling but I think at this point I’d still rather use rds/Postgres if
it has to be in a vpc anyway. Assuming I’ll save more time using postgresql
advanced features than money on the autoscaling.

I guess everyone is just using DynamoDB with their lambda functions, but I
miss a lot of the power Postgres has.

~~~
rmason
What's missing is support for Cloudformation. Why? Because then someone will
write a plug-in for the Serverless framework. Then using Aurora Serverless
will be simple, no trying to create VPC's as it will be all scripted behind
the scenes for you. There's only a single piece remaining and then Serverless
will totally boom.

~~~
dpeck
That’s only part of it, the long startup time is still there, even with cloud
formation support. You can easily hack it all together with serverless
framework now, and I’m really loving it for learning things but that startup
time is going to be a problem for anything customer facing.

------
red_dinosaur
Assuming this hasn't changed since the preview started:

> The cool down period for scale-down is 15 minutes since the last scaling
> operation. The cool down period for scale-up is 5 minutes since the last
> scaling operation.

~~~
mrep
From what I have read here / experienced personally, google scaling > amazon
scaling > microsoft scaling and each one is an order of magnitude greater in
performance (there is an article on microsoft functions vs Amazon lambda which
gives you the idea and don't even get me started on google tech versus
microsoft/amazon).

However, it is the opposite for sales/marketing/support apparently. It'll be
interesting to see where the market goes towards in the future.

~~~
ranman

        google scaling > amazon scaling
    

???

~~~
mrep
I've read many comments here about how googles load balancers are way more
reactive that amazons as amazons needs "warming" to get them ready. I've also
heard great things about app engines dynamic scaling. Unfortunately, I cannot
find any article purely related to it.

Here is the article about amazons functions being way more reactive than
microsofts:
[https://news.ycombinator.com/item?id=16099729](https://news.ycombinator.com/item?id=16099729)

~~~
NathanKP
Amazon employee here. I can confirm that ALB and NLB do not need prewarming
for most customers.

If you are Epic Games running Fortnite on AWS, and you want to direct millions
of player connections through a brand new, freshly provisioned load balancer
then you should definitely talk to support to make sure the load balancer is
prewarmed and ready for that level of traffic.

But 99.99% of websites and service won't need intervention or prewarming at
the load balancer level because the load balancer can and will scale up far
faster than your backend server provisioning and scaling, or your database
will. You only need to worry about prewarming a load balancer in the very
specific conditions where you are immediately redirecting millions of active
connections over to a new load balancer, and frankly there are very few
companies that have that problem.

Additionally even if you do have that problem Amazon gives you the tools to
solve it without needing any manual prewarming. Any blue/green traffic
switchover at massive scale should probably use a weighted Route 53 DNS record
set. You wouldn't immediately cut 100% of your traffic over to a new load
balancer, instead you should dial it up in percentage increments while testing
and monitoring the new stack. ALB and NLB can autoscale up gracefully and
automatically as you increase the DNS weight on the new DNS record.

------
bastawhiz
I have to say, it's really frustrating that Postgres is a second class citizen
in the AWS ecosystem. Aurora in general is often far behind on Postgres
support relative to MySQL. I hope this changes going forward.

~~~
hasa
Isn't AWS Redshift based on Postgresgl ?

------
dev_dull
> _Pay only for the database resources you consume, on a per-second basis._

> _You pay a flat rate per second of ACU usage, with a minimum of 5 minutes of
> usage each time the database is activated._

Cool idea, but an IO operation once every 5 minutes is considered (billing-
wise) a full time service.

This could unlock a lot of cool potential if it had finer granularity. I can
think of some IoT applications that have have infrequent operations which
would be cost effective with less minimum time.

------
guru4consulting
I had assumed that this would be a HTTPS service with a public end-point,
available outside a VPC. But, looks like it is still deployed inside the VPC
and accessible over JDBC. Did anyone find any configuration option to deploy
it outside VPC? Overall, it looks like good old Aurora RDS with an auto
start/stop option, and costing based on that start/stop duration.

~~~
nahname
Aurora requires a VPC, but you can create public subnets to allow for external
access to the database.

[https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-
public-private-vpc.html)

~~~
guru4consulting
but then, placing the instance in a public subnet is also not a good idea. And
moreover, JDBC connections are not optimized for serverless scenarios.. the
cost of connecting/dis-connecting are too high when compared to a light-weight
HTTP. And any vulnerability can be potentially exploited. I was hoping to see
a true HTTP service, highly optimized for Lambda clients.

~~~
sarvind
The blog post [1] provides additional insights that imply it can work well
clients despite its ephemerality, and is optimized for client connection
pools.

"Scaling operations are transparent to the connected clients and applications
since existing connections and session state are transferred to the new
nodes."

As for the public subnet concern, you can apply security groups to your
database instance to ensure only authorized CIDR ranges can call it.
Additionally, you can use IAM authentication [2] for DB callers (only
recommended for light-weight applications with low concurrency),

[1]
[https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Using...](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html)

[2] [https://aws.amazon.com/blogs/aws/aurora-serverless-
ga](https://aws.amazon.com/blogs/aws/aurora-serverless-ga)

------
paulddraper
AWS, why not PostgreSQL?

The bare minimum cost for Aurora PostgreSQL is $200/month. Aurora MySQL is an
order of magnitude less.

Why not PostgreSQL?

~~~
ranman
It's on the way.

~~~
paulddraper
That's good to hear.

Because Aurora PostgreSQL has already behind MySQL for years. Glad to hear
this isn't a perpetual condition.

------
ciferkey
Very exciting! I made a simple bot to scrape lunch specials menus off
Instagram and post them to my company's slack
([http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-
Bot...](http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot.html)).

One point point was making custom lambas to spin up/down the DB and everything
that was needed to use it (a NAT on EC2, and an elastic IP since the default
NAT aww provides is pricey). The bot only needs to run for an hour on weekdays
so there's no point keeping everything running 24/7\. Excited to try replacing
all of that with this!

------
tgp22
This definitely works from a cost pov. We use a small instance of AWS Aurora
which is mostly idle. If we go this route we will need to see how quick the
cold-start is and how quick the ACU scaling is. Serverless SQL DB - definitely
a nice way to to go.

------
oh_hello
I'm excited for this. I have a few small, low budget projects that would
either require expensive RDS instances or awkward data management to force the
use of Dynamo. Now there appears to be a low cost SQL solution for serverless
projects.

~~~
Cthulhu_
You say low cost, but if I'm reading that article right, one of the lowest
settings (2 availability units) is 12 cents per hour, so if it's being
minimally used 100% of the time, it's still a good $86 per month. You can get
a cheap DB instance or hosting outside of Amazon for a fraction of that. And
it goes up if the DB gets some actual load.

~~~
alexbilbie
If your DB load is consistent and the cost of serverless outweights standard
instance then yes go for a standard instance.

However if your DB load is very spiky then serverless Aurora might be cheaper
than running an over provisioned instance.

~~~
mwarkentin
Just need to give the option to run at 1 unit.. that'd get you in the ballpark
of a normal t2.small server..

------
smokedmeat
Is serverless Aurora available in a multi-az high availability form as well?

~~~
ranman
It is multi-az yes.

------
mooreds
For testing and development databases, would prefer to scale to zero. Weird
that they don't offer that as an option.

~~~
kondro
It does scale to 0 when there are no connections to the DB.

Cold-to-warm takes 25 seconds, but for dev/test this shouldn't be a problem.

~~~
mooreds
Ah, I totally misread this line: "Aurora Serverless can scale from a minimum
of 2 ACUs to a maximum of 256 ACUs. You can specify the minimum and maximum
ACUs your database can consume." I thought it meant you had to always run 2
ACUs.

My bad.

