
AWS Database Migration Service - hepha1979
https://aws.amazon.com/blogs/aws/aws-database-migration-service/
======
dantiberian
The truly interesting thing to me about this product is the pricing.
Competitors in this space sell software starting at ~$5,000 all the way up
into 5, and I presume 6 figures.

AWS is giving away their software and only charging for the hardware to run
this migration software on. The catch is that you need to be migrating to AWS
resources which you are paying for. I'm sure this tool will have less features
than the more established competitors, but for customers that fit the sweet
spot, it will be very attractive.

Disclosure: I know some of the people at
[http://www.dbvisit.com/](http://www.dbvisit.com/), a database migration
company, and competitor to this.

~~~
swehner
The lock-in is where the catch is right.

~~~
matt_wulfeck
There's not a whole lot of lock-in using an rds instance. It's MySQL.

~~~
nkozyra
... with automatically managed updates, backups, monitoring, auto-scaling,
auto-replication ...

The managed services are a huge part of the lock-in with AWS services, because
even though they generally cost more than the competitors, they can often come
out equal after accounting for those aggregate hours lost to doing those
things.

~~~
rubiquity
That's what you call "paying for a service." Lock-In is an entirely different
matter.

~~~
nkozyra
Lock-in happens when migrating would be expensive, too. And leaving a managed
service like this could introduce substantial cost.

~~~
Johnny555
When the vendor lets you export your data in a form that you can easily import
into your own copy of the exact same software, I fail to see how that is
"lock-in". If the vendor increase the cost of the service or changes other
terms in a way that's unfavorable to you, you can easily leave, that's the
opposite of lock-in.

You can't really accuse a vendor of lock in just because his product is priced
lower than it would cost you to do it yourself.

~~~
nkozyra
I think that's a very rigid definition of "lock-in" that implies it must be
done with some anti-competitive malice. That's not how I've encountered the
term in my past, so I'm welcome to other interpretations.

Per (not authoritative, obviously) wiki:

"In economics, vendor lock-in, also known as proprietary lock-in or customer
lock-in, makes a customer dependent on a vendor for products and services,
unable to use another vendor without substantial switching costs."

I agree with this. It shouldn't require that the service is purposefully
preventing you from changing; lock-in can occur because a service offers
advantages that prevent changing by virtue of some other cost.

~~~
Johnny555
"lock-in" implies some sort of lock. "I don't won't to leave because the
service saves me so much money" is not a lock, you _can_ leave, you just
choose not to because it's better for you to stay. And if the vendor increases
his price you can just take your data and go - your same application will run
on another vendor's hosted MySQL instance.

If I say I'll pay you $1 if you sit in my office for an hour, you can't say
that you were "locked in" by me, if Mary offers you $2 to sit in her office,
you can walk over to her office and get more money.

------
grapehut
One of the biggest problems I've had with RDS (postgres specifically) is not
being able to do incremental offsite backups. I wonder if the database
migration service can help with that, as they say the process is "reliable"
and "automatically restarts the process and continues the migration from where
it was halted"

~~~
darkr
You can, you just need to implement it yourself in EC2 (other compute
platforms are available) with trigger-based replication, using something like
bucardo or londiste.

[https://aws.amazon.com/blogs/aws/rds-postgres-read-
replicas/](https://aws.amazon.com/blogs/aws/rds-postgres-read-replicas/)

------
jschmitz28
One feature that I wish was supported in RDS is to migrate from a MySQL engine
to Aurora engine without having to create new database instance at all. It
looks like Amazon DMS makes this process easier, but since Aurora is fully
compatible with MySQL, I don't see why I have to create a new database
instance and point my application to talk to the new server when Amazon DMS
could do this behind the scenes on the one I already have.

Or am I allowed to set the source and target databases to be the one and the
same?

~~~
Xorlev
You can snapshot your RDS database and restore it as an Aurora DB. Only caveat
is it has to be MySQL 5.6.

[http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora...](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Migrate.html)

~~~
jschmitz28
Not quite the same, unless I'm misunderstanding. I'm talking about an online
migration on a single RDS instance with an engine switch handled fully behind
the scenes. The link provided doesn't seem to handle data changes since the
snapshot, and also requires a switch in the application to point to the new
instance which has been restored via snapshot.

DMS gets closer than the above link, I think, since it will handle replication
onto the target database for you. But, for two fully compatible database
engines, it seems like you should be able to make the switch without having to
fuss with replication configuration or spinning up a new instance at all.

~~~
ec109685
The on disk format for Aurua is likely different.

------
dsavinkov
I wonder, if it will be possible for me to "rollback" back to my "plain"
database server if level of service does not satisfy me.

~~~
toomuchtodo
Not unless you're ready to try to create a physical host replica, and then
fail over to it as a master during downtime.

~~~
aidos
Is it possible to run a read replica outside of RDS these days?

~~~
orillian
Funny this showed up on here. I just spent the last 4-5 days playing with DMS.

My company has been using AWS for a while, but management wanted a "backup"
backup off Amazon "JUST IN CASE". Due to amazon blocking replication
credentials for MYSQL servers we had to basically dump over scp to our off
amazon server and run the update on that machine via a script. We tried a
number of different options but none were reliable. All nasty stuff.

Anyway after setting up a dms.t2.medium replication instance, I was able to
create a number of tasks pulling from our amazon server to our off amazon
server.(You have the option to just pull a full dump, pull a full dump and
continue replication or simply replicate data.) It's been running for a little
under 24 hours now and has been, solid so far with the replication. I know not
even a day yet, but it's looking promising. Fingers crossed!

A small bonus to doing the setup for this. I found out the hard way that there
was bad schema in our database, which I spent the last couple days fixing. DMS
is rather sensitive, and will fail and not restart if it encounters to many
errors trying to replicate data.

Overall cost is looking like it's going to cost me about $150 a month for the
replication instance, which is only marginally more than the bandwidth costs I
was incurring doing full dumps to our off amazon server.

Benefits are almost instant replication and an interface that will give me
almost instant feedback on failed replication tasks, all within AWS which is
where we are hosting everything else at this time. I was also able to create
individualized tasks for separate schema so I can watch and manage errors on a
schema by schema basis which is nice.

Overall I'm happy with it, but only time will tell if it can continue to be a
reliable replication option.

O.

~~~
aidos
Thanks for all the info. Is your On Amazon DB on RDS?

We use Postgres so it might be that we couldn't do the same. Presently I run
the DB on EC2 instance(s) but one day I'm sure I'll switch to RDS. Just trying
to understand what an exit strategy might look like in the future.

~~~
orillian
Yes we are running a mySQL 5.6 Database on RDS.

If the Database you want to migrate/replicate from is running on Amazon you
should be able to make it a replication source.

I saw nothing that stated the database HAD to be on RDS. You use the servers
address to setup the endpoints. And provide a database username and password
to connect with so it should be doable.

------
obisw4n
Does anyone know how the MySQL migration & replication works under the hood?
How is Amazon doing all this remotely via just a DB connector?

~~~
gedrap
Typical MySQL replication works when slave is listening to changes on master's
binlog. Which, essentially, is a log of all operations performed on the
dataset. I did some quick googling and apparently it can be accessed quite
easily:
[http://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html](http://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html)
so I'd assume that this is how they are doing the replication, or something
very similar.

As for migration, don't know, you could export the database and listen to
binlog but that will lock table for a bit, depending on the database size. But
maybe that's acceptable.

Would be curious to hear from folks with more DBA experience :)

------
josephidziorek
Why is Amazon Redshift not a target for this service?

~~~
georgewfraser
Syncing to redshift for reporting is a somewhat different use case - you don't
always want to sync everything, you want to sync in larger batches, the
"migration" never ends.

There are several startups that will do this for you, including mine
(Fivetran).

------
eistrati
Finally! This is one of the most wanted functionality that we've waited since
AWS re:Invent announcement!!!

------
Narkov
The "Price and Availability" section doesn't seem to mention anything about
the price?!

~~~
brianwawok
Free!

~~~
Narkov
Not free! They have now updated the post to include pricing details.

------
znep
This has been in beta for a while, the key thing lacking for our uses, at
least with postgres, was full support for DDL.

The lack of "true" replication into postgres RDS instances is one of the
things blocking us from fully using it.

------
Negative1
Hearing a few mixed things and the page doesn't make it clear so can someone
clear it up for me, please; if you wanted to, could you migrate _away_ from
AWS using this service?

~~~
webdestroya
Yes, you could migrate away. As long as either the source or the target is
within AWS, you can use this.

------
jrgoj
What are my chances of a relatively successful SQL Server migration with
several hundred tables and complex views to PostgreSQL using their migration
tool?

~~~
manigandham
I tried this, SQL Server isnt supported yet... its just not even in the
dropdown options. However it is listed in the marketing and product overview
text so it seems like it's coming.

EDIT: looks like it's there now. will try it out tonight then.

~~~
jrgoj
Great! Let me know how it goes. It will be a couple of days before I can try
it. Hopefully I can migrate just the schema and cherry pick some relatively
static tables. We'll see.

------
drewpc
Looks like this could be a better way to migrate between major versions of
PostgreSQL on RDS than rolling your own.

~~~
znep
If you didn't catch it, AWS now supports upgrades between so-called "major
versions" (eg. 9.3 -> 9.4). [https://aws.amazon.com/about-aws/whats-
new/2015/11/rds-postg...](https://aws.amazon.com/about-aws/whats-
new/2015/11/rds-postgresql-now-supports-point-and-click-upgrade-from-
postgresql-93-to-94/)

~~~
drewpc
I didn't catch that, thanks for point it out. Like another commenter said:
this seems to be an in-place upgrade which is not advisable for major
PostgreSQL versions. Additionally, it looks like this involves restarting the
cluster (downtime). The Database Migration Service looks like it wouldn't
require downtime.

