
Amazon’s Consumer Business Turned Off Final Oracle Database - jeffbarr
https://aws.amazon.com/blogs/aws/migration-complete-amazons-consumer-business-just-turned-off-its-final-oracle-database/
======
AtlasBarfed
Amazon is a slightly annoying company that I rarely root for, but Oracle is a
predatory company whose customer relations strategy seems to actively involve
litigating them as a default assumption.

Amazon is considered a paragon of IT, so their expulsion of Oracle is a useful
tool at the CIO/CTO level of "nobody ever got fired for IBM" level of
reasoning.

Unfortunately, AWS is becoming the new IBM in that reasoning. Great that it
took ten years to reach that level of agreement, now hybrid cloud needs to
take over the grey haired CIO / CTO level.

Long live Postgresql!

~~~
gopalv
> Amazon is a slightly annoying company that I rarely root for, but Oracle is
> a predatory company

Amazon and Oracle couldn't be more different in this point of view.

Amazon's predatory instincts are pointed inwards and downwards, towards
employees and vendors, while being mostly great towards the customer.

Everything from their warehouse workers with timers, the crying software
engineers and all other things I hear about them are about running an
internally facing extraction economy.

The squeeze is being put on those who draw a paycheck (or at least, a profit)
from Amazon, while the consumers see instant access servers on EC2, Amazon Go
for lunch breaks in a city or One hour shipping of diapers.

In comparison, Oracle seems to be going after those who pay them already.

~~~
throwaway981211
I’m an engineer now at Amazon. I’ve been here 7 years. 5 years ago, our
culture was far more toxic. More recently, some orgs are horrible places to
work and others are much better. I don’t think we’re much worse than other
companies. Our pay for engineers is less than Google and Facebook, as far as I
know, but we beat other companies such as MSFT.

Amazon naturally attracts people who want to accomplish big things. It just
happens that many of these folks are assholes.

Is Amazon predatory to its employees? Some organizations are - yeah
absolutely. Are we worse than other companies I worked prior to Amazon? No.
Does that mean it’s okay? No. I’m just saying there’s a public perception of
Amazon being at the deepest level of hell and it’s not true.

We absolutely burn people out. We overwork employees. We don’t promote them
when we should. We try to find the tiniest bit pick reasons not to promote. We
practice stupidity and call it frugality? You’re an engineer making $200K
total comp a year? Your computer doesn’t work and is crashing? Oh too bad - we
won’t replace it for you. Yeah I mean that kind of fucking stupidity.

We encourage everyone to automate their role so we can replace those folks
with college hires.

Amazon is not a place you can have a long career at. You’ll have to jump teams
or orgs every couple years and eventually you’ll get tired enough that you
learn to manage upwards yourself and stick with a manager that you can manage.

~~~
AtlasBarfed
Sooooo.... it's not toxic anymore, but you still need to hop teams like crazy
or they kick you to the curb?

Your description sounds like my friends who worked there or just left, they
were used for 2-4 years and then nitpicked to deny the deferred compensation
stock. It's a rolling scam! But so is the rest of the new economy.

I'd hate to know what it was like before.

Interesting that you said "it's being fixed" traces right around the time the
various worker stories went viral. And yet you and several other people say
"it's not bad!".

I get there's no stats and its all anecdotal and situational, but the very
experience of being hired is terrible. Even though I had friends there, I
never bothered interviewing. Why waste two days?

At least google has free food. I'd waste an interview with them.

~~~
VRay
It was basically as that guy described when I was there through 2015, but it
really wasn't that bad. I saved enough money in the bank to quit and live off
it for 6 months, and after that it was less like an oppressive life and more
like a game I was playing.

I'll tell you what was REALLY stressful: Trying to find my first engineering
job after graduating from college. It took almost 9 months, and I ended up in
a really shitty low-paying internship. The whole time bills I couldn't pay
kept piling up, and my best friend in the world (my cat) ended up passing away
partly because I couldn't afford to pay for any more surgeries after putting
thousands of dollars' worth on my card.

If you can land a job at Amazon, though, congratulations: You've made it. You
can get a programming job anywhere in the world and make money in the 95th+
percentile. You've got health benefits, a 401k plan, life insurance, etc. If
you ever get so burned out that you're crying at your desk, just quit your
job, sell your shit, break your lease, and backpack around the world for a
year. (Or go get a laughably easy ~$100k/year job out in the Midwest)

------
irrational
We've been working on migrating from Oracle to Postgres for a few years now.
We are about 2 weeks from being finished. It is not for the faint of heart,
but it is totally worth it. The documentation is much much better, performance
is equivalent or better, the sql dialect is saner, etc. Other than moving the
data itself (ora2pg was invaluable for this), rewriting the queries is what
has taken the most amount of time. Some of our tips on differences between
oracle and postgres sql:

replace nvl with coalesce

replace rownum <= 1 with LIMIT 1

replace listagg with string_agg

replace recursive hierarchy (start with/connect by/prior) with recursive

replace minus with except

replace SYSDATE with CURRENT_TIMESTAMP

replace trunc(sysdate) with CURRENT_DATE

replace trunc(datelastupdated) with DATE(datelastupduted) or
datelastupdated::date

replace artificial date sentinels/fenceposts like to_date(’01 Jan 1900’) with
'-infinity'::date

remove dual table references (not needed in postgres)

replace decode with case statements

replace unique with distinct

replace to_number with ::integer

replace mod with % operator

replace merge into with INSERT ... ON CONFLICT… DO UPDATE/NOTHING

change the default of any table using sys_guid() as a default to
gen_random_uuid()

oracle pivot and unpivot do not work in postgres - use unnest

ORDER BY NLSSORT(english, 'NLS_SORT=generic_m') becomes ORDER BY
gin(insensitive_query(english) gin_trgm_ops)

Oracle: uses IS NULL to check for empty string; in postgres, empty string and
null are different

If a varchar/text column has a unique index a check needs to be made to make
sure empty strings are changed to nulls before adding or updating the column.

PostgreSQL requires a sub-SELECT surrounded by parentheses, and an alias must
be provided for it. - SELECT * FROM ( ) A

any functions in the order by clause must be moved to the select statement
(e.g. order by lower(column_name))

Any sort of numeric/integer/bigint/etc. inside of a IN statement must not be a
string (including 'null' \- don't bother trying to use null="" it won't work).
Concatenating a NULL with a NOT NULL will result in a NULL.

Pay attention to any left joins. If a column from a left join is used in a
where or select clause it might be null.

For sequences, instead of .nextval use nextval('')

~~~
JacKTrocinskI
I have a hard time believing PostgreSQL performance is better than Oracle, can
you give an example? How big is your database and how much did migrating cost
you?

~~~
stock_toaster
Think of how much you can spend on beefy hardware though, if you go with
PostgreSQL, versus spending that money on Oracle licensing.

~~~
adwf
Yep, it's surprising how much performance you can get when you're not paying
Oracle millions for licensing.

A lot of people who think Oracle is amazingly fast, think so because who puts
a million dollar DB on thousand dollar hardware? Put any major DB on
equivalent hardware and you'll get some pretty damn good performance.

------
the_duke
The video feels like a big middle finger to Oracle.

"stop oracle", and the URL
[https://aws.amazon.com/products/databases/freedom](https://aws.amazon.com/products/databases/freedom).

Not that I'm complaining, Oracle deserves all the hate they get for their
business practices.

But associating AWS with "breaking free" is extremely ironic. It's just as
much of a source for vendor lock in as Oracle.

If you are not extremely careful/dedicated with your system design, you are
never moving off of AWS ever again, or only with a huge engineering effort
that almost no-one will swallow. AWS also knows this very well: discounts are
harder to come by if they can tell you are totally locked in to their
infrastructure.

Combine that with the fact that AWS loves to fork OS projects and build
products out of them, without contributing back upstream.

AWS is a terrific platform in many ways, but it is certainly not something
that will lead to freedom of choice.

~~~
jpalomaki
This is kind of reply to Larry Ellison's comments [1] about how Amazon can't
run their business on their own databases.

[1] [https://youtu.be/xrzMYL901AQ?t=50](https://youtu.be/xrzMYL901AQ?t=50)

~~~
throwaway981211
I work at Amazon. Ellisons assessment is trash meant to fool maybe Wall Street
analysts, but as an engineer, his claims are laughable. In reality, database
migrations are hard. Removing that factor entirely for a moment, engineers
don’t want to work on lift and shift projects. They come to Amazon to build
interesting shit. Now imagine database migrations are easy and your engineers
love this work - well, now it turns out you’ve basically stopped real
innovation in your organizations to do this operational work for cost savings.

I could go on... there are so many factors why it took Amazon a while to get
off Oracle. Ellison is a heaping pile of dog shit fraudster. Before you
dismiss me as being biased, look at my post history. I’m critical enough of
Amazon.

------
orf
> Across the 48 hours of Prime Day, these sources made 7.11 trillion calls to
> the DynamoDB API, peaking at 45.4 million requests per second.

> The team added an additional 63 petabytes of storage ahead of Prime Day; the
> resulting fleet handled 2.1 trillion requests per day and transferred 185
> petabytes of data per day.

> Prime Day 2019 also relied on a massive, diverse collection of EC2
> instances. The internal scaling metric for these instances is known as a
> server equivalent; Prime Day started off with 372K server equivalents and
> scaled up to 426K at peak.

Woah.

[https://aws.amazon.com/blogs/aws/amazon-prime-
day-2019-power...](https://aws.amazon.com/blogs/aws/amazon-prime-
day-2019-powered-by-aws/)

~~~
apocalyptic0n3
The largest scale I've ever needed to work at was 2 EC2 instances (really just
1 for incoming and 1 for background SQS job handling). The numbers in that
post are simply staggering to me.

------
tootie
I wonder if anyone here has seen any net new adoption of Oracle DBs in the
last few years? It was an absolute juggernaut 10-15 years ago and has rapidly
tanked their reputation and spread their business across software, services
and cloud with mixed results. Is it actually still growing or just milking
their stodgy enterprise customers for more money?

~~~
drawkbox
In 2001/2003, I led a small team at an agency that converted Kraftfoods and
50+ brands from 25+ ATG Dynamo web/app CMS servers + 20+ Oracle servers to
.NET 1.0 CMS on 3 web servers, 3 app servers and 3 Microsoft SQL Servers.

Even then Oracle tools were horrible for developers and still are. The only
reason they had that many databases was Oracle's business is licensing,
massive, expensive licenses, and their sales pushed them sold through upper
management. Those days are almost over, developers choose the tech, the cloud
is here. Without Tom [1], Oracle was basically not even a great developer
community, all documentation was bad and the Java interfaces on the tools /
management studio were horrendous. Oracle is fast but for most things on the
web caching comes into play heavily and other databases are good enough.
Oracle is good for large datasets/reporting, cursors are extremely fast when
Oracle is tuned, the problem is it is like vehicle that needs tuning and
optimization all the time or it drives bad. The cost and upkeep make Oracle
mostly overkill and too painful.

Ellison laughed at the cloud like Ballmer laughed at the iPhone. That was a
major error. But ultimately Ellison is an engineer that forgot to make sure
developers like to use the product. That will ultimately be their fatal flaw
as now the developers choose the tools not what is sold in above them.

All other platforms are developer centric and cloud first now. Oracle the all
seeing one says the end is nigh for Oracle the database. No new projects will
be using it without force. Too bad Oracle also owns MySQL, Java etc. I really
wish someone else had bought Sun.

[1] [https://asktom.oracle.com](https://asktom.oracle.com)

~~~
Mountain_Skies
Oracle's new licensing requirement for Java is a bit threat to that platform.
My company for technical reasons can't switch to OpenJVM so we're going to pay
Oracle their new licensing money but we're not doing any greenfield Java
development and are going to work towards moving our Java applications to
another platform. That's an n=1 example but wouldn't be surprised if others
end up migrating away, causing longterm harm to Java.

~~~
chrisseaton
> My company for technical reasons can't switch to OpenJVM

The OpenJDK (not 'OpenJVM', that's not a thing) is literally 100% the same
build as Oracle JDK now. There can't be any technical reason.

~~~
toyg
It might be the magic "support" word - random openjdk builds don't bring any
commercial guarantees.

This said, the likes of RedHat, Amazon and Azul are pretty happy to offer
those guarantees, and I bet their prices are lower than Big Red O.

~~~
thenewnewguy
Yes, but that wouldn't be a 'technical' reason, would it?

------
LrnByTeach
Here is the meat of the Report:

>We migrated 75 petabytes of internal data stored in nearly 7,500 Oracle
databases to multiple AWS database services including Amazon DynamoDB,Aurora,
RDS, and Redshift.

>We kept careful track of the costs and the performance, and realized the
following results:

>Cost Reduction – We reduced our database costs by over 60% on top of the
heavily discounted rate we negotiated based on our scale. Customers regularly
report cost savings of 90% by switching from Oracle to AWS.

>Performance Improvements – Latency of our consumer-facing applications was
reduced by 40%.

>Administrative Overhead – The switch to managed services reduced database
admin overhead by 70%.

~~~
e12e
> Cost Reduction – We reduced our database costs by over 60% on top of the
> heavily discounted rate we negotiated based on our scale. Customers
> regularly report cost savings of 90% by switching from Oracle to AWS.

So Amazon saves 60%, when switching to paying "cost" for Amazon services - and
they still claim customers paying cost+markup save 90%?

I'm sure there's probably savings - but that math sounds optimistic.

Or more likely they're apple to oranges - porting a service that could run on
a single postgres node from oracle to anything will save you maybe 10k USD a
year in licensing for which you see a pretty slim return.

~~~
kondro
Oracle Enterprise (RRP) is $47,500 per unit. Software updates are $10,500 per
unit per year.

An Intel core is 0.5 units.

Running Oracle on single, 4 core CPU (or VM, because you can't partially
license cores and you'd be lucky to find a 4 core server CPU) is $40,000 per
year in licensing alone (amortised over 5 years)… for a single machine.

Want an active/passive replication set… double that cost.

~~~
fullstop
That's why we switched years ago. Going from single CPU to multiple ballooned
the costs so much that it was economically viable to do the work to port
everything to Postgres.

------
jnaina
Here's the key difference between Oracle and AWS:

\- Oracle will schedule periodic audits on their customers and legally
threaten them for inadvertent licensing oversights

\- AWS makes it a mandatory policy that all their sales folks periodically sit
with their customers to help optimise and reduce their ongoing AWS Cloud costs

If you are making a case that Oracle == AWS, you don't know what you are
talking about. Yes, I have dealt with both these companies up close.

~~~
philliphaydon
> AWS makes it a mandatory policy that all their sales folks periodically sit
> with their customers to help optimise and reduce their ongoing AWS Cloud
> costs

Their KPIs are to reduce costs for customers but also to increase usage of AWS
services to make more money. The KPIs seem to contradict each other.

~~~
jon-wood
I've been in some of these meetings, and strangely the KPIs don't actually
contradict themselves. Most of the recommendations we were given were around
switching workload from one way of running things to a service which would be
a better fit, and because we'd be using it as intended ended up being cheaper
than the original implementation.

------
mjw1007
Amazon's leadership famously pushed hard in the early 2000s for all its
systems to share data only via service interfaces (
[https://gist.github.com/chitchcock/1281611](https://gist.github.com/chitchcock/1281611)
).

I wonder how much of the reason for that was actually a wish to avoid getting
further locked in to Oracle (changing or removing a database that backs a
single service is very much easier than changing a database that also acts as
an integration point).

~~~
discodave
If the goal of SOA was to reduce Oracle lock in then it utterly failed. After
SOA practically every team had an Oracle db, so now they have to convince tens
of thousands of engineers to migrate thousands of databases.

Imagine what Oracle is going to try and charge Amazon per month for every
remaining DB given how publicly hostile Amazon has been.

~~~
cloverich
I think you missed the point slightly. If the interfaces between these
services are now Oracle agnostic, it makes migrating off Oracle (by mandate,
if necessary) more feasible. More pointedly, I struggle to imagine a scenario
where using Oracle of e.g. Postgres would be meaningful in any sense outside
of the specifics of Oracle's out of the box integrations. If they aren't being
used, why would anyone use Oracle at all?

------
darksaints
It should be noted that the email that Steve Yegge made famous was:

1) The birth of the service oriented architecture as a standard at Amazon

2) Took place long before AWS was used by Amazon's commercial business, and
most software systems were monolithic, requiring mainframe-like dedicated
hardware.

3) Was inspired due to the epic spaghetti monster known as FC Software, whose
entire non-architecture revolved around a massive Oracle database.

When Amazon hit the limits of scaling the Oracle databases with beefier
hardware, they realized what a clusterfuck they had created. There were
literally hundreds of applications, doing different things, but all connected
to the oracle database. The oracle database functioned as a database, but also
as a high throughput queue, as well as a message passing platform. At high
loads, different applications were hitting the same rows so frequently that
the transaction errors alone could bring down an entire fulfillment center.

And it should also be noted that it was the same FC Systems (the inspiration
for the move towards SOA) that took 12 years to finally get rid of Oracle.

Moral of the story: you need competent software architects, and at the very
least, a scalability plan. Also, fuck Oracle.

~~~
zzzcpan
_> The oracle database functioned as a database, but also as a high throughput
queue, as well as a message passing platform. At high loads, different
applications were hitting the same rows so frequently that the transaction
errors alone could bring down an entire fulfillment center._

Isn't that how everyone uses Oracle? I've seen exactly the same complains from
developers at banks. Although building software around big featureful RDBMSs
in general encourages poor architectures and messy software.

~~~
uxp
I've seen the same architecture implemented in MSSQL and MySQL. It's pretty
easy to paint yourself into a corner where your one database is doing all
sorts of stupid things because you never sat down and identified the problem
space in the larger context and moved to a wider configuration instead of
simply using a table with short-lived rows as a queue. It's the same arguments
about nails and hammers.

------
samfriedman
In my mind the true "hard part" would be in migrating business logic that
lives in Oracle stored procedures (PL-SQL). Are there any tools that do a
decent job in translating procs to non-Oracle dialects?

~~~
metadata
Yep, we built SQL Tran to do exactly that.
[https://www.spectralcore.com/sqltran](https://www.spectralcore.com/sqltran).
Finishing the rewrite right now which allows us to make changes much faster.

~~~
holykin
When is it out of Beta?

~~~
metadata
In a few months, it's not so far away. If you can share your database code
(data is not important), we can make sure that our parsers and translation
layer work with all the code constructs you use from day 1.

------
TrueDuality
Wow! Congratulations to the engineers. Oracle digs in deep. Bet that is a
massive headache off a lot of people's plate.

------
jedieaston
Is this not a case study that anyone can leave Oracle now, if an Amazon scale
install can?

~~~
mumblemumble
It is not.

Amazon accomplished the task by building their own Oracle replacement. You
have to be approximately as large as Amazon for that to be a realistic option.

If you'd like to use the alternatives that Amazon built, you can, of course,
migrate to Amazon Web Services. Though, if you dislike the vendor lock-in
you're experiencing with Oracle, and are considering Amazon as a good place to
flee to, you possibly haven't been paying attention. With Oracle, you're just
renting the software. With Amazon, you're renting both the software _and_ the
hardware, and renting the hardware is a precondition of being able to continue
using the software, AND they charge you a fee to move your data off of their
hardware. It's one of the most well-engineered enterprise vendor lock-in traps
the industry has ever seen.

~~~
cwyers
Most places likely aren't using Oracle at Amazon's scale, either.

I suspect that one of MySQL, Postgres or Microsoft SQL Server can fill in for
Oracle if you have access to the source code and can make changes yourself.
(If you've bought software that is Oracle-only, then it's not an option -- but
it wasn't an option for Amazon, either). I suspect that in a lot of cases,
this will pay for itself, but it might take 10 years to do so.

~~~
bilbo0s
> _Microsoft SQL Server_

Out of the frying pan, into the fire.

~~~
mumblemumble
MSSQL's really not too bad. Yes, it's a commercial offering, but there's
nothing inherently evil about charging money for software, and you do get what
you pay for.

I've worked in both MSSQL shops and PostgreSQL shops, and, while I wasn't in
charge of the budget, I wouldn't be at all surprised to find out that, once
you factor paying your programmers and DBAs for their time into the equation,
the MSSQL shop ended up having lower overall costs associated with its choice
of DBMS.

~~~
kirbypineapple
I'm not sure I follow this logic. Wouldn't both databases require the same
amount of care and feeding? Are you suggesting that you need less DBA's per
MSSQL database than for a PostgreSQL or Maria database?

~~~
mumblemumble
Yep. Because the tooling is stronger. The developer ergonomics are better. The
performance is better, meaning you'll be able to do more with fewer servers.
MSSQL has a boatload of useful features that PostgreSQL lacks. (Clustered
indexes and table-valued parameters are the ones I like to fixate on.) And so
on. I can only personally speak to the developer story, but my experience has
been that the productivity difference is huge. There's a night and day
difference in the information you can get out of an EXPLAIN on each of them,
for example, and I would get detailed emails warning me if a MSSQL query plan
unexpectedly changed in production. With PostgreSQL you're more likely to need
to build your own tooling for that sort of stuff, or just wait for a user to
complain. On the DBA side, I'm hazier on the details, but I've seen PostgreSQL
DBA's sinking hours' or sometimes even days' worth of effort into grinding
through tasks tasks that an MSSQL DBA can bang out by spending a couple
minutes clicking through a wizard dialog.

If you're a tiny startup, the differences won't amount to all that much, and
it will be hard to justify MSSQL's price tag, which is fine, because, yeah,
you can't afford it, anyway. If you're an enterprise or otherwise have large
database needs, the differences can very quickly become apparent, and easily
worth the money.

~~~
fanf2
Instead of a table-valued parameter, wouldn’t you use an array of a composite
type in PostgreSQL? What are the relative good and bad points?

~~~
philliphaydon
Yeah. And you don’t need to create a custom type to handle it as a parameter.
So it’s easier.

The only thing SQLServer has over PostgreSQL is multiply resultsets. That’s
it. I really wish that existed in PostgreSQL to prevent round trips.

------
IronWolve
Oracle goes after customers with 3rd party companies, if you use oracle expect
calls for audits and more licenses. You download virtualbox extension at work?
Thats a fee per user. Use an Oracle VM in a vmware cluster? They want paid for
all cores in your environment now.

Seriously, Oracle goes after their customers like criminals. I refuse their
calls at work, they really are just looking for something to make you pay.

~~~
toyg
_> Oracle goes after customers with 3rd party companies_

That's going to end soon - the partner ecosystem has been royally shafted by
Oracle's big "cloud push", people are moving away from a vendor who simply
sold them down the river; and in their Brave New Cloud World, there is no need
for license resellers and auditors.

Ironically, that might end up improving Oracle's reputation - they'll just be
seen like Google, where everything is automated and support goes from mediocre
to nonexistent.

------
CydeWeys
Migrating everything to the same externally-available cloud infrastructure is
exactly the thing that Google needs to do, but has been failing at for going
on a decade now. Yes, I'm bitter about it.

~~~
mattlondon
Is it not the case that it is actually the opposite? That GCP is running on
the internally-available cloud infrastructure at Google?

How would they turn that inside out? What would GCP actually run on if they
ran GCP on GCP? Or do you mean run search/gmail/youtube on GCP? Isn't their
internal infra basically kubernetes in all but name?

~~~
kenforthewin
I've heard that Google is bad at dog fooding their own cloud infrastructure
and even that Borg is still dominant over Kubernetes in their internal infra
but don't have any insider knowledge. If true this is one thing that Amazon is
vastly superior than Google at.

~~~
pradn
Internal teams are using some GCP services, but overall nearly all teams stick
with Borg. Many key services like Search are not able to switch to GCP b/c
performance degradation.

------
growlist
The video strikes me as an epic troll:

[https://www.youtube.com/watch?v=9yBP5gnnZi4&feature=youtu.be](https://www.youtube.com/watch?v=9yBP5gnnZi4&feature=youtu.be)

~~~
jeffbarr
That was the actual shutdown. I was there, it was not rehearsed, and there
were no retakes.

~~~
growlist
Sorry I wasn't saying I thought it was faked or something like that, in case
that wasn't clear. I was saying that posting a video celebrating removing a
vendor's software from your organisation could be seen as taunting them. Which
- given that it's Oracle we are talking about - is pretty funny.

------
victor106
> The migration gave each internal team the freedom to choose the purpose-
> built AWS database service that best fit their needs, and also gave them
> better control over their budget and their cost model.

Anyone know of a comprehensive resource that we could use to figure out which
DB to use?

~~~
jl6
Here’s an algorithm which I believe to be 95% accurate:

If only a single user/device needs to access the data: use SQLite. In all
other cases: use PostgreSQL.

~~~
haggy
Not sure how much sarcasm is built into your statement but as a long time user
& supporter of Postgres I have to disagree. Postgres is fantastic at many
things but there are times when it's not the best choice. An example would be
high-scale OLAP. I've tried to use PG for those cases but ingesting thousands+
of events per second into PG and then trying to perform rollups on them in an
online capacity (near real-time availability) requires a TON of extra legwork
to get it going and near continuous maintenance thereafter. Other purpose-
built DB's for OLAP such as Druid, Clickhouse, etc are much better suited for
this type of use-case. The main advantage these systems have over Postgres is
their column-oriented nature vs. row-oriented of PG and other RDMS.

~~~
wgerard
> Postgres is fantastic at many things but there are times when it's not the
> best choice.

To be fair, that would be the 5% :).

------
yannis7
wasn't Larry Ellison last year who was bragging that the whole of Amazon/AWS
runs on Oracle?

~~~
paulddraper
To be fair, AWS does run on Java.

~~~
kondro
Which is being migrated to AWS Corretto (if it hasn't been already).

------
nojvek
This is Jeff Bezos calling out Larry Ellison on his bullshit.

[https://www.youtube.com/watch?v=xrzMYL901AQ&feature=youtu.be...](https://www.youtube.com/watch?v=xrzMYL901AQ&feature=youtu.be&t=50)

------
throwaway981211
What that article doesn’t capture is how many engineers absolutely burned out
working against unreasonable deadlines and cut throat managers.

------
godson_drafty
The developer carrying it out still using 2015 MBP

~~~
hanniabu
My 2011 MBP is still my primary. This is the case for a lot of people because
the newer versions just don't compare. On to of that you can't even upgrade
the components in them since everything is integrated into the board.

~~~
alexvoda
In my opinion the best user friendly mac was the unibody early 2009 15"
MacBook Pro. It was the last Mac with a removable battery. Plenty of
technological improvements have been made since then but I still love that
model.

------
iRobbery
"This team moved 40 TB of data with just one hour of downtime, and realized
the same or better performance at half the cost, powered by Amazon Aurora."

Would have loved to have read some technical details on how 40 Tbyte was
transferred in an hour, even migrated.

~~~
Havoc
>Would have loved to have read some technical details on how 40 Tbyte was
transferred in an hour, even migrated.

Perhaps I'm missing something but that doesn't seem hard considering:

1) It's only like 3x high end consumer level 16TB HDDs

2) This is Amazon...they have AWS - really fat pipes and endless horsepower on
demand

~~~
kondro
Have you tried transferring 40TB from 3x spinning disk HDD… let alone an
actively running DB infrastructure?

The maximum _sustained_ transfer rate of a 16TB Seagate Exos X16 is 261MB/s.
That would take 17 hours to transfer at full speed.

Not that the data storage would look like that, but even at 100Gbps networking
operating at 80% (about what you'd achieve realistically) it would take 70
minutes to transfer 40TB at full wire speeds.

But we're not talking about full-wire speeds. We're talking about exporting
from an Oracle DB and importing to an entirely different DB. The file transfer
task isn't the hard bit, it's the actual data migration along the way. That's
a massive undertaking and I'm surprised any of the migration tasks were able
to complete with such a small amount of downtime.

------
TheKarateKid
As someone who has never used or worked with Oracle DBs, can someone explain
why migrating to another technology is so difficult?

With all the database engines available, is there really not another one that
can match Oracle without massive customizations?

~~~
ajkjk
Oh man. I used to work at Amazon at a time when this Oracle deprecation was
starting up. It's utterly non-trivial.

It's not really "switching relational DBs", it's "rebuilding all of the legacy
systems to not use relational DBs". Their scale had long since left the realm
where it was reasonable to run everything on massive monolithic DBs, but the
technical debt hadn't been paid down -- so they just kept putting more money
into bigger and bigger hardware and larger DBA teams to support them. For
instance when I last worked the there each warehouse ran on a single massive
Oracle DB. That isn't solved by swapping to another database; it was well
beyond reasonable.

I suspect that this article seems like a bit of self-promotion but it's
actually intended as a giant middle-finger to Oracle.

~~~
philshem
> but it's actually intended a giant middle-finger to Oracle

When they were all cheering, I actually felt bad for Oracle for a nanosecond.

~~~
tomashubelbauer
That sounds way too long. I think you should get that checked out.

------
mxschumacher
Will be interesting to see whether Amazon can make their insight into a
service. Countless companies would be glad about a migration path from Oracle
to AWS's solutions.

~~~
fooblitzky
Maybe this?
[https://aws.amazon.com/dms/?nc2=h_ql_prod_mt_dbms](https://aws.amazon.com/dms/?nc2=h_ql_prod_mt_dbms)

------
tyingq
Is "Amazon's Consumer Area" deliberately specific? That is, are they still
using Oracle somewhere else? (Despite 75 petabytes being significant)

Edit: I did see _" some third-party applications are tightly bound to Oracle
and were not migrated"_. Curious if that's all that is left, globally, at
AMZN.

~~~
sillypuddy
There are Oracle AWS offerings -
[https://aws.amazon.com/oracle/](https://aws.amazon.com/oracle/)

~~~
tyingq
Good example of what I'm asking for. Does AMZN use any Oracle RDS, or other
Oracle offerings, aside from what they just retired?

------
simonebrunozzi
It took Amazon.com, a large company with tons of incentives and a strong
mandate from the top floors, 7-8 years (at least!) to get out of Oracle.

I can easily imagine how "normal" companies will not have such an easy time
doing so. I guess we're going to have Oracle around for a long time.

------
senderista
Notice they don't say that AWS has completely migrated off Oracle. There was
plenty of Oracle there 5 years ago, including at the core of some of the most
popular services. (There was also a significant migration effort underway then
as well, mostly to DynamoDB.)

------
latchkey
They beat their own estimates:

[https://www.cnbc.com/2018/08/01/amazon-plans-to-move-off-
ora...](https://www.cnbc.com/2018/08/01/amazon-plans-to-move-off-oracle-
software-by-early-2020.html)

------
truth_seeker
Price is the only main reason I see which drive this migration.

About Oracle Database (not Oracle company), I am very much in agreement that
(PL)SQL can be used as one language to take care of Relational Algebra, Row
oriented, Column oriented, Document oriented and also Blockchain oriented disk
persistent tables with local node partitioning and distributed sharding, Graph
traversals queries using recursive CTEs, auto tuning of indexes, streaming
materialized views, distributed topic/queues.

Oh by the way with little effort you can accomplish almost all what Oracle
does with PostgreSQL and its community extensions.

------
exabrial
Oracle should have been doing everything to keep them as a customer and push
their databases through cloud offerings. They already push their software
through resellers, sort of the same thing, just automatic

~~~
WomanCanCode
Do you know what is happening to their cloud offering?

~~~
exabrial
From what I understand it's mainly being thrown at governments. Why do you
ask?

------
fierarul
There was recently a similar story about Amazon Fulfillment turning off the
final Oracle Database:
[https://twitter.com/jdarrow/status/1111629087947132928](https://twitter.com/jdarrow/status/1111629087947132928)

I wasn't aware they are doing this PR pieces on a department by department
basis.

Are there any more remaining Amazon departments still doing this process?

------
sofaofthedamned
Hate oracle personally. But when Amazon mention cost savings, I assume they're
not at list price for normal aws customers like us?

~~~
QuadrupleA
Article mentions that, yeah - cost savings are based on their already heavily
discounted deal with Oracle.

~~~
dastbe
I think OP means Amazon’s discount on AWS.

Many customers on AWS have contracts that guarantee discounted rates given
sufficient volume. People have commented on hn about negotiating them, and you
can see some examples from recent IPO filings. Amazon (the non-AWS parts) are
yet another customer of AWS, and wherever possible you should expect them to
operate like one. They even buy RIs!

~~~
sofaofthedamned
Thanks - yes that's exactly what I meant

------
suyash
Most of what Amazon is talking about is the Oracle Old database product, check
out Oracle's latest offering that is powered by AI - Autonomous Databases:
[https://www.oracle.com/database/autonomous-
database.html](https://www.oracle.com/database/autonomous-database.html)

------
edf13
Amazon Polly works well! First time I’ve used it ~ hit play and let it read
the article whilst I did a few other things!

------
danesparza
The very fact that they are dogfooding DynamoDB (with core features of their
business) makes me want to take it more seriously and use it more.

I also love that they didn't just replace Oracle with <insert competing RDBMS
here> but instead thoughtfully pieced together a strategy based on what the
data was going to be used for.

Well done!

~~~
mey
Have used DynamoDB a couple times, most recently for data processing.

It is extremely expensive, for data storage and performance. Queries are
limited and cumbersome to write.

If you are looking at it, put a reasonable amount of test data in and play
with it too see what costs look like compared to the performance you need from
it.

~~~
onefuncman
Expensive compared to what? DynamoDB has the cheapest 4ms writes I can find...

But yes, absolutely prototype and measure costs with real workloads

------
koolba
> Financial Ledger – This team moved 120 TB of data, reduced latency by 40%,
> cut costs by 70%, and cut overhead by the same 70%, all powered by DynamoDB.

Does this mean they have a custom set of books running atop DynamoDB or is
this specific to aggregate reporting and financial data warehousing?

~~~
denimnerd42
Would love to know also. My guess given the volume of data is data
warehousing.

------
jpalomaki
Oracle would have the set of tools for providing a interesting opinionated
cloud/onprem platform for microservices architecture.

Something along the lines of JavaEE, deployed on something derived from
WebLogic, bound together with reliable messaging bus (Tuxedo), backed by
Oracle database.

------
denimnerd42
We have a huge database problem at my company (department). There are only 4
databases for 200 developers. I find this extremely cumbersome to develop on.
Ideally there would be at least 1 DB per developer plus some shared
environments. What do others do?

------
dade_
The business case to stop managing Oracle contracts and licencing is probably
pretty good, especially when factoring in the sick days and stress leave of
employees dealing with their infuriating people, processes and tools.

------
matheusmoreira
This is a good thing, given this description of the Oracle database:

[https://news.ycombinator.com/item?id=18442941](https://news.ycombinator.com/item?id=18442941)

------
WomanCanCode
Any reason why Larry Ellison is still running the company?

------
arrty88
How much did the actual migration effort cost them?

------
decoyworker
The sooner that IBM and Oracle die the better.

------
choiway
Larry might need to sell his island.

------
hackernewsboy
is this purpose built database based on mysql or postgresql?

------
that_girl
In other words, AWS used the stones to sustain the stones.

------
hackernewsboy
is this purpose built database based on mysql?

------
vkaku
Obviously, enough Amazon bots are giving me bad karma, but my point stands:
No, Amazon is not a special snowflake because it did this.

This was an expensive exercise in spite, took a long time and a lot of people
- and everyone knows it.

They could have done this far efficiently already if they wanted to scale out,
but that company has too many conflicting internal priorities that they took a
heck of a long time to move away to anything that scales.

They never try to invent things and if not Oracle, they'd use some other
vendor built software. That is the true reality of this company.

So no, if you're thinking you're better than this company for moving away from
vendor built software, you are mistaken. In the worst possible way.

~~~
whoisjuan
"Amazon bots"... sure dude. Because all your opinions are popular by default.
Otherwise is just "bots" downvoting you.

Why people can't admit that their opinions can be unpopular sometimes. That's
not a bad thing. Not all the things you say need to be popular to have meaning
to yourself and others. It sucks to be downvoted but that's part of having a
healthy discourse in HN.

~~~
vkaku
Okay, all is fair in a really healthy discussion. The reason I write what I
write anyway is because I believe in them. Your point is taken and accepted.

I've slammed Google, FB and some other tech companies on certain occasions
BTW, and I absolutely never felt bad about it. Thanks!

