

Are cloud based memory architectures the next big thing? - helwr
http://highscalability.com/are-cloud-based-memory-architectures-next-big-thing

======
ekidd
That's an interesting article, although the level of hype for cloud computing
is rather high (and I'm an enormous fan of cloud computing): "When PCs and
client/server computing came along, the departmental computing guys (i.e.
DEC), considered PCs to be a special case of smart terminals... So the
batchguys are dead, the timesharing guys are dead, the departmental computing
guys are dead, and the client server guys are dead. Notice a pattern?"

That said, there are already several hybrid memory/disk storage layers that
might actually be a better choice than the pure in-memory approach that the
author suggests. For example, MongoDB is basically an in-memory document
database that's mmapped to a disk file and flushed once per minute. CouchDB,
IIRC, is basically an in-memory database backed by an append-only transaction
log on disk. These sorts of designs should give you many of the bandwidth
advantages of pure memory systems, but still maintain the ability to survive a
power outage.

------
jedberg
># Performance is better because data is accessed from memory instead of
through a database to a disk.

># Scalability is linear because as more servers are added data is
transparently load balanced across the servers so there is an automated in-
memory sharding.

># Availability is higher because multiple copies of data are kept in memory
and the entire system reroutes on failure.

># Application development is faster because there’s only one layer of
software to deal with, the cache, and its API is simple. All the complexity is
hidden from the programmer which means all a developer has to do is get >and
put data.

All but the first thing apply to disk based databases too... At least the new
ones like Cassandra, as well as the old ones with the right packages.

~~~
gxti
Buy enough RAM to fit the entire dataset in memory and any database will
scream.

------
jasonkester
I've been a fan of this sort of thing for several years now. Of course back in
the day it would get you complained at for violating the best-practice of not
keeping heavy objects in the Application or Session state (presumably because
back then you only had 500mb of it).

If you go play around on Twiddla today, chances are nothing you do there will
end up on physical storage, ever. All the sandboxes and guest sessions spend
their entire lives in memory, and only in very rare cases get persisted to the
database these days.

One side effect of doing this is that it makes scaling out a lot easier. If we
ever hit a traffic spike that bothers our server, I can spin up a few EC2
instances and hand off the actual Twiddling to them without having to worry
about what to do about getting the data written back to the database.

It also makes a nice differentiator between free and paid plans: If you need
the ability to save your stuff, we're happy to let you pay to do so. If not,
go nuts, since it doesn't cost us anything to keep you in memory.

------
jermy
Odd that SSDs aren't mentioned in the article at all - they would mostly avoid
many of the criticisms levelled at disks without requiring redevelopment

------
zitterbewegung
They may be the next big thing since the old guard databases although are
optimized for disk in memory eliminate that due to decreased latency. The
caveat is that the in memory databases aren't as mature as databases that
persist to disk.

------
0bfusct3
Can we just call it what it's been known as in computer science since oh
1960's distributed operating systems?

------
Khroma
So basically.. cloud based caching?

