
The Storage Tipping Point - jcreedon
http://storagemojo.com/2015/06/29/the-storage-tipping-point/
======
nickpsecurity
Funny he mentioned System/38, which largely eliminated users' storage worries,
then says that company's rep never heard of it. Funny cuz they still sell it:
IBM i. I posted a link [1] to original system on his site. It got so many
things right in a business machine it was hard to believe. That old design
still has better security and reliability than Windows or Linux boxes. Most
AS/400 deployments I run into just don't... have... downtime... They also
pretty much manage themselves.

I'd like to see FOSS do designs like that [minus the interface & API]. The
labor advantage might lead to something pretty awesome. Meanwhile, they keep
building complexity on top of UNIX architecture with a few stragglers on
better OS architectures. Not much hope there. Fortunately, CHERI (capability)
and SAFE (tags) are each building on mechanisms System/38 used for software
reliability and security. An academic team or company might build something
great on top of it. Still holding out some hope!

[1]
[http://homes.cs.washington.edu/~levy/capabook/Chapter8.pdf](http://homes.cs.washington.edu/~levy/capabook/Chapter8.pdf)

~~~
vezzy-fnord
Also worth noting is that IBM i had one of (if not probably the) first
implementation of containers, in the form of LPARs.

~~~
nickpsecurity
True. Add that many "inventions" in virtualization field were in VM/370 in...
the 70's. Including running itself on itself which has been celebrated in
several news articles for different prototypes over the past few years. Except
that was a production system. ;)

~~~
cbd1984
Add that IBM tried to kill VM multiple times, and wanted everyone using
System/360 to standardize around OS/360.

"The Mythical Man-Month" is the story of OS/360\. Hence VM still exists.

~~~
nickpsecurity
Lol @ last line. Well-put.

------
dang
The article says that log-structured apps and filesystems conflict with
underlying SSDs, because those are doing similar things and the two don't play
well together. But then it says that "products that already incorporate log
structured I/O will have a definite advantage" in the future. How can they
have an advantage if they're conflicting with the storage layer? Is this a
contradiction, or am I missing something?

~~~
tlb
The paper [0] has some suggestions: an API (already existing and somewhat
supported, called TRIM) to tell the disk that certain sectors are no longer in
use, and atomic writes.

I guess I buy the claim that you're better off starting with a log-structured
file system that will use new APIs to allocate and free chunks on the SSD,
than to use a conventional file system that the SSD tries to optimize

[0]
[https://www.usenix.org/system/files/conference/inflow14/infl...](https://www.usenix.org/system/files/conference/inflow14/inflow14-yang.pdf)

------
maccam94
Yup, storage is going to change a lot. RAID is nearly dead. That means a whole
lot of failure detection and correction needs to be reimplemented elsewhere in
the stack. We'll also need to build storage layers that can deal with the
crazy performance profile of SMR hard drives (at least, until SSD's are
cheaper).

------
narrator
Wait till memresistors get here. That will basically eliminate the distinction
between ram and disk.

[https://en.wikipedia.org/wiki/Memristor#1808](https://en.wikipedia.org/wiki/Memristor#1808)

"They can potentially be fashioned into non-volatile solid-state memory, which
would allow greater data density than hard drives with access times similar to
DRAM, replacing both components.[14] HP prototyped a crossbar latch memory
that can fit 100 gigabits in a square centimeter,[8] and proposed a scalable
3D design (consisting of up to 1000 layers or 1 petabit per cm3).[66] "

~~~
wmf
HP indefinitely postponed (canceled) memristors recently.

------
StillBored
I was under the impression that the storage model in the system/38 was part of
what was carried forward into the as400. Not the capability model, as much as
the lack of "files" because everything is just persistent objects/segments.

Is that not the case?

But I'm not really sure how this by itself solves the problems of modern
SSD's. Problems caused by two things, the differing segment/page sizes between
the upper layers (OS/filesystem/database/etc) and the need to rewrite static
portions of the SSD in order to wear level the entire address space. (and to a
lesser extent the need to update small portions in the middle of existing
pieces of data).

Its the second part that seems to be forgotten by lots of people suggesting
alternatives to the current storage stack. Whats really necessary is a higher
level communications protocol between the flash and the filesystem. One that
provides the metadata to the ssd about the sizes of individual "objects" so
that they may be keep together as atomic units. That way the flash layer knows
if a write is to a larger object, or a new object itself that can be managed
separately.

------
mark_l_watson
Good to see Robin Harris' article on the front page of HN (we live in the same
small town and a mutual friend introduced us).

A little off topic from the article: I think the biggest tipping point is the
decreasing cost of cloud storage because large companies like Google do so
much of their time critical processing with data in RAM (spread over many
servers) and these servers can re-purpose a lot of their disk I/O capability
to service low cost cloud storage.

~~~
dikaiosune
Can't believe I've seen two people from there on the same day on HN!
Flabbergasted. Grew up there, currently living just up the rim.

Re: your comment, I'm curious to see how the I/O capacity of those servers
could be distributed for cloud storage. I would imagine that even with 10Gbps
if those machines are really cranking, they could have a difficult time
provisioning network I/O for a storage service.

~~~
mark_l_watson
Email me when you will be in Sedona and we can have coffee. I can contact
Robin to see if he is available.

re: I/O capacity: I was commenting on how Google, etc. can provide such
inexpensive cloud storage, not access speed.

------
williesleg
I think the storage company Nimble may have figured this out.

~~~
stephengillie
What makes you think they have? Do you have a blog post or statistical data
that supports this notion? Please, share!

~~~
droopyEyelids
Not yet but the rep is taking us to the big game— a fully catered private box.
I'm sure I'll hear about it there.

~~~
JupiterMoon
If they had truly solved this problem they wouldn't need to take you to "the
big game" to do sales - you'd be taking them to "the big game" to be
customers...

