
Optane Memory – A bridge between DRAM and storage - wonderous
http://www.intel.com/content/www/us/en/architecture-and-technology/optane-memory.html
======
2bluesc
Anandtech, as usual, has a good article:

[http://www.anandtech.com/show/11227/intel-launches-optane-
me...](http://www.anandtech.com/show/11227/intel-launches-optane-
memory-m2-cache-ssds-for-client-market)

> The Optane Memory products are squeezing into a relatively small niche for
> limited budgets that require a lot of storage and want the benefit of solid
> state performance without paying the full price of a boot SSD. Intel notes
> that Optane Memory caching can be used in front of hybrid drives and SATA
> SSDs, but the performance benefit will be smaller and these configurations
> are not expected to be common or cost effective.

It doesn't sound that fast. Looks to be "slower" (throughput-wise) then modern
NVMe drives (i.e. Evo 960 and obviously smaller) and clearly trying to bridge
a price gap. Is the upside that it can act as an additional cache level
without OS support? I remember the first time I used Intel Rapid Storage Tech
(aka RST) ... it was also the last time as it's was convoluted with minimal
(or no?) upside on Linux.

As an "enthusiast" I'm assuming that my Linux desktop with plenty of RAM +
NVMe backed lvmcache will perform better.

I also assume that my laptop with reasonable RAM (because DDR (self)refresh
sucks enough power to annoy me) + high performance NVMe drive also won't
benefit.

Is this just marketing fluff?

~~~
2bluesc
Interesting comment from Anandtech [0]

> Keep in mind that consumer NVMe SSDs that boast throughput of 2GB/s or more
> generally do not reach their peak at low queue depth. Optane is supposed to
> be able to drive 1200MB/s read throughput at low queue depth (not sure why
> they listed QD4), so there is potential for some performance improvement
> here. Most consumer workloads never get out of low queue depth territory, so
> this could have some small real world benefit. Write throughput, however, is
> critically low.

> More importantly, these Optane drive are gear more towards lowering latency
> than transferring large files. Where HDDs access the data on the order of
> 10s of mS and SSDs access data on the order of 1mS (give or take), Optane
> should be able to access data on the order of 1s - 10s of uS. Where Optane
> will be useful is high numbers of small file accesses (DLLs, library files,
> etc.).

> That all said, I'd just as soon leave all the extra complications,
> compatibility issues, and inconsistencies on the table and get that 2 GB/s
> sdd that you mentioned until Intel figures out how to make these more
> compatible and easier to use without requiring a "golden setup". I don't
> want to buy a new W10, Kaby Lake, 200 series based system just to use one of
> these. My current W7/W10/Ubuntu, Skylake, 100 series system should work just
> fine for a good while yet.

[0] [http://www.anandtech.com/comments/11227/intel-launches-
optan...](http://www.anandtech.com/comments/11227/intel-launches-optane-
memory-m2-cache-ssds-for-client-market/537799)

~~~
sliken
Dunno, I find 1.2-1.6GB/sec on my samsung s950 all the time. The benchmarks
I've seen do not show any noticeable difference between SATA and NVMe. Sure
geeks like the idea of a lower head protocol, but truth is the protocol is not
the bottlneck.

Seems like Optane is a solution looking for a problem to fix. Normal M2 SATA's
do fine: # dd if=/dev/sdb of=/dev/null bs=131072 1953586+1 records in
1953586+1 records out 256060514304 bytes (256 GB, 238 GiB) copied, 159.18 s,
1.6 GB/s

(cold cache on a machine with 16GB ram).

~~~
wtallis
You're not getting 1.6 GB/s out of a SATA drive that's communicating over a
600MB/s link. I think you may be confused about which products are using SATA
vs PCIe.

------
ChuckMcM
The 'win' in theory for this stuff is that it is much faster than SSDs at low
queue depths and it is significantly more write tolerant.

An interesting interesting system design might be a 2GB main memory with a
32GB swap partition on this device. That would be interesting because in idle
mode it uses less power than an equivalent amount of DDR memory. So you could
potentially build a laptop with good performance on larger memory tasks but
longer standby battery life. It would also be interesting paired with an
equivalent amount of DRAM where it mirrors main memory so you could have
'instant' off and 'instant' on type booting and shutdown.

At larger pool sizes this becomes interesting as a non-volatile index for
NoSQL and other databases. Intel's claim has been in the past that its density
can exceed SSDs by a healthy margin.Not seeing that in these products but time
will tell.

I've been watching them struggle to productize this tech for about 2 years
now[1]. Now to see if they can get it to live up to the hype.

[1] [https://arstechnica.com/gadgets/2015/07/intel-and-micron-
unv...](https://arstechnica.com/gadgets/2015/07/intel-and-micron-
unveil-3d-xpoint-a-brand-new-memory-technology/)

~~~
mtgx
> Intel's claim has been in the past that its density can exceed SSDs by a
> healthy margin

Seems doubtful they would reach that goal until SSDs hit some kind of hard
plateau (kind of like HDDs did).

Toshiba plans to make a 100TB SSD over the next couple of years, and I believe
Samsung has already announced a 16TB SSD. I doubt 3D Xpoint can catch-up with
those in the given time frame.

~~~
wtallis
NAND flash memory already hit the wall for horizontal shrinks, and some
products plowed past that limit and into the regime where day to day use was
invoking the second level error correction methods to compensate for data
degradation.

3D NAND turned the clock back on horizontal scaling and cell sizes by several
years, and now the only scaling opportunity is to increase layer count. That's
not going so well. Samsung started with 24 layers but didn't ship retail
products until they hit 32 layers. Their 48 layers was about a year late and
caused several products to be scrapped entirely. Toshiba's on their third
generation 3D NAND and it's only just starting to show up in the market. SK
Hynix is doing worse than Toshiba. Intel and Micron started off reasonably
well with their 32-layer and are working on ramping up 64-layer.

By the end of this year, all four manufacturers should be shipping 64-layer
(72 for SK Hynix) 3D NAND in volume, or else it's time to admit that 3D NAND
just doesn't scale up as well as we hoped it would.

There's some reason to believe 3D XPoint memory might be able to avoid the
worst vertical scaling problems that are holding back 3D NAND flash. Each pair
of 3D XPoint layers can be pretty much isolated from the rest of the layers
except at the edges, whereas 3D NAND is building columns of bits that run
through all 64+ layers and have aspect ratios that are quite high.

~~~
markvdb
I can see some potential in things like RERAM to scale beyond 3D NAND and
bridge the SSD/RAM gap. I like the technological elegance of non-filamentary
RERAM [2] in particular.

[1] [https://en.wikipedia.org/wiki/Resistive_random-
access_memory](https://en.wikipedia.org/wiki/Resistive_random-access_memory)
[2] [http://www.4dsmemory.com](http://www.4dsmemory.com)

------
freeone3000
It looks to be an accelerator cache for HDDs, but slower than an SSD, with
fairly low capacity. And it requires a 7th gen intel processor, so this isn't
a performance boost for older PCs either. Exactly who is this aimed at?

~~~
valarauca1
Nobody.

You lose SSD transfer speed, while gain _some_ hardware response latency.

NVMe response latency (on hardware side, not OS) is around 14-15micro-seconds
on reads, and tops out at 200micro-seconds on writes [1]. Which is nearly 2-3
orders of magnitude below spinning rust. While Optane can halve this to
7micro-seconds on reads. The price/pref vs established NVMe tech is non-
existent.

When I can spend $X/GB to get a 2000% performance bump, its hard to justify
spending $X*4/GB to get 2050% performance bump. Most applications/data centers
have yet to transfer to NVMe. Offering marginal gains a sizable markup is a
risky strategy.

Furthermore Optane is nearly 2 orders of magnitude lower than RAM in latency,
and 5 orders of magnitude lower transfer. So the idea of "non-volatile RAM"
has yet to arrive.

\---

Also "Transparent to the OS" should strike fear in your heart. The OS's job is
to manage hardware, memory, and caches. Why drop this?

[1]
[https://www.cs.utah.edu/~manua/pubs/systor15.pdf](https://www.cs.utah.edu/~manua/pubs/systor15.pdf)

~~~
convolvatron
seems like it might be useful as a transaction log?

~~~
angry_octet
The latency of a regular SATA SSD is more than good enough if you want a log
of writes to spinning rust. The increased write endurance might be appealing
for something like a ZFS intents log (ZIL).

Too small to provide much of an cache, when you can have 256GB of SSD for the
same price.

It is a pity that the marketing docs don't address how this impacts on write
consistency. Instant write sync is the primary advantage of hardware RAID, but
still, how much better is this than a normal SSD?

~~~
Dylan16807
> The increased write endurance might be appealing for something like a ZFS
> intents log (ZIL).

These first generation products don't actually have better endurance, though.

------
codebook
The only thing could get from Optane is QD1 performance. It is impressive (5x
faster than current gen NVMe SSD). But everything else is just soso compared
to NVMe SSD.

------
smlacy
Quite a bit of marketing-speak here on the official site, without a lot of
technical details.

I know the Optane SSD drives are fast -- is this just device-side driver
software that allows you to virtually expand available "RAM" and mmap it to
the SSD?

Why just i7 CPUs? Is there a hardware component?

"Just download and install the driver..." (Linux?)

~~~
AdamJacobMuller
Seems like its some i3/i5/i7, only recent ones? Must be some hardware
component to it that makes the SSD look like memory to the system, perhaps as
another NUMA zone (could you have a huge NUMA zone with no CPU attached?)

~~~
the8472
Maybe the CLWB instruction?

------
amluto
I _think_ you can (pending driver support?) plug one of these in and memory
map it. On Linux, this means you'd see it as a "DAX" filesystem, and if you
mmap(2) a file, you get a bona fide direct mapping of the hardware into your
address space. This is very cool and would enable some interesting database-
ish use cases.

I'll believe it when I see the spec and the code.

~~~
wtallis
No DAX yet. Both this consumer Optane Memory product and the enterprise Optane
SSD announced last week are standard NVMe devices. For the enterprise market,
Intel is providing mmap capability with a hypervisor-based software solution.

3D XPoint DIMMs are on the roadmap for next year.

~~~
amluto
That's lame. Some competitors are working on DAX-capable PCIe devices. Intel
seems to be surprisingly behind here.

How would a hypervisor sulution work? Trap and emulate all accesses?

------
tetraodonpuffer
it seems this would be a great use case for DAWs and sample libraries in
general, many many small files to be accessed with extremely low latency: this
could make it possible to run large orchestral templates fully purged at all
times, which would be a great use case when your template has hundreds of GB
of samples but you only use a few of them at a time.

This is already kind of doable with SSD, but if optane has an order of
magnitude less access time it would of course make it even better for this use
case.

------
bushin
I thought I could throw away my DRAM and SSD, but it looks like for now it's
just another cache level.

~~~
readams
They're a long way from realizing the full potential of the technology. Right
now they're hugely limited by the interconnect. Also, if they reach some of
the goals they've been talking about, it might need differently-designed
operating systems to really work.

~~~
snuxoll
I mean, Single-Level-Storage is hardly a new concept - I don't think we need
entirely new operating systems to gain benefits, but for more applications and
enabling technologies to be designed around it.

LMDB, for example, would be a great fit with Optane already. Really, anything
that uses mmap() is.

------
dman
Seems like a questionable product launch from Intel -
[http://semiaccurate.com/2017/03/27/intel-releases-
consumer-m...](http://semiaccurate.com/2017/03/27/intel-releases-
consumer-m-2-xpoint-ssds/)

------
ksec
I think this is an Easy win for DIMM in servers where you could fit a much
larger capacity then DRAM. And this is only a first gen product.

But for consumers? I am not sure if they can feel the difference between a
decent SSD drive and Optane Memory. It fits the role of Page File. So you
could always have 2x - 3x the Optane Memory as Page Cache. System with this
would means less Memory required, and saves battery. While the controller for
SSD could be less complicated and focus primarily on Seq/Read Write. SSD could
also be DRAM-less, save cost and battery.

But even then, $44 for 16GB is just hard to convince most consumers.

------
ak217
If I'm reading this right, this device is slower than a Samsung 960 Evo across
the board ([http://www.anandtech.com/show/10833/the-
samsung-960-evo-1tb-...](http://www.anandtech.com/show/10833/the-
samsung-960-evo-1tb-review)). It seems that when Intel started developing
Optane, they didn't anticipate how much faster NVMe PCIe SSDs would become.
Couple that with significant clock frequency headroom for DDR4, and this
product really doesn't look competitive.

~~~
olavgg
You're reading the wrong numbers. Try benchmarking with PostgreSQL's
pg_test_fsync or with fio with the following options: --direct=1 --sync=1
--rw=write --bs=4k

Good random-write performance with 4k/8k blocks at QD1 is what matter for
SSD's, not if they can write 2500MB/s to a volatile cache.

Recommended read [http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-
test...](http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-
ssd-is-suitable-as-a-journal-device/)

------
Quequau
I think a lot of this next generation solid state storage level memory is
going to wind up being ill suited for the dominant interconnect standards
(e.g. DDR, SATA, or PCIe). I also really wonder if it isn't going to take a
really long time for the relevant parts of the existing software stack to see
serious development to effectively exploit this stuff.

------
whalesalad
Before the page could even finish rendering I had a pop-up to answer a survey.
What is this called and why do websites continue to do it? Does this not
massively destroy your conversion or retention rates?
[http://i.imgur.com/90VHYnU.png](http://i.imgur.com/90VHYnU.png)

------
patrickg_zill
2GBytes/second as RAM is as good as DDR1 memory.

So take a CPU with more memory channels on it, run at least 1 Optane per
channel, run hundreds of VMs on it, everything runs from this slower RAM, but
it is in memory and you only touch the SSDs for updating the disk. Once the
cache is warmed up performance should be just fine.

------
runeks
On the "Shop" page, there's a list of motherboards [1]. Is it not possible to
buy as a standalone drive?

[1] [http://www.intel.com/buy/us/en/audience/optane-
memory](http://www.intel.com/buy/us/en/audience/optane-memory)

------
wickedlogic
Reaction to 2nd video: It is like a kitchen, and a panty, with a sous-chef
which is like a conveyor belt. I mean if this runs ruby and also uses knife,
how fast could this really be. ;)

------
replete
So they've made a big fuss about something that: \- We can't just buy right
now \- Doesn't do the thing we were excited about (use optane as DRAM)

------
lostmsu
How much benefit does it give over using any other NVMe SSD (which would
probably be much cheaper) and some caching software like Diskache?

------
derefr
I'd be interested in knowing what the performance of the caching is compared
to e.g. using dm-cache with an SSD backing a HDD in Linux.

~~~
wtallis
If you've got suggestions for how to meaningfully and realistically compare
caching of a Windows boot volume against a Linux cache solution, please share.
My usual suite of tests won't work well for this, and I'm open to suggestions
for what to include in my new test suite.

------
tkyjonathan
With all the exciting background music, I couldn't help thinking "isn't this
solving a problem from 5-6 years ago?"

------
timvdalen
Was the entire pancake analogy a setup for the 'have your pancakes and eat
them too' joke?

------
kardos
I wonder if there is a use-case for this to be the storage device in a low-end
VPS?

------
daemonk
So this is really just a smart caching system for slower hard drives?

------
probablybanned
Does anyone know whether this memory has limited cycle life like Flash?

~~~
wallacoloo
It does. Somebody else linked to the Semiaccurate article higher in the
comments where they discuss this. It _looks_ as though the per-bit endurance
of the Optane drive is a bit less than modern flash, but they counteract this
by having the drive's firmware reserve some blocks that it can remap over
failing blocks.

~~~
wallacoloo
And here's that Semiaccurate article:
[http://semiaccurate.com/2017/03/19/intel-officially-
introduc...](http://semiaccurate.com/2017/03/19/intel-officially-introduces-
xpoint-dc-p4800x-ssd/)

------
solotronics
my understanding was one goal of this would be replacing DRAM with 3D XPOINT
so theoretically your computer would have 1 TB of this new memory in its RAM
slots.

~~~
valarauca1
Your understanding if a bit flawed, and largely based on marketting.

Intel has _hinted_ XPOINT will be usable in DRAM slots, but has yet to
announce a product.

Intel has _stated_ XPOINT will replace DRAM buuut based on it's marketing
material it won't. Having latencies go from 14nano-seconds to 9000nano-seconds
isn't acceptable for a DRAM replacement.

~~~
Moral_
A lot of the 9000 latency you're speaking about is traversing the PCI bus. The
Dimms should be lower latency.

------
faragon
How it compares to massive swapping to typical SSD?

~~~
dsr_
Badly. Suppose you have a 16GB laptop and the 16GB optane. When you close your
laptop to take it to a mid-morning meeting, you've just done a whole drive
write. Do it again at lunch, twice in the afternoon and again when you get
home, you're at 5 of your 6 drive writes.

Now imagine you open your laptop 8 or 12 times a day...

~~~
hoschicz
Closing laptop usually mean suspend to memory, not disk.

~~~
dsr_
Not in the regime Intel is proposing, because optane stores data without power
like a disk.

------
krn1p4n1c
Isn't this just another go at Robson and SRT?

------
monochromatic
A cache for spinning hard drives? Pass.

