
Integrating NVMe Disks in HopsFS (HDFS) - jamesblonde
https://www.logicalclocks.com/millions-and-millions-of-files-deep-learning-at-scale-with-hopsfs/
======
SirOibaf
"HDFS and S3 are designed around large blocks (optimized to overcome slow
random I/O on disks), while new NVMe hardware supports fast random disk I/O
(and potentially small blocks sizes). "

Interesting solution presented here - keep the block size constant, but put
the small files on NVMe disks.

------
jamesblonde
Disclaimer, one of the authors here. You can see a video with discussion of
the NVMe support in HopsFS in this Spark Summit talk:

[https://www.youtube.com/watch?v=tx6HyoUYGL0](https://www.youtube.com/watch?v=tx6HyoUYGL0)

------
senorcarbone
super cool. The world needs more success stories for NVM

