
Versioned HDF5 - todsacerdoti
https://labs.quansight.org/blog/2020/08/introducing-versioned-hdf5/
======
xen0
I've always been a bit curious as to what use cases people have for HDF5 files
that involve frequent changing of data.

I wonder how read/write performance scales with the _size_ of the dataset in
question, in particular, the number of chunks, and hence the number of virtual
dataset mappings. I never really explored how well the HDF5 library handles
obscene numbers of VDS mappings, and one per chunk on a large dataset is
probably more than the HDF Group had in mind.

------
nickysielicki
This is a huge shortcoming with HDF5 and I’ve personally been bitten by not
understanding this, and in spite of telling others to not get bit by it, I’ve
seen other people be bitten by it. Glad to see someone doing some harm
prevention, I hope it gets adopted and mainlined.

