
In-Memory Performance for Big Data (2014) [pdf] - lichtenberger
http://www.vldb.org/pvldb/vol8/p37-graefe.pdf
======
Rafuino
Has this pointer swizzling concept been adapted by any big data-focused
databases?

Would be interesting to see how this works with bigger pools of volatile and
persistent memory (disclaimer: I work at Intel on Optane DC SSD/PM-related
things).

~~~
lichtenberger
BTW: What's your take on random read speed? I think with SSDs now and maybe
NVRAM in the future we can address physical storage more fine granular than
with HDDs and for instance with [http://sirix.io](http://sirix.io) data is
only clustered during writes (batched and synced to the flash drive) and it
versions variable length in-memory and physical pages such that not every
record has to be copied and written again. For instance to reconstruct a page
we have to read from random places (maybe in parallel) to reconstruct a page
in-memory. Thus Sirix relies on fast random read speed and on find granular
access to the physical stored records.

~~~
Rafuino
My take on random read speed? Check out this paper from Lenovo testing the
performance of Optane DC Persistent Memory on their servers:
[https://lenovopress.com/lp1083.pdf](https://lenovopress.com/lp1083.pdf).

There's another good paper from a team at UCSD testing all sorts of usages and
file systems with NVRAM here too:
[https://arxiv.org/pdf/1903.05714.pdf](https://arxiv.org/pdf/1903.05714.pdf).

Sirix sounds like a good fit to take advantage of App Direct access on Optane
DCPM, but you're way more of an expert than me. Perhaps make a request for
alpha access at GCP to test Sirix with the new tech?
[https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-
aQUK2iVVi...](https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-
aQUK2iVVioClFX5N-061jqO46vzpHzAPHkjwzVw/viewform)

------
cl0ckt0wer
Had to look this up:

[https://en.wikipedia.org/wiki/Pointer_swizzling](https://en.wikipedia.org/wiki/Pointer_swizzling)

~~~
petergeoghegan
The term pointer swizzling isn't familiar to me, but it seems to be the same
technique by which variable-width tuples on a page are accessed via a fixed
width item pointer array (sometimes called an indirection vector) within most
database systems.

------
mpweiher
Interesting, sounds a bit like the way write-barriers made the combination of
copying garbage collectors and direct object pointers feasible.

