n=32 leads can lead to lots of sparseness, memory pressure, bad locality (given sparseness); its not going to fly easily as the default. The shared memory aspect is even a bigger hurdle: people share memory for a reason, its not very often accidental. FRP is hardly the answer, but I work on a competing paradigm so mark me as biased.
The systems people I work with do a lot of key value stores (sharded NoSQL-style databases), there is a lot of pipelining for data parallel processing (MapReduce, Pig), and there is still a lot of classic locking going on, throughput is important. Oh, and lots of Paxos, logging, state replication, etc... The normal stuff you could find at Google (note I don't work for Google).
On the HPC side, the people that really care about performance, there is some map reduce via MPI, but most of the big gains are to be had in CUDA, which is an extremely imperative programming model with very synchronized core executions and memory accesses.
I've learned that whenever the FP people talk about high performance, they usually mean something else (like concurrency that isn't so slow), rather than the real high performance stuff that the big kids (systems and HPC communities) are doing.
I never meant to imply FP is already the right choice for big data (which is what I think you refer to by 'systems people') or high performance computing.
The reality though is that well over 99% of the market works with data sets for which, with current hardware, the performance costs of these persistent data structures are very much irrelevant.
When you add FRP on top of that so that you can keep your managed state minimal you can actually often achieve quite dramatic performance improvements over the traditional hand-written data-to-UI synchronisation.
For persistent data structures to come into their own for big data quite a few more things are needed of course; but if you look at Nathan Marz's Lambda Architecture for example it's little more than a handwritten system for making large scale persistent data structures manageable with today's systems already.
I'm fairly sure that tooling for that will only improve over time until the very thought of manually managing derivation pipelines becomes as quaint as manual memory management.
As for HPC; yes... I would not be surprised at all if at that level mutable state no longer is a premature optimization. Even there I could imagine many of the more high-level control structures benefitting from persistent data structures though.
About the FRP part; which competing paradigm are you working on? I wouldn't be able to name a 'competitor' so I'm quite curious.
It actually involves versioned data structures (similar, but a but more complicated than persistent data structures, and outwardly they appear mutable):
I already had a feeling we were essentially arguing the same thing...
Yes; the managed time approach in Glitch is very interesting and indeed a competitor towards addressing the same problem I originally raised.
The core of my original statement/problem definition was that real mutable state, i.e. throwing away history by default, was no longer the smart thing to do given current hardware.
From what I can tell you would also agree with that...
I would, but it doesn't necessarily have to happen with persistent data structures, and I would also argue that they aren't really sufficient (the history has to be useful, after all). They are convenient, and I'm using trees in my current implementation.
The systems people I work with do a lot of key value stores (sharded NoSQL-style databases), there is a lot of pipelining for data parallel processing (MapReduce, Pig), and there is still a lot of classic locking going on, throughput is important. Oh, and lots of Paxos, logging, state replication, etc... The normal stuff you could find at Google (note I don't work for Google).
On the HPC side, the people that really care about performance, there is some map reduce via MPI, but most of the big gains are to be had in CUDA, which is an extremely imperative programming model with very synchronized core executions and memory accesses.
I've learned that whenever the FP people talk about high performance, they usually mean something else (like concurrency that isn't so slow), rather than the real high performance stuff that the big kids (systems and HPC communities) are doing.