On: Janas: Run LLMs larger than RAM by streaming experts from SSD

The problem is will the big ai companies siphon it all up before consumers even get a chance?

Same as big data centers siphoned up all the SSDs before their prices became reasonable for consumers, same with HDDs, or RAM every generation. They’ll be high before they’ll be reasonable, and then they’ll be cheap once their next generation comes along.

Did data centre SSD rollout really happen significantly before consumer SSD rollout?

I'd always assumed that the first generation of SSDs were really tested on consumers.

Yeah, started in the late 70s (1978) with the StorageTek STC 4305, which was 45MB in a large cabinet, cost $400,000, and was from what I can find the first SSD. From there up until the 2000s they served as expensive but crucial networking infrastructure, when in the early-mid 2000s the industry moved from RAM to NAND, and the price-per-GB of NAND became reasonable enough to enter the consumer space as a proper form of storage drive, and not just for USB sticks or memory cards.

For comparison, here is the rough trajectory of price:

1978: $9.1 million/GB

1985: $920,000/GB

1990: $50,000/GB

1995: $1,000/GB [0]

2005: $100/GB

2006 is when the first commercial SSDs hit market in laptops, and by 2010 they were ~$3/GB, which brought them to wide adoption.

[0]: 2000 was a stagnant year, prices stayed the same from my research between ‘95 and the new millennium before dropping in 2005.

Not all of it, but certainly enough to keep the price high.