The important thing here is they have a bonding method working for stacking DRAM with logic and apparently also solved the thermals well enough too.
Logic is usually rated to run up to 110*C but DRAM processes aren't rated for such high temperatures and perform worse the hotter they get in terms of retention etc.
There are also some novel approaches here in terms of banking choices and impact on refresh times as a result.
The end result that matters for normies is that if this process is turns out to be a success it will challenge and perhaps beat HBM.
That in turn matters because this process is vastly more wafer efficient than HBM for the same bandwidth. Which in theory would ease supply on DRAM wafers.
Except it is also more efficient in terms of energy/bit/s so very likely what will happen is chips will still target the same TDP and just use even more of this rather than consume less wafers because that is just how AI is today.
Disclaimer: I work for a competing company that also makes AI inference chips.
Depends on how well they translate that effort into promotions or company switches into better positions.
I averaged around 18mo to either a promotion or a move in my 20s, mostly off the back of working harder (to get the promotions/comp increases) and working smarter (to work out when they aren't coming and jumping somewhere else).
If this makes sense for someone has a lot more to do with their ambition/desire and ability to work the system than anything else. It also helps a ton if you are also legitimately very good at what you do. Then it's pretty easy because you are just demanding you are paid commensurate with impact.
Simply doing overtime and having hustle doesn't cut it though, you actually need to be able to leverage it into career progress or it's just wasted life.
They are slow walking it because you simply can't build the generation fast enough anyway. It's also why you are seeing a lot of project with their own generation plants on site, either stuff like the SpaceX/X/AI (I dunno what you call it man) datacenters - i.e very adhoc methane stuff or proper LNG generators ala Meta.
I think a large part is being close enough to existing utilities to save the installation costs of 40 miles of cable, telecomms etc (all duplicated of course). However, some data centres are built in the middle of nowhere but that is potentially then a nature problem because why spoil this nice countryside with a massive non-descript building?
I like Copenhill (https://www.visitcopenhagen.com/copenhagen/planning/copenhil...), where they decided that waste handling is not nice so why not make it nice by building a ski slope on it. I suspect that a bit more creativity could make people want data centres (free heating and hot water anybody? Hot spring baths? Free hosting for local schools and charities?)
I actually happen to know a lot about this! I have been in and around datacenters for ~16yrs and I work on direct liquid cooled hardware at $DAY_JOB.
Back when I was running my own startup I moved to Mascot, Sydney to be near Equinix SY3 that was being built which we had bought a decent chunk of committed capacity in. Mascot was originally an industrial area but was gentrified a lot over the last 20ish years but the datacenters, meat packers etc are all still there. It really adds to the charm of the place in a lot of ways, warehouses/factories that have been converted to cafes and breweries.
Which is to say I don't think living near a DC is bad at all, they aren't very loud and they just look like big warehouses with lots of lights, cameras and no windows. Oh and security dudes.
Anyways. Yes. The hardware I am working at $DAY_JOB is all direct liquid cooled.
The coolant that actually cools the hardware is closed loop, it's not expended. Which means you don't actually use any of it and it's considered good for the entire life of the hardware.
I normally don't directly talk about $DAY_JOB things because there are pretty strict rules so I can only use publically referenced sources but you can read a little bit about the coolant we use in this article [1].
The facility loop though is usually what people are worried about. A lot of datacentres that were built in low humidity environments use evaporative cooling which does indeed use a ton of water. However most data centers in more humid environments don't and either use loops that recycle the water but dump the heat (i.e rivers/lakes) or use closed loop systems with radiators etc.
It's worth mentioning though that evaporative cooling is extremely power efficient vs most closed loop solutions so there is a tradeoff here. I don't think Google could achieve their fleetwide PUE (Power Usage Efficiency, basically measurement of how much power goes to things in a DC that isn't a computer) numbers without relying on it heavily.
Given the difficulties getting DCs approved in much of the world right now and the backlog on generation capacity (i.e power) almost all modern DCs are being built to use as little water as possible.
To give you an idea I think evaporative cooling of the type you would find in a state of the art Google DC with a PUE of ~1.1ish might use 1-1.1L of water for every kWh of actual equipment load (this is a back of the napkin estimate, don't hold me to it).
Conversely true closed loop will use approximately zero.
So yeah, evaporative is a big waster of water especially at GW scale.
Thanks a ton, for the information. It's greatly appreciated.
I'm personally not interested in the details that you can't provide, that was not something I was aiming for, but putting a ballpark number for ~1.1L/kWh is very eye opening (I know it's not some accurate figure, but yeah). Considering a SOTA GPU uses somewhere from 700W to 2.5kW depending on the manufacturer and performance level...
Now I'm wondering how the dumped heat affect these water bodies (i.e. rivers, lakes, etc.). Their relative heat capacity w.r.t. this amount dumped heat can't be infinite so they can handle this forever.
> NIMBYs are the enemy of all that is good in the world.
Also see the Green parties / Greenpeace and the decline of nuclear power stations. We could have had clean, cheap energy across Europe and America, but the fear of nuclear is fully embedded in the public psyche.
Noise and air pollution are valid concerns with a lot of these datacentres being built as well. Some of these recent datacentres have been built close to residential areas, and the health harms are far more worrying than the property valuations IMO.
They need power and a lot of it. The one in Memphis built by Musk & co. currently uses 350MW & they are planning to expand it to 950MW. But these aren't even that bad because Google has data centers that use upwards of 2GW or more (Project Tembo (Wyoming)).
I vastly prefer Sol. It does what I tell it to almost exactly, pretty much every time.
I work on very low level stuff (think RTL/FPGA, firmware, software where optimising for nanoseconds is just normal).
For me Sol is the only cost effective model available. Fable 5.1 is indeed good and vastly better than original Fable (which refused to work on most of my stuff for 'safety' reasons).
It's very good at this sort of low level stuff to the point that I really can't understand/relate to people having a good time with Opus (which comparatively performs extremely poorly on my particular workload).
I also just don't like how lazy Anthropic models are. They will do 10% of what is asked and then summarily declare victory.
Sol on the other hand is more like "one of us", slight touch of the 'tism, extremely pedantic, will go to the edge of the known universe if that is what it takes to prove/fix/build what you asked for or run out out of credits trying.
It's a personal and workload dependent thing. For me right now Sol for 99% of stuff because Fable 5.1 still burns through $5k in credits a day.
Agree 100%. And I also work a lot on lower level / systems stuff (including RTL here and there, too). Opus is sloppy, and leaves negative cases all over. The GPT models in Codex have a more pedantic and detail oriented "personality." Often to a fault.
Sol will leave a mess of excessive redundant tests and isn't so great at abstraction ; but it produces more reliable working systems.
It's kind of nice to have access to both, but I don't have the $$ for that right now, so I just keep the Codex sub
Can confirm this as well, mostly VHDL and HLS. Sol and Fable can reason about performance and designs consistently. Whereas Opus and others seem to just throw generic optimisation techniques at the wall unprovoked (while hallucinating a justification + expected improvement) until the synth reports improve.
The plane was a significant factor but you really shouldn't look past the calibre of expertise in the cabin on this particular flight.
> 5 pilots were in the cockpit of this flight. In addition to the normal crew of pilot-in-command and co-pilot, there was an experienced relief pilot and two additional check captains; one was being trained as a check captain (CC) and the supervising CC, who was training the CC.
Captain Richard Champion de Crespigny had 35 years of flying experience at the time of the incident.
That is an insane amount of flight experience in the cockpit and all of the reports state this was a very significant factor. i.e by allowing distribution of the task load, plenty of time for one of the pilots to compute landing calculations manually etc.
THe A380 is a flying tank no doubt, but it wasn't the sole factor here - I think most people would agree this one was a shared victory.
It was probably helped by the fact that airbus didn’t try to pretend that the aircraft was just a slightly different version of some previous aircraft and therefore doesn’t require any additional training lol.
Logic is usually rated to run up to 110*C but DRAM processes aren't rated for such high temperatures and perform worse the hotter they get in terms of retention etc.
There are also some novel approaches here in terms of banking choices and impact on refresh times as a result.
The end result that matters for normies is that if this process is turns out to be a success it will challenge and perhaps beat HBM.
That in turn matters because this process is vastly more wafer efficient than HBM for the same bandwidth. Which in theory would ease supply on DRAM wafers.
Except it is also more efficient in terms of energy/bit/s so very likely what will happen is chips will still target the same TDP and just use even more of this rather than consume less wafers because that is just how AI is today.
Disclaimer: I work for a competing company that also makes AI inference chips.
reply