I also recently went through the specs and noticed "with sparsity" but I didn't quite understand what it specifically refers to - the premise is that a lot of weights in matmul operations will be zero or insignificant - also known as sparse matrices - and in that case A100/H100 has a circuitry that can boost the throughput up to 2x, essentially "skipping" half of the FLOPS as you say.
I am not an expert in LLM but I don't think you can end up having a significant amount of zeroed weights (~50%) in a converged network so I think it is safe to say that the theoretical throughput for 99% of cases is really ~800 TFLOPS and not ~1600 TFLOPS as advertised.
There are two populations of people reading the NVIDIA specs (and now you switched groups). If NVIDIA ever changes their marketing strategy and the asterisk denotes something else, there might be a third population because I know a lot of people that I suspect will keep dividing those starred FLOPS/s by two :-)
> this is one of the main things Rust users don’t promote enough
This is probably because using internal mutability not to achieve a valid design goal (such as a controlled side-effect on an otherwise immutable entity), but to side-step the borrow checker for ease of programming, is not considered idiomatic Rust and even though it makes a good educational tool, it should rather not end up in production codebases.
Firstly, when using regular references, you cannot create multiple mutable references. This rule prevents subtle bugs such as data races. When using internal mutability, you still keep that protection, but it is (undesirably) delayed from compile time to runtime, potentially causing an unavoidable crash/panic.
Secondly, when using regular references, you cannot create even a single mutable reference when an immutable reference already exists. This rule prevents subtle bugs such as unexpected mutation ("from under you") of data that was passed-in as immutable. When using internal mutability, you throw away that protection, since multiple immutable reference owners can request a mutable reference (even if only one at a time).
use std::rc::Rc;
use std::cell::RefCell;
let entity1 = Rc::new(RefCell::new(42));
let /* immutable */ entity2 = entity1.clone();
*entity1.borrow_mut() += 27;
assert_ne!(*entity2.borrow_mut(), 42);
A similar thing actually happened to me. I was receiving a small package (probably a phone case) about once every few weeks while living in a rental house.
The seller had put my address as their address, so all undeliverable mail came to my house.
I brought a bag full of these packages to the post office, explained the situation and gave them the packages. The entire pile of packages ended up back at my house a few days later.
I found the seller on Amazon and messaged them:
———
Me: I haven't ordered anything from you. I have received 6 packages that you sent but ended up at my house because the return address that you used is my address, not your address. Could you use a different return address?
Seller: Dear nulltype, We're sorry about this, we can't change the address at the moment because of the cumbersome policies on Amazon, but we'll change it when we have a new production facility. Please forgive us. You are free to use the packages. Sincerely,
If you read the section titled "Bizarre allegations by Reddit of Apollo "blackmailing" and "threatening" Reddit", he is directly addressing Steve. He starts with the transcript of the private mod call with Steve and then begins addressing Steve directly. The "you" in that section is the Steve Huffman he had calls with who heard the "threatening" bit
Didn’t the DGX-1 come out 9 years ago?
reply