On: The lottery ticket hypothesis: why neural networks work

[paulsutter]

That’s exactly how it works, read up on pruning. You can ignore most of the weights and still get great results. One issue is that sparse matrices are vastly less efficient to multiply.

But yes you’ve got it