On: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

[verdverm]

the analysis is more akin to analyzing with & without dropout, where a common number is to drop a random 50% of connections during a pass for training, thus forcing the model to not rely on specific nodes or connections

When you look at a specific input, you can look to see what gets activated or not. Orthogonal but related ideas for inspecting the activations to see effects