My read is that it wasn't one-shot, Astra was ran in a harness where it could change the weights, then Qwen would run with those new weights and be asked to do math, and its answers were fed back to Astra. So Astra could experiment with different weights until it saw better math answers from Qwen, with the actual benchmark run separately afterward.
And for the games, it was given as context the model architecture, game rules, environment description, etc. And since people assume Astra likely had knowledge in its training of existing interpretability research and weight-space programming, it can leverage that, combined with knowing the model arch and so on, it likely gives it a good base to determine what the weights do and how to change them.
And for the games, it was given as context the model architecture, game rules, environment description, etc. And since people assume Astra likely had knowledge in its training of existing interpretability research and weight-space programming, it can leverage that, combined with knowing the model arch and so on, it likely gives it a good base to determine what the weights do and how to change them.