Jan Leike's point is a bit more subtle there. He's not surprised it can learn translation with a relatively small number of training examples, but surprised that it seems to know some instruction with no examples.
And just a few Tweets later speculates it's likely because there are examples in the training data, and the success of the model to follow instructions in a non-English language from "nothing" is proportional to the presence of those examples in the training data [0].
I don't think he would disagree with anything in my statement.
And just a few Tweets later speculates it's likely because there are examples in the training data, and the success of the model to follow instructions in a non-English language from "nothing" is proportional to the presence of those examples in the training data [0].
I don't think he would disagree with anything in my statement.
0. https://mobile.twitter.com/janleike/status/16252150919889264...