Evaluating Large Language Models Trained on Code

[vok]

https://arxiv.org/abs/2107.03374

[tyingq]

A technicality perhaps, but their example of a python closest_integer() function chokes on some valid Python numeric literals. Feeding it "0.490_000", for example, pops a ValueError exception.