For those wondering, the RWKV architecture is an alternative to a transformer arch, and has the nice property of allowing very long inputs. Speculation here that it might be for code assistance would make sense. Early versions of RWKV that I played with took a long time to tokenize input strings, but generation was quick. I could imagine engineering finding a good fit with a codebase that’s going to be mostly static while an engineer is editing only parts of it.
