There's already open-source stuff [1][2] out there that can lipread video to transcribe text, but the word error rate is pretty high (and it may not be physically possible to get this down without doing something like subvocal and having other sensors near your face).
1. https://github.com/mpc001/auto_avsr 2. https://github.com/amywork777/lipflow