This reminds me of work by one of my professors at Rochester Institute of Technology (Dr. Richard Zanibbi). There is an application called Freehand Formula Entry System (FFES) available on his website for download (GPL source as well) [1]. He also published a paper on it titled "Recognition and Retrieval of Mathematical Expressions" (2012) [2].
[1] https://www.cs.rit.edu/~rlaz/ffes/
[2] https://www.cs.rit.edu/~rlaz/files/mathSurvey.pdf