
LaTeX, Instantly: Extract LaTeX from PDFs or Handwritten Notes - DyslexicAtheist
https://mathpix.com/
======
miki123211
Oh god. An app like that, with a full pdf to LaTeX file converter, would be a
godsend for the blind community. We usually sturggle a lot with mathematics in
PDF documents, as it's usually not accessible for screen readers at all. TeX
isn't the most pleasant thing to listen to, but it's at least plain asci text
that any screen reader can process.

~~~
nicodjimenez
Hey, Mathpix founder here. I'm doing some research on how we can do just that,
would love to chat if you're up for it, please ping me nico@mathpix.com.

~~~
georgewsinger
This is a 1-liner in Wolfram Programming Language:
‘Speak[SpokenString[ToExpression[texExpression, TeXForm]]]’, for
‘texExpression’ an arbitrary TeX formula.

See
[https://reference.wolfram.com/language/ref/SpokenString.html](https://reference.wolfram.com/language/ref/SpokenString.html)

I’ve been looking for a way to convert math texts to audiobooks, but haven’t
actually implemented anything.

~~~
georgewsinger
See
[https://www.wolframcloud.com/obj/george.w.singer/texToSpoken...](https://www.wolframcloud.com/obj/george.w.singer/texToSpokenString)
for a demonstration. Put any tex string into the form and it will be converted
into a spoken formula.

------
devwastaken
I've been looking for an open source solution to take drawn math and convert
to text. I don't know if this is it but we certainly need more of it.

Windows comes with one I don't remember the name of, but I'd rather like to
have a simple web browser canvas or webgl that could be drawn using a pen on a
smartphone.

~~~
oefrha
Last I looked into open source LaTeX OCR, albeit two or three years ago, the
landscape was hardly different from this 2010 answer:
[https://tex.stackexchange.com/a/1476](https://tex.stackexchange.com/a/1476)
i.e. pretty sad.

(This is not it because it’s a SaaS product.)

Edit: Also, on the commercial side, I’ve used several MyScript products.
They’re decent, but not amazing.

------
MayeulC
Well, this one is quite interesting, and would integrate nicely with tablet
inputs for IDEs, but I have been burned often enough by non-free tools or SAAS
(as well as the philosophy/ethics side) that I can't resolve to make closed-
source and nonfree software part of my workflow. As a one-off, though? Maybe.

~~~
nicodjimenez
Mathpix Snip is free up to 50 images per month. Also, we are open sourcing our
JS rendering stack very soon, which we hope will make publishing scientific
content online more competitive with sharing PDF links.

~~~
ssivark
FWIW, totally agreed with @MayeulC's comment (to the folks who might be
willing to pay for this, the content is _too damn important_ to risk platform
lock-in. For this service _to even get consideration_ , I would imagine that
everything must be cleanly exportable, such that one needs only pay for
continued use of the service).

That said, where I would seek a tool that _really_ adds value is auto-
converting notes (text and math interspersed) into LaTeX, so that (as a value
add) it can also be intelligently searched (coupled with an indexing+search
solution). I think many academics/knowledge-workers might be open to paying
for such a service.

Bonus if you can work well off just smartphone photographs of pages.

~~~
nicodjimenez
The content generated from Mathpix Snip can be pasted into any Latex
compatible editor, so that avoids the lock-in you sometimes get with
proprietary WYSIWYG editors. Plus as I mentioned we'll be open sourcing our
rendering engine soon.

Thanks for the suggestion about adding search capabilities, we're thinking
about this.

If you curious about it, I would suggest to just sign up and try it. If you
spend a lot of time typing up Latex it's definitely worth having, especially
for matrices / aligned equations.

------
meanmrmustard92
They recently implemented a pretty restrictive freemium subscription model
that has tempered my enthusiasm for it massively. 50 snips a month is nothing
for grad students.

~~~
danmg
Grad students will just just maple/mathematica to directly generate latex from
an expression.

~~~
jefft255
Which grad students? In CS the use of these is long gone (at least in the two
departments I’ve been in). I wonder how prevalent they still are in physics.

But I’m pretty damn sure that physicists, mathematicians and computer
scientists all just type their equations straight into latex unless it’s a
super complicated computer generated formula, which you don’t typically write
in papers..

------
PureParadigm
I used to think converting from handwriting to LaTeX would be a really useful
program and save a lot of time, but after typing LaTeX on a regular basis, the
input speed is really not an issue. Sometimes I'll encounter a symbol I don't
know, and then I draw it in here [1] and remember it for next time. When doing
math, it's definitely not the bottleneck in getting my work done. I would
definitely be slower with handwriting conversion, especially when you factor
in fixing errors in its translation. I only see handwriting conversion being
useful as training wheels for beginners, but would be interested if someone
has a different experience.

Automatic recognition from PDFs is more promising for accessibility and
reformatting, and also seems to be easier to do since typeset math is easier
to recognize than handwriting.

[1]
[http://detexify.kirelabs.org/classify.html](http://detexify.kirelabs.org/classify.html)

~~~
grzm
If you’re not at a keyboard, or prefer to use a stylus, typing really isn’t an
option. Or if you want to capture the actual working-through of a problem,
rather than working from an already-completed work. There are a number of use
cases you might not find useful with respect to handwriting, but others might
find incredibly practical.

~~~
abdullahkhalids
I am comfortable enough with latex that I often work through problems on
latex. For certain cases, with long expressions, it is actually simpler than
handwriting, because I can copy an equation onto the next line and edit it in-
place, hence, taking less time and ensuring I am not missing terms.

I have a friend who got so into latex that a few times he would mindlessly use
latex code while handwriting things, like writing "\alpha" on the board
instead of drawing the greek alphabet.

------
max_likelihood
Just curious if there is any plan for the Mathpix OCR/engine to transcribe
tablet notes in real-time to a LaTeX document? That would be the holy grail
for me.

~~~
nicodjimenez
Yes we're looking at this. What type of tablet do you own?

~~~
max_likelihood
Currently I'm not using a tablet. But, I'm doing an M.S. in Actuarial Science.
With > 90% of my notes being formulas, I would buy a tablet just for this
feature :)

------
16yoMLDev
I actually worked on transcribing math expressions to latex a few months ago
with a Resnet backbone + LSTM + Attention.

All my code to train the model is available On GitHub:
[https://github.com/bkkaggle/math-
recognition](https://github.com/bkkaggle/math-recognition)

If anyone wants to chat about math expression OCR, I'd love to talk about it!

It's a surprisingly difficult and interesting (and under appreciated and
unknown) area of research!

------
smsx
This doesn't seem to support Wayland? Trying to create a snippet corrupts my
whole screen. If I can get it to work, it seems quite useful.

This is what I get when trying to capture an image:
[https://cdn.mathpix.com/snip/images/OvK0viJ9Q4k4evoCpnfvtIFl...](https://cdn.mathpix.com/snip/images/OvK0viJ9Q4k4evoCpnfvtIFlyZA9n4W43tIysMr0LjI.original.fullsize.png)

~~~
nicodjimenez
Hi, no we do not support Wayland. You can, however, try pasting images
directly into our notes app snip.mathpix.com by copying the image to your
clipboard and then pasting directly into the document.

------
gammarator
I used this recently as part of a literature review I was doing. Worked
_amazingly_ well and sped my work up substantially. Highly recommended.

------
sokoloff
I enjoyed the Isaac Newton "testimonial" (the others as well, but that one was
the first I read).

------
heinrichf
@nicodjimenez Could you give the HN audience a quick idea of the technology
behind this? Recurrent ConvNets like in
[https://guillaumegenthial.github.io/image-to-
latex.html](https://guillaumegenthial.github.io/image-to-latex.html) ? I
remember seeing an "image to LaTeX" academic challenge, do you plan competing
in it, if it still exists?

------
dvt
FYI, I'm pretty sure this is wrong:

    
    
        \ell ( \theta ) = \sum _ { i = 1} ^ { m } \operatorname{log} p ( x ; \theta )
    

Given that we're summing over i = 1 to m (in the picture), I think that's
supposed to be x_i :) Impressive, nonetheless.

------
ygra
Just one note: Ctrl+Alt+M is a horrible choice of shortcut, as it conflicts
with typing characters on many keyboard layouts. μ on my keyboard, for
example. Ctrl+Alt is identical to AltGr and thus should be avoided for
shortcuts in general.

~~~
kait_m_cunn
You can record the shortcut to be whatever you'd like in your settings :)

------
thedudeabides5
Has anyone on this thread actually used this?

Would be interested in a product review

------
melling
Is there a recommended cloud or app solution that works well on the
iPhone/iPad for writing a lot of LaTex?

~~~
_ph_
There are several LaTex environments for iOS. I habe been using TexWriter, but
very lightly so far. It worked well for me.

------
nathell
I wonder if it could be (ab)used to LaTeX-ize plain TeX.

------
Iwan-Zotow
there is also
[https://github.com/falvaro/seshat](https://github.com/falvaro/seshat)

------
throwlaplace
i've used this. it's a really good solution to an odd problem. how often do
you need the latex from a paper or book or whatever instead of a screen cap?
if you're going to fold it into your own writing then you'll just write it out
yourself since most stuff isn't _that_ complicated (and because ostensibly you
know latex already). if you're not going to fold it in (and just use it in a
powerpoint or something) then max-zoom+copypaste works fine.

on the other hand if it could do tikz now that would be magical (but as well
know that's much much harder).

