
Stepping into math: Open-sourcing our step-by-step solver - shreyans
https://blog.socratic.org/stepping-into-math-open-sourcing-our-step-by-step-solver-9b5da066ae36
======
analog31
This seems interesting because it addresses the issue of "show your work."
Many years ago, I spent a semester teaching the freshman algebra course at the
nearby Big 10 university. This is the course that you take if you don't get
into calculus. My students were bright kids -- they were all admitted to the
state flagship school -- but not mathematicians.

There was huge variation in the preparation that kids brought with them from
high school. In particular, very few of them understood what "show your work"
means. They were told "show your work," but nobody told them what it really
entails. Is it just to provide evidence that you did some work, to deter
cheating, or is it something else? Many of my students were taught "test
taking skills" such as the guess-and-try method. So on one exam, a question
was:

x^3 = 27

One student's work:

1^3 = 1

2^3 = 8

3^3 = 27

Answer = 3

I asked the professors to tell me what "show your work" means. None of them
had a good answer! These were the top mathematicians in the world. I wanted to
talk with my students about it, but I'm not even sure that my own answer was
very good.

But if we did well in math, then we just know what it means. It's not just
evidence that you did the work. It doesn't mean "turn in all of your chicken
scratch along with the answers." It means something along the lines of
supplying a step-by-step argument, identifying the premises and connecting
them with the conclusion, in a language that is "accepted," i.e., that mimics
the language of the textbook / teacher. In fact, the reason to read the
textbook and attend lectures, is to learn that language. (It's not so
different in the humanities courses).

At least, that's my take on it, as just one teacher with one semester's worth
of experience.

In my view, a problem solving tool that actually addresses the process of
building the _argument_ and not just determining the answer, would be
beneficial to students.

~~~
yequalsx
Show your work means that the process for doing the problem works in a
sufficiently generic situation. The student's methodology would not work for
solving the equation:

x^3 = 7

Hence I would mark the student's answer wrong. At least at the level of a
college algebra course. Guess and try is fine for getting an intuition for a
problem but it is not fine as a solution.

EDIT: I don't care about down votes but I'm interested why there is
disagreement with what I wrote. I've been teaching mathematics at the college
level for 20 years and I believe my answer above fits with what most college
level teachers think about the topic.

So, why do you disagree with what I wrote?

~~~
Eliezer
I HATED "show your work". You want a proof, ask me for a proof. It's not my
fault I can solve your problem in my head. _Especially_ horrifying is if you
want me to solve the problem a particular way. There is no better way of
turning the fun and freedom of math into drudgework. You are licensed to ask
me to arrive at the correct answer, or to prove a theorem from premises;
telling me how to do it is mathematical abuse.

~~~
yequalsx
The answer is not important. What is important is that one knows a process for
solving the general situation. Knowing how to get the answer in a problem with
easy numbers is not the point. Demonstrating how to find it no matter what
numbers I used is the point.

There is also the fact that I'm not just teaching the student so they can pass
this class. I'm teaching them so that they can pass the next class too. In
beginning algebra, our lowest level course, we start with basic problems.

For instance, you drive for 3 hours and travel 150 miles. What's your average
speed. Almost every person gets this right. But many can't do this problem.
You drive a car for 4/3 an hour and travel 100/6 miles. What is your average
speed? Now many can't do this problem. They ask, which way does the division
occur?

Our goal is that they know and understand a general process. We are setting
them up to solve more advanced problems. Problems that can't be done in their
head. If you can't write the steps out in the simple case you'll never
understand the harder problems. Problems that involve quadratic functions or
trig functions.

Mathematics is a human activity and communication is part of it. Knowing how
to communicate what is in your brain to another person in a way that they can
understand is very valuable. I give the students nice numbers and in exchange
they are expected to tell me a general way of solving that type of problem.
Instead of feeding them for a day I want to teach them how to fish.

I strongly disagree with your belief that this mathematical abuse.

~~~
webmaven
_> Knowing how to get the answer in a problem with easy numbers is not the
point. _

So, don't use _easy_ numbers?

 _> I give the students nice numbers and in exchange they are expected to tell
me a general way of solving that type of problem._

If this quid-pro-quo is explicit, that seems fair. But I still don't see why
"nice numbers" are necessary.

~~~
yequalsx
In this case nice numbers are not necessary. In other types of problems they
are. One does not want to burden students with needlessly complicated
calculations when the main point is a simple one.

------
tgb
Has anyone done a study to see if this kind of aided solving actually helps
students learn? I'm worried that "Eh, I'll just write this solution down
today, I'm sure I'll learn it tomorrow" is what's happens.

Awesome software though.

~~~
empath75
Yeah, seems like it's going to be used for cheating on homework. (ie: "Show
your work")

~~~
maxerickson
It should be used to replace homework!

A system that can break a problem out into steps should be able to assess the
students understanding of the steps and even give them appropriate practice
problems.

------
jorgemf
Some years ago I tried to do something a bit more complex:
[http://telauges.appspot.com/mathsolver/](http://telauges.appspot.com/mathsolver/)

My idea was to use planning and A* search to solve any type of math problem,
even create probes for things like the quadratic equation
[https://en.wikipedia.org/wiki/Quadratic_equation](https://en.wikipedia.org/wiki/Quadratic_equation)
. I gave up after learnt the search space was so big for it that it was
impossible to solve. If I had to do it today I will explore deep learning as
heuristic, but I think it probably wont work.

I always like to see this type of projects, I hope they succeed where I
failed.

~~~
jahewson
Perhaps that was an over-ambitious project given that Automated Theorem
Proving is an entire field of research.

[https://en.m.wikipedia.org/wiki/Automated_theorem_proving](https://en.m.wikipedia.org/wiki/Automated_theorem_proving)

~~~
jorgemf
It was during my first months in my PhD when it wasn't clear what I will do.
My idea was to define the theorems as operators of planning algorithms and
then to use the planning techniques to create new theorems or probes. But
planning is not good for long plans as theorem proving, so we moved to
something simpler: linear equations and A* algorithm. Then we found the math
equations have different representations for the same thing and the branch
factor is quite big. Moreover the heuristic of trying to minimize the
equations doesn't give good results some times. That is why the prototype
works for the example, but for other examples the a* gets lost in the search
because of branch factor and equation representation.

The problem with the equation representation is that if you don't find a good
one, then you cannot make searches in hash tables efficiently. You end up with
a lot of equations duplicated with different representations. And the
representation, I used trees for it, was important for the operators.

Planning was a very nice idea because the algorithms already deal with the
heuristics. But algorithms as FF
[http://www.cs.toronto.edu/~sheila/2542/s14/A1/hoffmannebel-F...](http://www.cs.toronto.edu/~sheila/2542/s14/A1/hoffmannebel-
FF-jair01.pdf) wouldn't work due to the branch factor and the relaxation of
the problem it performs.

------
yequalsx
It's a nice program and I can see it being both helpful and harmful. From my
perspective, as a teacher of mathematics at a community college, students are
unwilling to engage in thought about a problem. If they can't see the solution
in a few minutes then they want to look at a complete solution. Mostly they
are not willing to struggle through a problem.

I vacillate on whether, with the advent of computer algebra systems, it is
necessary for students to master algebraic manipulations. I started to think
that conceptual questions are better.

For instance, give me an example of an equation with no solution. Explain how
a baseball player can have the highest batting average the first half of a
season and in the second half of a season but not have the highest overall
average. Draw the graph of a function defined on [0, 1] but has not maximum or
minimum.

Students can't do those types of problems either. They are very frustrating
problems for students because it requires you to really think about what the
words mean and to think of extreme situations. So I've reverted back to the
traditional style of teaching math. Manipulation of symbols.

~~~
bigger_cheese
When I was in year 12, 15 years ago, my math teacher introduced our class to a
website called "Wolfram's Integrator" which could simplify complicated
integrals automatically for you.

I just googled it and it still exists:
[http://www.wolframalpha.com/calculators/integral-
calculator/...](http://www.wolframalpha.com/calculators/integral-
calculator/?redirected=true)

At the time I thought it was pretty cool in a passing trivia kind of way but
didn't make much use of it.

A year after that I was a first year university student all of our linear
algebra tutorials were taught in the labs using a computer program called
"Maple". I really struggled to wrap my head around it. I didn't do well in the
class until I started writing out the problems myself and solving them on
paper.

I found at least for me personally that inputting problems into a computer and
having it spit the answer out wasn't teaching me anything (besides which
functions to call), in other words I was learning the programming language and
not the underlying concepts.

Nowadays I work with FEA and LP solvers and I rely on computer assistance all
the time to do my job. I'd like to think having a firm grasp of the underlying
math is advantageous and makes me a better engineer but I know there are
people around that get by just by "plugging things into Ansys".

------
stdbrouw
Worked on something like this as a hobby project a while ago, but to avoid the
complexities associated with solving arbitrary exercises, instead I had it set
up as an algebra exercise generator: you start with the solution, which you
then (algorithmically) obfuscate by splitting terms and recombining things for
a couple of rounds. Never got around to finishing it, but the neat thing is
that you've already generated one possible way to solve the problem, it's just
how you generated the exercise in reverse.

Another thing that's quite easy to do is to check intermediate steps in a
solution for equivalence. You don't even really need CAS, just brute force the
problem by probing the equations: set all variables to randomly chosen values,
n times and if the sets of results are the same for both equations, you're
good.

Anyhow, Socratic looks great and a great deal more advanced and useful than
what I came up with, so kudos!

------
benbristow
I'm jealous of kids these days... homework would've been so much easier with
this.

You could always use a calculator but the whole 'show your own working' catch
meant you had to do it all manually. Not any more!

~~~
maxerickson
Hopefully the kids of tomorrow end up getting less homework because of things
like this.

Consider a system that combines practice and assessment. It could
individualize both, reducing the need to force students that have mastered a
concept to do repetitive practice.

It might be a big challenge to get such a thing to work well, but let's not
look back at our schooling as an anchor for what students today must do.

~~~
jerf
I once designed myself a system for teaching math online. In addition to using
an engine like what they're showing here, my plan was to start from more-or-
less the beginning, but also track and categorize the errors a given student
makes over time. Errors are not generally made randomly, there's a pattern to
them.

Then, we could allow the student to "solve" an equation the way they really
should, by skipping over two or three steps at a time, but when we see them do
something wrong, we can use our table of "errors the student is most likely to
make" to explore the space of possible errors between step 4 and step 5, and
give focused feedback about what they did wrong. Using that table there's
really only a couple hundred possibilities; if that fails we can always ask
the student themselves to break it down more tightly. Presumably if someone
were making a business out of this, there would be someone on the lookout for
errors the computer can't figure out to add what rules they can to the system.
(Though there will always be an irreducible residue of incomprehensible
error.)

Teaching a student math would then be about reducing each of these errors to
zero over time. You'd have the computer custom create problems that hold a
constant probability of the student making an error at some point during
solving it; say 20% or so. Then as the student demonstrates mastery, you
naturally make the problems more complex as you have to put in more steps to
make the probability of error go that high.

Instead of a klunky, chunky "ok now we learn this and you blindly practice it,
now you learn this and you blindly practice it, and we hope at the end you've
learned everything we taught", you would in theory get a naturally
progressive, customized difficulty curve that keeps the student continuously
engaged with being about 80% correct, but always progressing forward. This
approach also naturally ensures that just because we're covering the quadratic
equation this week does not mean you get to forget fractions; once you've seen
the simple stuff with integers, we're naturally going to fold fractions back
in to the problems, for instance.

There's some elaborations on the theme after that, such as pre-examining the
generated problems to ensure that the most likely mistakes are all
distinguishable by producing different error output.

But I don't have time to do this. I'm at least reasonably confident it would
work, though.

~~~
nooron
Diagknowstics is working on this in Ann Arbor. Not sure how far along they
are, but it's a very bright team.

~~~
jerf
Interesting. I'm literally 15 minutes' walk from them right now. I wish them
well; I'd love to have something like this for my own kids in the 3-4 years
it'll take for it to be useful for them.

------
therealmarv
Does anyone know if there is a good open source library for making equations
(Latex, MathML) out of pictures like in their demo?

~~~
auria
Hi! This is Evy (I wrote the post)

I know this doesn't answer your question, but at Socratic we use an API that
we pay the creator of [http://mathpix.com/](http://mathpix.com/) for. It would
definitely be super cool to see an open source library for this :)

~~~
therealmarv
Thanks anyway. Getting equations into the laptop (editor) or smartphone is not
easy.

------
equalunique
My academic math journey stopped at pre-calc, and I had been a C student for
quite a long time. HS Algebra II would never have happened for me if I hadn't
discovered XMaxima, an emacs-based CAS. Fortunately I took a Discrete Math
course before dropping out of college, and it gave me a new admiration for
math.

In spite of my weak math background, this has been the most enjoyable comments
section on HN I've read so far.

------
MichaelBurge
People here keep saying this will change learning and be good for the
students, but the only real difference is it's open-source. You can already
get step-by-step solutions for more types of problems from Wolfram Alpha, and
you can already get API access if you're a 3rd-party developer who needs it:

[http://www.wolframalpha.com/input/?i=2*y+-+x+%3D+(8+*+x+%2B+...](http://www.wolframalpha.com/input/?i=2*y+-+x+%3D+\(8+*+x+%2B+2\))

I don't think it will have any real effect.

~~~
chipperyman573
I think you need the paid Wolfram Alpha to get step-by-step solutions

Source: Paid for it for that feature.

------
chriswarbo
Very interesting work, and well-explained in the post.

Like many others here, I suppose that in it's basic form this would mostly be
used for cheating on homework; although it would certainly be useful for those
(few?) students who are truly motivated to self-learn the material, rather
than just pass the tests.

One thing which springs to mind is "Benny's Conception of Rules and Answers in
IPI Mathematics" (
[https://msu.edu/course/cep/953/readings/erlwanger.pdf‎](https://msu.edu/course/cep/953/readings/erlwanger.pdf‎)
), which shows the problem of only focusing on answers, and on "general
purpose" problem sets. Namely that incorrect rules or concepts might be
learned, if they're reenforced by occasionally giving the right answer.

I think it would be interesting to have a system capable of some back-and-
forth interactivity: the default mode would be the usual, going through some
examples, have the student attempt some simple problems, then trickier ones,
and so on.

At the same time, the system would be trying to guess what rules/strategies
the student is following: looking for patterns, e.g. via something like
inductive logic programming. We would treat the student as a "black box",
which we can learn about by posing carefully crafted questions.

Each question can be treated as an experiment, where we want to learn the most
information about the student's thinking: if strategies A and B could both
lead to the answers given by the student, we construct a question which leads
to different answers depending on whether A or B were used to solve it; that
gives us information about which strategy is more likely to be used by the
student, or maybe the answer we get is poorly explained by A and B, and we
have to guess some other strategies they might be using.

Rather than viewing marking as a comparison between answer and a key, we can
instead infer a model of the domain from those answers and compare _that_ to
an accurate model of the domain.

We can also use this approach the other way around, treating the _domain_ as a
black box (which it is, from the student's perspective) and choosing examples
which give the student most information about it.

------
Steeeve
Now... the only thing remaining is to translate this to common core :).

I say that in jest, but doing so would make common core much easier for
parents AND teachers to grasp. There's an enormous divide between those who
get it and those who hate it, and providing parents/teachers with something
that would help them understand the benefits of common core concepts would be
a gigantic win.

~~~
mkesper
What's common core in this context?

~~~
omaranto
It's a math and English curriculum used in the US up to the 12th grade.

[https://en.wikipedia.org/wiki/Common_Core_State_Standards_In...](https://en.wikipedia.org/wiki/Common_Core_State_Standards_Initiative)

------
aidos
That's so cool.

Reminds me of how different the learning experience is now. When we were at
school (80s/90s), there was nowhere to turn if you didn't have the answer. My
parents had an Encyclopedia Britannica set, so at least there was a paragraph
to go on. It's amazing how good you became at fleshing out that paragraph into
an essay :-)

------
gravypod
Now that this exists I think it's worth creating an opensource version of the
TI-Nspire for engineers & mathamaticians. Something based on cheap hardware,
runs linux, and can implement this + a theorum prover to basically make the
most handy lab calculator.

~~~
rkcf
Could base it off of a raspberry pi with a touch LCD. It would make a good
teaching project for a highschool class.

------
poseid
that feels like a nice application of AI in a way. we often use a computer
that can help in making a plan (e.g. a kind of map or "steps" as here). this
might be nice to help understand problem solving in general. also, nice to see
the project is in javascript, that means quite a few non-professional
programmers could learn from it.

------
JotForm
This is such an inspiring software.

------
GrumpyNl
It looks like Sheldon came through.

