
Tell HN: Learn about the state-of-the-art in Program Synthesis - saurabh20n
Hi HN:  Program synthesis generates code from specifications: test-cases, I-O examples, and assertions. On September 19-20, synthesis experts will speak about recent breakthroughs and practical tools. The talks will be aimed at software engineers who haven’t encountered synthesis before. We want to extend an invitation to the HN community because the periodic posts on program synthesis and Z3 show interest in the technology.<p>Speakers include researchers from Stanford, Berkeley, UW, MIT, and Google Brain. As a preview, here are 3 topics that will be covered: hw-exploit synthesis (https:&#x2F;&#x2F;www.cs.princeton.edu&#x2F;~ctrippel&#x2F;#publications), end-user web programming (https:&#x2F;&#x2F;schasins.com&#x2F;papers&#x2F;), verification and synthesis of OS level code (https:&#x2F;&#x2F;unsat.cs.washington.edu&#x2F;projects&#x2F;).<p>Technical abstracts and registration is on: https:&#x2F;&#x2F;synthetic-minds.com&#x2F;pages&#x2F;conference&#x2F;2019&#x2F;#program. After the conference we’ll post the slides of the talks.<p>The foundations part will explain why synthesis reduces to solving an \exists\forall query. More specifically, the query asks “does there exists a program P such that for all inputs x, P(x) computes the correct output?” A solution to the query is the synthesized program P.  How to solve the query? The Z3 prover works well for boolean satisfiability (i.e., one \exists query). To solve an \exists\forall query, one approach is to have a two Z3 solvers communicate: one solver synthesizes a candidate program P’ that is correct on a sample of inputs, while the other verifies that P’ is correct on all inputs. If P’s is incorrect, a counterexample input is added to the sample on inputs.  The two solvers iterate until the latter is satisfied with the correctness. This process is not unlike GANs; it is called CEGIS (counterexample-guided inductive synthesis) and was invented 2005, and built on an earlier CEGAR (counterexample-guided abstraction refinement) technique from the 90s.<p>Please comment; or email us with questions.
======
saurabh20n
I am the OP. Along with Ras and Henele, we are leading efforts in program
synthesis at Synthetic Minds (YC S’18). I started working on synthesis in
2007; and Ras has been developing these techniques since the late 90s and is
an ACM fellow for his work in program synthesis.

The researchers will talk about their peer-reviewed work in web automation,
hardware security, operating system extensions, programming for non-
programmers, automatic code translation, and superoptimization. Hopefully,
this will illustrate the power _and_ limitations. We'd love for people to
extrapolate from these onto their own domain-specific automation needs.

By touching upon foundational techniques (making imperative code functional,
symbolic compilation, SMT encodings, partial evaluation), hopefully the leap
to "code synthesis" will seem less like magic and more like an obvious next
step. In addition, open-source frameworks exist (e.g., Rosette, Sketch) that
abstract away these foundations, and the program will cover those in hands-on
workshops.

We’d love to hear insights into application from people for whom synthesis is
new. Some problems are exciting to us (Synthetic Minds is working on smart
contract synthesis); and we’d love for the community to brainstorm
applications to their domains.

------
kyboren
Cool! Very similar techniques can be used in the hardware domain. For these
types of problems, at least, I have found SRI's yices solver to be much, much
more performant than Armando's CEGIS solver. Do you have any tips on other
solvers you've found to be faster for exists-forall problems?

Here's a raw example of a 32-bit adder super-optimized according to rough
delay, gate complexity, and wire complexity models embedded in the 2QBF:
[https://share.riseup.net/#xg2ySn41zmhtHtmrlQWn5Q](https://share.riseup.net/#xg2ySn41zmhtHtmrlQWn5Q)
. According to post-PAR results from Innovus, it can actually (barely) beat
DesignWare adders at some points in the trade-off space.

~~~
rbodik
Hi, this looks like a strong result. Congratulations!

Short answer: to scale up hardware synthesis, it may be necessary to change
the encoding rather than look for a better solver.

More details: I should first say that I have limited experience with
synthesizing hardware. My lessons come from [1], where synthesized a small
Wallace Tree multiplier. What that taught me is that hardware arithmetic
circuits should perhaps not be formulated as a 2QBF problem -- because you
might need too many counterexample inputs to terminate the CEGIS loop.

Instead, I believe that the circuit synthesis should use algebraic reasoning
(for correctness) and combinatorial reasoning to explore the space of
candidate circuits. Since one symbolic input is sufficient to show
correctness, the problem simplifies from 2QBF to SAT.

This idea is briefly explained in Sec 4.1 in [3], a project that synthesizes
software expressions that look very much like hardware circuits (permutations
and such) [2].

I am happy to discuss this in person if it might be helpful to your work. Both
Mangpo (the author of [2,3]) and I will be at the conference.

[1]
[https://ieeexplore.ieee.org/document/5227085](https://ieeexplore.ieee.org/document/5227085)

[2] [https://github.com/mangpo/swizzle-
inventor](https://github.com/mangpo/swizzle-inventor)

[3]
[https://github.com/mangpo/mangpo.github.io/blob/master/paper...](https://github.com/mangpo/mangpo.github.io/blob/master/papers/swizzle-
inventor-asplos19.pdf),

------
AthLado
This is very interesting - my eyes stuck on the programming for non-
programmers as we in MOOCTORS
([https://www.mooctors.com](https://www.mooctors.com)) we are building a
tutoring platform for MOOC students who are not necessarily programming
experts. We invite technology experts and everyone who can code to register as
tutors on their free time, help students on their MOOC courses, and earn an
extra income for doing that. How Synthesis could power tech/coding tutoring?

best, Athanasios

~~~
rbodik
Hi Athanasios, program synthesis has been used to automatically repair
introductory programming assignments (a sample of papers is at the end). This
technology could make tutors more productive: [1] says that 81% of synthesized
repairs were of high quality. This is very impressive but the tutor is still
needed to check the quality. Also, synthesis cannot repair all programs (an
empty submission cannot be repaired). Finally, this automatic repair allows
the tutor to focus on giving guidance, rather than on finding the bug in a
programming assignment.

[1]
[https://arxiv.org/pdf/1603.03165.pdf](https://arxiv.org/pdf/1603.03165.pdf)
[2]
[https://rishabhmit.bitbucket.io/papers/icse18.pdf](https://rishabhmit.bitbucket.io/papers/icse18.pdf)

~~~
rbodik
I should add that Rishabh Singh, one of the lead authors of of the repair
papers, will speak at the conference.

~~~
rishabhs
Thanks Ras for the above pointers! I also wanted to add that there is a demo
of the system with different levels of feedback mechanism hosted here:
[http://sketch2.csail.mit.edu/python-autofeedback/new-
encodin...](http://sketch2.csail.mit.edu/python-autofeedback/new-encoding/)

It uses constraint-based synthesis to search for small repairs. We also did
some follow up work on learning from other student's solutions to learn to
correct similar mistakes:
[https://rishabhmit.bitbucket.io/papers/sarfgen_pldi18.pdf](https://rishabhmit.bitbucket.io/papers/sarfgen_pldi18.pdf)
[https://rishabhmit.bitbucket.io/papers/dyn_iclr18.pdf](https://rishabhmit.bitbucket.io/papers/dyn_iclr18.pdf)

