
The present and future of semantic code search - Jefro118
https://www.sourcesort.com/interview/hamel-husain-on-semantic-code-search
======
mloncode
Hi, this is Hamel. I'm happy to answer any questions about the dataset or the
project. I think this dataset is exciting for the following reasons:

\- Its really large: Parallel corpus is 2M code snippets, and the unpaired
corpus is 6M code snippets Its real data generated by people in the wild and
gives everyone a chance to exercise skills with cleaning a messy dataset.

\- Its unique: its [code, comment] pairs; part natural language, part code.
This presents unique challenges. This data has seen very little utilization so
far which means lots of opportunities for the community to make tools for code
navigation, search, bug detection, automated documentation, and program
synthesis (some of the problems being significantly harder than others).

\- Other niche datasets from specific domains require a non-trivial amount of
domain knowledge before being effective with the dataset. For example, to
utilize radiology images it is extremely helpful to learn a little about the
domain to help with cleaning and exploring the data and debugging your models.
However, many machine learning folks are already familiar with writing code so
you have a head start!

\- The most painful part of cleaning the data is done for you: parsing the
code and separating out the comments from the code.

We are really excited about the possibilities of this dataset and what folks
are able to do with it. Looking forward to your questions.

~~~
socrates667
What is semantics in this context? When I use the word, I generally think
about "what I mean" when I use a word, or a symbol. Then there is the
alternative "interpretations" of that symbol. The hope with good semantic
tooling is that these things match up. If they don't then there is ambiguity.
At what levels are you attempting to resolve ambiguity? Concept, code, etc.
And what challenges remain?

Finally, with your findings you could theoretically turn around and produce
your own "semantics friendly" language/toolings, are there plans for that?

Thanks for your time.

