
Building Sourcegraph - A large-scale code search engine in Go (2014) - philip1209
https://text.sourcegraph.com/google-i-o-talk-building-sourcegraph-a-large-scale-code-search-cross-reference-engine-in-go-1f911b78a82e#.1wvq28y42
======
sqs
This was my talk. Thanks for posting it! Happy to answer questions.

To try out Sourcegraph, sign up at
[https://sourcegraph.com](https://sourcegraph.com) and it'll take you on the
tour.

Also, I'll shamelessly plug our Chrome extension, which "Sourcegraphifies"
GitHub (adding jump-to-def and cross-references):
[https://chrome.google.com/webstore/detail/sourcegraph-for-
gi...](https://chrome.google.com/webstore/detail/sourcegraph-for-
github/dgjhfomjieaadpoljlnidmbgkdffpack). It's the #1 Chrome extension for
GitHub.

Finally, check out
[https://sourcegraph.com/beta](https://sourcegraph.com/beta) to get early
access to some new languages and super neat features we're adding soon.

~~~
kozikow
Extension is cool. I know that hackernews comment is not the most appropriate
place for bug report, but pressing DAG in
[https://github.com/apache/incubator-
airflow/blob/master/airf...](https://github.com/apache/incubator-
airflow/blob/master/airflow/example_dags/example_bash_operator.py#L27) or
set_downstream in [https://github.com/apache/incubator-
airflow#L37](https://github.com/apache/incubator-airflow#L37) does not work.
Goto definition works with both locally with emacs-ycmd.

~~~
sqs
Sourcegraph CEO here. Filed! Thanks. I've taken bug reports and feature
requests from airplane seatmates and when walking down the sidewalk in SF, so
this is actually not a bad place to post them. :)

------
barpet
I do not understand these free tiers + steep pricing. This is like you are
looking to bank on those that really really need your stuff..

But I do not want to invest myself in your ecosystem just to see it die
eventually. I want to pay you because I do not want the service to die. Do you
plan to serve ads in the free tier ? Why not have the pricing more dynamic ?

~~~
sqs
Sourcegraph CEO here. We have a generous free tier that lets you see the value
of the product (and get a lot of use out of it). Happy to help you get your
company to see the value, too, if that's what you're interested in. If you
want to chat further, my email is sqs@sourcegraph.com.

------
boyter
I find Sourcegraph really interesting as a different approach to code
indexing. Opengrok, Krugle, BitBucket Server and my own poor attempt
searchcode and the downloadable server version simply index the source code as
is which is to just index everything thats text (with some smarts on top
usually). By comparison Sourchgraph builds parse trees and attempts to
understand it.

I am not sure as to which approach is better for the average user, but I do
think that if you are using a language currently supported you should give it
a try and compare to one of the afore mentioned solutions.

~~~
sqs
Sourcegraph CEO here. Thanks for the kind words! Ben here is just being
modest...searchcode is fantastic. It has come in handy in many crucial moments
when I couldn't find the code I needed anywhere else. Here's a link for those
who aren't familiar with it:
[https://searchcode.com/](https://searchcode.com/).

------
jschrf
Looks great. I've been building almost the exact same idea for fun as a side
project, but initially focused on deep analysis of TypeScript and ES2015+
specifically. Runs in Docker as well, spinning up containers to do analysis
work, but written in TypeScript and run via node.

Definitely going to be following the blog. Thanks for sharing!

~~~
sqs
Very cool. Would love to chat and see if we can push this stuff forward
together. I can't seem to find your email address. Could you please email me
at sqs@sourcegraph.com?

------
kozikow
Is there a way of "git cloning" the
[https://sourcegraph.com/sourcegraph.com/sourcegraph/sourcegr...](https://sourcegraph.com/sourcegraph.com/sourcegraph/sourcegraph)?
It is linked in the article. It does not seem to be at
[https://github.com/sourcegraph](https://github.com/sourcegraph) .

~~~
shadowmint

        apt-get install docker protobuf-compiler
        mkdir foo
        cd foo
        export GOPATH=`pwd`
        export PATH=${PATH}:`pwd`/bin
        mkdir bin
        mkdir -p src/sourcegraph.com/sourcegraph
        cd src/sourcegraph.com/sourcegraph
        git clone https://sourcegraph.com/sourcegraph.com/sourcegraph/sourcegraph sourcegraph
        cd sourcegraph.com/sourcegraph/sourcegraph
        make
        make serve-dev
    

good luck.

It doesn't actually run for me, just crashes, but you'll get kind of... most
of the way there with that.

(oh, you probably dont need protobuf unless you run `make generate`)

~~~
sqs
Sourcegraph CEO here. We're going to do a big source release soon. Then this
should all be easier. We're growing quickly and pushing out a lot of product
releases soon and have been busy on those. Email is in my profile if you
really want to figure it out, and I can direct you to the right person to
help.

------
CptMauli
How can one add other public repositories not hosted on github (for instance
from the Eclipse foundation)?

~~~
sqs
Sourcegraph CEO here. Right now, just email us (hi@sourcegraph.com) with the
Git clone URLs and we'll add them. Soon, you'll be able to add them yourself.
Thanks!

~~~
CptMauli
Ah great, I'll do that! Thx.

------
the_arun
Sorry for my naive question. What are the features this is adding over
GitHub's source search?

~~~
sqs
Sourcegraph CEO here. Unlike GitHub, Sourcegraph actually understands the code
like your IDE. Unlike your IDE, it has all the code on GitHub and some other
repo hosts already loaded up and built. (We're working on adding more
languages beyond Go, etc.; check
[https://sourcegraph.com/beta](https://sourcegraph.com/beta) for early
access.)

Try out jumping to definition and seeing hovers at
[https://sourcegraph.com/github.com/gorilla/mux/-/blob/mux.go...](https://sourcegraph.com/github.com/gorilla/mux/-/blob/mux.go#L93).
This helps you understand code that you're browsing.

Check out global usage examples and refs of any function, such as those at
[https://sourcegraph.com/github.com/golang/go/-/info/GoPackag...](https://sourcegraph.com/github.com/golang/go/-/info/GoPackage/net/http/-/ResponseWriter/WriteHeader).
You can get to this page by right-clicking on tokens in the code view above.
This helps you quickly figure out how to use a piece of code and see best
practices.

Finally, the code search is also semantic and is ranked by popularity. Try it
out at
[https://sourcegraph.com/search?q=new+http+request](https://sourcegraph.com/search?q=new+http+request).

Code analysis and search at this scale is tough, but we're going to do it
right, because it'll help developers learn more quickly, write better code,
and reuse more existing stuff. (We're not totally there yet, but if you want
to help by adding language support or joining our team, email me at
sqs@sourcegraph.com.)

~~~
the_arun
Thanks for the details. Is this available as a self hosted application or
enterprise edition? Useful for enterprises.

~~~
cuddy
Hey there,

James from Sourcegraph here. Right now we don't offer a self-hosted solution
and won't have one for the foreseeable future. Please email me your contact
details so I can let you know when we do though.

james@sourcegraph.com

Thanks, James

------
electic
This seems quite limited. I went to the site and it seems to work only on a
few languages. Honestly, the best code search out there is GitHub. I learn so
much on there and it works across every language.

Right now I am coding in Obj-C, Elixir, Ruby, and doing web development in
React. That is my day to day so this seems quite limited.

~~~
sqs
Sourcegraph CEO here. Support for more languages is our #1 request by far.
We're going to have more language support very soon. If you want to get early
access, head over to
[https://sourcegraph.com/beta](https://sourcegraph.com/beta) and we'll be in
touch.

