
Google’s revamped Cloud Source Repositories in beta - Sami_Lehtinen
https://cloud.google.com/blog/products/application-development/introducing-new-cloud-source-repositories
======
dabei
Being able to search code so efficiently across the whole company’s code base
is a huge productivity gain. But I found people who hasn’t worked in Google or
other companies that have this capability generally don’t believe it’s a big
deal.

~~~
kngspook
Well, you need to have a fairly large codebase with a fair amount of in-house
infrastructure and interdependencies before you have both the need to find
code that you're not working on and have a dense enough source tree where
search makes more sense than just learning the organization and navigating to
where you need to go.

~~~
joatmon-snoo
Honestly, it's not even the monorepo and tooling. Simply the ability to
navigate our code IDE-style in the browser, and move back and forth between
different revisions, makes a huge difference.

I've been mucking around with some code in javac recently and was trying to
show someone something I'd found, and browsing the openjdk mercurial repo was
just so painful compared to hunting around google3.

------
mark_l_watson
Very nice. Reading that makes me miss Google’s internal dev environment. I may
not much use anymore Google Search and gmail, but Google’s paid products like
GCP (and Google Play Music, Movies, etc.) are awesome. Also, amazing permanent
free tier levels. I need to use github for the repos for my books because they
are already linked, but for my personal private repos I think I may switch.

~~~
bogomipz
Could you share why as a former Google dev you no longer use Google search and
gmail? I'm curious why you mentioned that in response to a post about cloud
source repositories. Thanks.

~~~
paulgb
I'm not OP, but I'm also a former Googler. I switched to DDG and Firefox after
I left because I think it's important that alternatives exist. Google may be a
better actor in monopoly situations than some other companies, but they're
still a company trying to make profit for shareholders.

(As a bonus, I find ddg+Firefox snappier than Google+Chrome)

~~~
srean
> Google may be a better actor in monopoly situations than some other
> companies

That made me think, so thank you and yes I agree.

~~~
paulgb
I'm glad you quoted me after I edited "Google is" to "Google may be". I am
willing to contend that it's not a settled point :-)

~~~
srean
I think I would have agreed even if you had said 'has been ... so far'.
Monopolies rarely behave well.

------
jrockway
Nice. Is the code search the same as what Google has internally? I do miss
that.

(Though honestly, the value from the internal code search was that
_everything_ was in there. So if you got an error message from a service you
were a client of, you could just search for the error message in their code
and see what it actually meant. "Aha, this 'optional' field that's marked
'deprecated' isn't actually optional or deprecated," most of the time ;)

~~~
duality
You might also want to check out [https://kythe.io](https://kythe.io) and
[https://github.com/google/kythe](https://github.com/google/kythe).

~~~
romed
It’s great that they opened Kythe but as someone who deployed it from scratch
I say it’s like ordering a DIY aircraft carrier kit. Many steps before you get
something useful.

------
lunch
Sounds great but at this point I've been burned by Google EOLing products that
I use too many times.

~~~
client4
I was going to make this same joke. It's hard to put faith into a product when
Google keeps killing them off. Wave and Google Code are two top of mind
products that I missed when they were gone.

Conversely, I think it's important for Google engineers to be able to try new
things and publish products without having to worry about supporting them
forever. Is there some sort of happy medium? Like a Google labs of products
that are in various states of whimsical testing?

Cloud source looks nice, and for GCP projects I'm sure it will have a nice tie
in, but for now I'm 100% more likely to push code to Gitlab until it makes
sense to do otherwise.

~~~
theptip
I know that is the meme, but are there any examples of Google dropping
enterprise (i.e. paid) offerings? I can think of a few free services that they
have dropped, but that's a different story than dropping a commercial service
that you're selling to enterprise users.

~~~
client4
I don't know. A quick perusal of the graveyard[1] shows what I would consider
mostly free services.

[1]
[https://en.wikipedia.org/wiki/List_of_Google_products#Discon...](https://en.wikipedia.org/wiki/List_of_Google_products#Discontinued_products_and_services)

------
prepend
This looks interesting but impractical for me to mirror hundreds of repos here
just to get better search.

Has anyone played with this or similar tools that can provide search by
indexing at source. It’s a little surprising that Google doesn’t just index
everything public in Github, gitlab, etc.

I would like a good code search solution especially because I have projects
across lots of different repo servers but want developers to be able to find
code regardless of repo home. A colleague indexed it with solr and that was
ok, but had limited semantic ability and definitely nothing like what Google
describes where your recent search history and activity is displayed to each
user.

~~~
russellwolf
Author of the blog post here.

Thanks for taking a look. I'm glad you find Cloud Source Repositories
interesting. We'd love to Cloud Source Repositories more practical to use. Can
you explain what is impractical for you? Is the initial process of setting up
multiple mirrors? Or the routine of having to go to another site for search?

(Disclaimer: I work at Google and am the PM on this product)

~~~
prepend
Thanks for responding. I work in an organization (happy to PM) with a few
Github orgs and about 300 repos. Dozens of project teams work in them and use
various GitHub features.

I have no easy insight or overview into code across these repos without
scripting. The impractical part is in cloning mirroring all of these repos to
get insight from your new function because user behavior is baked into all the
workflow around the Github repos.

The go to another site isn’t so bad as this is a new function and not
available in Github, so programmers will likely check it out when they need.

But manually forming everything and syncing. It would be neat if I can import
an org in one fell swoop.

That being said, I do have scripting forking everything over on my list of
things to do when I have cycles on the weekend.

------
kyrra
For those that haven't used Google's code search before, I believe this will
be very similar to Chromium's code search[0] (which is based on Google's
internal one).

[0] [https://cs.chromium.org/](https://cs.chromium.org/)

------
pjmlp
> For Java, JavaScript, Go, C++, Python, TypeScript and Proto files, you’ll
> see result suggestions indicating whether the match is an entity such as a
> class, method, enum, or field.

Apparently they forgot about Flutter users.

------
bryanlarsen
One nice thing is how trivial it is to mirror a repository from Bitbucket or
GitHub. I wish it was as easy to mirror a repository from Bitbucket to GitHub
or GitHub to Bitbucket.

~~~
briffle
It would be nice if it was so easy to mirror bitbucket server, or gitlab, etc.

------
hjuutilainen
Quickly glanced through the blog post and the docs but didn’t find any mention
of Git LFS. Does Cloud Source Repositories support mirroring LFS content?

~~~
russellwolf
Author of the blog post here.

Cloud Source Repositories currently supports very large repositories. The same
backend scales to the needs of the Android open source project, which
regularly checks in massive binary files. There are many APKs and VM images in
the Git repositories we host. However, Cloud Source Repositories does not
support LFS or the mirroring of LFS content.

LFS is not deeply integrated into Git, creating usability problems. Because
the content is not part of the object graph of the repository, you have to
decide in advance to use Git LFS. You don't get the benefit in existing
repositories with large files. You also can't back out --- once you're using
Git LFS on a medium-sized file, you can't change your mind and instruct Git to
send it inline in fetches, without rewriting history, which breaks existing
clients that have cloned the repository.

For the same reason, history mining commands like "git blame" and "git log -S"
don't have access to the object.

In addition, it complicates migrating to another host. Usually in Git, you can
take out your content by running "git clone --mirror one-url and then "git -C
directory.git push --mirror another-url". With Git LFS, this copies over the
pointer files but not the underlying large file content and you must remember
to take extra steps to instruct the new host about where the blobs are stored.

Google's Cloud Source Repositories team believes that Git itself needs to deal
better with large files. The first step of this work within the Git project
has been partial clone, which is supported in Cloud Source Repositories and in
the public Git 2.17 release (for best results, please use Git 2.19, released
September, 2018). If you run "git clone --filter=blob:limit=512M <url>", files
larger than 512M will be omitted from the initial clone and fetched
automatically on demand when needed (for example during checkout operations).
See [https://crbug.com/git/2](https://crbug.com/git/2) for more details about
this feature. We are continuing to work with the community on adding other
features related to large file support into Git.

(Disclaimer: I work at Google and am the PM on this product)

------
themihai
Why would someone move from bitbucket?

~~~
russellwolf
Author of the blog post here.

We've generally heard that developers move from Bitbucket to Cloud Source
Repositories because they want to use other Google Cloud services and it's
simpler for them to manage their source in the same place where they debug,
build and deploy that code. Developers often mention that they appreciate the
unified identity, IAM permissions, and integrations that Cloud Source
Repositories has with Cloud Shell, Cloud Build, Cloud Debugger and others
Google Cloud services.

However, developers don't need to move from Bitbucket to take advantage of the
utility provided by Cloud Source Repositories. You can mirror any number of
your repositories into Cloud Source Repositories and take advantage of the
code browser, code search, and various integrations with Google Cloud
Platform.

(Disclaimer: I work at Google and am the PM on this product)

~~~
kngspook
FWIW, it'd also be great to have mirroring with Gitlab (especially since I
think Google Cloud has a deal for users with them?).

------
etaioinshrdlu
Google code is back? They really do indicate they can host your repo in the
post!

~~~
skj
GCP has been able to host your repo since before the turndown of Google Code.
In fact, the original GCP Source Repo team and the Google Code team were the
same team (though the current product has a different backend and team).

~~~
zaphar
Yep, I used to work on the Google code specific features but there was a lot
of collaboration between the devs.

------
wasd
This is pretty cool. What can we expect in terms of additional language
support?

~~~
russellwolf
Author of the blog post here.

I'm glad you like it. Which languages would you like to see support added for?

Note that search works today across all languages but the semantic
understanding of source which enhances search is limited to Java, JavaScript,
Go, C++, Python, TypeScript and Proto files.

(Disclaimer: I work at Google and am the PM on this product)

~~~
wasd
Thanks for the post Russell. Our SaaS application is running on Rails /
Typpescript frontend. We're patiently waiting for App Engine to add Standard
support for Ruby before we go all in on GCP. I would be most excited to see
Cloud Source support Ruby. Hope your team will consider it.

------
foobaw
Similar to OpenGrok, I'm assuming?

------
jdlyga
Why would I want to host code on Google's Cloud when they shut down Google
Code? What's to stop them from shutting down Google Cloud in a few years?

~~~
tomassre
I believe the difference is this is part of GCP and their terms for GA
products are that they will support for at least one year on deprecated
versions / products and not just instantly remove something.

------
devoply
They should list expected months until closure on the product page so people
can plan accordingly. I kid, but not really.

~~~
apoorvgarg
how many gcp products has google abandoned in total?

~~~
theDoug
I know we deprecated the prediction API with a year's warning while bringing
out Cloud ML Engine. And Cloud SQL changed a bit when the second generation
came in and supported MySQL, PostgreSQL and more.

Can't think of others off the top of my head, so that is two if we want to
consider major upgrades "abandoned." Perception often doesn't match objective
fact.

(Disclosure: I work at GCP)

