
Go at Digital Ocean - dcu
https://speakerdeck.com/farslan/go-at-digitalocean
======
sytse
Very interesting presentation. From my perspective it was interesting that
they used GitHub Enterprise with Drone for CI and Concourse and GoCD for CD.
At GitLab we're planning a 'CI Only' mode
[http://bit.ly/2Aj70zv](http://bit.ly/2Aj70zv) because companies like Datadog
are using GitHub Enterprise with GitLab for the CI. Based on this presentation
I've asked the team to rename it to CI/CD only to stress that people are able
to use one application for both CI and CD.

BTW I'm not sure if this comment is adding to the conversation or if it is too
self promotional. If it gets down voted I'll delete it.

~~~
testb
Any way to get notice when CI/CD Only mode comes out? My org would definitely
be interested in using GitLab for CI while keeping our repos on GitHub

~~~
sytse
Thanks for being interested. We don’t have a set date. The issues that are
linked from the presentation give an indication. You can follow them.

------
arianvanp
The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also
the fact that imports are full git URLS sounds totally crazy. Is `dep` fixing
all that stuff? Last time I checked out go a few years ago, I didn't continue
because I found the story of actually installing dependencies and keeping
track of versions very confusing. You couldn't even `go get` a specific git
tag or something.

I'm happy that there seems to be some movement. But now we have godep,
govendor, glide, dep. and every project uses one or the other. Are they all
compatible? Or is it a minefield?

~~~
imsofuture
Go vendoring gets _such_ a bad rap undeservedly. In practice it's the simplest
and most straightforward dependency system (since 1.4/5 added vendoring) I've
used.

Step 1: Check out code that you want to use. That's actually the only step and
you can do it by hand, or with one of a multitude of tools. All the tools
differ slightly, and any metadata or manifests or things like that they do
aren't compatible, but the code they put in your repository is, enabling
anyone to clone a repository, and instantly build the exact version of all
your dependencies.

Imports also _aren 't_ full git URLs, they _can be_ as a convenience that
tells you where to get a package. You can't `go get` a specific tag, but
that's fine, because `go get` isn't a package or dependency manager, it's just
a convenience to grab code to your system, not to freeze a version as a
dependency into your project.

Frankly, the amount of people that put their nose in the air at go is just
fine with me, just makes it more of a 'secret productivity tool' I guess.

Edit: not to say it wouldn't have been nice if `dep` had been the one blessed
path at version 1. Still, I have extremely little to complain about with the
language and tooling since 1.4/1.5.

~~~
alain_gilbert
The fact that the full url is used to import packages make it impossible to
fork a repository.

Once you forked it, you have to change all the imports across the repo, and
then it's kinda either very hard to make Pull Requests, or pull the new
commits from the original repo.

This is one thing I really dislike about go dependencies.

If you have a solution for this, let me know, I would be very interested to
know more about it.

But overall, I love go. I use it all the time anyway.

~~~
imsofuture
You don't have to change the package name just because it's forked.

I threw together an example of doing this with `dep`. I forked Fatih's color
package, and am using it as `github.com/fatih/color` no problem:
[https://github.com/sofuture/colortest](https://github.com/sofuture/colortest)
(forked dep here:
[https://github.com/sofuture/color](https://github.com/sofuture/color))

~~~
alain_gilbert
As I said in my other comment
([https://news.ycombinator.com/item?id=15632049](https://news.ycombinator.com/item?id=15632049))

It works for you because it's a small project and you are not actually
importing other of your own packages inside it.

But if you look at "echo", it import some other "echo" packages inside of it.

~~~
imsofuture
Okay, updated my example with `echo`. Check it out:

Using the original package name:
[https://github.com/sofuture/colortest/blob/master/main.go#L7](https://github.com/sofuture/colortest/blob/master/main.go#L7)

I can refer to my fork:
[https://github.com/sofuture/colortest/blob/master/Gopkg.toml...](https://github.com/sofuture/colortest/blob/master/Gopkg.toml#L8-L11)

~~~
alain_gilbert
Oh I see.

Thank you for the example :)

------
neom
When I joined DO it was transitioning out of it's very Perlly beginnings - one
of the engineers was pushing for rust. I guess Go won, and it's nice to see
they have picked up such an awesome competency in building with it.
[https://github.com/digitalocean?language=go](https://github.com/digitalocean?language=go)
(also Grumpy MacB reference who wrote the first go service at DO is one of the
best engineers I've ever met, and also one of the nicest dudes:
[https://github.com/macb](https://github.com/macb))

~~~
hardwaresofton
While rust is the safer, and more featureful language, I think Go is quite
possibly the better pick for large corporations just due to it's simplicity.
Rust to me is the more interesting language, but it definitely offers more
power and more choice -- they probably did the right thing by going for Go
IMO.

Also really cool to see all the tooling they've built up -- all of the things
mentioned seem really cool, "who's gonna clean up all the stale branches" is
definitely a question that gets asked (and answered) at more companies that it
probably should.

~~~
yjftsjthsd-h
Another possible reason: my understanding is that Go works really well for web
apps (including the standard library covering http and templates), while Rust
isn't as strong there. But I might just have seen less web-focused Rust; are
there any good frameworks for web apps in Rust?

~~~
rapsey
In my experience Rust works well as a C/C++ replacement. As a high level
language much less so. Go is clearly the better choice for web apps and
probably will remain so.

~~~
nicoburns
I'd definitely agree that Go is currently clearly the better choice for web
apps. I'm not so sure it will stay that way though. Rust has some really
powerful abstraction abilities, which can make for super nice apis.

I reckon it will end up like the choice between python/js/php/ruby on the
backend, where each have their strength and weaknesses, but none are
universally better.

~~~
hardwaresofton
I agree and disagree, for the reason you stated. I think Rust is a good
language for web apps, with the caveat that you must understand the syntax and
the power afforded first. That makes the language harder to learn, but for the
purposes of webapps that makes it better.

If you start from bare micro-framework (set a route, attach a handler), you're
eventually going to write some generic function that fetches an entity from a
data store. In Go, this becomes a bit of a kludge (interface{} + casting +
etc), but in rust it's robustly supported (traits/generic functions).

Of course, if you pick the right library for the database you wouldn't have
that issue, but that's just the kind of abstraction/complexity-hiding problem
you run into building webapps (from scratch at least) that I think rust is the
better tool for. Like I mentoined in another post, basically all the micro-
frameworks have the same shape to me at this point: add route(s), add handler
function(s), and start the server.

------
throwawaysml
Am I alone in finding it unsurprising but unfortunate that all those addons to
the official go toolchain are created by everyone to paper over the
limitations of the Google Go implementation (which naturally reflects Google's
development process and needs more than anything)?

EDIT: E.g moving .git/ back and forth or adding extra vetting/linting tools
instead of extending `go vet`.

~~~
iends
The most painful parts of the Go ecosystem are directly tied to the fact that
the Google is making Go for themselves first and foremost and the community is
an afterthought.

~~~
throwawaysml
True and the more surprising aspect is that to land a develop position at
Google you need to be on top of all CS theory and fresh in memory, just to
unlearn everything and use Go for unspectacular business/enterprise projects,
unless you're on certain teams like V8 or DeepMind for example. I think Go is
meant to replace Google's Java coders with Go coders and have a language that
fits exactly into the mold of their coding guides and rules for their monorepo
and all the business/enterprise code written by the hordes of the rest of
their developers.

Google's also opensourced Abseil, their C++ standard library (not meant to
completely replact STL, to be clear), which contains all kinds of classes
which were copied into many existing Google C++ projects in some form or
fashion and sometimes incompatibly (e.g. Google's StringPiece found in several
projects).

Either way I applaud them for doing a lot to open source projects. It's not
something we can take for granted.

~~~
throwawaysml
I suspected this might get downvoted and wanted to make clear I'm not
demeaning the 90% of Google developers, but I failed, so I deserved the
downvote. Just to make it clear I'm aware of where my comment failed to
express what I was trying to communicate. Will try to be less lazy next time.
It's hard to explain the purpose Google built Go for without considering where
it's not used, and I failed to write a good comment.

------
twic
_Coffee and bag geek_

I'm a coffee geek, but i'm intrigued at the idea of being a bag geek. How does
one geek out over bags? What are the cool things bag geeks know that ordinary
people don't?

~~~
throwawaysml
I'm not a bag geek, but I know firsthand how hard it is to find the right bag,
if you care about functionality more than form.

For instance, I still haven't found the right carry on backpack that's sturdy,
doesn't cost $200+, allows me to take clothes for a couple days and a laptop.
It's either too big, too expensive, too heavy, too fragile, take your pick. So
I can understand how someone can become a bag geek if they have to travel
professionally on their own (without assistants who carry your baggage). Bonus
points if I can detach a small laptop bag to take with me in order not to
leave it at the hotel with the rest of the bag. So far I've always had to
carry around the backpack and leave clothes in the hotel room. Pack, arrive,
unpack, carry laptop, repack, go home.

~~~
js2
I don’t understand the objection to price if it fulfills all your other
requirements. A quality bag will have a lifetime warranty. Have you looked at
Go Ruck?

~~~
throwawaysml
Past experience makes me wary to risk spending that much and be disappointed
two trips later. Lifetime doesn't mean they give money back if unsatisfied,
except one or two companies. Also, upping my budget didn't magically reveal
viable options either. So I could have skipped the price tag mention and
conclude that it's just hard to find the perfect bag.

I've seen the Go Ruck GR2 and it doesn't fit my requirements.

~~~
js2
Gotcha. Perhaps REI? They carry a handful of brands and you can return
anything you aren’t happy with.

~~~
throwawaysml
Thanks. Basically I want the sturdy laptop backpack you got from HP 10 years
ago with a clever way to pack clothes and ideally waterproof bottom
(rubber/plastic) so that I don't worry when having it sit on random floors at
airports or train stations.

Having a subdivision of the main compartment bigger than 2 is also important.
Laptop, papers, cloth, food maybe.

I'm in Germany and looking for a shop nearby where they have a diverse
selection of bags to inspect. I'm kinda tired of ordering bags and having had
to send them all back. Amazon is threatening a penalty for returning too
often, so there's that :).

First world problems, right?

------
logicallee
Can someone explain slides 54-58, about how it takes many minutes to lowercase
strings?

slide 55 reads: "each string operation takes 21 seconds" (an amount of time
which I translate conservatively into tens of billions of operations or
gigabytes of in-memory lookups).

How can performance be that bad - I would think it's trivial? Like, I'm not
getting what lowercasing can possibly be doing that is so resource intensive.
Any ideas?

~~~
golangnews
Looks like the output of pprof, so perhaps the slide means all calls to this
function (when run 10000 times say during checking dependencies over a very
large codebase), take 21 second in total, reduced to 4 seconds by storing the
results of comparisons in a map and using that instead. Go strings are
immutable so calling ToLower repeateadly would copy them each time.

------
ShabbosGoy
I'm curious as to why they went with the monorepo approach. This might sound
really stupid, but couldn't you just have a seperate repo for each
team/service?

My approach is to shove everything in $GOPATH/src.

~~~
farslan
Author here. The approach of a single repository for each project has its
downsides, which I’ve tried to explain in the beginning of the slides already.

------
gtaylor
If anyone over at DO is reading this, we've got some technical and conceptual
overlap at Reddit. Would love to compare notes.

------
cristaloleg
Are there any chances to see `gta` tool in open source?

------
alphaalpha101
This seems like an enormous amount of tooling complexity to simply have a few
dependencies for your program.

~~~
farslan
Author here. It’s not by any means a few dependencies. There are over 2
million lines of vendored packages, you can imagine how much effort it’s
needed to upgrade them, be sure each team uses the corrext version, has no
security exposures, etc...

------
innocentoldguy
I really like Digital Ocean and use it for all my small to mid-sized projects.
I agree with a lot of the other comments that languages like Rust (and I'll
add Elixir) are far more interesting, fun to program in, and feature rich than
Go, but I really don't care what DO choses to use, as long as their offerings
continue to be great.

~~~
innocentoldguy
Hmm. Must have made sensitive Go programmers cry. Go is a boring language, and
it isn't as good at concurrency as other, better designed languages. I'm
sorry, but that is a fact.

~~~
hliyan
As someone with both a C++ and JS background, Go's boring nature is the best
thing about it. It gets out of the way and lets you focus on the application
logic. Programming languages are tools, not ends unto themselves.

~~~
innocentoldguy
Don't insert words into my commentary that I didn't say. I said Go was boring.
You agreed. I said Go wasn't the best designed language, which it isn't. Look
at its package/dependency management, memory usage for processes, inability to
guarantee a process will relinquish control so other processes can run. Go
enforces code bloat that other languages don't. For example, a program in D
will often times be half the length of a Go program that does the exact same
thing.

You are right. Programming languages are tools, not ends unto themselves
(something you said, not I), and just as I can use tools from Snap-on or tools
from Harbor Freight, I'm personally inclined to use tools with superior
engineering. Tools that are designed and manufactured to be robust and to
last, because they make it so much easier for me to build meaningful things.

I also think it is an unfair comparison to take two of the most awful
languages in computer history and use them to make a case for Go's
superiority. It is like saying a Fiat 500L is an awesome car because it isn't
a Yugo.

