

Golang dependency hell in Haunts The Manse game  - laurent123456
http://www.kickstarter.com/projects/2066438441/haunts-the-manse-macabre/posts/373466

======
aston
"Golang dependency hell in Haunts The Manse game" is not only a very
editorialized title for this link, but it's also factually incorrect. Here are
some choice quotes about the codebase that have nothing to do with
dependencies in Go:

Syntax errors:

    
    
      "Firstly, the source simply doesn't compile."
    

Performance problems:

    
    
       "...it's horribly slow."
    

Bad version control:

    
    
      "... the original code [has] ... potentially significant differences"
    

Those sorts of mistakes seem consistent with the type of coder who wouldn't
set aside the 3 minutes it takes to understand Go's dependency system and
either use it correctly or build a super-simple makefile to pull the correct
versions of code from elsewhere.

Also, pedantic but worth noting: Golang is not the name of the language.

~~~
bdcravens
I've got 3 minutes. I've spent more than 3 minutes searching, and I can't find
a way to specify a version in an import (though you can tag the go version,
you still can't tie the import to a specific tag or version or commit). Am I
missing something? It seems the only way around this is to check out a point
in time release and import that (which would solve the problem the article
mentions). However, that's a workaround for a problem that package managers
like NPM and RubyGems have trivially solved.

~~~
aston
You can't. The right way to spend the 3 minutes is to read the docs. [1] [2]

Also, just because it doesn't work the way NPM does doesn't mean it doesn't
work.

[1] <http://golang.org/doc/code.html#remote>

[2] <http://golang.org/cmd/go/#hdr-Remote_import_path_syntax> (linked from
[1])

~~~
bdcravens
Neither of those links address the issue the author mentioned, nor do they
describe the need to use your own local copy of a package in case the package
authors introduce non-backwards compatible changes. Yes, it needn't behave
like NPM, or RubyGems, or Aptitude, or Homebrew, or any of the dozen or so
systems out there for install remote packages that let you indicate version.

------
LPCRoy
This has nothing to do with Go. It sounds like a case of poorly managed
dependencies and build rules and likely weak engineering. The fact that they
wanted to support the iPad and chose Go also shows this wasn't well thought
out.

~~~
beatgammit
I would say bad engineering:

* Go 1.0 had a terrible GC, especially on 32-bit platforms (their target platform)

* No manual memoray management (and unlikely to ever exist)

* Can't bind to C++, can't cross-compile cgo (gccgo can though)

Go is really cool, and I use it every day, but it doesn't make any sense for
games, especially not for their target platform. Maybe in a few years the GC
will be more reliable, but I wouldn't try to build a game in Go if I thought I
might want to try to sell it.

------
crcsmnky
"There is nothing special in Go that makes it particularly well suited to
video games, but on the other hand there is nothing inherent in the language
that makes it a bad choice, except for the fact that it's new and not well
supported."

I would say that "new and not well supported" makes this a potentially bad
choice for shipping code. Not to say that it can't be used but the problem
with new technologies is that not all warts have been properly exposed and you
can open yourself up to major headaches.

------
bentcorner
Previous discussion: <https://news.ycombinator.com/item?id=5707019>

~~~
kristianp
And the March 5 update on the Kickstarter page seems to imply the project has
been abandoned: [http://www.kickstarter.com/projects/2066438441/haunts-the-
ma...](http://www.kickstarter.com/projects/2066438441/haunts-the-manse-
macabre/posts/420311)

------
MetaCosm
My summary from the last time it was posted.

"Original author didn't localize dependencies, obviously caused problems."

------
NateDad
"it makes a local copy of that external library. And if you have a local copy,
you can edit it. And meanwhile, that person you copied the OpenGL code from?
They are also changing their version (fixing bugs, adding features, adding
bugs). Therefore there is no guarantee that your version and their version
remain compatible: it takes work to maintain compatibility."

Remote packages in Go are like DLLs for other languages. Yes, if you edit a
DLL locally, and other people download the public version of that DLL, they
won't be the same. DON'T DO THAT.

Make your own copy of the public Go repository, so you can merge in changes
from the main branch as you see fit. This is akin to making a copy of the DLL
and using that to distribute with your code, rather than just telling people
"go download the newest version of the DLL from github".

It's also good practice for go authors to keep any one repository backwards
compatible, make it clear when the code is in flux and may have breaking
changes... and to make new branches if they have to make significant changes
to their code.

------
paulsamways
> and put import ("github.com/go-gl/opengl/gl") directly into your code.
> Viola! Thus hilarity ensues.

Of course it does! All they had to do is fork the project (which they have)
and reference that it instead. Viola, problem solved!

~~~
MetaCosm
No need to fork, it puts a copy of those libraries under ./src, so you always
have a sealed (hermetic) referentially complete copy.

You don't bet your production software on things randomly downloaded from the
internet at compile time. This is why go never auto-updates when you do "go
get", you have to explicitly use "go get -u".

Always localize what you depend on, you never know when a github repo could be
pulled, go offline, be renamed, a lawsuit could break out making the library
vanish. There are TONS of reasons you should ALWAYS localize your dependencies
for building software.

EDIT: This has the amazingly cool side-effect of meaning even on complex
projects, you can just do a git clone and then go build FOO and it just works,
no nonsense, no fuss, and it is EXACTLY in the state it was when it was
checked in -- no surprises.

~~~
paulsamways
> No need to fork, it puts a copy of those libraries under ./src, so you
> always have a sealed (hermetic) referentially complete copy.

While that is true, it doesn't solve the problem with distributed teams. The
library path MUST be pointing to a 'localized' repository to solve the problem
the Haunt guys are having.

~~~
MetaCosm
What? It ONLY looks for localized copies.

When you have import "github.com/gorilla/mux"

it will look in

$GOROOT/src/pkg/github.com/gorilla/mux and $GOPATH/src/github.com/gorilla/mux

if it is missing, you can fulfill it with

go get github.com/gorilla/mux which will place it in the first place in
$GOPATH

What are you talking about breaking with distributed teams? Since
$GOPATH/src/github.com/gorilla/mux is in source control you NEED NOTHING and
it will be a known working git clone.

EDIT: Additionally, it will never auto-update those dependencies (and break
your build) unless you explicitly tell it to with "go get -u".

~~~
paulsamways
I know how it works :) I'm not new to Go.

BUT, what you are describing is exactly how the Haunt guys used Go and
consequently the issues they have run into. Please re-read the update from
Rick and read my comments again, you should then understand the issue and why
changing the import path to a fork/clone/localized copy is beneficial (though
not required).

~~~
MetaCosm
I don't understand, so explain it to me like I am a child, I am certain I am
missing something basic and the developers on the project aren't as dumb as it
seems.

Seriously, it seems so goddamn basic. I am going to walk you through step by
step how I would add an external library to my Go project.

#1. Switch to the go project directory #2. Fork for the new feature
(adding_mux) #3. Run go get github.com/gorilla/mux #4. Add import
"github.com/gorilla/mux" to my code, use mux for great goodness #5. Commit my
code (this is commit of ./src which includes two things, the addition of the
new github.com/gorilla/mux/* and my code using that new library) #6. Merge #7.
Push

At this point, the commit that was just checked in doesn't need a change to
its import path, will never break and will work as it was committed FOREVER.
git clone FOO; go build BAR -- victory!

If at some point I want to update that library (new cool feature!)

#1. Switch to go project directory #2. Fork for updating library (update_mux)
#3. Run go get -u github.com/gorilla/mux #4. Write code using new awesome
features #5. Commit my code (again, "my code" in this case is both the update
to the mux library, and my new usage of it, again committed together for great
goodness!) #6. Merge #7. Push!

I feel like one of us is bat shit insane, and at this point, I am just hoping
it isn't me.

EDIT: Another cool side effect of this is you end up with localized hermetic
copies of your code with references for posterity to where you got it, bonus
points!

EDIT 2: For reference, there are various forks of the source he "released"
(abandoned? Sold for 28k? whatever) ... I think you can start looking at the
github network from here: <https://github.com/runningwild/haunts>

~~~
paulsamways
So the key part of the update I was referring to was the following:

    
    
      Because of course that's not how it really works. Instead, it makes 
      a local copy of that external library. And if you have a local copy, 
      you can edit it. And meanwhile, that person you copied the OpenGL code 
      from? They are also changing their version (fixing bugs, adding features, 
      adding bugs).
    
    

In short, original developer has the clone of the OpenGL library in their
'src' directory. While working on Haunts his making changes to it and not
pulling updates from upstream. All is well, things will always build on his
machine because as you say, it won't update unless he does 'go get -u'.

Fast forward to when the community takes over, they clone the Haunts project
and attempt to build it. Now they are using the HEAD of the 'github.com/go-
gl/opengl/gl' repository, as specified by the imports path and they find that
it no longer compiles.

This is where the problem exists.

The simplest way around this problem is to simply fork the OpenGL Go library
(like they have), and reference the fork in the import path. That way no
matter what happens in the upstream repository, the Haunts project will
continue to compile. The original developer could of then just pushed the
changes he was making in his local 'src' directory back to the fork and
everything would be fine. When they are ready to take updates from upstream,
they simply update their fork and then 'go get -u' on all developer machines.

By forking, or maintaining your own copy of library, you get a level of
isolation from the upstream developers.

What you are saying is 100% correct, the Go tools automatically give you this
when running 'go get', it clones the library into your 'src' directory. What I
am saying is that when working in a team/community environment, you need an
additional level of isolation (the fork), so that the entire team can be using
the exact same version of the library.

    
    
      > I feel like one of us is bat shit insane, and at this point, I am just hoping it isn't me.
    

I don't think (hope) either of us are, just not thinking on the same track :)
I hope what I've written is clear enough that you can see what I was saying.

~~~
MetaCosm
> In short, original developer has the clone of the OpenGL library in their
> 'src' directory. While working on Haunts his making changes to it and not
> pulling updates from upstream. All is well, things will always build on his
> machine because as you say, it won't update unless he does 'go get -u'.

The important addition is -- he does not check this clone into his OWN source
control, so his source control doesn't have a complete hermetic version of the
application, it has a "partial" application... or really "application bits".

> Fast forward to when the community takes over, they clone the Haunts project
> and attempt to build it. Now they are using the HEAD of the 'github.com/go-
> gl/opengl/gl' repository, as specified by the imports path and they find
> that it no longer compiles.

If he had checked it into his version control system (you might call it
forking, I call it localizing), they would have had the EXACT same version as
him and they would not have had to fulfill that dependency on their own. Also,
the same import path would continue to work off this local checked in version.

> By forking, or maintaining your own copy of library, you get a level of
> isolation from the upstream developers.

I thought that maintaining your own local copy of libraries under /src was the
norm for go developers, as the value of hermetic checkouts goes hand in hand
with some other Go'isms like static built binaries.

------
EugeneOZ
Sorry, but problem not in Go. Code of external libraries shouldn't be edited
(or should be forked before changing).

