What we really need is for language-level package managers to support signed immutable packages, and for that to start being taken seriously above the OS-level.
Edit: author changed the title, it used to say something about a "GitHub's security facepalm"
If golang allows to import from Git URLs over HTTP or SSH, then it is the same problem with DNS as with the github usernames, isn't it? If the domain name, from which you import, will disappear, then someone else can register it and serve malicious code. This is also pretty hyperbolic.
IMO It's simply solvable by refering to a commit hash, like govendor does it (or the gemfile in ruby for that matter). I am actually not sure why it's not possible to add commit hash to a golang import, I thought that $GOPATH/src contains only git repos (but I'm not sure about this).
For example, I may publish something as an online backup or a mirror of my real repositories, without any kind of guarantee whatsoever. ONLY sharing because I want people to be able to see how the things I do were made. I mean, even most FOSS licenses explicitly state that there are no guarantees of any kind.
IMHO I think people should stop with the "if it's on internet, it must retain compatibility" mindset, and instead switch to a "this will break or disappear at any time unless explicitly stated otherwise" mindset.
It really ought to refer to a signed tag that the package manager verifies before installing the package.
Do you never update your dependencies?
Say you want to check out some older version of the code for bisecting, and it doesn't even build anymore because it worked with some version of the dependencies that was the latest years ago, good luck figuring out what commit they were all on at the time.
It's trivial to just update your own project to point to the latest upstream SHA-1s and commit that, this is why git's own facility to do this (submodules) pins you at specific upstream commits.
What about the author having their github account compromised? I agree that GitHub account names should not be released so quickly, but if you're seriously worried about that possibility then I'd think it's also wise to be worried about the possibility of upstream being compromised in other ways.
Thank you. GitHub shouldn't be the only line of defense between a malicious coder and your software.
+1 Emphasis on immutable. Can someone explain why popular package managers like npm don't offer this? During the leftpad fiasco, it boggled my mind that someone could just "liberate" their packages one morning, and transitively break the build for thousands of projects. Immutable packages would have prevented this entire problem.
Thanks for the tidbit, seen it been added May last year to npm5. Hard to keep up if you don't use the tooling activity and only depend on it for other tooling.
Anyone can actually 'reflect' dependencies hosted with various VCS with this method, and gopkg.in does essentially this (while adding version pinning).
Is there somewhere I can read more about this tag? Opened up the source of one of my repos and I couldn't find it.
<meta name="go-import" ...> tags
Makes it more difficult to implement a handler for the BitTorrent protocol to `go get` from a DHT, just for the sake of mad science.
Has anyone tried to do something like this before? I would absolutely run a torrent server to serve up my own and others' open source packages.
I think it’s abandoned though.
You could probably write "plugins" or shims/wrappers for most package managers out there pretty easily.
And a great MVP would just be the ability to install from a magnet link.
But yeah, I'd really like for someone to develop a VCS like that (even more if it's in Go, with no Cgo, and under BSD, MIT or Apache license). It'd make distribution, immutability and mirroring so much easier.
Unfortunately, "non-profit" hosts are somewhat in a short supply.
I haven't heard any news of git moving to SHA3-256.
Why shouldn't this be built in?
Here's an old thread from 2017-03-04: https://public-inbox.org/git/CA+dhYEViN4-boZLN+5QJyE7RtX+q6a...
git commits can be gpg signed: https://git-scm.com/book/id/v2/Git-Tools-Signing-Your-Work
Your mentioned method signs the git commit hash. A signature is only as good as the hash you use, and SHA1 is not suitable.
Also see https://github.com/git/git/commit/721cc4314cb593e799213ad5f9...
I was always bewildered by this choice , because many, many other package management systems have independently realized this to be a bad idea -- and that it's valuable to have an extra layer of indirection between package identity and package location to prevent this exact situation. The component that solves this indirection is called a package repository, which can then institute strict rules about immutability, deletions, and naming, if so desired. NPM didn't, for a long time, until they got burned. GitHub is not a package repository, but due to the Go community's guidance, is effectively being used as such in a large volume of Go code in the wild.
 https://github.com/golang/go/wiki/PackageManagementTools  https://golang.org/doc/articles/go_command.html#tmp_1  https://news.ycombinator.com/item?id=12189356  https://news.ycombinator.com/item?id=15677338
Indirection doesn't solve the problem it just moves it to the centralized repository.
Go designers recommend vendoring packages which solves the problem.
As far as I know, this is true , and can be done with the 'source' rule in Gopkg.toml. However, the guidance for dep says :
"A source rule can specify an alternate location from which the name'd project should be retrieved. It is primarily useful for temporarily specifying a fork for a repository. source rules are generally brittle and should only be used when there is no other recourse. Using them to try to circumvent network reachability issues is typically an antipattern."
This doesn't sound like a glowing endorsement of this feature.
> "Indirection doesn't solve the problem it just moves it to the centralized repository."
Precisely; we're in agreement. A repository can then choose to institute sensible rules, if they so desire: Maven Central has what I would consider to be sensible rules, while NPM prior to 'left-pad' did not. Without a repository, the community has no place to make this decision point, so everyone's left to fend for themselves.
> "Go designers recommend vendoring packages which solves the problem."
Can you point me to guidance from Go designers which substantiates your point? I have been searching for this and have been unable to find it.
> If you're using an externally supplied package and worry that it might change in unexpected ways, the simplest solution is to copy it to your local repository. (This is the approach Google takes internally.) Store the copy under a new import path that identifies it as a local copy. For example, you might copy "original.com/pkg" to "you.com/external/original.com/pkg". The gomvpkg program is one tool to help automate this process.
Would you statically link to an ftp server in your code build process? What about to a random website for a piece of code? No, you wouldn't, and this is equally unprofessional.
Just with a delay until you run update.
A central-repository package manager that doesn't either enforce immutability or at least have strong protection against a deleted project being replaced with a same-name project poses the same risk, only with itself rather than GitHub and the target.
There are package managers that are safe against this, but I don't think it is yet the norm.
And yes, NPM allowed name reuse, and that was bad, but that was a problem on NPM's side, not Github's side, because NPM is supposed to be a robust package repository and Github was just designed to be a lightweight social network for programmers.
For f*s sake, the Java ecosystem is horrible and yet even they managed to get this right.
Maven Central doesn't allow deletion (except in very rare cases, and in those the packages were available in other repos), as result you can always rely on those packages staying available. You can import, and be done. Every dependency you declare has an explicit version as well.
The go and JS dependency management solutions are pure madness
Oh, what a low move. Whatever you consider the Java ecosystem to be, it's used by most large companies, millions of developers and has been one of the most successful, scandal-free ecosystems in the programming world... if you think that's horrible, you must explain yourself, otherwise you're just being a dick.
I am definitely against this nonsense. Don't depend on some third party for your solutions who never said that their urls are here to stay. Quite the contrary!
Put another way, I would _also_ upvote an article that said "The Go package manager is insecure because it assumes github URLs never change owners."
Admonishing how? The change of policy would motivate them to do nothing at all, perpetuating the mistake.
Using GitHub HEAD is one of many ways it can be used. It’s not a very popular approach. People with important projects that often use other ways, such as vendoring via a tool like dep.
Even projects using dep continue this pattern of pulling things from github master. I just browed through the top projects on Github for go and pretty much all of them that contained a gopkg.toml pulled in one or more things from github directly from master
These include projects like graphana, cockroachdb and hugo.
Literally the first example of the Go tour  tells you to run "go get". "dep" is not (yet) an official part of Go's tooling. So when the only official tool fetches HEAD and calls it a day, I think it's perfectly fine to say "Go does that".
It was never intended to be a part of your production build process. If you push its outputs directly to production, you're holding it wrong.
The funny thing to me is that this was always pointed to as a problem of Lisps: Lisp is so powerful that a student can implement an object system as homework, and the issue is that every student will. It looks like this problem isn't limited to Lisp: any sufficiently-powerful language enables programmers to reïnvent the wheel rather than forcing them to collaborate (e.g. as with C, where doing anything is so incredibly difficult that it's easier to pitch in with someone else's project than start your own).
Also, it may not be hanging but merely running for a very long time. Sometimes I'll start dep, then head to lunch.
It's … not the fastest thing ever.
"I promise you can trust me" is not something you can trust.
If you're interested in the topic, there are many good papers and studies on how we can't even trust ourselves when it comes to basic things like memory and honesty (with ourselves).
GitHub has a name squatting policy for a reason. I have multiple "names" because some guy's repo who abandoned GitHub years ago was released and I was able to reuse it for my own brand.
This is like saying if I buy a domain it should be mine for eternity even if it is no longer useful for me.
More about the squatting policy here: https://help.github.com/articles/name-squatting-policy/
The real issue is Go just pointing to "whatever is the current HEAD" which is insane.
Because you're not building a list of just username and repo names for software dependent purposes, right? That would be a massive security failing on your part.
I cannot use git's built in functions for secure applications. If git was meant to be used only as Linus describes, then this should not be possible as it is not secure:
Instead, I have to build something external from git, including "commit hash", in order to have a secure identifier for a commit. I would have a one off solution that isn't adopted widely.
The new hash stuff can't come soon enough:
Not the parent, but the problem specifically with using github repos with a hash to identify a specific commit is that if someone can make a collision, they can make the code anything they want.
Linus doesn't consider this a problem, because he doesn't use untrusted repositories.
That is correct. The hash of a commit is based on the parent commit's hash, the hash of the tree (file state), and commit message (and maybe more, I don't remember).
>It's not that they can make a repo with arbitrary code of their choice cause a collision, correct?
They can make a repo with arbitrary code, but they need to change make specific (potentially wierd) commits to get a specific hash.
> The chance of the collision be something meaningful in the context of the code at hand is vanishingly small, isn't it? Is it the concern that this can happen at all?
The chance by normal users is really small. But someone malicious could intentionally try to manipulate it. This could be done by varying commit messages.
So if you trust such a person does not have control over the repo (Linus's position)its fine, but if a hash isn't cryptographically strong, malicious actors can make repos with commits pointing to arbitrary code with a specified hash.
`git clone some-user/some-repo`
There are so many scenarios were you can end up cloning malicious code when you're simply relying on cloning alone. What if someone hacked the author's github account and committed malicious code? And why do you trust that the author has no plans in the future to inject malicious code?
What signature verification would one have when accessing a public repo? If I tell somebody there's a really great tool at fakename/faketool, they are screwed.
Your https connection confirms you got something from github, but you have the ability to prove the thing you got from github was from the same individual.
You can gpg sign your git commits. This is not a new feature...
Until git uses something other than SHA1, this is insecure.
You can gpg sign code, but do not sign a SHA1 hash.
I understand the "omg sha1" frenzy, but 1) you still need an incredible amount of compute power to generate a collision (you read your linked article, right?) and 2) even if you make something that collides, it still has to compile into the project.
Perhaps nation states can do this today. If that's your concern then, well, you've already lost because they've likely found other ways to compromise you with much less effort. For everyone else, sha1 for commits is still not _that_ bad.
That article sets the "best case", not the worse, and that article is "old". The GPU/ASIC industry is bananas, and we have no idea what other breaks there are. MD5 is a great example of how quickly these things become dangerous.
SHA1 was a terrible choice when git was made, it's a horrific one now. SHA1 cannot be used for anything security sensitive. That domain is always larger than what we may first suspect.
>the alternative is... do nothing?
No, the alternative is to use gpg with a secure hash over and above anything git provides. This is a bad choice because there is no standardized git infrastructure built around anything secure.
This fragments the industry (like Go building their own, again) where git could instead continue to unify, making an interface that has a lot of eyeballs, visibility, and a secure, universal implementation. Instead, fragmentation remains the solution. Git could have easily taken care of this by simply listening to people back in ~2005 saying, "gee, maybe we shouldn't use a hash that's already on its deathbed". https://www.schneier.com/blog/archives/2005/02/cryptanalysis...
It would be nice just have used SHA-256. I hope Snowden didn't show us nothing.
> And why do you trust that the author has no plans in the future to inject malicious code?
Signature verification is used to protect against malicious authors now?
No! This is the go ecosystem's fault! It was knowable that GitHub managed usernames this way and yet they still chose to introduce this security risk.
When you consider github.com/foo/bar you see that foo has established a reputation by developing code for third party use. Can foo break upstream builds that rely on HEAD? Absolutely. However, given that foo has a reputation for high quality content is it likely for them, barring an exceptional event, to insert malicious code? Probably not.
The issue I have with github allowing username reuse in a broader context is that the reputation from a trusted user is transferred to a third party. That third party could be neutral, well meaning, or malicious. All the same, they should not be able to take the reputation developed by that user. In this example, if foo decides to become a bespoke tailor and deletes their github, someone should not be able to use their reputation.
In the context of go using urls as canonical locations for code, is it smelly? Oh yeah. Does that mean that username reuse isn't a problem? No. A lot of people here are pointing at either go developers or github. You can blame both and eventually get a robust solution from both.
github's general policy is that if an account has been dormant for a year (which might mean not just means commits, but also fetches from it) its can be recycled. This is because they dont want to enable username squatting.
But really I kinda shudder to think of how much stuff is hackable just by stealing the right laptop and phone while someone is on vacation.
GitHub should definitely not allow username reuse, but most prominently it's a Go issue, where they use GitHub as a package manager.
1 : http://www.drinchev.com/blog/alert-npm-modules-hijacked/
Point in case: treat GitHub as part of the filesystem, so you don't need to host a package manager. I suspect that the end game is the opposite of Java (com.example/package/whatever) and in the long run you're expected to publish your source to a domain you control. Starting with GitHub and services like gopkg.in just gets people acquainted with the idea but in the enterprise you'll be importing `enterprise.com/factory/builder` or some such.
 - https://github.com/golang/dep
The only thing I get from the article is "people treat URLs as eternal, even if the site does not provide strong guarantees for that."
But that is not news and moot to debate, as a site's policy can change anytime and then you are still screwed. And nothing of this is specific to GitHub.
One day, after many years, I emailed the github guys to know if I possibly get it, and they immediately, and very kindly agreed and renamed me.
I have faced this problem with github. I changed my username. The very next day, someone created an account with my old username and now all my resume, old profiles etc. were pointing to an empty github account. :(
> What did GitHub say when you reported it per https://help.github.com/articles/github-security/ ? ...
> I did not report it. ...