Hacker News new | past | comments | ask | show | jobs | submit login
Writing secure Go code (jarosz.dev)
381 points by gus_leonel 31 days ago | hide | past | favorite | 297 comments



As the article also mentions: instead of checking if your program has a dependency on something that contains vulnerabilities, govulncheck checks if vulnerable code is actually reached. I find that so awesome. (And I know, someone is going to point out that hipster language foo does this too and better — it’s not the norm).


If the code cannot be reached, what is the point of having it as a dependency?

Does it know which part of a dependency has a vulnerability and check, if the execution reaches _that_ part? Then it would make sense.


> Does it know which part of a dependency has a vulnerability and check, if the execution reaches _that_ part?

Yes, govulncheck does symbol-level reachability static analysis, and the vulndb is manually annotated with affected symbols for each vulnerability.

(So glad to see a comment about this at the top, I have sometimes feared we made a mistake in designing a low-noise vulnerability scanner, because I've often seen complaints that "it doesn't work" because it doesn't show as many vulnerabilities as its more popular, less accurate alternatives.)


My understanding is that the primary goal is to determine that if a program is pulling in a dependency, and only using a small part of it, to determine if that part is vulnerable or not. This allows a program owner to know if they need to do an emergency version bump in the face of a CVE or something like that. For some businesses doing emergency deployments is a massive deal.


> hipster language

Funny, I always considered Go a hipster language for Google fanboys.


It kinda is if you're thinking about the manual-coffee-grinder-french-press hipster who eschews automatic coffee makers. Rob Pike doesn't believe in syntax highlighting and to date the Go website / interactive editor doesn't have any. "When I was a child, I used to speak like a child, think like a child, reason like a child; when I became a man, I did away with childish things."

Anyway, that's fine, I like Go and I like grinding coffee manually on occasion.


Funny, I have a similar analogy when it comes to mice: Small children lacking verbal communication skills can only point at things, which is the equivalnet of using a"pointing device". When they grow up, they learn to speak meaningful sentences to express themselves. Which is equvalent to learning to use the command line...


> french-press

How is that hipster? Did you mean aero-press?


A lot of people see anything other than a Black & Decker drip coffee pot or a Keurig pod machine as "hipster coffee," somehow. But being perceived as hipsterish is the only thing that makes something hipsterish, so they can't really be wrong.


Perhaps, but all I really care about is having a complied, strongly-typed language with a fully-featured modern stdlib and good cross-compilation support that includes wasm. If that comes with an automatic admission to the Google Fanboy Club, then sign me up.

What other well-established languages do we have that meet this criteria? I know .net is a strong contender but do we have other options?


Rust & Java also come to mind (yes, Java can be AOT compiled). Erlang too if you want more fearless concurrency if you’re OK with JIT languages. There’s lots of alternatives to Go in its space but it does have mindshare and there’s nothing wrong with staying on the well trodden path even if it’s full of if err != nil instead of sane error chaining built into the language.


I have nothing against "if err != nil", in fact, I like it.

As for a replacement of Go, I would have to say Erlang or Elixir. I use Go code for non-serious distributed software, and Erlang and Elixir for more serious ones. That is not to say that Go cannot be used for serious stuff though.


Regarding Java, since early 2000 to be more precisely, although it required paying for commercial JDKs like Excelsior JET.

Nowadays besides the more well known GraalVM, there is OpenJ9 and its cousin Android since version 5.

PTC and Aicas remain as two well known commercial Java vendors, with AOT toolchains, alongside bare metal and real time GC support, although their focus is embedded deployments.


Erlang is not strongly typed and cross compiling apps with native dependencies is not straightforward. Running it on WASM is not common (is it possible?). It does not have a comprehensive standard library like Go.

It is compiled though.


Erlang is most definitely strongly typed [1]. Perhaps you confused static & dynamic typing? Easy mistake to make. Similarly, from what I could find Erlang is typically run on BEAM [2] which is an interpreter virtual machine that executes BEAM byte code (with a JIT option).

Since I’m not an expert on either language, here’s my take of how ChatGPT summarizes Erlang vs Go on various options.

Go’s standard library is primitives driven for general purpose programming while Erlang’s is purpose driven for distributed programming. So it depends on what you mean by “comprehensive”. For example, out of the box Erlang provides an environment for writing correct, robust distributed programs. If comprehensive means having a bunch of knives & start juggling that’s a different use case.

[1] https://learnyousomeerlang.com/types-or-lack-thereof#:~:text....

[2] https://www.erlang.org/blog/beam-compiler-history/


Erlang is interpreted. It is in the same class of performance as Python and Ruby. If you want a relatively high-level and performant alternative with great concurrency support your options are C#/F# (you are likely find the tooling pleasant) and perhaps JVM languages once they adopt structured concurrency (but you are likely to find the tooling less pleasant).

Graal Native Image support is very niche and does not provide the same level of experience as .NET’s NativeAOT nor has tricks up its sleeve like static linking with native libraries.



Thanks! I was briefly aware that BEAM has JIT capability but performance numbers usually put it next to other languages with interpreted bytecode so I assumed it was either not enabled or used in some specific scenarios. I should update my knowledge.


It's possible your previous knowledge was based on HiPE, which to my understanding was kind of sucky.

The new JIT in Erlang/OTP 26 is called BeamASM and is based upon asmjit


Really...? In my experience, whilst Erlang is slower than most AOT languages, its an order of magnitude faster than Python or Ruby. Most benchmarks I've seen also back that up.


Unlike Python it scales with cores perfectly, which makes sense given that’s what BEAM is designed for, but the baseline cost of operations is in the same group.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

It’s a bytecode-interpreted language. If it were JIT and statically typed we would have seen drastically different results. Also JIT output being slower than static compilation is a myth. When compilation happens does not dictate the kind machine code the compiler can produce (mostly, compiler throughput and JIT-time optimizations do influence this, but are not a strict limitation).


Erlang is JIT compiled since 2021.

Grandparent is also correct in that it tends to be faster than Python et al. If we have a deeper look at the benchmarks [1][2], as long as there is no significant amount of bignum arithmetic (where both call C code) or standard IO involved [3] it's consistently faster than Python, and often by a large margin.

[1]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[2]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[3]: Standard IO goes through several indirections to make it work with remote REPLs; other forms of IO do not suffer from this.


For your convenience:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> no significant amount of bignum arithmetic

There is none shown in the charts. There is none shown elsewhere apart from where aribitrary precision arithmetic is shown explicitly: pi-digits.


The memory taken and timeouts for Erlang are concerning in those benchmarks. And this is when compared to Python!


Perhaps when programs written for pre-R12 are now at R27.


You can statically link with native libraries if you have static versions of them in GraalVM:

https://www.graalvm.org/latest/reference-manual/native-image...


Does it let you bring your own .a/.lib and statically link it into the final product? In .NET you can pass DirectPInvoke + NativeLibrary build properties and link with anything provided the imports don't conflict (because at the final step it's the system-provided linker that statically links together the components nativeaot binaries comprise of, so you are effectively just adding another compilation object).

For example, I can take a mimalloc.lib/.a, link it into the binary and the pinvokes into its mi_malloc, mi_realloc, etc. will all be just direct calls + branch checks for GC poll if applicable (well, you need to suppress gc frame transition but it's still about three steps in total). It will be just a single static bundle at the end.

I know that conceptually GraalVM Native Image and NativeAOT are similar tools, but they mostly seem that way at a distance and at closer inspection they only partially overlap, much like C# and Java themselves do.


You can yes although the docs don't make that obvious.

https://www.blog.akhil.cc/static-jni

You can also use the FFI to avoid JNI.

I tend to feel that static linking is overrated. The moment you want easy upgrades of your app you need extra infrastructure anyway, and every tool for distributing such programs can handle directories as well as files.


> I tend to feel that static linking is overrated.

I agree. My response was just meant to indicate that NativeAOT has comparatively more effort and focus in .NET ecosystem than GraalVM's Native Image in JVM's, and as a result is an option that is easier to opt into where it makes sense. There's an intention to make it play as nicely as possible with most common scenarios and ideally not require user input when enabling it even with difficult to analyze reflection scenarios.


> I tend to feel that static linking is overrated.

For me, the big win of static linking is startup time. Just mmap the executable and go. No need to slow down the startup process with the very branchy process of resolving and loading DLLs.


That's true, although pre-linking means the bad old days of apps spending most of their time in the dynamic linker during startup are mostly behind us even when a program uses a reasonable number of libraries.


Go / golang added https://pkg.go.dev/errors

Which includes nested / stacked errors and helper functions for checking them.

It doesn't implement error classes, but you can create a stacked chain of errors which achieves the same sort of 'Handle a classification of error' (anything which includes that class).

Older libraries don't use these features, as far as I know. So it's sort of like the half-baked enumerate everything sort of generic functions that older stable versions (like on hacker rank) ship.


The %w printf verb. It yields much more than a stack dump. Get meaningful error annotations from every step back up the callstack.


I think you missed my complaint was that unlike more modern languages like Rust, Go has way too much boilerplate for error handling and not only does it not have error chaining via a `?` operator, it doesn’t even force you to check the error meaning I’m sure there’s plenty of missed error checks in production code leaving all sorts of vulnerabilities lying around. The package you linked in no way addresses what I wrote.


> have error chaining via a `?` operator

Although I do frequently find typing in the boiler plate of _every_ _single_ _error_ a bit of a faff, it does prompt me each time to really think "what if an error really happened here". I'm inclined to think that something like the ? operator makes it much easier to just toss in the ? and not consider the implications of an error.

> even force you to check the error[,] meaning I’m sure there’s plenty of missed error checks in production code

Something the equivalent of "#[must_use]" would certainly be an additional aid, (as would const pointers).

EDIT but one of the tools mentioned in the blog post, golangci-lint, will warn you of unchecked errors.


You’d really like the C community. They like to say things like “although I do find the setfaults annoying, it really makes me think carefully about memory ownership and layout”. The problem is that if you don’t have a consistent way to accomplish a task correctly, something like errors that could happen a nearly every function call, then you’re very likely to make a mistake. Coupled with that, most people ignore testing for error conditions or simulating errors, so the error handling has a very high likelihood of having bugs.


> Coupled with that, most people ignore testing for error conditions or simulating errors, so the error handling has a very high likelihood of having bugs.

Er, is Rust any different in that regard? As I said, I tend to think the `?` operator would make that worse, as the error path is so much less visible. In Golang if you don't test your error path, at least it will be listed as having no coverage -- is the same thing true in Rust?


The `?` operator is even worse than that as it leaks implementation details by default (the default you are undeniably going to lean on if testing/simulating error cases is more than you can handle), which is going to make someone's life a miserable hell when requirements change in the future.

But it seems most programmers here are only around for the MVP and are content to let the future of maintaining their garbage be someone else's problem. Which, I expect, is also the fundamental aversion they have to Go, which is trying to create a world where you still want to work on the same codebase in 30 years.

Not that Go is perfect. It most certainly isn't. But you are going to spend more time obsessing over things that aren't even all that important in practice when your "launch and dash" lifestyle is being threatened.


Rust: this `Result` may be an `Err` variant, which should be handled.

I am not at all a fan of Rust but dissing the `?` operator and the compilers error/warning messages regarding error handling and somehow thinking Go is superior here when its objectively not is "inconsistent with reality"


> The problem is that if you don’t have a consistent way to accomplish a task correctly

I know of no programming language that provides a consistent way to deal with values correctly, if the venerable if statement (or whatever is equivalent) is not it.

What is this magical fairy language that you speak of or envision?


I've been interested in learning more about Rust, but so far haven't had a project that seemed like it'd be worth learning a whole new language structure.

So, I was responding to my _understanding_ of what you had written, which apparently didn't adequately explain what you sought to those who haven't seen the thing you were trying to reference.

I do occasionally use a helper function in golang like 'nilOrPanic()' which if it's given an Error type that isn't nil causes a panic(); which isn't so useful outside of development or toy exercises.


A language like Rust makes the Error and Option types first-class. It’ll be a compiler warning (or error? Don’t remember right now) if you ignore the return from a function that returns one of these. Go requires a separate linter and relies on uncaught variables. Minor distinction but important one because defaults matter.

If you want to panic on error/option (i.e. you don’t think it’s going to happen), you add an exclamation mark after the error. If you want to unwrap but propagate the error effortlessly, add a question mark. This syntactic sugar is a pretty common ideas at this point not unique to Rust. What is a bit more unique is that Error and Option are sum types. This means that you can’t just access the value without unwrapping them somehow and unwrapping requires you to either handle the error by a match or conditional statement, propagate (?), or panic (calling .unwrap() function). But you have to make that decision and can’t ignore it so while you have to think about what you want the error handling to look like, you can’t ever accidentally forget (& since mostly you forward, ? Makes things easy even if you need to bridge different error types).


They are going to add boilerplate free error handling sooner or later. There are many proposals for "Go 2" already.


Zig is pretty good


You either die a hipster or live long enough to become mainstream.


Go is a retro nostalgia language, taking programming languages back to basics, removing syntax highlighting and advanced concepts like exceptions and function shorthands because that's what it was like in the 70's.


Pray tell, what is it about Go that stops editors from using syntax highlighting when displaying Go code?

Seriously, IMHO Go is less "retro nostalgia" and more trying to stick to proven concepts (e.g. there was no test driven development in the 70s, and Go has testing/documentation/examples built into the language) while leaving out things like exceptions and inheritance that, while widespread, have significant disadvantages.


I'll give up my Go when you pry this PDP-11 emulation from my cold, dead hand.


Great tips in here - I was not aware of `go vet` nor `go test -race`.

FWIW, while go is not memory safe, I do find that it's much easier to be safe in go than it is in other languages. Its verboseness lends to a very clear understanding of what's happening in any given function. I absolutely hated this at the start, but now ~3 years into maintaining a go codebase, I find it quite nice both for debugging as well as editing old code. I know exactly what each function does, and what the structure of data is in any given context.

Another interesting side effect is that AI tools seem to work amazingly well with golang, given how context is often local to the function.


Go very much is memory safe in the absence of data races.

Data races cause issues in all languages, though it's fair to say that Go is affected slightly more than languages like Java. Rust is a bit special by making data races hard to trigger (impossible in safe code IIUC), but this is not typical.


Kind of, regarding Rust.

It is impossible in the context of having all threads accessing in-process memory.

If the data can be accessed externally, regardless of the guarantees being uphold on the Rust side, there are no guarantees from third parties accessing the same data.

It also doesn't prevent other race issues with external data.


Memory like that needs to be wrapped with unsafe for access, there is the volotile crate to mark stuff like that so the compuler won't optimize it away.

Other than rust haskell seems like the other primary candidate for memory safety even across threads.


Yes, but it doesn't guarantee changes occurring from third parties, even if everything is done correctly on Rust side, and all invariants are correct, so corrupted data can be still be seen as valid.


Is there any defense at all against what you are talking about? I mean, I could use a firewire controller to modify memory without the processor or OS being aware. I suppose you could sign every block of memory using the tpm, but you'd have to the signatures in the tpm, and the code to check the signatures, and so on.


The point is that Fearless Concurrency comes with some footnotes when doing the full spectrum of systems programming.

Which tend to be ignored when talking about how Rust is so much better than anything else.

Ye it has improved some concurrency/parallelism scenarios, not all of them.


Yeah, and C is memory safe in absence of memory safety bugs..


> I absolutely hated this at the start, but now ~3 years into maintaining a go codebase, I find it quite nice

I've heard this so often. Thanks for sharing :)

I find going back to other languages and trying to read other people's code is a trial. There's always the temptation to write "smart" code that is terse but takes a lot of parsing to understand.

I love that I can pick up anyone's Go code and it's going to make sense almost immediately because everything is explicit, and laid out the same way (including that rhythm of "do the thing, test the error, do the thing, test the error")


Go is memory-safe. It's not the definition of "memory-safe language" that it's impossible to write memory-unsafe code, only that ordinary code is memory-safe by default.


> ordinary code is memory-safe by default

What does that mean? What constitutes "ordinary"? I'm not sure there is any official definition of memory safety, but I would consider it to mean that aside from code that is explicitly marked as unsafe it is impossible to write code that has undefined behavior.


Good definition. I've seen Go beginners trying to append to a slice from multiple goroutines. It works as well as calling push_back on the same vector from multiple threads in C++. It can easily corrupt GC state and lead to segfaults. The beginner didn't use any advanced trickery or the unsafe package. Therefore Go is not a memory safe language.


> Therefore Go is not a memory safe language.

Interesting.

To quote the NSA [1], "Some examples of memory safe languages are Python, Java, C#, Go, Delphi/Object Pascal, Swift, Ruby, Rust, and Ada. Memory safe languages provide differing degrees of memory usage protections, so available code hardening defenses, such as compiler options, tool analysis, and operating system configurations, should be used for their protections as well."

The narrow definition of memory safety here is:

Go has garbage collection, so you won't have memory leaks or use-after-free.

Go is powerful enough that beginners can cause segfaults by accidentally abusing internals, okay.

I'm not sure this is a very redeeming property of Go: Being able to crash the GC, without the flexibility of manual memory management.

But I'm not sure I'd categorize it as "not memory safe" for the same reason C/C++ aren't (a trade-off).

Because I don't believe that you can generally leverage this for the kinds of memory exploits made in C/C++.

I recall that some ML dialects (Standard ML and OCaml) have a library function Obj.magic : 'a -> 'b which escapes the type system. Using this can easily cause segfaults. Does that mean Standard ML and OCaml are not memory safe? Generally, no, they're extremely safe if you avoid that feature, which is most likely. This is arguably less safe than Go, since you most likely won't accidentally run that function.

[1]: https://media.defense.gov/2022/Nov/10/2003112742/-1/-1/0/CSI...


I'm trying to provide some commentary to OP's original term of "ordinary code" three comments above. While this term is inherently ambiguous and subjective, my personal opinion is that appending to slices simultaneously from multiple goroutines count as "ordinary code" but Obj.magic does not.


I don't think that's ordinary code. In any language, if you don't use a thread safe container and mutate it from multiple threads you'll get problems. This isn't an issue of memory safety but rather thread safety. You have to check the documentation for thread safe operations or otherwise use a lock. This goes for C#, Java, Go, you name it - the one singular exception being Rust. But, even Rust does not fully get around it.


You missed the point.

> In any language, if you don't use a thread safe container and mutate it from multiple threads you'll get problems.

Yes I agree there will be problems but what kind of problems do you get? Can you potentially get a memory safety problem, or are you guaranteed that the problem is not a memory safety problem?

The point is that thread safety problems in Go lead to memory safety problems. That's not the case in Java. You can crash the whole Go program by doing that, but you cannot crash the JVM by doing the same thing.


Crashing the whole program is actually memory safety. Because then the program can't get into an undefined state where parts of the program have access to memory they shouldn't.


Crashing via SIGSEGV is not memory safety.


Crashing with SIGSEGV can be perfectly memory safe. For example, writing to a NULL address is defined behavior on almost all platforms and typically causes the OS to send SIGSEGV.


Appending to slices concurrently, without synchronization, is unambiguously invalid code.


Yes. And: you will run into correctness bugs quickly if you mutate shared references in Go code. It's only my contention that you won't create a security vulnerability, in the colloquial understanding of the term (ie: a panic doesn't count).


You can, though it's much harder than in C or C++ or unsafe Rust for this to be exploitable. A data race on an interface value can give you a corrupted interface value, overwriting the vtable with struct contents. This can happen to lead to arbitrary code execution if you're unlucky enough, though in most cases it would be a SIGSEGV. It's also very hard for an attacker to craft a payload that can be guaranteed to reach this, though with a microservixe architecture with automatic restarts of failed services, they might get a lot of tries.


If I can induce a race that corrupts a data structure so that it leaks data back to me that I shouldn’t have access to, does that count?


How can you do that?


I mean, a very serious security vulnerability is/was row hammering, where an attacker was waiting on flipping a bit they have no access to by continuously flipping neighboring ones. Compared to that a race condition is "trivial" to exploit.


To add to Go being memory safe, it automatically blanks/zeroes memory, unlike C.


Could you share more of your thoughts on why that kind of memory corruption wouldn't be exploitable? Do go have something in place that prevents it?


Appending from multiple goroutine to an in un-synchronized slice is "memory safe", it's completely different from c/c++.

It behave exactly like Java or C# which are also memory safe.


I'm not sure of C#, but Java has stronger memory guarantees than Go, even in the presence of a data race.

In Java, all primitive types (including Object pointers) are atomically modified. And since all Java writes are primitives (Java doesn't have structs), you can never corrupt a data structure at the Java level. Of course, you can still corrupt it at a logical level (break an invariant established in the constructor), but not at the language level.

Go has a guarantee that word-sized reads/writes are atomic, but Go has plenty of larger objects than that. In particular, interface values are "fat pointers" and exceed the word-size on all platforms, so interface writes are not atomic. Which means another thread can observe an interface value having a vtable from one object but data from another, and can then execute a method from one object on data from another object, potentially re-interpreting fields as values of other types.


> Which means another thread can observe an interface value having a vtable from one object but data from another, and can then execute a method from one object on data from another object, potentially re-interpreting fields as values of other types.

If this were the case, then surely someone could construct a program with goroutines, loops and a handful of interface variables—that would predictably fail, right? I wouldn't know how to make one. Could you, or ChatGPT for that matter, make one for demo's sake?


I am also curious, I keep reading from this thread folks talking that this is possible, but I can't see to find anything searching in Google/DDG.

There is this document from Golang devs itself[1], that says:

> Reads of memory locations larger than a single machine word are encouraged but not required to meet the same semantics as word-sized memory locations, observing a single allowed write w. For performance reasons, implementations may instead treat larger operations as a set of individual machine-word-sized operations in an unspecified order. This means that races on multiword data structures can lead to inconsistent values not corresponding to a single write. When the values depend on the consistency of internal (pointer, length) or (pointer, type) pairs, as can be the case for interface values, maps, slices, and strings in most Go implementations, such races can in turn lead to arbitrary memory corruption.

Fair, this matches what everyone is saying in this thread. But I am still curious to see this in practice.

[1]: https://go.dev/ref/mem

Edit: I found this example from Dave Cheney: https://dave.cheney.net/2014/06/27/ice-cream-makers-and-data.... I am curious if I can replicate this in e.g.: Java.

Edit 2: I can definitely replicate the same bug in Scala, so it is not like Go is unique for the example in that blog post.


> Edit 2: I can definitely replicate the same bug in Scala, so it is not like Go is unique for the example in that blog post.

Could you share some details on the program and the execution environment? Per my understanding of the Java memory model, a JVM should not experience this problem. Reads and writes to references (and to all 32 bit values) are explicitly guaranteed to be atomic, even if they are not declared volatile.


    import java.util.concurrent.Executors
    import scala.concurrent.{ExecutionContext, ExecutionContextExecutor, Future}

    trait IceCreamMaker {
      def hello(): Unit
    }

    class Ben(name: String) extends IceCreamMaker {
      override def hello(): Unit = {
        println(s"Ben says, 'Hello my name is $name'")
      }
    }
    class Jerry(name: String) extends IceCreamMaker {
      override def hello(): Unit = {
        println(s"Jerry says, 'Hello my name is $name'")
      }
    }

    object Main {
      implicit val context: ExecutionContextExecutor = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(2))

      def main(args: Array[String]): Unit = {
        val ben = new Ben("Ben")
        val jerry = new Ben("jerry")
        var maker: IceCreamMaker = ben
        def loop0: Future[Future[Future[Future[Any]]]] = {
          maker = ben
          Future { loop1 }
        }
        def loop1: Future[Future[Future[Any]]] = {
          maker = jerry
          Future { loop0 }
        }
        Future { loop0 }
        while (true) {
          maker.hello()
        }
      }
  }

Here. I am not saying that JVM shouldn't have a stronger memory model, after thinking for a while I think the issue is the program itself. But feel free to try to understand.


Sure, here is an example:

https://go.dev/play/p/_EJ4EvYntr2

When you run this you will see that occasionally it prints something other than 11 or 100. If it doesn't happen in one run, run it again a few times.

An equivalent Java program will never print anything else.


Thank you, that's really illuminating.


Not at all. Java or C# can end up in a logical bug from that, but they will never corrupt their runtime. So in java you can just try-catch whatever bad stuff happens there, and go on afterwards.

Go programs can literally segfault from a data race. That's no memory safety.


No. Go is worse than Java in this regard. I'll just give you a random example from my five seconds of search: https://github.com/golang/go/issues/37484


> corrupt GC state

I understand this to mean the runtime's internal state, not visible to user code. If so, in general we should expect almost any sort of crash mode to be possible. Seems fair enough to call this "memory-unsafe".


You'll be using an idiosyncratic definition the rest of the industry does not use, but you do you.

What I think is happening here is another instance of a pattern that recurs all the time in communities like this: a term of art was created, "memory safety", to address the concept of languages that don't have buffer overflows, integer overflows, use-after-frees, double frees, controllable uninitialized pointers, and all the other memory lifecycle vulnerabilities. People unfamiliar with the state of the art heard the term, liked it, and have axiomatically derived their own definition for it. They like their definition better, and are not open to the idea that the term exists to serve a purpose orthogonal to their arguments.

Another recent instance of the same phenomenon: "zero trust".

Just as happened in the Zero Trust Wars of 2022, people, hearing the industry definition and intent of the term, scramble to reconcile their axiomatic definition with the state of the art, convincing themselves they were right all along.

The problem they have in this particular argument is: where are the vulnerabilities? Go is not a niche language. It is a high-profile target and has been for over a decade. I saw Go security talks at OWASP Chicago(!) in 2012(!). People have all sorts of hypotheses about how a memory corruption vulnerability --- not "memory corruption", but a vulnerability stemming from it, implying valuable attacker control over the result of whatever bad thing happened --- might sneak into a Go program. Practitioners hear those axiomatic arguments, try to reconcile them with empirical reality, and: it just doesn't hold up.

Just for whatever it's worth to hear this, if at Black Hat 2025 someone does to Go what James Kettle does to web frameworks ever year and introduces a widespread repeatable pattern of memory exploitability in Go race conditions, about half of my message board psyche will be really irritated (I'll have been wrong!), but the other half of my message board psyche will be fucking thrilled (there will be so much to talk about!) and all of my vulnerability researcher psyche will be doing somersaults (there will be so many new targets to hit!). On net, I'm rooting for myself being wrong. But if I had to bet: we're not going to see that talk, not at BH 2025, or 2026, or 2027. I'm probably not wrong about this.


> You'll be using an idiosyncratic definition the rest of the industry does not use, but you do you.

What definition are you using that you seem to think is the one definition of memory safety that is canonical?

> don't have buffer overflows, integer overflows, use-after-frees, double frees, controllable uninitialized pointers, and all the other memory lifecycle vulnerabilities

Any guarantees about this are dependent on the language not having undefined behavior in its safe subset. Once you have undefined behavior any other guarantees made about memory safety are significantly weakened.

> where are the vulnerabilities?

I don't know of any other than code written to demonstrate the concept. But I imagine if you look at any large Golang codebase you will find race condition bugs. So the fact that you have potential undefined behavior resulting from an extremely common coding error seems like it might be something to be concerned about (to me at least). Especially given how little Golang helps you write safe concurrent code.

That's not to say that Go is therefore totally useless and everyone should stop using it now because it's "insecure". But it also seems ... unwise ... to me to just pretend it's nothing because it is hard to exploit or that we don't have any (known) examples of it being exploited.


You will find race condition bugs. You will not find memory corruption vulnerabilities. Go look.

The argument is not about whether languages admit vulnerabilities --- all of them do. The argument is about whether they admit the vulnerabilities that motivate the term of art "memory safety". Go does not, at least not in any non-contrived scenario not involving "unsafe" or FFI.

As for definitions, I like what Alex wrote about this; or, you can look at ISRG's writing about it.

https://alexgaynor.net/2023/oct/02/defining-the-memory-safet...


You linked me to an article that defines an unsafe language as:

"a programming language which, by default, allows code to introduce memory-related vulnerabilities (use after free, buffer over/under flow, use of uninitialized memory, type confusion) and undefined behavior,"

Which was my whole point. The "and undefined behavior" part is absolutely essential to the definition because you can't guarantee the first part without it.

> Go does not, at least not in any non-contrived scenario not involving "unsafe" or FFI.

It absolutely does. You can find examples of code that triggers undefined behavior in this thread. You can hand-wave them away as being "contrived" (which is literally correct) but that doesn't just make the problem go away.

We had a whole collective meltdown (pun intended) over spectre and meltdown a few years back even though AFAICT there have never been any known exploits in the wild. But most people who are knowledgeable in these things still took it seriously because they realized that:

1. Just because you haven't found a specific instance of an in-the-wild exploit doesn't mean there isn't one 2. You don't want to necessarily wait because if something it exploitable in theory is will almost certainly be exploited sooner or later. And it takes time to sort this stuff out

Is this relevant to your average working dev slinging micro-services in go? No, probably not and it's probably not something you should even think about for the most part. But if you're writing a complex, mission critical system with very hard security requirements? Yeah maybe you should worry about little bit about the fact that Go is, by the definition you yourself provided me, not a memory safe language.


For a variety of reasons I'm confident in my interpretation of the link I provided.


The problem with naming the language "go" is that my initial interpretation of your comment was to think that "go look" was a valgrind-esque tool for finding such things.

Too bad it's not.


Go lets you use `unsafe.Pointer` (or indeed, assembly intrinsics) if you really want to, but those are certainly not used "ordinarily".


It's not just about that. Data races can expose an object in a state that was never written from any thread in Go, potentially corrupting even internal details not exposed. Simply writing a struct value from two different threads can expose this.


An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.

Another example would be code specifically contrived to highlight a soundness problem in the language.

I used the term "extraordinary" to avoid exactly this kind of bickering over corner cases that aren't relevant to day-to-day software development (or at least, not in ways that aren't immediately evident when they come up.)


> An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.

That's my point though. Of course calling non-memory safe native code over FFI can lead to memory-safety problems in any language. Likewise using the "unsafe" subset that basically every language has. But none of that is required in Go. It is only required that you mutate shared state from different threads, which is something that I would imagine happens in a lot of Go code codebases since it is an extremely easy mistake to make.

To be clear I think:

1. Go is mostly a memory safe language because it does in fact prevent the most common memory safety issues in C/C++ (UAF, buffer overflows, etc)

2. It is LESS memory safe than other modern memory-sage languages (Rust, Java, C#, Python, etc....)

3. The memory safety issues in Go are very difficult to exploit in code that is not specifically crafted to surface them


But ordinary go code is not memory safe. Data racing can trivially happen just by using the language's primitives. It requires no special keyword like unsafe, or native FFI like in other, actually memory safe languages (rust, or of the GCd kind, java, c#, Js)


You're using a definition of 'memory safety' which is not common.


How safe is it? It has pointers and they are widely used (more than Rust where pointers are unsafe, but there are other reference types). Are those safe?


Generally.

That is as it does not have pointer arithmetic, unlike C, and arrays / slices are bounds checked. So one will get a crash from a null pointer deref.

The other risk with null pointer access is struct member access via such a pointer, but again due to lack of pointer arithmetic, that can't be easily triggered. The one way would be to have a massive struct, say much greater than the page size, and deref through that - fairly unlikely.

The other reference types (slices, maps, interface values, channels) are safe unless subject to data race issues (multi goroutine update). However channels are safe there, as their role is to be used from multiple goroutines.

So the path to lack of memory safety would be a data race, leading to type misinterpretation, hence type unsafety, then incorrect access and/or spatial and temporal unsafety as a consequence.

Apart from poor design / implementation of explicit multi threaded apps, the most likely data race strikes me as accidental lexical capture by a goroutine, hence movement to the heap, and a resultant race. The sort of thing which was mentioned in a paper (by Uber?). Those should be amiable to detection by linters.

The other case of races from poor threading design would be harder to automatically detect, but also harder to trigger. Probably avoidable by correct use of mutexes around access to the shared types (slices and maps), or simply by following an Actor or CSP design model.


Thanks for the summary, that is very helpful.

At a language level though, it is either safe or unsafe. If it is "generally safe" provided you use it correctly, I would say it is not safe, in the strict sense.

I don't think data races on pointers are allowed (looking at the memory model: https://go.dev/ref/mem) but I am not sure I have understood your scenario fully. Maybe I should read that paper you mention.

Thanks again for the detailed response!


That is an absolutist position, which some of us don't agree with. Taking the view that in practice there are degrees of "memory safety". That is generally my position, and that the largest benefits come from spatial safety, then temporal safety, in that order.

On that absolute position, there possibly are no "memory safe" languages, not even Rust as until it's borrow checker "bug" is fixed, it fails the absolutist position. If such a bug is left unfixed for long enough, one can deem it as de-facto "won't fix".

The Go example code provided elsewhere in the thread included a memory race on an "interface value", that being a form of "fat pointer". It was that I was referring to, updating only half of value, so making it internally inconsistent.


Yes.


> I know exactly what each function does

Isn't this basically the same argument that C people have been using since, what, 40 years?


Don’t forget about capslock: https://github.com/google/capslock

Assess your 3P modules for dangerous capabilities


Nice, thanks a lot for that!


Semgrep is another great option to get value out of static analysis checks against both the language and a few common frameworks. It remains a popular choice for security folks writing static detection rules (and contributing them to the commons).

You can check the open rules here; https://github.com/semgrep/semgrep-rules/tree/develop/go


Does go have a bad security reputation?

I get that anything can be insecure and its a constant battle as this article suggests, but i thought it was quite secure and stable generally (say on a par with .net or any other tool you may use to make a web app at least?)


It has essentially the same security properties of all the modern non-C-languages (ie, C, C++, ObjC), with the added bonus of largely being designed after the deserialization pandemic that especially hit Java, Python, and Ruby. ~All these modern languages are fine for security (though: be careful with serialization formats in anything but Go and Rust).

Arguably, Rust and Go are the two "most secure" mainstream languages, but in reality I don't think it much matters and that you're likely to have approximately the same issues shipping in Python as in Rust (ie: logic and systems programming issues, not language-level issues).

Be wary of anyone trying to claim that there are significant security differences between any of the "modern" or "high-level" languages. These threads inexorably trend towards language-warring.


I'd point out that one advantage Go has over Rust in terms of security are the coverage of standard libraries. Go has great support for HTTP clients/servers, cryptography primitives, SSH, SQL, JSON, secure RNG, etc. all in officially maintained standard libraries. The Rust ecosystem has some standards here but the most widely used HTTP client, just as an example, is mostly maintained by one guy[1]. I think that adds considerable security risk vs Go's net/http.

1. https://github.com/hyperium/hyper/graphs/contributors


My own experience is that the Go stdlib has resulted in worse security than, for example, rust.

The reason for that is that both the Rust and Go stdlib have a stability promise, so anything built into them can't change if it's insecure.

For example, the 'tar' package in go by default returns unsanitized paths, and has led to a bunch of CVEs: https://github.com/golang/go/issues/55356

The go stdlib can't change the tar package to make it secure by default because it would be a breaking change to do so.

Rust, on the other hand, has a tar package outside of the stdlib, and so it can evolve to be more secure and over time find a better interface.

We've seen that with various other packages, where the Go stdlib HTTP implementation defaults to no timeouts, and thus makes it easy to DoS yourself. Ditto for tcp. The tls package has similar backwards compatibility warts that make it less secure by default.

Forcing backwards compatibility with network protocols by baking them into the stdlib has largely not been a security win in my experience.

You can argue that people can build packages outside of the Go stdlib too, like if the stdlib "image/draw" package is so bad it can't be used, they can make "golang.org/x/image/draw", or if the stdlib crypto package is bad, they can make "golang.org/x/crypto"... and they did, but people still reach for the stdlib because it's easier to, which makes it an active security trap.


No, I'm not going to give Rust security credit for vulnerabilities it avoided in library functionality that it simply didn't provide.


I'm not giving rust credit, I'm giving Go a demerit for having a large stdlib which it does not have a good path to evolve around security problems.

We do have stuff like `golang.org/x/<etc>` and `rand/v2`, both of which people don't really use, which are I think clear indications that the go team screwed up here.

Things like tls and http should have been separately versioned packages from the beginning, allowing infrequent breaking changes, and for users to update at their own pace independently of the compiler version.

As-is, every time I update the go compiler, I also have to worry about setting a bunch of new GODEBUG flags (like 'x509sha1=1') to perform the compiler update without breaking stuff, and then separately deal with the breakages associated with those flags. Practically every go version in recent memory has had a breaking http or tls change which has caused issues for me.

But of course they're all tied together, so to get a CVE fix in one package, I have to update the entire stdlib at once, so I have to accept some broken http change in order to fix a tls CVE or whatever.

If tls were a separate package, I could update it separately from the compiler and http package and consume security updates more quickly, and also actually update my go compiler version without worrying about how much of my code will break.

As I said, I'm not giving rust extra-credit, it did the reasonable normal thing of saying "the stdlib is for stuff we're pretty sure is actually stable", while go instead said "idk, will net.Dial ever need a timeout? Who knows, let's promise it's stable forever anyways" and "the default zero value for tls version should be 1.0 forever right", which I think deserves an obvious demerit.


For what it's worth, I don't believe there's any meaningful security difference between Rust and Go.


Good point. If you consider the size of your dependency graph as a risk, especially for languages that encourage large dependency graphs like JS and Rust, then Go has a very clear advantage.


> Be wary of anyone trying to claim that there are significant security differences between any of the "modern" or "high-level" languages. These threads inexorably trend towards language-warring.

Hm, I think this is a reasonable take but taken too far. Presumably this out of a desire to avoid people arguing about this-language-feature vs. that-language-feature, but in practice "the language" also gets conflated with the tooling and the ecosystem for that language, and having good tooling and a good ecosystem actually does matter when it comes to security vulns in practice. Indeed, anyone can write SQL injection in any language, but having a culture of finding, reporting, and disseminating those vulnerabilities when they happen, and then having mature tooling to detect where those vulnerable packages are being used, and then having a responsive ecosystem where vulnerable packages get swiftly updated, those are all things that make for more secure languages in practice, even among languages with near-identical feature sets.


What is the "deserialisation pandemic"? It doesn't have obvious web search results, and I'm struggling to imagine what about deserialisation what be common between Java and Python (except that, in both cases, I'd surely just use protobuf if I wanted binary serialisation).


In the early 2000/2010s there was a popular idea that it'd be neat to have (de)serialization functionality that could perfectly roundtrip your language's native objects, without requiring that the objects be whatever the language uses as plain old data storage. In the happy case it worked super well and basically every language sufficiently dynamic to support it got a library which let you take some in memory objects, write them to disk, then restore them exactly as they were at some later time.

This had the obvious-in-retrospect major problem that it meant that your deserialization was functionally equivalent to eval(), and if an attacker could ever control what you deserialized they could execute arbitrary code. Many programmers did not realize this and just plain called deserialization functions on untrusted data, and even when people did become aware that was bad it still turned lots of minor bugs into RCE bugs. It was often a long and painful migration away from insecure deserialization methods because of how darn convenient they were, so it continued to be a problem long after it was well understood that things like pickle were a bad idea.


See https://en.wikipedia.org/wiki/Log4Shell , but also historically the mess that is pickling/unpickling in Python (see the big scary warning at the top of https://docs.python.org/3/library/pickle.html#pickle-python-... ), and more broadly any dynamic language that exposes `eval` in any capacity.


For many years, these were the most widespread serverside RCE vulnerabilities; Rails YAML might be the best-known, but there were a bunch of different variants in Java serialization, and a whole cottage subfield of vulnerability research deriving different sequences of objects/methods to bounce deserializations through. It was a huge problem, and my perception is that it sort of bled into SSRF (now the scariest vulnerability you're likely to have serverside) via XML deserialization.


You said that Go and Rust managed to avoid these issues. Is there anywhere I can read about how they avoided it? And why other popular modern languages can't?


Elixir is "more secure" than Go due to its isolated processes, functional processing, and immutable data.


Given the enormity of Elixir's runtime, that seems extremely unlikely. The kinds of bugs you expect to see in interpreted/VM code are different than those in compiled languages like Rust; someone is going to find memory corruption, for instance, when you index exactly the right weird offset off a binary, or do something weird with an auto-promoted bignum. We still find those kinds of bugs in mainstream interpreted languages built on memory-unsafe virtual machines and interpreters.

I'm not saying Elixir is insecure; far from it. It's a memory-safe language. Just, it would be a weird language slapfight to pick with a compiled language.


My comment isn't about compiled vs. bytecode languages. It's about memory management. For example:

• In Elixir, each process runs in isolation, has its own heap, and prevents one process from directly accessing or corrupting the memory of another process. In contrast, Goroutines share the same address space, which means that a bug in one goroutine can potentially corrupt the shared memory and affect other code.

• Elixir uses immutable data structures by default, so nothing can be changed in place. Go, on the other hand, allows mutable state, which can lead to race conditions if not managed correctly. In other words, Elixir is inherently thread safe and Go is not.

• Elixir uses a generational garbage collector with per-process heaps, meaning that the garbage collection of one process can't impact another process. In contrast, Go uses a mark-sweep garbage collector across its entire memory space. This can cause global pauses that can open a window for denial-of-service attacks.

• Elixir uses supervisor processes to monitor operational processes and restart them if they crash. Go's error handling can lead to memory leaks and other undefined behavior if not carefully managed.

• Elixir inherently protects against race conditions, whereas Go relies on tools like the race detector and developer onus to avoid them.


Yeah, none of this is plausible. These are mostly true statements about Elixir that have no bearing on the memory safety of Go, in the sense that term is used in vulnerability research.


No.

Ironically, a flip side of the complaints about how Go lacks power is that a lot of the "standard" security vulnerabilities actually become harder to write. The most obvious one is lacking the "eval" that a dynamic language has; more subtle ones include things like, there is no way to take a string and look up a type or a method in the runtime, so things like the Ruby YAML vuln are not assisted by the language level. To write something like that into Go, you'd have to actually write it in. Though you can, if you try hard enoough.

But, as sibling comments point out, nothing stops you from writing an SQL injection. Command injections are inhibited by the command only taking the "array of strings" form of a command, with no "just pass me a string and we'll do shell things to it" provided by the language, but I've dispatched multiple questions about how to run commands correctly in Go by programmers who managed to find []string{"bash", "-c", "my command with user input from the web here"}, so the evidence suggests this is still plenty easy enough to write. Putting the wrong perms or no perms on your resources is as easy as anything else; no special support for internal security (compare with E lang and capabilities languages). And the file access is still based on file names rather than inodes, so file-based TOCTOUs are the default in Go (just like pretty much everywhere else) if you aren't careful. It comes with no special DOS protection or integrated WAF or anything else. You can still store passwords directly in databases, or as their MD5 sums. The default HTML templating system is fairly safe but you can still concatenate strings outside of the template system and ship them out over an HTTP connection in bad ways. Not every race condition is automatically a security vulnerability, but you can certainly write race conditions in Go that could be security vulnerabilities.

I'd say Go largely lacks the footguns some other languages have, but it still provides you plenty of knives you can stab yourself with and it won't stop you.

I've been running govulncheck against my repos for a while, and I have seen some real vulnerabilities go by that could have affected my code, but rather than "get arbitrary execution" they tend to be "didn't correctly escape output in some particular edge case", which in the right circumstances can still be serious, but is still at least less concerning than "gets arbitrary execution".


> I'd say Go largely lacks the footguns some other languages have

With the glaring exception of "I forgot to check the error code", which you need a linter (e.g. as provided by golangci-lint) for. It's critically important for security that you know whether the function you just called gave you a meaningful result! Most other languages either have sum types or exceptions.


No it's not. This is what I meant, cross-thread, when I suggested being wary of arguments trying to draw significant distinctions between memory-safe-language X and memory-safe-language Y. Error checking idioms and affordances have profound implications for correctness and for how you build and test code. Programmers have strong preferences. But those implications have only incidental connections to security, if any. Nevertheless "security" is a good claim to throw into a "my language is better" argument.


I don't even use Golang, I maybe read two Golang repos a year, I find these errors in almost every repo I look at (probably because of the selection effect: I only look at the code for tools I find bugs in). One of them I remember was a critical vulnerability of exactly this form, so :shrug: Perhaps I'm just grotesquely unlucky in the Golang projects I see, but that makes maybe 10% of the Golang error-handling bugs I've found to be security bugs.


Sounds memorable. Say more about this critical vulnerability?


I'll gesture at it. It's not an open source tool, so I can't point at the code (and in fact I just checked and I don't have perms to see the Jira ticket I caused to be raised!), and I am wary of describing security bugs in company-internal code. But in general terms it was a service that attempted to check whether a request was allowed, and it ignored errors from that check. (I just searched history for a bit to find the error in the absence of any actual details about it, but it was a while ago and I failed.) Sorry this is not a very satisfying answer.


Any language where errors are returned as values will allow you to ignore errors (if you don’t have proper linting set up, and unless it has something fancy like linear types). I’ve even seen a similar error in Haskell code, where someone called an isLoggedIn function inside a monad with the expectation that it would short-circuit evaluation, whereas in fact it just retuned a Bool.


Very true, but I do think there is an issue in the margin about how easy it is to ignore errors. For example, in Java you might have something like

``` void checkPermissions() throws AuthException ```

so you have to actively ignore errors by catching the exception. Likewise in Rust you can do

``` fn check_permissions() -> Result<(),AuthError> ```

In that case you can just use the `?` operator to short-circuit (and clippy will warn you if your forget to do that).

In other words, while language design can't fully prevent you from ignoring precondition checks, it can make it harder to forget or even force you to actively ignore precondition failures


But isn't the idiomatic Go solution something like this?

    func checkPermissions(success func())
Like anything, you can still screw it up if you try hard enough, but it should nudge most in the right direction. The talk of error handling seems like a distraction or a case of someone confusingly trying to write code in another language using Go syntax.

Obviously you are not forced to think of the user when designing an API, but you don't have to be mindful of the user in any language. Not even Haskell can save a developer who doesn't care, as noted in an earlier comment.


Go linters do a pretty good job of spotting where error return values have been ignored, so I'd suggest that the kind of bug the OP is referring to is pretty unlikely to happen in a Go project that's properly configured.


Sure - my question is "why do you need to set up the third party linters when they're so critical to the correctness of your program" really. It's the general "yeah we'll give you these footguns which are the first thing every developer will learn about during their first incident; good luck, and I hope you know you need to do things this way!" attitude I object to.


> my question is "why do you need to set up the third party linters when they're so critical to the correctness of your program" really.

No doubt the same reason it is also critical in Haskell (see comment about isLoggedIn function): Developers not knowing what they are doing.

If you work within the idioms and generally accepted programming practices this isn't a problem. It only becomes a problem when you get a developer who wants to "go their own way" without understanding why the norms exist. The linter is a crutch to support their "bad habits".


There’s no simple solution when it comes to ignoring errors. Some errors should be ignored and some shouldn’t. So your only lines of defense are linting heuristics and tests.

I would agree that languages which handle errors via exceptions have an advantage here, as they make not ignoring errors the default behavior. But even then, it’s obviously still possible to indicate error conditions of various kinds via return values, in which case they can still be ignored thoughtlessly. (And you also have all the bugs caused by unhandled exceptions to deal with.)


> Some errors should be ignored and some shouldn’t.

Assuming the function author followed Go conventions, you never need to consider the error for the sake of using the function. Granted, there are some bad developers out there who will do something strange that will come to bite you, but that is not limited to errors (or any particular language).

You may still need the error for your own application requirements, but application requirements are pretty hard to forget. At very least, you are going to notice that your application is missing a whole entire feature as soon as you start using it.


Any function that returns an error obliges its callers to check that returned error before attempting to use any other returned value.

This is Go 101 type stuff.


While you might be forced to if faced with a developer who doesn't know what the hell they are doing, that is not the convention.

Consider the case of (T, error). T should always be useable, regardless of error. At very least, if there is nothing more relevant to provide, the function should return the zero value for T. And we know that in Go the zero value is to be made useful. Go Proverb #5.

In practice, this often means something like (*Type, error), where the zero value for *Type (nil) is returned on failure. In which case the caller can check `if t == nil`. No need to consider the error at all. It is there if your requirements dictate a need for the error (e.g. reporting the failure), but for using the result it is entirely unnecessary.

If you leave the caller in a state where T can be invalid, you screwed up horribly or are purposefully being an asshole. Don't do that. That is Go 101 type stuff.


> Consider the case of (T, error). T should always be useable, regardless of error.

Go convention dictates that T is only valid (usable) when error is nil. Equivalently, if error is non-nil, then T is invalid (unusable). In either case, the caller is obliged to verify error is non-nil before trying to access T.


You're still only halfway there, as we discussed earlier.

Go calling convention is that you have to assume T is invalid, unless proven otherwise, because we know there are developers who have no business being developers that will screw you over if not careful. The Go style guide promotes documenting in a function comment that your function does return a valid T so that users don't have to guess about whether or not you are competent.

But the convention on the function authoring side is to always ensure T is valid. Just because others shouldn't be writing code does not mean you need to be among them.

As Go promotes limiting use of third-party packages by convention, as a rule most of the functions you call are going to be your own. So most of the time you can be sure that T is valid, regardless of error state. Yes, sometimes you are forced to rely on the error, as we discussed already in earlier comments. Such is life.


> the convention on the function authoring side is to always ensure T is valid.

I'm not sure where you got this idea from. It's not (in the general case) true. But do as you like, of course.


Isnt this the same as any language though.. check if have permission then ignore the result seems like something that the language cannot protect you from?


I mean, Golang has an unused variables compile error, which presumably is trying to do precisely this. It's like they got so close to forcing the user to acknowledge the possibility of errors, and then stopped just before the end!


Mmm, that's fair. I tend to forget about it because it's not something I personally struggle with but that doesn't mean it's not a problem.

I'd still rate it well below a string eval or a default shell interface that takes strings and treats them like shell does. You assert down below that you've seen this lead to a critical vulnerability and I believe you, but in general what happens if you forget to check errors is that sooner or later you get a panic or something else that goes so far off the rails that your program crashes, not that you get privs you shouldn't. As I say in another comment, any sort of confusing bit of code in any language could be the linchpin of some specific security vulnerability, but there are still capabilities that lead to more security issues than some other capabilities. Compared to what I've seen in languages like Perl this is still only "medium-grade" at best.

And I'm not trying to "defend" Go, which is part of why I gave the laundry list of issues it still has. It's just a matter of perspective; even missing the odd error check here or there is just not the same caliber problem as an environment where people casually blast user-sourced input out to shell because the language makes it easier than doing it right.

(Independent of language I consider code that looks like

    operation = determineOperation()
    if !canIDoOperation(operation) {
        // handle failures
    }
    doOperation(operation)
architecturally broken anyhow. It seems natural code to write, but this is a form of default allow. If you forget to check the operation in one place, or even perhaps forget to write a return in the if clause, the operation proceeds anyhow. You need to write some structure where operations can't be reached without a positive affirmation that it is allowed. I'd bet the code that was broken due to failing to check an error amounted to this in the end. (Edit: Oh, I see you did say that.) And, like I said, this is independent of Go; other than the capabilities-based languages this code can be written in pretty much anything.)


I think it's a reasonable observation but it isn't a fair comparative security criteria. The subtext behind error checking critiques is that languages with idiomatic sum type returns avoid authz vulnerabilities, in the same way that memory-safety in Go eliminates UAF vulnerabilities. But authz vulnerabilities are endemic to the mainstream sum type languages, too; they're much more complicated as a bug class than just "am I forced to check return codes before using return values".

Sum types are one of the few things I miss when switching from other languages back to Go. I like them a lot. But I think they're wildly overstated as a security feature. Sum type languages have external tooling projects to spot authz vulnerabilities!


> "I forgot to check the error code"

How is it that people "forget to check errors" but not other types, even though they are all just 1s and 0s? Or, to put it another way, why do programmers forget how to program as soon as they see the word "error"?

It seems to be a real phenomenon, but I can't make sense of how it can happen. It is not some subtle thing like misspelling a word in a string constant. You are leaving out entire functionality from your application. It is almost on the order of forgetting to add the main function.


I would think it's a mix of not being sure exactly what to do on error and not wanting to undergo the effort of writing error logic. You have to switch from "basic skeletal structure of the program" to "cover all bases", which isn't simple. So it's easy to have no or rudimentary error handling, and by the time you want to change it, it's hard to change. Like, "malloc can fail, but it would be a lot easier right now if I assume it won't".


as if DoS by exception is any better...


Depends on the application! There's a reason we have the concept of "failing closed" vs "failing open": sometimes (very often, in fact) it's correct to shut down under attack, rather than to open up under attack.


The subtext of that comment cuts against the argument you're trying to make here: a panic following a missed error check is always fail-closed, but exception recovery is not.


One thing to note about data races in Go is that the safe Go subset is only memory-safe if you do not have data races. The original post alludes to that because it mentions the race detector. This situation is different from Java where the expected effect of data races on memory safety is bounded (originally due to the sandbox, now bounded effects are more of QoI aspect). Data races in Java are still bad, and your code may go into infinite loops if you have them (among other things), but they won't turn a long into an object reference.

The good news is that the Go implementation can be changed to handle data races more gracefully, with some additional run-time overhead and some increase in compiler complexity and run-time library complexity, but without language changes. I expect this to happen eventually, once someone manages to get code execution through a data race in a high-profile Go application and publishes the results.


These arguments would be more compelling if they came with actual exploitable vulnerabilities --- in shipped code, with real threat models --- demonstrating them, but of course the lived experience of professional programmers is that non-contrived Go memory safety vulnerabilities are so rare as to be practically nonexistent.


About footguns, I'd like to mention an important one: in Go, it's hard to deserialize data wrongly. It's not like python and typescript where you declare your input data to be one thing, and then receive something else. It's a feature that makes server code, which after all is Go's niche, considerably more reliable.

Safety isn't 0% or 100%, and the more a language offers, the better the result. Go is performant, safe, and fairly easy to read and write. What else do you need (in 99.9% of the cases)?


> It's not like python and typescript where you declare your input data to be one thing, and then receive something else

In Python that's likely to lead to a runtime TypeError, not so much in TS since at runtime it's JS and JS is weakly typed.

Besides, Python has Pydantic which everyone should really should be using. :-)


Only if you use a deserializer that's tied to your classes, and not put everything in a dict. And then only if the data encounters an operation that doesn't accept it. But many operations accept e.g. strings, arrays, ints and floats. Is there even an operation that throws a TypeError when using a float instead of int?

Pydantic only helps (AFAIK) when you're letting it help, and you actually use the correct type information. It's not difficult to use, but it's optional, and can be faulty.


> I'd say Go largely lacks the footguns some other languages have

It does have a couple of its own. Like ((*SomeStruct)(nil)).(SomeInterface) != nil.

And yeah, the error handling is fucked up.


I was referring specifically to security footguns like having a string eval. While one can construct code in which that is the critical error that led to a security vulnerability, that can be said about any confusing bit of code in any language, and I would not judge that to especially lead to security issues.


This actually is a security footgun. In Java or C# you can't get security issues by trying to update a reference from multiple threads, because it's always atomic. In Go you can create type confusion because interface pointer updates are not atomic.


Point to a real, exploitable, public vulnerability that exploits this behavior, and then we'll all be talking about the same thing.


This sets the bar ludicrously low for "security footgun". If this is a "security footgun" then what is string evaluation in a dynamic scripting language, a "security foot-nuke"?

Granted, there is no sharp line that can be drawn, but given my personal career I'd say I've encountered it personally at least once is a reasonable bar, if not quite excessively low. (tptacek would have to set the bar somewhere else, given his career.) Concurrency issues causing a security issue because of type confusion on an interface in a Go program is not a "every time I crack open a program, oi, this security vulnerability again" like bad HTML escaping or passing things straight to a shell. I mean, "concurrency issues causing type confusion on an interface" is already not something I've ever personally witnessed, let alone it actually being a security issue rather than a difficult-to-trace panic issue.

And I will reiterate, I already say that any bug can become a security issue in the right context. That doesn't make them all "security footguns".


> This sets the bar ludicrously low for "security footgun". If this is a "security footgun" then what is string evaluation in a dynamic scripting language, a "security foot-nuke"?

Not really. Apart from dangerous serialization formats (e.g. Python's "pickle") it's not at all easy to eval a string in modern scripting languages.

String evals are also not widely used anymore.


> i thought it was quite secure and stable generally

It is, but security isn't a "given" anywhere. XSS, SQL Injection, Dependency etc can be done by any language, regardless of how "secure" it claims to be.

The headings are all pretty general (versioning, tooling, scanning, testing) but the contents are Go-specific.

It's a pretty good article IMO and could/should be replicated for other languages as well.


You can write SQL injection in any language.


> Does go have a bad security reputation?

Depends on who's behind the keyboard.


Reminds me of when SQL injection was the hot security problem, which was mainly caused by PHP, but not the language itself but reams and reams on low quality online tutorials trying to keep things simple by just concatenating GET parameters straight into an SQL query.


You can use outdated dependencies in any language.


TIL about `gosec`.


I use VSCodium when I am programming in Go, using the extension, because it has everything I need, and that includes gosec.


Wow!


does it work


I've been maintaining a Go app for about 9 years now and I can just upgrade the Go version + mod for vulnerabilities (GitHub tells me about them automatically idk) and it works with no changes 99% of the time. I can't overstate how this makes maintaining it very stress-free.

My JS apps on the other hand...


My few tiny steps in JS world were alienating in that sense: having a brand new install of all tools, doing a “npx create-react-app” and got greeted with “congrats, your app is initialised, it has 13 vulnerable dependencies”.


Tbf those are development deps rather than production server deps, and the vuln will be something like "DOS possible if you let users craft their own regex string as input to lib.foo(re) in a server ctx" rather than "by using this in development to build your static js app, people get remote access to your dev machine."


It is a bit silly then that it reports them as vulnerabilities by default.


Worse CRA goes from saviour to deprecated, "use nextjs or vite instead" in a blink. Meta should maintain it. Nextjs will probably morph again in the future so you hope investing in learning vite is the answer. JS has this way.

Meanwhile Rails is so old it is thinking it needs to find a partner, settle down and buy a picket fenced house.


Unfortunately, in Rails, your major breaking changes without a migration path come from the core team.


create-react-app is not maintained AFAIK


The very first Go code I ever wrote, way back in 2011, still compiles and runs perfectly. It's glorious.


    package main

    import "fmt"

    func main() {
        fmt.Println("Hello world")
    }



I've had similar experiences, but I've noticed my Node.js applications which have few-to-no dependencies behave in the same way as my Go apps in that regard. I might get some deprecation logs from Node letting me know about future changes, but generally they do just work. The apps with a heavy dependency graph are a different story however.

This is still a feather in Go's cap given the fact that the standard library offers so much out of the box. I don't find myself reaching for dependencies that often.


I shudder to think the amount of thousands of engineering hours are spent in my FAANG to keep our Java services just running as-is with updates.

And now we're moving more to Typescript on Node...UGH.


I thought Java was robust. What's the hassle?


Not OP, but typically Spring and transitive dependencies. Some package that you don’t even use is pulled in and has a CVE. Or you upgrade major Spring versions and the API changes underneath you.


I recommend Spring Boot, it provides a "blessed" set of dependencies that work with each other. When you want to upgrade, you just need to increase one version (the Spring Boot version).


if you don't pull in 50 jars to replace the "new" keyword (aka spring DI), then this ceases to be a problem


This Spring hater (me) thinks that's a fair summary. It also eliminates much of the safety the compiler can give you for free. You will only find out during Spring init, or even later, that your program is broken.


Have people considered frameworks implementing JAX-RS instead? Or does the breakage happen specifically in extensions to Spring?

The only inconvenience I have experienced upgrading a Quarkus backend is renaming javax.* package imports to jakarta.*. Hopefully the next major version requires just as little effort (if not less).

I am sure there would have been a lot more work if the project used extensions like the Kubernetes client. But overall, I have had the best experience with tools like Maven (for Java) and Leiningen (for Clojure). It helps to avoid libraries that hack or access JDK internals (e.g. Lombok, or ancient libraries using sun.* internal packages for reflection)


The main problem is Spring Boot and some other Spring projects like Security.

If you would use Spring MVC directly, it is very possible that one could upgrade Spring versions for many years with minimal or no changes at all.

However Spring Boot regularly breaks code. And given the fact that it's very popular, it means that any Java upgrade is pain. You need to rewrite code, sometimes a lot of code.

If you just use JAX-RS, probably simple Spring setup would suffice, but people usually want to slap database, security stuff and other things and everything is provided by Spring, so it's not apples-to-apples comparison.


I've been working with Java for the last decade and for the past 5Y used the latest LTS versions in a very regulated environment (we have very strict patch deadlines for most CVEs). Rarely we hit issues with migrating to different versions of our dependencies. The most painful one was a small API change in Spring that revealed that we were doing something very bad so it took me 1-2D in between meetings to investigate. It is true though that every few weeks we are hit by a new CVE and we have to patch a lib version, but TBH this is what I expect from a language that has so many eyes on it's ecosystem.


Java is fairly robust and plenty of libraries are very low-velocity.

JVM itself, however, has had several breaking changes recently. So a lot of organizations are stuck on an ancient version of the language.


Not really true imo.

I speak from the experience of supervised the upgrade of thousands of services from JDK8 to JDK17

There’s few quirks added but:

1. JDK17 will happily run JDK8 code without any changes 2. Most of the issues I observed were due to project jigsaw (and were resolved by adding —add-opens as needed)

I would expect 17 > 21 upgrade to have basically no issues as an upgrade in place

I hate Java but backwards compatibility isn’t one of the reasons why I hate it


This unfortunately is not true for large codebases. The language and the basic library are extremely stable, but the overall runtime is not. So the 8->17 switch resulted in lots and lots of regressions.

So companies either pay Oracle to maintain the old JDK8, or use something like Amazon Corretto. It's so bad that there are companies promising JDK8 support until 2031 at least.

And yeah, upgrades past 17 are easy.


> It's so bad that there are companies promising JDK8

Come on, that's absolutely not the reason behind. That just means that there are banks and such that still run goddamn windows XP completely firewalled off from the internet just because. Similarly, for some companies not touching that ancient codebase and just having it safely run worth the hassle and the money.

Java is the most backwards compatible language and it is not even a close competition.


> Similarly, for some companies not touching that ancient codebase and just having it safely run worth the hassle and the money. > Java is the most backwards compatible language and it is not even a close competition.

It's amazing to have two contradicting sentences right next to each other.


> Java is the most backwards compatible language and it is not even a close competition.

In competitions consisting of Java, PHP and Python, I presume?


In competitions of real world code in a language that has been in use for close to 30 years, and I can find a java 1.1 program that will both compile as source on the latest version, AND the original compiled version itself will run on a modern JDK as is.


And I assume this mythical 1.1 program does of course do a lot more than System.out.println and the reason that it took until recently, in part thanks to the Log4J fiasco, for 8 (almost 11 years old) to no longer be the most widely used version, was just superstition?


A lot of BigCo people's (myself included) perception of Java is tainted by the challenges of old, inherited code bases. Java has been ubiquitous for a long time, and it's not surprising to accumulate code bases that have been underserved maintenance-wise over the years. Updating dependencies on a Java 8 codebase isn't much fun, especially because semvar wasn't widely followed back in those days.


> GitHub tells me about them automatically idk

GitHub tells you about published CVEs which represent a small fraction of actual patched security vulnerabilities in the wild, which typically never get a CVE.


I'm still stuck in JS world - it's difficult to get a Go job if it's not already your day job - and I hate it.

Currently I'm adding a React Native component library to an NX monorepo where I want it to work with Storybook for which I need to add Expo but I can't just run the generator, I need to extract the relevant bits from a template project and cross my fingers it works.

I long to go back to the simplicity of my Go project where I'd start my day by running `make watch` and it would just work. (mind you, it took me a while to find a file watcher that worked properly)


You can run govulncheck as part of your CI pipeline too


If JS apps are the standard you measure against you'll be happy with most things.


Except 'enterprise' Java, the indirection there is insane if you're using e.g. Spring. I last worked with it five or so years ago and I had no idea what I was doing.


Don't get it. Is it because your Go app relies in fewer dependencies? If so, it's just a matter of numbers I believe. JS apps tend to rely on more dependencies on average... but that doesn't need to be that way. I have plain JS apps that still work like the first day (even better than Go apps, since there's no compilation step involved).

TypeScript apps on the other hand, yeah, they tend to be more fragile (at least from my perspective: the tsc package has dozen of dependencies, so anything can go wrong)


You can do that in practically any language however that doesn’t mean it’s easy nor the norm.

JavaScript has a culture of move fast and break things. Whereas Go has a culture of moving slow and backwards compatibility.

It also helps that Go has a pretty extensive stdlibs whereas JavaScript is really more like several distinct language ecosystems wrapped around a common specification. So what works on one JavaScript runtime might not even work on another.


> but that doesn’t need to be that way

It kind of does though. If you need to do something with security implications, reinventing the wheel is usually higher risk than using a popular dependency. So it’s not like you can realistically avoid this issue. At least not without causing bigger problems.

It’s also not just a coincidence that Go apps have far fewer dependencies. The comprehensiveness of the std lib (along with officially maintained /x/ packages) means that you need fewer direct dependencies. And just as importantly for the overall size of the tree, all the dependencies that you do need themselves have fewer dependencies. This can easily mean an order of magnitude difference in total transitive dependencies for a significant project.


Go is nice, but the recent trend of using generics for many stuff is making harder and harder to keep Go code readable imho. See an example here https://eli.thegreenplace.net/2024/ranging-over-functions-in...

I'm not saying it's hard to read, but it's harder than previous Go code that used little or no generics at all.


Your example of go code that's harder to read is iterators, and I agree with you. There's no denying that code like this places a high cognitive load on the reader:

  func (al *AssocList[K, V]) All() iter.Seq2[K, V] {
    return func(yield func(K, V) bool) {
      for _, p := range al.lst {
        if !yield(p.key, p.value) {
          return
        }
      }
    }
  }
But the code that actually uses iterators is in my opinion more readable than its non-generic counterpart. So it's really a question of how often you're expected to write (or read) iterators. And I don't expect that most programmers will be writing (or reading) iterators that often.


On further reflection, I think what makes this example particularly difficult to understand is not so much its use of generics, but the way it uses functions. It's a function that returns a function that takes another function as an argument. The generic [K,V] type arguments are actually pretty straightforward.


I often feel this way about heavy use of typescript generics. The more you lean into the crazy (and awesome) world of generics, the more inscrutable the code becomes to anybody who isn’t a generics wiz. It’s really like an extra language stacked on top of JS. I’ll come back to code I wrote a year ago, and it’ll take me a full day to figure out the types.

But the simplicity of using a library or set of functions that have really nice generics? So awesome. The intellisense and type errors alone can almost be a decent form of documentation.

The source becomes hard and weird to change, but the end result is a very nice DX


I'll admit I've only ever done one serious Go project but I've thankfully never felt a need to use generics, before generics there were the builtin list and map types that were themselves generics.


Meanwhile, error handling still can't get any sort of syntactic sugar


That's because nobody has yet solved the side effect problem of the sugar.

All the proposals that have ever been given have ultimately boiled down to essentially `return err`, which, while suitable for meme comments on an internet forum, cannot be used in a real production application for many obvious (and some not immediately obvious) reasons.

At least under the direction of rsc (the new leadership is still settling into the role so that is less clear), the will to add such sugar was there if a good solution was found. But the solution has yet to be found.


I don't know what the syntax should look like.

But the most common pattern is a sequence of calls to functions that return an optional error plus the happy path value, followed by a short circuiting check of the error, followed by a call to another function with the happy path value as an argument. It's very common to have a chain of these kinds of calls making up the body of a function.

It seems like "return err" is very useful for this pattern, if I understand you correctly. A function returning the error from the first call it makes that fails, or the happy path value if all the calls succeed. Seems like it should be possible to bake that pattern into the language, but its tricky doing it a way that doesn't obfuscate the underlying semantics, which is very important to many Go developers.


> I don't know what the syntax should look like.

I'm not sure the syntax is all that significant. There have been numerous proposals, but the syntax was never the reason for rejection. It is that the entire concept is unusable in the state that it is understood.

That's not to say the problems can't be solved, but nobody has yet.

> It's very common to have a chain of these kinds of calls making up the body of a function.

Yes, like in Rust, for example. But it also has defined traits and other features on top of the chaining to deal with the same problems Go would suffer from it had such syntax. Theoretically Go could introduce the same, but it remains unclear how to do that in a way that makes sense in the Go language.

Again, there is probably a solution out there, but nobody has come up with it yet. Surprisingly, these kind of things aren't sent down from the heavens by a magical deity. It takes human effort, which isn't there because they are busy ranting on HN.

> It seems like "return err" is very useful for this pattern

Where would you find it useful (memes aside)?


There were many proposals but none of them were an actual improvement over the simplicity and straightforwardness of the existing. `if (err != nil) {` is simple, short and to the point, and adding language features for only this use case wasn't deemed worth the cost in the end.


The problem with this syntax is that it's not required anywhere, any time. It also makes the logic extraordinarily complex for what it is. You can very quickly get into branch hell. I hate to say this, but often the control flow is much simpler and easier to understand with exceptions. The "if" works fine for one level, but any deeper than that and it's no fun.


As with real sugar, we humans don’t have sensors that would tell us when there’s "too much sugar".


I'm curious about your objection to the proposal. Sure, generics mean that libraries need a bit more syntax - that's true in all languages - but the actual consumption of the AssociationList type here is clean and readable.

Most types don't need to be generics. Containers do, and I prefer a bit of generics syntax to copy/pasting the container ten times for ten types.


You spend more time reading code that writing it. Optimising for the later is a mistake. I guess the noticeable pushback against including generics was not unwarranted, people are just now starting to see the ripple effects we were warned about.


Generics are, IMO, necessary for even a semi-modern language. Okay, you don't need a turing complete templating sublanguage like C++, but you do need at least a way to take generic functions and create generic containers.

In application code you will almost never write generics. To me, it's always been a non-issue.


It's optimized so that it's easy to read the code that you read all the time: code iterating through the containers.

It is dramatically less common to read through the implementation of containers.


Indeed. I still try to avoid generics whenever possible, and prefer a solution that doesn't use them. Thankfully, there aren't many scenarios where they're absolutely indispensable.


Write a generic instantiator that scans your codebase for generic usage, and makes one copy of a generic for every type it's used with. Then you can remove all the generics and go back to copy and paste.


Writing custom iterators always looked bad and overly complicated.

If it's not essentials I'd rather not allow code like this in my codebase and use some other solution that is more readable.


Somewhat related, I learned a surprising fact recently: Go is not actually memory safe. In particular because atomicity is only guaranteed for word size values, double word values(interface pointers, slices) can introduce memory unsafety in the presence of concurrency[0].

It's one of those things that feels obvious when you see it.

0: https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...


Here is code to circumvent Go's memory safety without importing unsafe.

get() reads a byte at an arbitrary address and set() writes a byte at an arbitrary address.

This is excerpted from BUGFIX 66 ("Hack This Site"):

  func racer() {
      var (
          ptr1 *uintptr
          ptr2 *byte
          race any
          done = make(chan struct{})
      )
      put := func(x any) {
          for {
              select {
              case <-done:
                  return
              default:
                  race = x
              }
          }
      }
      go put(ptr1)
      go put(&ptr2)
      for {
          var ok bool
          ptr1, ok = race.(*uintptr)
          if ok && ptr1 != nil {
              close(done)
              break
          }
      }
      get := func(addr uintptr) byte {
          *ptr1 = addr
          return *ptr2
      }
      set := func(addr uintptr, to byte) {
          *ptr1 = addr
          *ptr2 = to
      }
      if get(0xdeadbeef) == 111 {
          set(0xbaaaaaad, 222)
      }
  }


"Without importing unsafe" is doing a lot of work for examples like this.


This comes from a webpage where the challenge is to compromise the site, despite the fact that Go imports are disallowed (including unsafe). It's a puzzle game.

To clarify, I think Go is magnificent and I use it for everything. The racer() code is just a curiosity.


Right, it's a cool trick. It's just not material to real threat models, which is what people imply when they say "Go isn't memory safe".


The fact Go has UB under data races has practical implications for sufficiently complex concurrent software. If you can induce a race on a non-trivial object, that's UB instantly - you can probably blow up the Go runtime and all bets are off.

I would not characterise this fact, which is a design choice in Go, as similar to say a Rust soundness bug, which will sooner or later just get fixed. They aren't going to somehow magically fix this problem in Go, it's part of the design.


> They aren't going to somehow magically fix this problem in Go, it's part of the design.

I wouldn't be entirely pessimistic.

Russ's post https://research.swtch.com/gorace mentions a conservative representation for Go's data structures (essentially: more indirection) that would make it possible to implement them in a way that was robust to races, at an obvious large performance cost.

More recently others have been investigating the possibility of using 128-bit atomic writes (on ARM and x86) to reduce the cost. Go's strings and interfaces are both 2-word structures. Slices are three words but by changing the field order atomicity can be achieved with 2-word writes. Of course it would break a lot of code that assumes the representation or the ABI.


That code is usually internal.


I mean, C is also memory safe when run within valgrind.. at an obvious large performance cost.


To make this work you need to re-define "memory safety" to the point where it loses any value in a discussion about programming

Valgrind has no way to detect trivial global or local array bounds misses so long as they don't stray out of the defined memory. It can't spot this because the resulting executable (the only thing Valgrind sees) is not doing anything that's prohibited - it's nonsense because you used a non-MSL, but the actual executable has some defined behaviour.


My point has nothing to do with whether the language will achieve "soundness". It's that this is behavior that has not over the last 15 years produced exploitable vulnerabilities, despite extremely high incentives for those vulnerabilities to be unearthed.


You don’t need to blow up the runtime to cause a vulnerability due to a data race in Go:

https://security.snyk.io/vuln/SNYK-DEBIAN13-GOLANGGITHUBGORE...


That's a completely different type of vulnerability than the UB that's being talked about.

> The call to sync.Pool.Get will then return a bytes.Buffer that hasn't had bytes.Buffer.Reset called on it. This dirty buffer will contain the HTTP request body from an unrelated request.

This is just a good ol' logic error, that just so happens to also be a race.


These type of UB bugs in Go are a bit of a red herring, since most race conditions arise from improper use of shared mutability, and would still be a problem even in the presence of full memory safety, for instance:

https://github.com/golang/go/issues/37669

https://github.com/golang/go/issues/48340

These types of race conditions cannot happen in Rust. Not because Rust does not have UB, but because Rust does not allow multiple writable pointers ("mutable borrows") to the same memory region. If you want shared AND mutable access to memory, you must use a thread-safe construct such as Mutex or Cell — or drop into unsafe code.

Rust does not prevent all types of errors of course. Dirty buffer reuse (as in the GP example) is still possible in Rust. You could still have situations where a buffer is returned to a pool without resetting it. But this could only be a pure logic error where you've forgot to reset the buffer and it would occur consistently and thus would be easy to reproduce and debug. In addition, with idiomatic Rust, you could enforce proper buffer cleanup in Rust by wrapping the Buffer with a type that implements Drop.

More specifically, the vulnerability mentioned in GP is not possible in Rust. The description is a bit misleading, but the issue was not that the buffer was returned to the pool without being reset, but rather that the same buffer was returned to the pool TWICE under certain conditions, due to a data race. This is not possible in Rust. You cannot put the same owned buffer twice in a pool, due to Rust's move semantics (affine types). And if we want to be completely honest, you'd probably won't need to pool buffers in Rust to begin with, since you don't need to avoid garbage collection (there is none). In most cases, malloc is going to work good enough as your "pool".

We have a serious problem as an industry, where there is a popular conception of memory safety and type safety as a binary property: a language is either safe or unsafe, either sound or unsound. But it's more of a spectrum, and not even a contiguous one at that. This comments thread is split between people who say that large size atomicity UB is not a major issue in practice and people willing to completely rule off Go's memory safety based on that. But we could just say Go sits near the safe end of the spectrum of memory safety — it certainly does far better than C. My security concerns with Go, after nearly 9 years of using are mostly about race conditions, memory leaks and lack of better mechanisms to enforce type safety (such as sum types and affine types).


Given the total lack of empirical evidence (that is: language-specific vulnerabilities in the myriad large high-profile Go projects run all over the Internet; instances of bug classes not found in other mainstream memory-safe languages --- "memory-safe" here just to factor C/C++ out of that set) for those security concerns, why do you prioritize them?

We of course continue to find SQLI, authz, SSRF, metacharacter parsing and cryptography vulnerabilities in Go codebases, the same way we do in literally every general-purpose programming language; what the the vulnerabilities we actually see, over 15 years of explosive growth in use, that are distinctive to Go? It's been 4 years since I was a full-time software security person, but I keep up, and did a lot of work in Go before then, and I'm not aware of anything beyond "if you skip the ,ok on a type conversion you might panic something".


I must say that I really appreciate your patience in addressing these comments. If the possibility of race conditions leading to UB and the lack of sum types in Go are so bad for security, then it shouldn't be difficult to observe exploitable vulnerabilities in real-world Go code bases.


In a specific scenario, I have made use of interface values, and type switches as a form of "tagged union" / "sum type".

All it requires is that the module defining the interface, and structs implementing it, are in the same package, and the interface requires a private method.

I used that to have a message type, passing instances of it over a channel of that interface, and demuxing based on a type switch of the message.

One could use a similar scheme for return values from functions, just that for the simple error / no-error case it would not be idiomatic, however that should not prevent one from doing so if desired.


Yes, that’s a possible pattern to emulate a sum type in Go.


I think these are "if a tree falls in a forest and no one is around to hear it, does it make a sound?"-type musings.

Whatever the case, it doesn't really affect anyone and it doesn't really matter.


It’s a matter of time. Spectre / meltdown were also considered extremely hard to use in practice. Yet they are considered vulnerabilities.

In Golang case the data race window to corrupt memory is extremely narrow, so it makes it very hard to trigger it. That together with Go being still quite a niche language results in the fact we see no exploits… yet.


I note that of the Spectre/Meltdown and similar hardware vulns, even the hard-to-swallow kinds of mitigations for Spectre primarily prevent user-to-kernel hijacking only, which is the most important single property but doesn't cover inter-process hijacking. We can more or less patch these vulns completely, but there is a (huge) performance penalty to be weighed as a drawback. I do not know enough to say whether the Go data race bugs are an acceptable risk. Although, not everyone may accept it, namely if it strikes them just once.


Even if some sort of security bug is discovered tomorrow, then we're talking about one issue every 15 years or so. Whoop die doo. That barely even registers in the noise.

That it "may" lead to a problem and that it's not "sound" is basically just meaningless.


How many spectre / meltdown related vulnerabilities were detected between 1990 and 2010? Zero. So those chip vendors must be paranoid they patch them - were talking about one issue per 20 years xD Similarly how many hashmap collision attacks existed prior to 2010? Zero, but once people learned they are not just a theoretical problem, suddenly plenty of vulnerabilities were found.

Seriously, it doesn’t work like that. It’s not linear. During the first half of those 15 years almost no one heard about Go, and forget about using it in critical systems where vulnerabilities would matter. Even at Google it was (still is?) very niche compared to Java, Python and C++ and is used mostly for userspace clis and orchestration, not the core stuff. There is simply very little incentive to attack systems written Go, when there exist 100x more less secure networked systems written in C or C++.

Considering this memory unsafety thing in Go is fortunately very hard to exploit, there is no doubt why attackers don’t target this weakness and it has been so far only a technical curiosity. Also data races in Go are easy to make and can lead to vulnerabilities in a much more direct way, without corrupting the heap. I bet those are exploited first (and there exist CVEs caused by races in Go).


Well, time will tell. As Go usage increases, it becomes a more tempting target, which means that more malicious third-parties will start poring over the code of the std library and the frameworks looking exactly for this kind of vulnerability.

The same goes for Rust, Swift or Zig, of course.


I agree.


How is it not material? You only need to accidentally write and read a map at the same time in language that is supposedly for concurrency (which is why not the same as parallelism, in its case it does largely correlate).

This is a ridiculous design issue with big ramifications.


You shouldn't be modifying any variable concurrently without a mutex. The only exception to this is if the variable is (1) less than or equal to the CPU word size; (2) is at a CPU word size aligned address; and (3) atomic memory access functions are used to read and write the variable.


Even when a value satisfies these architecture-dependent requirements, the language still does not guarantee atomicity of concurrent reads/writes, and programs which rely on that assumption are buggy.


Isn't this part of the Go memory model (https://go.dev/ref/mem)?


Logic bugs != memory safety bugs.

E.g. in java you can mess up your logic with data races, but the racing itself is safe and can never cause the VM to enter an invalid state.


Memory safety as long as you don't violate certain rules is what C and C++ also have. The problem is that programmers make mistakes because we're all human.


No, the "mistakes" we talk about with C/C++ are so common that it's hard to think of a major C/C++ project not to have them, and the "mistakes" we're talking about with Go or "unsafe" Rust are contrivances built to demonstrate things an actively malicious programmer could do. Equating the two is, obviously, a sleight of hand.


To add to this: the built in go race detector is very good at catching data races. It’s a runtime, but I’ve never had a race that couldn’t be reproduced in the race detector trivially.

But yes, in theory Go has a memory safety problem because of it. In practice though, it’s that people don’t use the race detector, which is ridiculously easy to do.


Ordinary non-race-checked Go code is memory-safe in the sense that we mean it in the field of software security.


"We"? Do you have a mouse in your pocket?


It's only on Go, leave Rust out of it. Rust's safe part is entirely memory safe. Unsafe is the escape hatch, which pretty much every language has in the form of FFI.


That's not true: idiomatic Rust projects use `unsafe` much more liberally than other languages use FFI, because of shared xor mutable. That's not a knock on Rust. I couldn't be less interested in Rust vs. Go; I use both and would use them both in different situations.


I doubt that "idiomatic Rust projects use unsafe liberally". It is a more liberal construct, perhaps, but IMO actual usage is usually reasonable. Unless you mean the standard libary's use of unsafe?


I'm not saying it isn't reasonable, just that it serves a different role in Rust than unsafe/JNI would in Java: there are things you naturally want to express in Rust, not having anything directly to do with interfacing with external code, that want `unsafe` in order to (carefully) bypass shared xor mutable.


> Memory safety as long as you don't violate certain rules is what C and C++ also have

There are numbers between 0% and 100%, thus it's possible that Go can be less than 100% memory safe and still far safer than C or C++.


"100% memory safe" is mostly not a thing; it's not a concept that gets quantified this way. The closest thing I think you get to a compromised notion of safety that's actually noteworthy is Zig's allocator behavior (which can in ordinary code theoretically still produce UAF bugs, and UAF bugs --- lifecycle bugs more generally --- are the most pernicious kind of memory safety vulnerability). Most practitioners would still call Zig "memory safe". You can see how much bigger a deal that behavior is than the one we're talking about.

I think the basic takeaway here is not to tie yourself up in nots trying to quantify memory safety. There's a reason Prossimo calls Go memory safe (not "mostly memory safe"), along with Rust, C#, Java, Swift, Python, and JavaScript. Ordinary code written in any of these languages is just not going to have exploitable memory corruption vulnerabilities. Other vulnerabilities, yes!


I'm curious to hear why most practitioners would call Zig's allocators memory safe? Do you mean the std.heap.GeneralPurposeAllocator which protects against use-after-free when building in debug and release_safe mode (not release_fast)?


It's easy to demonstrate contrived abuses of Go concurrency that break memory safety, but despite the enormous popularity of the language, actual shipping vulnerabilities --- mistakes in concurrency, not deliberately-engineered pathological cases, that yield attacker-controlled control over memory --- are basically nonexistent (I can't think of a single one; there must be one somewhere!).

Basically this is about as credible an argument as claiming that Rust isn't memory safe because its libraries have so much `unsafe` code. And that claim: not super credible.

Basically, the takeaway in both cases is that it's not safe to allow an attacker to write code for you in the language. But everybody assumes that's the case anyways, because it's the case with virtually every other language (with one very notable, fraught, and well-understood exception), too.


Instead there’s a whole host of subtle footguns which while not leading to true memory unsafety will lead to complete junk data.

https://www.uber.com/en-SE/blog/data-race-patterns-in-go/


I don't care to litigate program correctness and ergonomics. Those are extremely subjective, and I don't feel like I ever get anywhere useful in those kinds of conversations. The most popular backend programming language in the industry is almost certainly Python, and it barely even has types. I still wouldn't dunk on it.

This thread is about a much narrower question, which is code security. There, I feel like I'm on much firmer ground drawing and defending conclusions, and my conclusion is that there isn't a mainstream general-purpose modern language that is meaningfully more secure than Go (or than Rust, or than Python, etc).


What is that exception?


Browser Javascript.


Also embeddable scripting languages like Lua


Go is memory safe by modern standard.

If I show you a UB in Rust without the use of unsafe does it means Rust is unsafe?


I believe UB without unsafe is considered a bug by the Rust language team.

I should’ve said in my original comment, but I don’t mean to dunk on Go. In practice the issues illustrated in the blog post I linked seem unlikely to cause problems in practice, they are interesting nevertheless.


What does that mean?

If I follow correctly, assuming that there are no bugs in the compilers/interpreters, Go is less memory-safe than Java, C#, Python (with GIL), JavaScript or Rust. The only languages that are less memory safe would be C, C++ or Zig.


That would mean it, yes. And yeah there is a bug in rust's borrow checker which can trigger something like that for some very special, "no human will ever write code like that" case. But this is an implementation detail for a semantically memory safe language, while in go's case having UB is a language primitive here.


The trigger for Go is exactly "no human will ever write code like that".


This is also true of most other "safe" languages.


That's like... the first thing you learn about the language and the primitives the language was built upon. Yes mutex are a thing.


Sure race conditions in general, but the subtlety that it can cause memory unsafety, not something I recall being mentioned.


Please note, currently, there are no tools to detect the new footguns created by the new semantics of 3-clause "for;;" loops: https://github.com/golang/go/issues/66156

> The second step is to keep the Go versions in our projects current. Even though we don’t use the latest and greatest language features, bumping the Go version gives us all security patches for discovered vulnerabilities.

It is not always a good strategy to use the latest toolchain version. There are often some fresh bugs in it. From the security perspective, it is better to use the previous version, which is also still being maintained.


Good grief, you're still banging on about this. It's been well over a year. Literally no one agrees with you. That happens sometimes. But please, accept that and give it a rest already.


Copernicus was in the same position.

:)


you're literally the only person who's making a mountain out of a molehill coming up with ever more convoluted code to "prove" the change was a bad thing.



a lot of words and yet no real world issues identified


The problems are right there. But some people choose to turn a blind eye.


.0 is for local development, .1+ is for production


There might be some bugs in .2: https://github.com/golang/go/issues/70035


The examples in that ticket are convoluted, who would write code like that? Has this issue been spotted in the wild?

I agree that there is some issue and a lint should probably warn you about these, but I doubt a lot of people will run into it.


You should read that article carefully.

Some Go core team members don't agree with you: https://github.com/golang/go/issues/66156





Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: