
What's Coming in Go 1.15 - ra7
https://lwn.net/SubscriberLink/820217/47ed80088c03b18d/
======
greendave
> Michael Knysze significantly increased throughput of memory allocation for
> large blocks by redesigning the memory allocator's "mcentral" data structure
> to reduce lock contention. The new allocation code is more than twice as
> fast for blocks of 12KB or larger.

Excellent! Nice to see these sorts of infrastructure enhancements.

------
zaarn
The fun thing about the Go compatibility promise is that it doesn't work as
well as the go developers probably imagine; just last week one of my older Go
project was discovered to be broken; wouldn't compile anymore.
golang.org/x/crypto was responsible, they used some code that wouldn't work in
more modern go compilers anymore. Granted, I had not maintained the project in
3 years but I also had everything vendored just in case go decided to break
anything upstream, and the broken parts didn't even touch on anything that the
compatibility promise excludes.

I think the Go devs should improve in that section a bit.

~~~
jeffsmith82
I had the same issue. The issue was a security one
[https://nvd.nist.gov/vuln/detail/CVE-2019-11840](https://nvd.nist.gov/vuln/detail/CVE-2019-11840)
which required a breaking change to fix so their options where leave everyone
vulnerable but could compile or break anyone that uses this code and fix it. I
think they made the right choice.

Also golang.org/x/crypto is not a core library so doesn't actually fall under
the compatibility guarantee but they seem to try their best not to break it.

~~~
zaarn
Neither of those things applied though; I had x/crypto vendored, so it didn't
change it's code either and the CVE is unrelated to the actual code that broke
(I used blake2b) and again; it was vendored, so my code wouldn't break but be
vulnerable.

~~~
Scaevolus
How did vendored code that didn't change break your build?

~~~
TheDong
The easiest way to accomplish that is definitely with //go:linkname

You can use a comment of the format '//go:linkname [localname] [method]' to
link some function as another one. Notably, this includes unexported private
methods in the go runtime and stdlib.

I've seen code in the wild that uses this to grab go's map hash algorithm, get
monotonic time, and other things.

The go authors have used this hack themselves in the stdlib often enough
because the runtime doesn't expose some knob, and rather than thinking that
perhaps other developers may want that knob too, they instead use such hacks,
but they also don't support said hacks (because how could you, obviously
keeping all private functions stable forever is silly and impossible)

------
EE84M3i
>On Windows, Go 1.15 now generates executables that use address-space layout
randomization (ASLR) by default. ASLR uses position-independent code to
randomize the addresses of various data areas on startup, making it harder for
attackers to predict target addresses and create memory-corruption exploits.

Impressive this is only landing for Windows now after PPP made a spectacle of
the lack of ASLR back in 2013. [http://pwning.net/2013/04/21/exploiting-a-go-
binary/](http://pwning.net/2013/04/21/exploiting-a-go-binary/)

~~~
zxcmx
Ehh I agree with you but I also haven't seen an "actual" exploit for a go
memory corruption vulnerability outside of POCs.

Like I know it's possible with effort and maybe it was just lack of interest
or adoption (like lets be fair its not a browser writing or pdf rendering
engine type of language...) so while that situation was not great it never
struck me as a burning problem with practical impact in the ecosystem either.

Interested in any pointers if possible.

------
jeffdavis
What are some examples of medium-large Go projects where the code is well-
written? By that, I mean code that would make a good example for someone
trying to learn the "right" way to do things.

I was briefly learning Go a while back. I'm curious how well gorotuines and
channels have held up as building blocks, and how extensively they are used in
real projects.

~~~
benhoyt
These days when I see a fast, reliable CLI tool I think "this has got to be
written in Go" and a lot of the time they are. Some examples: the new GitHub
CLI, Docker, Hashicorp tools, ngrok (though v2 is not open source), Caddy, the
list is quite long. I can't vouch for the quality of that code, but I suspect
some of those projects are good starting points. And I've found the Go
standard library source to be really helpful (and generally fairly easy to
understand).

~~~
megameter
Speaking from some experience, Go is probably the best option these days at
"do one thing well" command line tools - the mix of library support, native
binaries, and the simple, stable language make it a good fit for a weekend
project that opens files, munges strings and pushes them over standard
protocols.

When Go has to support a large application that defines a platform in itself,
the downsides of the language rise to the forefront, but that level of
expression usually isn't entailed in a Unix-style CLI tool.

(Rust also has more potential for performance scaling in this space, but with
the "if you have the time for that" kind of caveat.)

~~~
majewsky
> Rust also has more potential for performance scaling in this space, but with
> the "if you have the time for that" kind of caveat.

When you have a CLI for some network-based service, the network latency is
going to be magnitudes larger than the performance differences between Go and
Rust anyway. (Or at least in 99% of cases.)

~~~
tonto
some suggestions are that in the future, the I/O latency is going to get very
low, network speeds very fast, etc.

[0]
[https://www.youtube.com/watch?v=SM3PhGGllL0&feature=youtu.be](https://www.youtube.com/watch?v=SM3PhGGllL0&feature=youtu.be)
[1]
[https://www.ece.rice.edu/~willmann/teng_nics_overview.html](https://www.ece.rice.edu/~willmann/teng_nics_overview.html)

~~~
majewsky
Unless that 10-gig NIC messes with the speed of light, I don't see it reducing
the RTT between me and some server in the US by any significant amount.

------
hactually
A good presentation did the rounds a while ago[0] that does a deeper dive but
the main things for us are the performance, smaller binaries and the
improvements in tooling.

The one thing I wish they added though? `go new` and have it initialise a new
project structure that stopped tiny package syndrome and bizarre Makefiles in
repos! [1]

[0]
[https://docs.google.com/presentation/d/1veyF0y6Ynr6AFzd9gXi4...](https://docs.google.com/presentation/d/1veyF0y6Ynr6AFzd9gXi4foaURlgbMxM-
tmB4StDrdAM/edit#slide=id.g57812b4045_0_66)

[1] [https://rakyll.org/style-packages/](https://rakyll.org/style-packages/)

~~~
benhoyt
Re "go new": I think part of the problem with that is that people have quite
different ideas of what a good project structure is. It also varies a lot
between a simple CLI tool (for which main.go and maybe go.mod is enough) and a
large multi-package server project.

Can you explain what you mean by "bizarre Makefiles"? I've never used
Makefiles for my (admittedly fairly small) Go projects, because "go build"
just works.

~~~
hactually
I've seen a number of projects where the root directory is a load of other
folders and then there's a Makefile and it's a kafkaesque nightmare.

I agree on the toplevel `main.go` vs the `./cmd/myapp/main.go` style needing
catering for that only comes out as the project evolves... maybe it should be
a go vet addition instead :D

~~~
stubish
Go modules are removing the need for much of the Makefile magic. You still
have use of a Makefile if you need vendoring or are dealing with non-public
branches, but things can be much simpler.

------
3fe9a03ccd14ca5
> _A new time /tzdata package was added to allow embedding a static copy of
> the time zone database in executables._

This is a BIG improvement. One of the big benefits of Go is static binaries.
Having the TZ data includes will make deploying even easier.

~~~
TheDong
The heck? This seems silly, and like an active nuisance. Timezone data changes
multiple times each year. You can see this by the tz-announce email dates [0]

Why would I want to have to update recompile and deploy a go binary 2-10 times
a year just because Morocco decided to remove DST from one of its regions?

I'd much rather the go authors not encourage me to do something dumb like
encode tzdata in a nonstandard un-updateable location in my binary rather than
having it in a standardized location that the OS will update for me as
appropriate.

[0]: [https://mm.icann.org/pipermail/tz-
announce/](https://mm.icann.org/pipermail/tz-announce/)

~~~
4ad
Go will use the system timezone data, this is just a fallback the programmer
must _opt-in_.

~~~
3fe9a03ccd14ca5
Tz data is not always available (see alpine base image, or scratch). When it
is available, it may be out of date. It may never get updated in fact.

------
FiloSottile
I'm also pretty happy about what landed in the cryptography packages.

Session ticket keys, which are a major weak link in Forward Secrecy in TLS
1.0-1.2 [0], are now rotated automatically every day, and dropped after 7 days
[1]. Session keys are also dropped once they get too old [2], instead of
becoming an increasingly powerful Forward Secrecy liability.

The tls.Config.VerifyConnection callback is a much more powerful and usable
way to customize peer verification [3].

X.509 root CAs are now extracted on macOS by linking directly against
Security.framework with some assembly glue, instead of using cgo [4].
(Warning: awakens Cthulhu.)

crypto/x509 now ignores the 20-year-deprecated Common Name field by default
[5] and has a consistent rule about invalid hostnames [6].

crypto/elliptic now provides MarshalCompressed and UnmarshalCompressed for
compressed point encodings [7].

The clever TLS 1.3 downgrade canary in the server random is now enforced
client side [8].

PrivateKey and PublicKey types implement Equal, giving a consistent method to
build type-safe interfaces for them [9].

math/big has a new Int.FillBytes method for fixed size, zero allocation byte
serialization [9], which we used all over the crypto packages.

RevocationList and CreateRevocationList were added to crypto/x509 to generate
spec compliant CRLs [11].

crypto/ecdsa has two new functions, SignASN1 and VerifyASN1 that operate on
encoded signatures instead of big.Ints that everyone was encoding/decoding
anyway [12].

Plus some performance, cosmetic, correctness, and documentation improvements!
Not a bad cycle overall, even if it made sense to focus on smaller scope tasks
for a while.

[0]: [https://blog.filippo.io/we-need-to-talk-about-session-
ticket...](https://blog.filippo.io/we-need-to-talk-about-session-tickets/)

[1]:
[https://go.googlesource.com/go/+/43f2f5024b](https://go.googlesource.com/go/+/43f2f5024b)

[2]:
[https://go.googlesource.com/go/+/6ea19bb668](https://go.googlesource.com/go/+/6ea19bb668)

[3]:
[https://go.googlesource.com/go/+/62a3f2e27c](https://go.googlesource.com/go/+/62a3f2e27c)

[4]:
[https://go.googlesource.com/go/+/6f52790a20](https://go.googlesource.com/go/+/6f52790a20)

[5]:
[https://go.googlesource.com/go/+/d65e1b2e41](https://go.googlesource.com/go/+/d65e1b2e41)

[6]:
[https://go.googlesource.com/go/+/9d1e120c42](https://go.googlesource.com/go/+/9d1e120c42)

[7]:
[https://go.googlesource.com/go/+/5c13cab36b](https://go.googlesource.com/go/+/5c13cab36b)

[8]:
[https://go.googlesource.com/go/+/a6c6e59655](https://go.googlesource.com/go/+/a6c6e59655)

[9]:
[https://go.googlesource.com/go/+/b5f2c0f502](https://go.googlesource.com/go/+/b5f2c0f502)

[10]:
[https://go.googlesource.com/go/+/c9d5f60eaa](https://go.googlesource.com/go/+/c9d5f60eaa)

[11]:
[https://go.googlesource.com/go/+/5d47f870a6](https://go.googlesource.com/go/+/5d47f870a6)

[12]:
[https://go.googlesource.com/go/+/8c09e8af36](https://go.googlesource.com/go/+/8c09e8af36)

~~~
mholt
> Session ticket keys, which are a major weak link in Forward Secrecy in TLS
> 1.0-1.2, are now rotated automatically every day, and dropped after 7 days.

Wooooah, that's awesome! Up to now I think only Caddy has been doing this
automatically & by default, I'm happy to see the std lib able to offer this
too. Maybe we can simplify some of our code base. (Although, on second
thought, Caddy can rotate the keys distributed across a cluster so as to
improve TLS performance behind load balancers, for example; so we might still
need to use our own home-brewed rotation. Which is fine!)

> The tls.Config.VerifyConnection callback is a much more powerful and usable
> way to customize peer verification [3].

Also looking forward to using this in Caddy! Right now we use
VerifyPeerCertificate which is... fine, and definitely better than without it,
but I am sure plenty of users will be happy to have the extra flexibility and
added robustness of VerifyConnection.

Everything else looks stellar too -- keep up the good work!

------
mholt
This is a well-written article for lwn subscribers, by HN's very own @benhoyt.
If you read lwn, I definitely recommend subscribing; it consistently has high-
quality technical content.

------
yegle
I was surprised to see ASLR was not enabled by default on Windows. Is enabling
ASLR by default a common thing on Windows?

~~~
mappu
It is commonly enabled for C++ compilers, but ASLR defends against attacks
that Go code is not generally vulnerable to.

~~~
zaarn
I'm not sure if that is accurate, ASLR defends against quite a range of
attacks that are the result of code execution, which AFAIK can still occur in
Go, Rowhammer comes to mind here as a universal crack against languages.

There is no downside to activating ASLR either, so I'm not sure why it isn't.

~~~
swiley
Does addr2line still work with ASLR? Because that could be a reason not to.

~~~
zaarn
addr2line works depending on the code and compiler/libc. Your libunwind may be
able to output relativ code addresses, in which case addr2line will work if
you use the stacktrace. GDB should handle this as well. If your c compiler
doesn't support it you can just disable compiling PIEs which automatically
disables ASLR but that should be reserved for debug builds.

------
sime2009
Speaking of the future, does anyone here have an idea as to the the time frame
when generics could land in a release?

~~~
phwak
An year or two.

