I'll add, too, that anyone who wants to try TLS 1.3 right now can do so with the latest source builds of the Caddy web server. We just merged support for TLS 1.3 a few minutes after the Go 1.12 release: https://github.com/mholt/caddy/pull/2399
If you have a domain name pointed at your machine:
$ caddy -host example.com
(Roughly equivalent to python -m SimpleHTTPServer but with HTTPS, and production-ready.)
crypto/tls is designed and developed to be fit to be exposed directly to the Internet. The advice I wrote here is mostly dated (I really need to write a follow-up), but it's still indicative: https://blog.gopheracademy.com/advent-2016/exposing-go-on-th...
[Updated to add] TLS 1.3 is opt-in because despite years of improvements, it can still hit compatibility issues with broken or legacy peers. (Search for "TLS 1.3" on the Go issue tracker if you are curious.)
0-RTT is blocked on figuring out good, safe APIs for it. I feel like I am 80% of the way there for servers (just have to figure out how to make it opt-in per http.Handler instead of opt-out), while for clients I have no idea how to fit resending in io.Writer.
> just have to figure out how to make it opt-in per http.Handler instead of opt-out
Hrmm, I can think of a few ways (assuming I'm understanding the problem). One is a ZeroRTTHandler iface w/ a single ServeHTTPZeroRTT function and if Server.Handler impls that iface in addition to Handler, then that function is called instead. Another way is just to have a ZeroRTTAllowed property in the Server struct of type "func(Request) bool" w/ nil as an assumption of false. If it has to be on the handler instead of on the server, do something similar to the context package and have a "func WithZeroRTT(h Handler, allowed func(Request) bool) Handler" that returns the new handler w/ the internal check. Just spitballing, I may not understand the problem (can't escape asterisks here, but Request is meant to be a ptr).
> for clients I have no idea how to fit resending in io.Writer
This is tougher (assuming I understand the problem). Maybe a TLS config or http.Request option that the entire byte slice of first write can be resent as a whole. The other option being a tls.Conn.ZeroRTTFirstWrite method that the http req's Write can use if a setting is present before falling back to regular write, I dunno.
I know and understand the philosophy behind limited exposure/customization of the handshake process. However, there is a lot of value in implementing TLS extensions, adding early data on client hello, etc. I saw [0] but didn't see any details. Is it possible/reasonable to extract some of the internals to /x/crypto/tls or similar? Or is it either too hard to maintain or is there just too much fear about misuse?
Most of it is that it's harder to come up with useful, safe, not overfitting APIs the deeper you go into internals. For example, registering extensions is rarely useful on its own as extensions are allowed to change literally anything else in the protocol, and without hooks for everything else you did not get much value.
But! Hearing about what people would like to do and then putting together an API if it makes sense is my job! So please do open an issue saying what you'd like to do but can't.
For example, I mean to extract Client Hello parsing and a capturing middleware net.Conn, as a lot of passive things can be implemented that way.
Is there any performance issue that we should be aware of when turning on TLS 1.3? Go's standard library has many assembly implementations (to leverage CPU special opcodes). What is the speed of the TLS 1.3 crypto stack in Go?
What's the plan for pairing based curves now that the Extended Tower Number Field Sieve has made BN256 unusable for many applications[1]? Do you plan on integrating BLS curves any time soon?
hehe I guess so. Was a bit shocked to read SSL is deprecated! Still had that on my "Things to learn about one day" list! ...Ah, I did get this error trying to curl something earlier today "error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version" - seems my computer hasn't heard of it either.
SSL and TLS are really the same protocol, except that newer versions are called TLS. Many software stacks still refer to both as SSL, but these days we only use TLS.
> libSystem is now used when making syscalls on Darwin, ensuring forward-compatibility with future versions of macOS and iOS. The switch to libSystem triggered additional App Store checks for private API usage. Since it is considered private, syscall.Getdirentries now always fails with ENOSYS on iOS.
Hooray! I'm surprised it took as long as it did, since IIRC this was something that broke constantly…
It appears to be ~3-4 ms faster when counting repositories like redis which is rather nice for me as a free speed boost. Probably related to the changes to GC listed here https://golang.org/doc/go1.12#runtime
I'm impressed every release by Go's GC improvements. IIRC, they started with a terrible conservative GC. It was slow and (particularly on 32-bit platforms) ineffective. But then they made it increasingly precise, then targeted 10 ms pause times, then sub-millisecond, and so on. All with far fewer knobs than the Oracle JVM.
I understand there's still a cost compared to non-GCed languages, but I think it's mostly RAM usage (perhaps affecting CPU cache effectiveness and thus efficiency) rather than tail latency.
I feel that Go really excells in getting out of your way. I rarely find myself fighting with the language or runtime. The few occasions I have have been related to native interop and generally mean I'm having to look at the internal fields of slices or strings to modify data effeciently, and even that isn't too diffit.
I often find myself fighting with the language but the runtime more than makes up for it.
In particular the GC & thread scheduler are great. The fact that they are accessible and digestible by the average go dev is so impressive a testament to their authors and their focus.
> But then they made it increasingly precise, then targeted 10 ms pause times, then sub-millisecond, and so on. All with far fewer knobs than the Oracle JVM.
The number of "knobs" in Go's GC is the same as that of G1: max pause time and max memory usage. (HotSpot has more intuitive names for its configuration settings compared to Go's confusing names.) More importantly, Go's GC sacrifices a very large amount of throughput in pursuit of low latency at all costs. This is not a very good tradeoff for most applications.
I haven’t checked since 1.10 but at that time you could definitely see what I presume was the impact of lack of compaction on heap allocations over time. That is allocations would take longer as your program ran longer.
This is mitigated quite a bit by how much gets stack allocated but I imagine there are workloads that this hampers a lot, in particular large in memory data sets.
That’s not to disagree with your point the GC improvements are great for my workloads but only to point out there is a cost to them.
There are GC paradigms that do compaction and do not stop the world (https://www.azul.com/products/zing/). They are expensive in other dimensions though (actual cash).
Again, not disagree-ing with the choices the golang team are making on GC, they work great for my workloads but they aren't without trade off for other kinds of workflows.
Also this release introduces ABI versioning, it should be a first step to introduce register based calling convention and deliver a 5% performance boost.
Glad to see syscall/js is still seeing updates. This seems like a excellent chance to share server and client code by being able to cross compile to wasm.
Go team: please reduce the binary size of wasm files!
The large binary size is mainly due to the Go runtime being packed into the binary itself. You can compile with "-ldflags -s" to make it slightly smaller. But it will almost never be comparable to Rust/Emscripten binary sizes.
OTOH, there is a new project called TinyGo (https://tinygo.org) which generates LLVM byte code from Go. This allows a subset of the Go spec, sans any GC or runtime, and generates binaries comparable to Rust/Emscripten.
I was the one who converted Google's CLA system from "Please fax us this form" to a web form. At the time (2007, 2008?) I added all the fields from the fax template to the web form. But we dropped the phone number & physical address years ago. And they were optional even before that.
Nowadays we ask for your name, email address, and optional GitHub username.
I think he's referring to the fact that maps are iterated in random order. Now if you print them directly, you may be lead to believe that the order is _not_ random.
If you print them one by one while iterating you would quickly discover the truth. Not sure why this would be worth making the string rendering of large maps much harder to use and compare (eg in log entries). Since any rendering implies an order (which is not guaranteed to remain consistent through the program's lifetime), then any string rendering of a map should be considered misleading by your logic.
Given Python's success with its insertion-order-retaining "compact dict" implementation[1], I've wondered why more languages aren't adopting that design into their maps/hashtables.
"The order-preserving aspect of this new implementation is considered an implementation detail and should not be relied upon"
Early in Go's development, they intentionally randomized the iteration offset to make sure developers didn't become dependent on a behaviour they didn't want to be tied to.
You see this in Erlang also, where iteration order is deterministic (though not based on insertion order), but documentation makes sure to say iteration is arbitrary.
So, the answer to your question is that most people don't want their dictionaries implementation to be tied down due to this feature.
There's a big difference between "iteration order is deterministic" and "iteration order is <very specific high-level behaviour>" even if the latter is a side-effect of an implementation details. In particular, iteration order being either insertion order or sorted are useful and sought-after properties, while other implementation-dependent deterministic orderings are mostly implicit dependencies and sources of bugs (not unlike e.g. the scheduling sequence of goroutines in Go).
I wonder if this isn't going to be seen as a mistake in the future. What if a faster map is discovered in 3 years that happens to not preserve order? This stuff is really hard to detect, so there's no going back on it.
That was likely weighed and I'm guessing the answer is that they would introduce a new type under collections, but that the new dictionary makes most sense for users while also currently being faster
In Python, they'd probably just make it a separate class, just like `OrderedDict` is/was. People who needed the performance would use it and everyone else would enjoy the convenience of ordering.
To me it's a deeper issue that this and breaches the single responsibility principle.
fmt should only print, not manipulate the data it is asked to print.
I believe that the spec says that maps iterate over keys effectively at random and so that should be the result of any operation that iterates over keys.
The `fmt` packages is described as being for "formatted i/o". It seems that ordered maps, while not "i/o" is reasonable to describe as "formatting", so it's fine to do it in this package I think.
`fmt` has always made things look prettier, that's part of what formatting is about.
You could equally argue that `fmt` should not round floats or pad numbers with zeros or spaces.
The data itself is not being "manipulated", though. By definition it is unordered. You are not guaranteed that whatever order you get when iterating the keys will remain the same. But any string rendering of the map would put the keys in an "order" and thus would be "manipulating" the data according to you. Why not have the order be useful to humans?
You're missing the point entirely. Maps have no defined order. You cannot "change" the order of a map, because it has no order. `[foo:1 bar:2]` is the same map as `[bar:2 foo:1]`, there is no difference.
For printing purposes that means that it doesn't matter which order you print something out in. ANY order is equally correct. So you might as well choose the one that makes the most sense to human eyes, which is to print them out ordered alphabetically on the keys.
Just because the printing function is choosing alphabetical order to print things out in, it's not "manipulating" anything. Literally nothing about the map changes.
How are you reading "vitriol" from these comments? "Print" isn't changing the order of the data it's given--the data is unordered by definition. It has to print it in an order, so it orders it by keys as a practical courtesy to humans. Nothing is manipulated except perhaps data structures internal to the print operation.
You can print in sorted order without modifying the map at all. I don’t know what definition of “manipulating” you’re using, but I struggle to imagine a definition that is both useful and consistent with your position.
I'm confused about why this would be a problem, though. The majority of use cases I can think of for using fmt functions to render maps into strings would not be harmed in any way by presenting the map in sorted-key order. For my own use, such usage is mostly for debugging or logging purposes, in which case a sorted map is vastly preferable.
Given that the order of keys in a map is not guaranteed, I also wouldn't consider rendering them in a fixed order as "manipulation". The order of the keys is undefined, unimportant, program-wise. So presenting them in a certain order implies meaning (presumably there is one, whether it be creation order, LRU, or physical-memory layout). But if you know the keys are sorted before being printed, then you can continue to assume you don't know the actual order, or to put it another way, you can continue to suspend your disbelief about whether the keys are truly "unordered" in the system.
But to get back to practical concerns, I'm genuinely curious what harm you see from presenting the unordered keys in a predictable order that allows humans to get more use from the particular formatting?
It is not random, it is undeterminate. Any order follows that spec. Whether you display items in alphabetical order, in insertion order, randomly, or in any other way, you're good.
But sometimes golang docker images are late for a few days which is very annoying if your ci build pipeline depend on them. It would be great if Go team would be able to update docker images as well. And I am sure that this automation would not be so hard to do.
For some organizations unpredictable docker image releases are considered unreliable to use at all. Sometimes the image is updated the same day, sometimes after a few days, earlier even after more than a week.
The gap between developer tools and infrastructure support for them should be as small as possible this days. It's 2019.
You can always make your own base image with go installed that you update the instant you need to, if you really can't wait for the team responsible for the accepted docker image to do it...
I think that the question is, why wouldn't you? There are already a lot of work and testing done on projects that depend on 1.12 features, but using beta and rc releases. Final 1.12 version would just confirm that everything works, but without docker image, testing on staging infrastructure is delayed even the final version is out.
This is why I have https://NewReleases.io configured to track both Go GitHub releases as well as golang image tags on Docker Hub.
So you do not need it as you do not want to use it too soon. That is cool. Others may need docker image release aligned with regular release, and I think that there is nothing wrong with that. Especially for the people that are trying to contribute to Go by testing beta and rc releases.
That’s what Go’s bets and rc builds are specifically for - to test your code against a new version before it’s available. The only changes to Go during an rc are ceirical bug fixes.
If it’s a critical part of your infra, building your own container images isn’t a huge ask. Use gimme[0] to install your preferred version of Go into the container.
Either as a base for a development environment, or a disposable container to compile your go programs.
For example, compiling your program with a different version of Go can be done just by changing the tag on your compile command, and doesn't require that you have any development tools or compilers installed on your machine (except docker).
It's super useful to be able to build go binaries in a CI system. It provides the Go toolchain in a container so that you can just use it as a mini-executor. For instance, in Jenkins you can hook up the Jenkins master to a Kubernetes cluster and run your CI in a docker container. The Go docker container is great for this. I can run `go test ...` etc.
Also, when building a container with a Go binary, it's a great base to build from. You can use Docker Multistage Builds[1] to build your binary in the Go container, then copy it out into a minimal container all in the same Dockerfile.
Been meaning to make the dive into learning Go for a while now. My semester is finally slowing down so about to make the jump. I'll be doing a lot of digging on my own, but please link any resources that you feel were helpful for you.
Very happy to read about the improvements in releasing memory back to the OS. Hopefully this will help prevent our go apps in containers from getting OOM killed.
> A contribution to Go is made through a Google account [...].
Google accounts can either be Gmail e-mail accounts, G Suite organization accounts, or accounts associated with an external e-mail address. [...]
The fact that Go is mostly controlled by Google doesn't bother you, but that does?
(I consciously avoided getting a Gmail account for years until I bought an Android phone which requires such an account, so I understand where you're coming from but this issue seems meh to me.)
> (I consciously avoided getting a Gmail account for years until I bought an Android phone which requires such an account, so I understand where you're coming from but this issue seems meh to me.)
Gmail accounts are not required for Android phones. To utilize the Google services on an Android phone, which 99% of people will choose to do, a Google account is required.
A Google account can be but doesn't have to be a Gmail account. You can use any email address to create your Google account.
It doesn't bother me that it's "controlled" (someone has to lead the project anyway) by Google. being Google doesn't disqualify them from doing good things, but needing a Google account makes it seem like it's a closed community only for Googlers.
Please don't do this. All code has to have a CLA signed legitimately. Many many projects do this, and all Google projects do. You're basically going to poison the project you are trying to improve.
Disclaimer: Used to work in Google's Open Source office, which administers the CLA.
Sorry, I didn't intend that you'd use false data filling the CLA. Didn't realise you relied upon the Google Account data in the CLA. If that's the case then yes, don't do that.
Sort of... it works with GoMobile but because it doesnt use LLVM there is no support for bitcode which means WatchOS and tvOS are off limits and maybe iOS someday.
This is a rather unique take on Go that I haven't seen before. A quick scan of your post history indicates you feel quite strongly (negatively) about Go. May I ask what inspired this particular take, and what language background you have?
I'd be happy to answer any questions about it, or about the rest of the Go cryptography libraries!