I think only #1 really counts as a gotchya. #3 is literally just "spend at least 2 seconds knowing what a slice is" (which you should do! there's only like 4 data structures in Go built in, so get to know 'em all). #2 is similar.
My favorite gotchya is that assigning a nil pointer to an interface will not give you a nil interface.
Once you know what's going on, it totally makes sense, but usually you don't start out with that deep an understanding of how interfaces work, and it violates natural-looking assumptions about algebraic equality, which makes for very easy counter-intuitive behavior.
Example:
package main
import "fmt"
func main() {
var x *ErrorX = nil
var err error = x
if err == nil {
fmt.Println("nope, this won't happen ")
} else {
fmt.Println("here's my non-nil error:", x)
}
}
type ErrorX struct{}
func (x *ErrorX) Error() string {
return "hi i'm ErrorX"
}
Under the covers, interfaces are implemented as two elements, a type and a value. The value, called the interface's dynamic value, is an arbitrary concrete value and the type is that of the value. For the int value 3, an interface value contains, schematically, (int, 3).
An interface value is nil only if the inner value and type are both unset, (nil, nil). In particular, a nil interface will always hold a nil type. If we store a nil pointer of type * int inside an interface value, the inner type will be * int regardless of the value of the pointer: (* int, nil). Such an interface value will therefore be non-nil even when the pointer inside is nil.
> Once you know what's going on, it totally makes sense
Anything “makes sense” from an operational point of view, so long as you internalize the rules. But does Go's handling of `nil` denote anything you ever actually want to express, regardless of language?
> it violates natural-looking assumptions about algebraic equality
No, Go's handling of `nil` doesn't violate anything about equality: everything (besides floats, but you can't fault Go for those) is equal to itself and to nothing else. But the syntax (i.e. using a single unadorned token `nil`) does a very poor job of reflecting the language's semantics (the existence fo multiple `nil` values, all different from one another).
I'm sure there are uses for it; in this case, since making nil pointers not work would require extra rules, it doesn't seem worth doing. A suggestion I saw on reddit was to use a different word for "nil" for interfaces ("unset" or something), which would make the gotchya have less bite.
And I wasn't trying to say nil-to-interface violated anything real, just that it violated natural-to-make-but-actually-untrue assumptions (which is I guess pretty much the definition of a gotchya):
If we have
a = nil
b = a
it's natural (but incorrect) to assume that we have
b == nil
Basically, I agree with you that the syntax is being poorly suggestive of the semantics. I like the "unset" suggestion because then we'd have:
to have x repeatedly be a new reference to each of xs members, it seems like we'd avoid 2 gotchyas (capture of x by a closure inside the for loop not doing what we want, and #1 in the original article).
I think the downside would be the concept of references doesn't exist anywhere else in Go (so the "for ... range" loop would have to be its own thing, rather than explainable as a transformation to simpler code) & maybe also implementation concerns I don't know about.
But if we were willing to pay the complexity cost it would make the language nicer, IMO. (I'm not saying we should be willing to pay the complexity cost though -- a lot of Go's charm is how much the language leaves out.)
> Hypothetically, if the language defined for i, x := range xs to have x repeatedly be a new reference to each of xs members, it seems like we'd avoid 2 gotchyas
I'm not convinced. I believe what you'd end up with is that someone will write a "Common Gotchas in Go" article about how, if you think you are operating on a copy in the loop, you are actually operating on a reference.
Really, it seems very non-obvious to me, why one would be a less surprising behavior than the other.
(The fact that the loop variables are not scoped to the loop body - i.e. closures will share the references - is another issue and that I would pretty unambiguously call a gotcha that should be fixed…)
You think so? It just feels like I basically never want it to be a copy, but you do frequently want to modify the element you're iterating over.
It actually pushes you to design things in ways you might otherwise not. E.g., instead of []X you'll have []*X just so it'll be easier to modify from inside a loop.
> It just feels like I basically never want it to be a copy, but you do frequently want to modify the element you're iterating over.
Feels differently to me :)
> It actually pushes you to design things in ways you might otherwise not. E.g., instead of []X you'll have []* X just so it'll be easier to modify from inside a loop.
Never do that. Instead I use indexes when I actually want to access the element.
(Where I do do that is in maps, but not because of range, but because index-expressions over maps are not addressable)
I agree on all of these. #1 would even count as gotcha when it would the loop would yield a reference instead of the a copy, since lots of code might expect a copy. I think in some other languages (e.g. C#) there was also some confusion and change about that.
There are of course also the usual gotchas when capturing things in a closure (is it by value or by reference?). But this is not Go specific.
You explained the main Go-specific Gotcha in my opinion: The dual-nullability of interface types, and the fact that non-null interface objects might contain null pointers to structs that implement the interface.
I think only #1 really counts as a gotchya. #3 is literally just "spend at least 2 seconds knowing what a slice is"
Most programming language gotchas are literally, "just spend {small time increment} learning about {thing}." Those things build up and get lost/misplaced. Programming "nickels and dimes you to death." Design is about the emergent interaction of myriad details. Programming language design is exactly that as well.
i'd say a "gotchya" is something in a language where there's a subtlety that confounds what you would reasonably expect to happen, after obtaining a casual famiarity with the language.
the slice examples doesn't qualify because it's integral to the fundamental idea of what a slice is. they only way this could confound your expectation is if you'd literally spent zero time learning what a go slice is.
(so i guess it is a gotchya if you see "slice", know python slices, and assume the concepts are the same and don't bother to investigate any further.)
by the way, i agree with you that "Design is about the emergent interaction of myriad details"; by that criteria go is very well-designed...
How do Go programmers deal with the fact that it is hard to reason about whether slices alias?
Go lacks const, so there's no way to prevent your caller from modifying the slice you return, or the callee from modifying the slice you pass it. Also the result of append() may or may not alias its parameter, depending on the allocator's mood that day.
Do Go APIs just defensively copy, or pepper APIs with "do not modify" warnings, or rely on runtime tests? Given the fact that append() may mask aliasing bugs, is there a way to make append() alias maximally?
It's rarely a concern in practice. Public APIs should document if a slice may or may not be modified, for example: `io.Writer` says ` Write must not modify the slice data, even temporarily`, and `bufio.Scanner.Bytes()` says `The underlying array may point to data that will be overwritten by a subsequent call to Scan`.
The `append` builtin may allocate a new array and copy, or reuse the same array, but the result should almost always be assigned back to the original variable. The only time you wouldn't assign it back to the same variable is when you are carefully managing the slices yourself for some specific reason, which should never be exposed via an API.
Do you also rely on function documentation telling you whether it modifies its arguments, “even temporarily”? Or do you just read the source for every function you use, transitively?
It definitely can lead to hard to find bugs. I even made an issue out of one case where it could happen in very unexpected ways (which is now fixed on the go master branch): https://github.com/golang/go/issues/21149
Apart from the issue above I've found that it's usually quite easy to reason about though. Usually when you take a slice you either use it temporarily without modifying or you throw away the original one. In the rare case where you do still need both slices to outlive the same function you just copy one of them manually defensively.
I make defensive copies when I'm worried about it.
Append will reuse the current backing array when it has enough capacity to hold whatever you're appending. So to alias maximally like you suggested, you could try creating all your arrays/slices with lots of extra capacity. That would be pretty expensive, but you could try doing it one-off just to see what happens in your test suite?
On #3 “slicing”, on a related note for Python, remember after a new list is created from slicing, each element/item in the new list is the exact reference to the corresponding element/item in the original list. This is espeically problematic when you try to do things on a mutuable object in either list, but expect the other list remains completely shield from change. You have to do deepcopy (but for immutable object deepcopy has no effect).
Makes sense and probably is a given for most programming languages.
1. Don't use private attributes for struct types if you ever want to serialize/encode them (exception: Mutex and Channel Attributes). Just had to refactor a whole lot of code just, because I decided I wanted to save the struct to disc. Btw. the type itself can and probably should be private.
2. Don't use Pointers in map key structs. As nesting maps is a pain the simple solution is to use structs as keys. But if you do so please remember not to use any pointer fields within those structs. Again when you encode and decode maps with pointers within the key structs those pointers will bite you.
For #1, the "internal" package magic name allows you to have "private public" types; types that are public in terms of what reflect thinks about them, but are private in the sense that external packages still can't get at them (which is the operational definition of "private" in Go) so it's safe to have all of their attributes be "public", i.e., start with capital letters.
I often use the "internal" when I have a package with a lot of internal-only marshaling behaviors, so that the godoc for the package isn't made up of 80% internal implementation details and 20% payload.
"As nesting maps is a pain"
Spend a few years in Perl with its autovivication and it'll seem a lot less painful. However, this is just an observation, not an actual recommendation.
The private attributes thing has also but me in the ass...encode/decide functions won’t touch private variables and won’t throw an error if all variables in a struct are private. What kind of issues did you run into with pointers inside of a struct in a map?
In my case it wasn't just that I used pointers which might work in some cases. Instead I used a pointer to a global variable containing a function reference (directly using a function reference was not possible as those are not 'compareable'). Therefore, whenever I started the programm the global variable had a different location and therefore it didn't macht the location it had when the file was saved (during an earlier execution).
Not GP: I'd guess if you briefly forget that pointers are memory values you'll run into some Fun when trying to re-create a struct and use it as a key.
I liked how this was written. The way this was written sort of reminds me of go. It's as simple as approriate, and written in a way to be memorable. Nice UX trick deadbeef!
Also check out Elixir. Testing as first-class baked in, data randomizer for tests baked in, formatter baked in, BEAM VM, pattern matching, the list goes on and on and on.
The last gotcha sort of leads to a different gotcha. That copy statement will only work as expected if the destination slice is large enough - thus always initialise it with the length of the (sub)slice you want to copy over. You won't get any errors if it's smaller, but you will surely not get what you desired.
I got nearly the same ratio as the github example. (CPU is overclock to ~4 GHz)
$ cat /proc/cpuinfo | grep 'model name' | uniq
model name: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
$ go version
go version go1.9.1 windows/amd64
$ go test -bench . --benchtime 10s
goos: windows
goarch: amd64
Benchmark_AllocWithMake-12 20000 870349 ns/op
Benchmark_AllocWithAppend-12 30000 561120 ns/op
They may be referring to the idea that the result of that operation is defined based on the y's capacity, cap(y). (Capacity is the total amount of space in the slice backing y whereas length is the number of entries used in y; append will not allocate until the new length > capacity)
If the capacity is less than the new length, a new backing array will be be allocated. This results in x != y
However, if the capacity is sufficient to contain the new values then x == y.
Normal usage of append is overwriting the initial slice variable so you don't need to worry about this, x = append(x, ...). If you use two variables, as here, you can potentially have two different slices used as though they were equivalent in later code.
Thanks for this. It gets weirder! The slice grows/doubles if capacity is insufficient. So if you initialize the original slice all the way to capacity (len=4, cap=4), it works correctly. But if len=3 and cap=4, it shows bad behavior.
https://play.golang.org/p/qf3lAoin4dM
it also takes some figuring that the first character of anything needs to be uppercase if 1) you try to a function outside of the main source file, and 2) use an element from a struct in a template, i.e. {{.element}} does not work...
This would not work actually because it would only be modifying the copy that exists in the inner loop of the for loop. You'd be taking a pointer to the for loop stack variable and modifying it.
If they were already references as you first mention that way simply article.legs = 99 would work yes.
The problem you are trying to solve is, that with a statement like
for i := 0; i < n; i++ {
doAThing(i)
}
you want `i` to be valid inside the loop body, in the for-clauses but not outside the for-statement. Go solves this by saying that an if/for/switch statement has an implicit block surrounding them and that block is what scopes the loop variables. AFAIK no one considered, that this would have this consequence in relation to closures.
Performance wouldn't really matter, because compilers tend to be pretty good at optimizing these kinds of things. They already reorder when they check for the condition and how they jump non-intuitively and the naive instruction sequence would involve freeing some stack-space at the end of the loop, reallocating it in the next iteration and then writing the new value to it. Figuring out that you can save the actual stack-pointer operations isn't that hard.
Sure. It's an easy problem to fix, now that the team is aware it. I'm pretty sure they'll solve it for Go 2. In the meantime, it would break the compatibility promise, so it's just a kludge we have to live with.
I basically just wanted to point out that the problem isn't so much an implementation question (or about performance). But that no one thought of it and it's now mainly a question of how to best phrase the spec for this :)
My favorite gotchya is that assigning a nil pointer to an interface will not give you a nil interface.
Once you know what's going on, it totally makes sense, but usually you don't start out with that deep an understanding of how interfaces work, and it violates natural-looking assumptions about algebraic equality, which makes for very easy counter-intuitive behavior.
Example:
https://play.golang.org/p/BB1b2_HDb6I