
Go and Simplicity Debt Redux - xyzzy_plugh
https://dave.cheney.net/2017/06/18/simplicity-debt-redux
======
alkonaut
> Everyone who wanted to do basic things like read input or write output would
> have to understand how option type, in addition to discussion of what
> templated types are

I don't necessarily think that's a bad thing. Something that may be null _is
already an option type_. It's "some T or null". Making the compiler force you
into checking which it is, is no difference from checking for the presence of
null. There is literally no difference in complexity - it's actually simpler
because you don't have the cognitive overhead of remembering where a certain
variable has and hasn't been null checked (After a null check, your variable
has magically changed type to "some T, definitely not null".)

    
    
       x = getsomething  
       if x != nil
         dosomething(x)
    

and

    
    
      x = getsomething
      if x.hasvalue
        dosomething(x.value)
    

The programmer alone has to remember this - remember to check, and the status
of each variable that can be null. Was x checked before? scroll up to see...
Add 10 variables and some nested ifs and ask the programmer which variables
can and can't be null at any given place. This is the job of a compiler, not a
programmer. And if my experience is any guide, the less experienced
programmers, which are the ones we are discussing now, would be happy to have
100 locals or 12 levels of nested ifs in a method. They need this added help
_more_ than the people who ask for it.

~~~
catnaroek
No, nulls aren't options. Options nest. You can have Some(Some(Some(foo))).
The non-compositionality of null makes it impossible to abstract it away.

~~~
brandonbloom
Huh? What kind of "nesting" are you talking about? How is that different than
nesting structs containing nullable pointers?

~~~
catnaroek
In ML, if you have a type “foo”, then you can also use the type “foo option
option option” without creating any new nominal datatypes. In particular,
using a value of type “foo option option option” is just as easy as using a
value of type “foo” or “foo option”:

    
    
        case wrapped_foo of
            NONE => ...
          | SOME NONE => ...
          | SOME (SOME NONE) => ...
          | SOME (SOME (SOME foo)) => ...
    

On the other hand, with your proposed alternative, you have to create a
wrapper struct for every nullable kind of thingy:

    
    
        type MaybeFoo struct { foo *Foo }
        type MaybeMaybeFoo struct { foo *MaybeFoo }
        type MaybeMaybeMaybeFoo struct { foo *MaybeMaybeFoo }
    

And, as if that weren't offensive enough, you have to manually pack those
pointers into structs and then unpack them back.

~~~
brandonbloom
1) You're not "abstracting away" anything like that. If you're talking about
operations you can abstract over `a option`, that has nothing to do with
nesting.

2) I would consider explicitly nested option types to be a code smell. If `foo
option option` happens incidentally via a module functor or something, fine,
but if you ever write that by hand, you probably want to explicitly flatten
that to a 3 or 4 case sum type with explicit constructor names.

3) Nested structures like this are very unlikely to have synthetic, abstract
names like MaybeMaybeFoo. Instead they are more likely to have operational,
domain-specific, concrete names names. These structs are likely to _already
exist_, since Go lacks type inference for function parameters: You're already
going to have to create types in order to talk about these things.

4) You're "manually" packing/unpacking by writing (SOME (SOME ... constructor
calls and in your pattern match. The difference between &Foo{&Bar{...}} and
(Foo (Bar ...)) is irrelevant. Meanwhile, pattern matching has a syntactic
advantage over a `== nil` check, but again, I'd argue that intentional nesting
of numerous expected absent values is a code smell. If you really need that,
you'd probably employ the null object pattern, which is easy to support in Go
because methods can be called on nil instances. I'll say: I've never had to do
that in tens of thousands of lines of production Go.

~~~
catnaroek
1) When I make an abstract type “foo” whose underlying implementation (hidden
to clients) is “bar option”, I'm abstracting over an option type.

2) What if, in client code, I want to make a value of type “foo option”,
completely unaware that foo's underlying implementation is “bar option”? Is it
still a code smell? Do I have to know what abstract types defined by other
people are implemented as options, so that I don't wrap them in options of my
own?

3) Meh.

4) My pattern matching is completely safe. OTOH, with your proposal, you can't
safely unwrap those nested structs of nullable pointers in a single step,
because you need to unwrap the first layer just to check whether the second
layer is safe to unwrap.

\---

2) The problem exists whether there exist non-disjoint types or not.
Oftentimes I _do_ want to use nested types, because that's the way the data is
naturally structured, i.e., that's what shortens the description of the
operations that act on the data. Flattening the nested type into a single sum
with 4 constructors would merely impose on myself of undoing the flattening
every time I want to operate on this sum.

3) It's abstraction for the sake of making code easier to reuse and verify.

~~~
brandonbloom
1 & 2) Read my comment again. My remark about functors (in the ML sense)
addresses this point. I'm quite well versed in how type abstraction works.
Your comment about being aware of underlying type details is completely non-
sensical with respect to nesting. The problem you're worried about applies to
non-disjoint union types, ie dynamic typing, not Go's typical use of type-safe
pointers. Nil is type-safe in Go, there's no risk of confusing:

    
    
        a nil *Foo with a nil *Bar
    

3) You say "meh", but abstraction for the sake of abstraction is exactly what
drives working programmers away from so called "better" languages.

4) Meh.

Anyway, I'm done with this thread.

------
grasleya
More golang Stockholm syndrome. These arguments are always weird to me. Go has
plenty of complex features that go developers don't seem to think cause too
much cognitive burden (automatic gc, structural subtyping, etc.). Why is it
that relatively simple and ubiquitous language features like exceptions and
generics are just way too much and cause an unacceptable amount of complexity?
I just don't buy it.

Turn the argument around. Are there any Java, C++, etc. developers arguing for
the removal of these features from their languages? Are there people who say
that even though these languages have generics you shouldn't touch them
because they're bad? Do any devs with option/maybe types really want to go
back to unsafe code that can fail if you forget to check for nil/null? Show me
someone who hasn't drunk the go kool-aid who still makes these sorts of
arguments and I might actually start listening.

~~~
bredov
Yes, google c++ style guide
([https://google.github.io/styleguide/cppguide.html](https://google.github.io/styleguide/cppguide.html))
recommends to avoid exception as well as complex template metaprogramming, and
if one has to be used, user visible api should avoid templates if possible.

~~~
grasleya
I can buy using generics sparingly (especially in the form of complex template
metaprogramming). But never using it at all? Can you imagine convincing the
C++ community to give up Boost because its use of templates is too complex?

They also seem to admit that if they were to do it over again and start from
scratch they'd use exceptions:

> Things would probably be different if we had to do it all over again from
> scratch.

~~~
kjksf
If exceptions are so great, why Rust and Swift aren't using them?

It seems that thinking on exceptions in low-level languages is firmly in the
"no way" camp.

Contrary to popular thinking, exceptions are not free.

For example, when Chrome disabled rtti and C++ exceptions in Chrome codebase,
it resulted in saving 6MB (20%) of code. See
[https://bugs.chromium.org/p/chromium/issues/detail?id=19094](https://bugs.chromium.org/p/chromium/issues/detail?id=19094)

This is when exceptions where not actually used in the code. The bloat is
merely from enabling C++ compiler flags to generate rtti and exception support
code.

~~~
cwzwarich
> If exceptions are so great, why Rust and Swift aren't using them?

Technically speaking, Rust does have exceptions with its unwinding feature.
The panic! macro throws an exception, and catch_panic allows you to catch it.
It is implemented exactly the same way as C++ exceptions, with the
accompanying code bloat and performance pessimization, and you have to think
about exception safety exactly the same way when writing unsafe code.

~~~
kibwen
Rust has unwinding, which indeed opens a whole can of worms, but to say that
it has exceptions is to miss the point. Rust has nothing anywhere near close
to first-class resumable exceptions as surfaced in, say, Java. Not only is the
`catch_unwind` function (it's not called `catch_panic`, btw) deliberately
difficult to use (specifically to deter people from using it as a general
error handling mechanism), the language deliberately defines an alternate
compilation mode where unwinding does not exist and hence any attempt to
"catch" it will fail. As a result, I have never once seen Rust code APIs that
use `catch_unwind` as a mechanism for error handling; the purpose of that
function is to prevent unwinding from crossing FFI boundaries.

------
YZF
A few random thoughts:

\- C++'s std::array is an example of marrying fixed sized arrays with
templates.

\- I'd say keep the built-in types and their behavior unchanged but offer a
way of providing those for user types. I guess the Deletable interface is one
idea but maybe the "delete" override should have nothing to do with
interfaces. I think something like func (t _Type) operator delete(k_ KeyType)
? I guess in Go a function _is_ an interface but I wouldn't start with the
interface here.

\- Allowing e.g. delete or slicing on user types does not need to imply those
types are polymorphic with the built-in types. This might be a little
confusing but I think it's still worth while to draw that line. I don't think
it's necessary to allow creating a function that takes a Deletable to take
either the built-in map type or a user type.

\- range support requires some sort of iterators. Could be done through co-
routines like Python's generators or something more like C++ iterators? co-
routines could be a cool addition to the language :)

I think you'd want to try and "contain" the change as much as possible. More
syntactic sugar than fundamental changes and as little change as possible to
the built-in types. Otherwise you end up with a completely new language.

~~~
kjksf
And how many decades did it take for C++ to support std::array?

What people seem to ignore is that templates are a bottomless pit of
complexity.

Swift is on version 3 and they still didn't finalize the semantics of
generics. Not due to lack of trying. Swift was trying to design and implement
generics from day 1 and they still haven't.

That's how complicated designing generics is.

I prefer stable language with fast compiler to Swift/Rust situation, with
language taking years to mature and very slow compiler. Judging by popularity
of Go, so do many other people.

While the languages probably will mature after couple more years, I don't see
much hope for making the compiler as fast as Go, regardless of the effort. See
C++ compilers.

~~~
pjmlp
> And how many decades did it take for C++ to support std::array?

About one decade.

In the mid-90's already most C++ compilers had their own version of
std::array, for example Borland compilers.

------
xyzzy_plugh
Previous article in series: [https://dave.cheney.net/2017/06/15/simplicity-
debt](https://dave.cheney.net/2017/06/15/simplicity-debt)

And corresponding discussion:
[https://news.ycombinator.com/item?id=14560471](https://news.ycombinator.com/item?id=14560471)

~~~
m_sahaf
Actually, the "Go and Simplicity Debt Redux" is not the same as the one you
referenced. The current post is part 2.

~~~
xyzzy_plugh
That was what I was going for :) I'll clarify.

------
ori_b
This all sounds rather awful to me, honestly. If this is the path that Go
takes, I'd rather not have generics added.

Which is a shame. I like generics, and I implemented them in Myrddin
([https://myrlang.org](https://myrlang.org)) for a reason.

------
vanderZwan
> _Right now, to understand how io.Reader works you need to know how slices
> work, how interfaces work, and know how nil works. If the_ if err != nil {
> return err } _idiom was replaced by an option type or maybe monad, then
> everyone who wanted to do basic things like read input or write output would
> have to understand how option types or maybe monads work in addition to
> discussion of what templated types are, and how they are implemented in Go._

I can't really follow the argument here: how would using option types replace
the existing idiom? Instead of err being a pointer, err is an option type and
the only thing that changes is that instead of writing _if err != nill_ , you
write _if err_. And if comparing nill vs option types works in isolation, then
I also don't see where the jump to _" in addition to what templated types
are"_ comes from, or the connection to slices and maps.

In fact, if Go had option types, would it still need nill at all? By which I
mean, on the user-side of things (under the hood I suppose null will be used
for implementing an option). Because if it doesn't, and I strongly suspect one
doesn't in a language without pointer arithmetic (which Go happens to be),
then we can simply replace every nil with an option.

The discussion on how templates interact with existing features and complicate
things is fairly clear, but the option types are treated as a lot more complex
than they really are. In practice it is a compiler-enforced if-not-null check
on pointers, because nill is a type instead of a value. That's about it. That
level of compile-time enforcing also seems to be perfectly in line with the
safety the Go authors want their compiler to enforce, given how strict it is
with errors.

You can easily use them without really understanding what a monad is - I'm
living proof of that. The only extra thing you somewhat need to understand is
sum types, which are a lot simpler. Then again, Go doesn't have them and their
usage overlaps with interfaces - how to deal with _that_ might be a more
useful discussion regarding option types in Go.

------
hota_mazi
> A powerful motivation for adding generic types to Go is to enable
> programmers to adopt a monadic error handling pattern.

That has never been my impression. The main drive for generics is safer code,
period. It doesn't necessarily translate to a different way of handling errors
(Java and Kotlin mostly use exceptions despite supporting parametric
polymorphism).

Rejecting generics because they would force a switch to monadic error handling
is completely misunderstanding the value of generics, a sentiment that seems
to be widespread in the Go community.

------
comex
Rather than extending number-of-return-values overloading to user-defined
methods, I’d say just deprecate it altogether. It might have made sense in a
world where generic functions are magic and weird and so you want to minimize
the number of them (but on the other hand don’t necessarily mind making them
even more magic). But in a Go with generics, operations like “access value
that may not be present” can just be regular named methods. That applies to
both user-defined collections and the builtin ones. Would help pay down the
debt...

~~~
YZF
Aren't these just two different operator[] ?

    
    
        value, exists := usertype["something"]
        value := usertype["something"]
    
        func (t *usertype) operator[](k KeyType) (value)
    
        func (t *usertype) operator[](k KeyType) (value, bool)
    

Should work, no?

(EDIT: usertype would be the templated type, so <userType> or whatever the
syntax ends up for that. so the above would be the instantiation of the
template for a particular type)

~~~
TheDong
If you have sum types, multiple return is less useful.

if you have proper tuples and destructuring assignment, you can entirely
remove that abomination.

Right now, it's impossible for a user to define a method that can return 1 or
2 values depending on how it's called.

Only stdlib types, like map and channel, get to do that.

It's fuggin horrific, and spreading that further to user defined functions
would be a mistake.

~~~
YZF
My point is that you don't really know if the stdlib types actually have one
method that returns 1 or 2 value or two methods one returning 1 value and the
other returning 2 where the compiler chooses based on the call site which one
to call. Just because [] looks like one method to the user doesn't mean it has
to be one method.

Having two different functions is a way of dealing with this while minimizing
change as it's just syntactic sugar and not a language change. Variable length
tuple types are a much bigger change. They'd also presumably be a lot more
expensive. Ideally a map type implemented by the user should be as performant
as the built-in map type.

~~~
comex
Tuples wouldn't be more expensive if implemented the way most languages do,
where a tuple type has a fixed length and series of element types, e.g. '(Foo,
error)'. In other words, structs with some syntax sugar.

But they're the wrong abstraction for this anyway. A better choice would be a
type like maybe<Foo> (as mentioned in the post), that only lets you get at the
contained Foo if it exists, rather than the current practice of returning a
fake default Foo value in the case where it doesn't.

Having two overloaded functions would work, but it would increase the
complexity of the language compared to dropping the overloading feature. Yes,
for consistency you'd want to make that change to the builtin types as well,
and that would be churn. But only in code that's already being churned: if
generic collections are added (and maybe some immutability stuff), I think
you'd want to inspect just about any code that uses slices or maps to see if a
new collection type might work better or better follow the new idioms. (Mind
you, I don't suggest actually breaking backwards compatibility, just leaving
some of the existing stuff in a permanent supported-but-deprecated state. So
"change everything that uses slices or maps" isn't as bad as it sounds - it'd
be a recommendation, not a requirement.)

~~~
YZF
maybe<Foo> or it's C++ equivalent optional<Foo> is more expensive. You now
either have a bool + a value, or a pointer that can be nullptr. So you're
consuming more memory at the very least and if you want to panic on accessing
a value that doesn't exist that means an extra comparison as well. Granted
there are situations the compiler can optimize this away in C++. I also find
that starting to use optional in C++ code leads to it being used everywhere
and for everything which introduces new run-time failure modes that can't be
detected on compile time and in general IMO messes up the code.

An alternative is to split find and access into two separate operations like
C++ or to provide "in" like Python. Is there any language where a map/set
access returns an optional/maybe?

FWIW I agree the current solution is clunky. It's clunkiness was evident prior
to the template/generics question :)

~~~
comex
I was envisioning that the maybe-returning version would be a separate method,
equivalent to the current two-return-value variant, which of course already
has that overhead.

Having the default return a 'maybe' would be a possibility; I'm pretty sure
the practical performance difference would be completely negligible, given all
the other stuff a map lookup has to do, but it might have worse ergonomics. In
that case you'd probably want a builtin optional-unwrapping operator like some
languages have, so it's not too verbose if you expect the element to be there.

> Is there any language where a map/set access returns an optional/maybe?

Swift is one. It has ! as an unwrap operator, along with other syntax sugar
for optionals, so it's not verbose:

    
    
          5> let q = ["a": "b"]
        [snip]
          6> q["a"]
        $R2: String? = "b"
          7> q["x"]
        $R3: String? = nil
          8> q["a"]!
        $R4: String = "b"
          9> q["x"]!
        fatal error: unexpectedly found nil while unwrapping an Optional value

------
DelightOne
Go doesn‘t want to decide on generics, Swift doesn‘t want to decide on multi-
threading.

Put them together in a room, maybe they can help each other.

------
coldtea
> _Right now, to understand how io.Reader works you need to know how slices
> work, how interfaces work, and know how nil works. If the if err != nil {
> return err } idiom was replaced by an option type or maybe monad, then
> everyone who wanted to do basic things like read input or write output would
> have to understand how option types or maybe monads work in addition to
> discussion of what templated types are, and how they are implemented in Go._

God forbid we'd all have to learn such a simple concept (in a language with
several non-orthogonal concepts and special cases and all the channels
coordination nuance that lack of generics prevents of abstracting in a
standard library to boot).

> _Obviously it’s not impossible to learn, but it is more complex than what we
> have today._

Well, adding a feature we'll always be more complex than not adding it. In the
end, simple assembly style instructions are the simplest both for
implementation and for learning fewer concepts as a developer. But
abstractions make things easier to write and read _after_ one has learned
their concepts. And it's 2017, even "lowly" JS developers understand such
functional concepts today...

> _What began as the simple request to create the ability to write a templated
> maybe or option type has ballooned into a set of question that would affect
> every single Go package ever written._

This is more similar to how some molehills grow into mountains. People have
been asking for generics as a standalone feature first and foremost --
orthogonal from replacing error handling. Now suddenly we can't discuss adding
Generics without taking into consideration the big burden of changing the
error handling too?

> _On the surface this sounds like a grand idea, especially as these types are
> leaking into the standard library anyway. But that leaves the question of
> what to do with the built in slice and map types. Should slices and maps co-
> exist with user defined collections, or should they be removed in favour of
> defining everything as a generic type?_

Whether the answer, the mere addition of the ability to have standard user
defined generic collections and collection libraries sounds awesome in itself,
regardless of what would happen to the ill-thought maps and slices that
specialcased genericity in the backend.

This doesn't seem like a discussion of how to best add generics, but more like
analysis paralysis.

> _David Symonds argued years ago that there would be no benefit in adding
> generics to Go if they were not used heavily in the stdlib. The question,
> and concern, I have is; would the result be more complex than what we have
> today with our quaint built in slice, map, and error types?_

A better question is: would some more complexity hurt us, especially beginning
with in an overly simplistic language, and even more so, when said complexity
actually unifies and harmonizes ugly special cases.

~~~
andmarios
>People have been asking for generics as a standalone feature first and
foremost -- orthogonal from replacing error handling.

There are many languages out there that have generics. Go is build on
different principals. Just use something else, no need to enforce your ideas
on people who don't want them.

Maybe go users should start going into every language discussion they can find
and ask for generics to be removed.

~~~
pcwalton
If you can successfully argue that generics should be removed from other
languages, then you should feel free to argue that.

The reason why nobody is arguing that is that it's hard to argue successfully,
because simple ML-style generics are incredibly useful and have negligible if
any drawbacks in virtually every statically typed language.

~~~
andmarios
I would participate in an argument only for a language that I am invested in.
When it comes to go, most people who argue about generics, do not write go and
probably wouldn't touch it even if it had generics.

Also, an important issue with such arguments, is that generics is a feature,
so obviously it is an easy side to choose for an argument. You can either have
or don't have a feature. People will instinctively select the have option
because common sense dictates that it is better to have something even if you
don't use it. People want to quantify an argument and having is better than
not having.

People don't want to argue about language design —there are few people on the
planet that could really argue about language design anyway-, they need
language MHz to compare.

~~~
danaliv
_> most people who argue about generics, do not write go and probably wouldn't
touch it even if it had generics._

I wrote Go at work for two years and the lack of generics made me want to
scream on a near-hourly basis.

~~~
grey-area
Can you give some examples of areas where this lack really annoyed you, given
your experience with other languages?

Was generics your only problem with Go?

~~~
littlestymaar
Not the gp, but I think my own experience may also be relevant.

Since the birth of Go it's been impossible for many years to create something
like a thread-safe reusable hashmap[1], which is pretty annoying when all your
code is multi-threaded in goroutines. And creating such hashmap was impossible
_due to the lack of generics_. If you wanted to build your own concurrent
hashmap, you had basically 3 bad options :

\- you use `interface{}`, which is bad because of the dynamics dispatch
(slow)[2] and it's also not type-safe.

\- you use code generation, which is bad because I haven't left the JavaScript
world to start using babel for Go …

\- you create a specialized hashmap for every type you need, this basically
means copy-pasting the same piece of code ever and ever and changing the type
signatures, which is error prone and a hassle to maintain.

[1] this is eventually gonna get better in the next release, since a
concurrent hashmap will be included in `sync`.

[2] an illustration of the performance problem with dynamic dispatch can be
found here :
[https://github.com/nieksand/sortgenerics](https://github.com/nieksand/sortgenerics)

~~~
grey-area
I too feel like containers are the main area where it is painful. The sort of
built-in generics we have in Go are useful in most circumstances, but it'd be
nice to have custom containers which could take any type, instead of having to
specify them separately. Errors is the other area where it feels like they
could help (though the article points out possible problems with this).

That said your third approach (make custom concrete types as required) has not
been a huge problem for me in practice, and is probably the best choice for
now.

------
taeric
I certainly appreciate the desire to avoid two main ways of doing things.

Curious about the race conditions being common. Is this from go encouraging go
routines? Seems most programs don't need or use multiple threads for a single
task.

------
kmicklas
Basically all these articles about Go make it sound like it was designed to be
some kind of teaching language with a low conceptual barrier to entry, except
Google intends it to be used in real settings, which is the problem.

~~~
computerex
What articles are you talking about? This couldn't be further from the truth.
Go is a practical language meant to be used in production settings. It's not a
haskell. For example, it's support for protocol buffers is second to none imo.

~~~
mhh__
Haskell is probably the worst teaching language I can think of (Of the set of
languages that are considered good/production worthy)

~~~
leshow
Uh, what? Learning Haskell covers a huge surface area, many things you learn
are applicable to tons of other languages. How does that make it a poor
teaching language?

~~~
mhh__
I'm interpreting "Teaching language" to be the first language students learn.
How can one explain the need for monadic IO properly without explaining how
regular printf and the like work. Also, Haskell is quite scary to the beginner
(Although I personally find it much simpler/more consistent than the Java and
C++)

If it's (say) the third language they learn, then Haskell is absolutely
perfect for the exact reason/s you have stated. Personally, I actually learnt
Haskell by accident (Read SPJ's book about lazy functional language
implementation, realized SPJ started Haskell then learnt more from there).

~~~
wtetzner
> Also, Haskell is quite scary to the beginner (Although I personally find it
> much simpler/more consistent than the Java and C++)

Is it? Or is it just harder for people who have already learned a mainstream
programming language?

