
Go and Simplicity Debt - ingve
https://dave.cheney.net/2017/06/15/simplicity-debt
======
jasode
_> But there is equally no question that adding these features to Go would
make it more complex. [...] I have no doubt that adding templated types to Go
will make it a more complicated language, _

I don't know if adding generics to Golang is the right thing to do but I don't
agree with how Cheney has framed it. I wish programmers would use a more
precise mental framework about "simplicity/complexity".

It's not important if generics make the _" language specification"_ more
complex. What's really more important is if it makes the real-world _language
usage_ more complex.

What matters is the sum of _total complexity_ : the language spec + adhoc
idioms/patterns/conventions/workarounds used in actual codebases.

As an example, if you make a super simple language like Javascript in 1995
with hardly any features, what you eventually end up with is _complexity in
npm_ with dozens of modules for things like leftpad(). Or you can insist that
"prototypes are simpler than OO" but you actually end up with a dozen OO-
simulation libraries with different syntax out in the wild. Instead of the
overall JS landscape being simpler, it is more complex.

This does not mean you must put everything language feature including the
kitchen sink into the language. The lesson learned from history is to talk
about about simplicity/complexity in a _more holistic way_.

~~~
stcredzero
_What matters the sum of total complexity: the language spec + adhoc idioms
/patterns/conventions/workarounds used in actual codebases._

This is a great one sentence summary of the history of C++. (Would also do for
a history of programming.) Any hoary Ruby veteran probably has a story about
the misuse of missing_method. (Likely one in which they were the antagonist.)
The way that the programming field has shaken out, it turns out that there is
a group of programmers who are fine with application programming and using
well thought out libraries. However, only a minority of these programmers have
the ability to engage in the thinking about other-POV and 2nd/3rd order
effects that are needed to make a good library or meta-level tool. (Not to
mention the community management skills needed to keep people happy while not
giving them absolutely everything they want.) As a relative noob, I was
listening to hoary Smalltalk veterans talking about this in the early 2000's.
This situation is pretty analogous to the one surrounding crypto, except that
the consequences of hubris are somewhat milder.

 _The lesson learned from history is to talk about about simplicity
/complexity in a more holistic way._

Also from the early 2000's -- The _Design Patterns_ guys were often saying
that design patterns and other kinds of common incantations were often
symptoms of deficiencies in programming language design.

~~~
buzzybee
I've been musing about these sorts of issues of "what is really complex"
lately, too. I started evaluating new languages for low-level application code
again after spending a lot of time overly close to web browsers, and then
ended up deciding that Free Pascal beat them on the stuff I cared about,
relative to C compilers as a baseline:

* Best-in-class compile times

* Modules; preprocessor is relevant, but not dominant

* Generics, variant records, function and operator overloads, exceptions, safe string and dynamic array types

* Libraries for common things, with a reasonably conventional style: APIs, containers, bindings, etc.

* No GC; pointers are available, but language features discourage casual usage. RAII pattern is achievable.

* Strong + static types, nominal types and type aliasing.

* Good IDE support, mature tooling, compiler has very broad platform support.

The new languages do win on a lot of the language-feature aspects(particularly
memory safety when performing deep optimizations), but they don't have as good
a story on the tooling. The compile speed is a major feature and is only
likely to remain advantageous over time - and the features already in place
are quite effective at limiting source code size, boilerplate, and dangerous
constructs. This despite "Pascal is verbose" being memetic.

And if I want more high-level power, I've found that writing a FSM is the most
probable path to achieving it. A FSM that embraces the problem domain is a
tiny compiler: input source constructs, FSM checks them, applies appropriate
algorithms, then emits a list of actions. More power? Add a second FSM to make
it a two-pass compiler. Add a stack, or attach it to a database. If it has to
be fast, it can generate code in the last step, too. Those things add tons of
power to model the problem and aren't heavily dependent on language-side
support.

~~~
pjmlp
Yup, we already had all of that in Turbo Pascal for MS-DOS, hence why I never
saw any value in C back in 1993.

On those days outside UNIX, C was just yet another systems language, with most
home micros being written mostly in Assembly.

Also Turbo Pascal OOP features were adopted from Apple's Object Pascal, used
to write the initial versions of Lisa, Mac OS and known software like
Photoshop.

------
nicpottier
I know Go gets a lot of heat for not having generics, but having done a few
fairly complex things in Go now, I really haven't missed them, nor have I had
to bail out to interface{} in ways that I wouldn't have had to in other
languages.

The main reason for that is that golang lists and maps CAN be typed and while
they aren't perfect, they are pretty darn great for most use cases. Certainly
I understand some people have the need for specialized data structures, but I
wonder how often a project needs that data structure to be totally generic, as
opposed to just doing the work to implement it (yes even perhaps copy/pasting
some other implementation) and moving on. That's ugly yes, but it just isn't
that big a deal in my experience.

In any case, the gains from having a great runtime that compiles to a binary,
an excellent standard library and awesome concurrency primitives far outweigh
the negatives for me. Really enjoy Go.

~~~
xyzzy_plugh
I find Go is quite similar to C, and C also lacks all of these features. It's
rare to see C code using some library for generic containers. Most developers
roll their own, as needed.

In practice, bugs and safety aside, this tends to work swimmingly. Look at
Redis, or git, or postgresql, or the many other projects getting by just fine
without generics.

While I agree, and believe generics offer unparalleled power, they often
aren't remotely necessary. If anything, their presence creates a tendency to
over-use them, resulting in even greater complexity.

~~~
pcwalton
No, it doesn't work swimmingly. All of those projects have ended up either
using void∗ or macros, or both, in order to roll their own _generic_ data
structures.

> Look at Redis,

[https://github.com/antirez/redis/blob/unstable/src/adlist.h](https://github.com/antirez/redis/blob/unstable/src/adlist.h)
— generic doubly-linked list with void∗

[https://github.com/antirez/redis/blob/unstable/src/dict.h](https://github.com/antirez/redis/blob/unstable/src/dict.h)
— generic hash table with void∗

> or git

[https://github.com/git/git/blob/master/hashmap.h](https://github.com/git/git/blob/master/hashmap.h)
— generic hash table with void∗

[https://github.com/git/git/blob/master/list.h](https://github.com/git/git/blob/master/list.h)
— generic doubly linked list implemented with macros

> or postgresql

[https://github.com/postgres/postgres/blob/master/src/include...](https://github.com/postgres/postgres/blob/master/src/include/lib/ilist.h)
— generic doubly linked list implemented with macros

[https://github.com/postgres/postgres/blob/master/src/include...](https://github.com/postgres/postgres/blob/master/src/include/lib/rbtree.h)
— generic red-black tree with void∗

~~~
KirinDave
I agree strongly.

"It works" is an encoding of, "I can write tests which perform correctly" and
not "My code is stable in the face of aggressive input."

Go definitely follows along with C in this tradition. It's better than C,
because you have slightly more safety interpreting memory as executable and in
bounds checking. However, it does very little to stop destructive bugs or let
programmers escape copy-paste code hell where regressions keep resurfacing
because it's most expedient to copy-paste code.

------
frou_dh
After seeing Rich Hickey's excellent material on the matter^, I can no longer
read anything talking about Simplicity and Complexity in programming without
suspecting the author of being fast and loose with what those terms
specifically mean.

As it stands, they are recipe for different camps and sub-camps of programmers
to talk past each other endlessly.

^ [https://www.infoq.com/presentations/Simple-Made-
Easy](https://www.infoq.com/presentations/Simple-Made-Easy)

~~~
catnaroek
Simplicity is very easy to objectively measure. Write down a formal semantics
for the programming language in question, and count how many pages you used.

But, of course, nobody will actually do this, because it would expose the
inherent complexity of designs advertised as simple. Many people's feelings
would be hurt in the process.

~~~
MereInterest
I would argue that is a good measure of the simplicity of the language itself,
but not a measure of the simplicity of the use of the language. By that
measure, Malbolge is a simple language than C++ by a factor of ~1000. However,
it is still much simpler to write code in C++ than in Malbolge.

~~~
catnaroek
I said absolutely nothing about _ease_ of use.

~~~
codygman
For many simplicity is defined by ease of use though.

~~~
catnaroek
Ease of use is subjective. It depends on people's goals, skills and even
tastes.

------
benhoyt
I agree with the sentiments of this post (you can't "just" add generics; there
are serious trade-offs), but I think his assertion that Python is "on the
wrong side of history" is untrue. More people are using Python now than ever
before, in more and more problem domains. It's great for certain things, just
not very good at concurrency (alleviated by async) and particularly
parallelism (though in practice you can often work around it with C extension
modules and multiprocessing). But to say Python has "missed the boat" and is
"on the wrong side of history" is not true.

~~~
pdimitar
Anecdotal evidence from the Pythonistas I've known in my career -- they stick
to it because they like it, it's old (to them that means it's a proven tech)
and because there's too much intertia in their organization to move on to
something else.

Technologies aren't only popular because of their quality, or shall we bring
PHP once again to prove this point?

~~~
kodablah
Yup, this is what I hate about all of the we-must-be-right-look-at-our-
adoption-rate arguments. From the blog post:

> There is a reason Go programmers choose to program in Go, and I believe that
> reason stems from our core tenets of simplicity and readability

Not the reason I choose it, I choose it for tight cross platform binaries and
nice stdlib. Arguments made against generics should be technical, not
political... complexity may be subjective, but the justification for the
definition can still be technical.

~~~
pdimitar
As I mentioned in my more lengthy comment right here --
[https://news.ycombinator.com/item?id=14563597](https://news.ycombinator.com/item?id=14563597)
\-- popularity is very loosely correlated to the quality of the popular thing.
There's ease to pick up, there are pressing business needs fixed by a creative
junior Python dev, there's the businessmen loving quick solutions (and future
be damned of course), there's inertia, there's skepticism ("but our current
system is working! why switch?"), there are tens of other such phenomena that
can be attributed to the very buggy automatic brain processes.

So yeah, I hate the _we-must-be-right-look-at-our-adoption-rate_ arguments as
well. :)

------
rwj
The facet that is missing is the impact on users' code complexity. It is all
well and good if Go is simple, but if my code is horrendously complex to
compensate, it isn't a net win. I exaggerate, but a lot of the complaints for
Go already stem from this trade-off.

Focusing entirely on the language itself is a failure. Keep on eye on the
language's users,

------
jhpriestley
No one knew about generics until languages like Pony and Crystal proved that
they were important? This may be a pedantic point, but technically, generics
or templated types were already being used in some obscure, academic languages
like Java, C#, and C++.

~~~
cratermoon
As I read it, the author wasn't discounting those languages or saying nobody
had heard of generics, he was saying that newer languages demonstrate that
generics are now expected. Remember, both c++ and Java got generics added on
(and c# followed Java's lead).

The introduction of languages that have generics designed in from day one is
key. Even though Go doesn't (yet?) have them, the fact that its designers
wrestled with the question from the start is an indicator that the concept has
become central to programming language design.

~~~
pjmlp
You got your history wrong.

C++ got templates during the time it was being designed.

Between C++ ARM and ANSI C++98, C++ compilers experimented with them, and its
design reflected on the standard work.

Borland was one of the first PC compiler vendors to support templates, still
on their MS-DOS/Windows 3.1 compilers, with the release of Borland C++ 3.0.

Microsoft Research already had a .NET prototype with generics support, by the
time v1.0 was about to get released, but they decided it was more relevant to
ship v1.0 than waiting for the generics support to be fully done. Don Syme
from the F# team has a few blog entries about this.

One of the first languages to get generics was CLU in 1975, shortly followed
by ML and Ada in the early 80's.

~~~
rst
C++ did not have templates when the first commercial compilers shipped and the
first books were published. It got templates around 1990, five years after the
first release of a commercial C++ compiler and the first publication of "The
C++ Programming Language".

ref: [http://www.stroustrup.com/hopl-almost-
final.pdf](http://www.stroustrup.com/hopl-almost-final.pdf)

~~~
pjmlp
C++ was only set in stone in 1998 with the release of ANSI C++98.

Until then, it was a moving target, with C++ compilers catching up with CFront
and ongoing work at ANSI/ISO meetings.

------
ridiculous_fish
Google cache link:
[https://webcache.googleusercontent.com/search?q=cache:__B53p...](https://webcache.googleusercontent.com/search?q=cache:__B53pKdoxMJ:https://dave.cheney.net/2017/06/15/simplicity-
debt+)

------
josteink
Go: the only place in the known universe where you can find generics described
as complex.

Even though modern VB-programmers manage to deal with them just fine.

So pardon my French, but what's the fucking issue?

    
    
        var typesafeListOfInts = new List<int>();
    

How is that hard?

~~~
majewsky
The point is not List<int>. The point is that the primary intention of Go is
to push developers to write the most readable (and hence, maintainable) code
possible. No one argues that List<int> will not make the code more readable
(and in fact, Go has some very limited generics for lists, maps and channels
for these usecases). The problem is that once you give people parametrized
types, they're going to parametrize _the shit out of them_ , leading to
byzantine messes such as the C++ standard library or Boost.

And that's not saying that something like Boost isn't an accomplishment in
itself. But it's beyond hard to read, and when your language allows to write
code like that, it's bound to appear in your codebase as well, and make
maintaining it much harder than it needs to be.

~~~
josteink
> The point is that the primary intention of Go is to push developers to write
> the most readable (and hence, maintainable) code possible.

Lack of parameterized types leads directly to code-duplication. How is
duplicate code more maintainable?

> The problem is that once you give people parametrized types, they're going
> to parametrize the shit out of them, leading to byzantine messes such as the
> C++ standard library or Boost.

An extraordinary claim with no backing what so ever.

Just because C++ turned out terrible doesn't mean everything else has to. Look
to C# and Java. They're doing _just fine_.

Also: how is littering your code with "interface { }" any less Byzantine than
simply using "T"?

------
gtrubetskoy
I'm glad the article begins with mentioning the GIL, the elephant in the room
of Python and Ruby. I often wonder why it isn't getting the attention I
believe it deserves - either the problem is just too hard to solve and has
been written off as not worthy of addressing, or perhaps developers do not
understand how much (unnecessary) complexity goes into deploying, running and
monitoring a production Python/Ruby app only to work around the GIL
limitation. I devoted a section of my recent blog post to the GIL problem here
[1] if you're interested in why I believe it is such a problem and how cool it
is that Go does not have it. It's not the only thing I like about Go, but it
is one of the most important ones in my opinion.

[1] [https://grisha.org/blog/2017/04/27/simplistic-go-web-
app/](https://grisha.org/blog/2017/04/27/simplistic-go-web-app/)

~~~
baq
It can be argued that the GIL doesn't matter, because Python is too slow if
you have a performance sensitive problem anyway. I'll give numpy as an
example.

------
coldtea
> _By the time this decade rolled around, Node.js and Go had arrived on the
> scene, highlighting the need for concurrency as a first class concept.
> Various async contortions papered over the single threaded cracks of Python
> programs, but it was too late. Other languages had shown that concurrency
> must be a built-in facility, and Python had missed the boat._

While concurrency is nice, some citation is needed here. For one, Python is
not going anywhere, and remains spectacularly more popular than Golang.

> _But, no matter how important and useful templated types and immutability
> would be, integrating them into a hypothetical Go 2 would decrease its
> readability and increase compilation times—two things which Go was designed
> to address._

Well, they sacrificed compilation times to get a native compiler (which has no
real utility to the users of the language, on the contrary it makes
bootstrapping a little more messy).

Sacrificing it for Generics seems like a no brainer, but it seems they're not
yet ready to bite the bullet.

As for "sacrificing readability" I don't think that having 30 lines of code
just because you need to make it work with different types is any easier than
having 10 lines of the same code with parametric types. Not to mention NOT
having to write any code at all, because the standard library can finally have
code (e.g for math, collections, etc.) that works across all relevant types.
With Generics people could just reuse a few generic implementations of channel
uses to handle 90% of their concurrency needs.

And it's also much better and easier to parse than using dreadful string
templating kludges or interface{} to get the same behaviour.

I mean, the author mentions all those languages with generics ("Rust, Nim,
Pony, Crystal, and Swift showed that basic templated types are a useful, and
increasingly, expected feature of any language—just like concurrency"). Does
he see the users of those languages complaining about generics being a burden?

Why do people always have to lament the terrible burden Generics will bring to
Go in advance, when millions of us (literally, just add Java and C#
programmers) just use generics in other languages without thinking twice about
it?

There's a proverb in my country, that "He who'd rather not knead the dough (to
make bread), keeps sifting the flour for days".

In programming terms it would translate roughly to: "He who is too lazy to
build a bikeshed or doesn't like the prospect of building one, will keep on
discussing the color it should give it".

Which I think is the case whenever anybody from the core Go team discusses
Generics.

> _As it stands now, generics or immutability can’t just be added to Go and
> still call it simple._

It's 2017. Neither are particularly difficult to grasp for a modern
programmer. Even enterprise Java programmers have been using the former for a
decade already, and they're not known for the wild experimental nature...

~~~
PaulRobinson
> For one, Python remains immensely more popular than Golang.

And it's worth looking at the reasons for that. Firstly, age is a factor -
Python is much older.

However, most Python growth has come from data science and machine learning
fields. There is no reason Go can not grow the pie and make such fields more
accessible to those with an interest in computing concurrently. And surely, if
there is one field where concurrency might be a boom it might be data science
and machine learning.

Now, Go has some problems in that area: principally it does not have data
structures as "friendly" as Python lists or NumPy arrays, and therefore people
will have to jump through hoops. Could it get them? Yes. Would that be more
powerful than generics? Absolutely. Will it get them? It's not obvious to me
it will, other than through a third-party library which is fine.

In fact Python's successes might not be Python's. They might be NumPy's and
Pandas' successes. Go can have the same thing, and do it better.

~~~
Daishiman
I think you underestimate just _how_ important is Python's support for various
numeric data types is for NumPy's, success. There is no possible equivalent in
Go that would not look like a horrendous mishmash of various different
functions to declare all the different numeric types and various forms of
matrix slicing and traversal operators.

~~~
PaulRobinson
Go does not support multi-dimensional slices, and that is a problem.

I am not sure the numeric data types thing is such an issue.

Python supports int, float, long and complex types. So does Go, with the
slight annoyance you have to care a little more about the size of your number,
and so you don't just label something 'long' and walk away safe in the
knowledge it's kinda big. Numpy just makes Go's numeric data types available.
The clever thing it does that Go does not is around arrays and operations on
them - something we need to consider in Go land, carefully.

The other stuff Numpy and Pandas makes available could be made available in
Go, too and could abstract away the horrible stuff under the hood where you're
resizing slices and creating multidimensional versions.

I think the moment multi-dimensional slices are available, you can do
everything you want, and with concurrency and therefore able to trivially
exploit multi-core systems - that could be the killer feature. Maybe.

------
pjmlp
The good part of Go is having a programming language, which DevOps are
adopting in place of C, allowing for safer userspace applications.

The sad part is Google supporting Java, Kotlin, TypeScript, Dart and C++
progress o one side, all with lots of people knowledgeable in language design
and then there is Go.

------
kbutler
While the author's main idea is good (it can be more complex to work around
missing features than to have them designed in from the beginning), starting
from an invalid premise really weakens the article.

Node.js illustrates the need for concurrency, but is even more single-threaded
than Python.

Somehow, in spite of the GIL, Python has continued to increase in popularity,
probably because the workarounds are pretty easy (use multiple processes or c
extensions that give up the GIL or jython or...).

Threads are not the only route to parallel execution, just as templates are
not the only route to generic data structures.

------
YZF
Wrt/ to generics I think one interesting exercise would be to look at list and
see what would need to happen such that it behaves more like the built in map.
You'd want to be able to declare:

\- list[type]

\- delete should work.

\- len and cap should work. (rather than .Len() )...

\- range should work.

\- What do we do for iterators?

I think it would make code that uses lists more readable/grokable. So you do
get something in return for the debt you possibly took on to be able to
implement list. In a sense that's also the way C++ trades things off, the
library code is more difficult but the code using the library is
simpler/safer.

~~~
majewsky
I don't understand the question. All of these operations (except for delete)
work on []T and have always worked on it.

~~~
YZF
[] is a slice... not a list... I'm talking about
[https://golang.org/pkg/container/list/](https://golang.org/pkg/container/list/)

------
deckarep
This is all about trade-offs right? Choose to have a simpler language when it
comes to concurrency and multi-threaded programming and the outcome is a
harder time troubleshooting at runtime.

Or, make the compiler more powerful and annotate the code in such a way that
the compiler does way more upfront on your behalf...now this means you don't
have to work so hard to troubleshoot code in production.

Personally, I'd rather work harder up front--even if it means a higher
learning curve.

------
ShabbosGoy
I'm finding it difficult to accept the premise that generics and immutability
are needed in Go.

In my case,

    
    
      var foo map[<type>]interface{}
    

solves 99% of issues where I would normally need generics.

The author also doesn't really specify what kind of generics, there's a huge
difference between parametric polymorphism (i.e. Haskell type classes) and
Java/C++/C# generics.

The sheer rapid pace of development and simplicity has made Go our primary
stack.

~~~
codygman
> var foo map[<type>]interface{}

> solves 99% of issues where I would normally need generics.

But it also opens you to to runtime conversion errors, spurious if checks for
type information compilers with generics have, or slow reflection to find the
type.

> The author also doesn't really specify what kind of generics, there's a huge
> difference between parametric polymorphism (i.e. Haskell type classes) and
> Java/C++/C# generics.

Java/C++/C#/Haskell generics all provide parametric polymorphism though.

Haskell type classes provide ad hoc polymorphism, though I guess maybe they
provide parametric polymorphism as well.

Per Wikipedia:

> In programming languages and type theory, parametric polymorphism is a way
> to make a language more expressive, while still maintaining full static
> type-safety. Using parametric polymorphism, a function or a data type can be
> written generically so that it can handle values identically without
> depending on their type.

------
grabcocque
I don't think Cheney properly understands the concept of "simplicity" fully. A
language is not simple because it meets some arbitrary measure of smallness.

A language is simple if it decomplects unrelated concerns. Which go does
admirably in certain areas, but its absence of generics certainly adds to
cognitive overhead and unnecessary complexity elsewhere.

