Hacker News new | past | comments | ask | show | jobs | submit login
Go Chainable: .map().filter().reduce() in Go (github.com/neurocollective)
50 points by davidashe on March 7, 2022 | hide | past | favorite | 58 comments



Hmm. Implemented on a custom type that wraps []T, so you have to create a List to get the methods, meaning more boilerplate (a common theme in Go); eager, like JavaScript’s Array methods rather than like the iterator methods in Rust or most/all functional programming languages; and since methods can’t have generics (which really surprised me in the Go generics proposal), Map() can only work on the same type (T → T, rather than T → U), and Reduce()’s output type is a generic parameter on the list, so you can only ever reduce to one type (unless you deconstruct and reconstruct the List) which must be specified at instantiation time.

As one who works mostly in Rust and JavaScript and is passingly familiar with Go (I used it for a couple of projects eight and nine years ago), these seem some pretty severe limitations.

Rust’s trait-based iterator system is delightful, so that you can map, filter, reduce, &c. on any iterator, lazily, and even define an extension trait to define your own methods on any iterator, thereafter accessible by just importing that trait.

In the end, I think the current scope of generics and interfaces won’t be enough to produce any feeling other than “shoehorned” for this functional style in Go. It’s just not a style that works well in all types of programming languages.


I don't love libraries like this and feel like they work against the idiom in Go; I'm skeptical that things like this will be part of the idiom going forward. But Rust isn't all wine and roses with this stuff either; obviously, it's a much more powerful generics system, but closures are much more annoying to work with than they are in Go, where everything just magically escapes to the heap when you need it to.


I used to hate the closure from rust but I learned to love it. With golang its a PITA to track down performance issues die to heap escape where it is explicit in rust.


Right, go is just not designed for this kind of functional style, and it will always require a lot of boilerplate and fighting the language to try to write code like this.

I predict that production go code will continue to use for loops for this kind of thing, even once generics are widely established.


I agree, the current implementation is really not sufficient, at least for this. I played around with lazy iteration a while back and found the current restrictions would make working with it an annoyance.

I've placed some thoughts in the Readme [1] so I don't forget. Hopefully at least some of the restrictions will be dropped.

1: https://github.com/urandom/iter


After taking a stab at some more functional approach with generics in Go — https://github.com/gtramontina/go-extlib — I do agree that some of it does feel shoehorned. Although for some other constructs, it feels quite nice. As I mentioned on the readme of the linked repository, this post https://hypirion.com/musings/type-safe-http-servers-in-go-vi... presents pros and cons nicely.


https://github.com/AlexanderYastrebov/stream is my take on this that does not re-allocate full slice on every filter/map operation


Are there people with experience in a wide variety of languages that prefer Go?

I've only used it in passing, but everytime I see examples they're verbose and look clunky.

For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code.

Do you find Go preferable to other languages for solo projects?


>Are there people with experience in a wide variety of languages that prefer Go?

Other than Go, I have varying levels of experience in C, C#, Common Lisp, Python, (embedded) assembler, Fortran, Tcl, PHP, and maybe a few other languages I screwed around with for fun.

At first I really really liked Go, and I grew to loathe it. I still think Go probably excels in large software development shops where some of its design decisions make a little bit more sense. For a solo project, unless you're trying to get hired to write Go professionally, I would avoid it.

Some complaints in no particular order:

- An unused import is a compile error, which means every time you comment out a variable while debugging, you also have to go fuck with your imports

- SemVer is baked right into the language, but Go simply cannot handle major versions > 1.x in any sane way. The documentation[0] actually recommends that you copy-paste your entire codebase into a separate v2/ subdirectory and then maintain that.

- No function overloading

- Annoying type/interface system

- Smug, snotty community (mostly #go-nuts on freenode/libera)

That's about all off the top of my head but it was enough to turn me away from go for good. If I was writing an httpd or something similar, as part of a large team, I might choose Go. For just about any other use case I would not, because it sucks for those use cases.

[0] https://go.dev/blog/v2-go-modules


> The documentation[0] actually recommends that you copy-paste your entire codebase into a separate v2/ subdirectory and then maintain that.

That's not the documentation though. Just a blog post. The documentation [0] recommends a branch be created. Creating a directory is not even mentioned.

[0] https://go.dev/doc/modules/major-version


> No function overloading

In my experience, non-type-specific function/operator overloads ultimately end up being a pain point. They’re generally wonderful —- until they’re abused, for which there’s a tendency. That’s certainly true of C++, and likely true in other languages which support overloading to varying degrees. With generics, the potential value of overloading in Go should now almost always be (hopefully) close to zero, so I think this criticism may be less notable than it was before.

> Smug, snotty community (mostly #go-nuts on freenode/libera)

I wonder if that’s not specific to those communities rather than the Go community as a whole. My general pulse on things is that the community Go isn’t less inclusive than others, but that may be a biased viewpoint.


>I wonder if that’s not specific to those communities rather than the Go community as a whole.

Maybe. My experience with IRC communities, especially tech-focused ones, is that they tend to be pretty welcoming and friendly. For example #lisp and its sibling channels, on libera, are some of the nicest places around.

By contrast everyone I interacted with on #go-nuts seemed to have a huge chip on their shoulder and a quick snarky answer always at the ready for why Go is always better than $language in every way, no matter what.


We're using go for some projects at work and I'm not a huge fan. I don't like things like the compiler just exiting 0 when a build completes with absolutely zero output by default. Things like logging feel too complicated for what they are. Errors that I ran into tended to be cryptic and hard to resolve.

One of the main reasons we went with go for a particular project was the ease of distribution and being able to build it for all platforms. The "all platforms" part isn't actually easy at all. It's a relatively simple app but does use GTK which is a nightmare to get working correctly on Windows, and a bigger nightmare to get working with our CI system that expects builds to be dockerized. In my opinion, it's not worth it.

At least it's fast when it does run, which isn't 100% of the time since sometimes it was just exiting with no output until we set up Sentry inside the app.

I feel like it's similar to Java, where the "write once, run anywhere" thing doesn't actually apply in practice. Combine with how GTK wants root stuff handled (TL;DR you have the app that runs in the background do the actual work that requires elevated privs and run the GUI as a separate app then do IPC / expose an API) and how different platforms have different ways of running a persistent daemon + how that complicated distribution (we're at the point now where we have an MSI for InTune for Windows and a JAMF package for Mac + the binary itself just works on Linux and we can push that with Salt) and...if it had been my decision, I would have just used python or node.

As a bonus, none of the libs for accessing our $BIGCO internal services are ported to go, so for some things we can use GRPC but other stuff just sucks to integrate with. It's so so so so so much easier to just use the most-supported or at least a well-supported language within a $BIGCO for the network effect.


>I would have just used python or node

I've always stayed away from node because the ecosystem looks like such a shitshow from the outside, but the speed of V8 appeals to me.

Python is great for quick stuff that doesn't need to be performant. I use magic-wormhole and pgAdmin4 probably every day and they're great, but I shy away from Python for anything that needs threading.

One thing I will say in go's defense is that I found goroutines to be a fairly pleasant and straightforward experience (at least as far as multithreading can be straightforward and pleasant)


Sure, I’ve used a variety of other languages (C#, Java, Swift, C, even Scheme etc) and like Go, and even chose it for a solo project.

To me it feels like C with some extra features, kind of like Objective-C. If you need to do some C-like things but can afford the small latency etc overhead introduced by Go, what Go provides (GC, large standard library, green threads, package management, etc) feels like absolute luxury compared to using C. C# is another GC language with the ability to go lower level, but the syntax can also be pretty verbose.


I'm a broad enough polyglot, and I find myself writing quite a bit of go.

I don't think go is a good language. It's not enjoyable to write. It's annoying to read. It doesn't bring me happiness.

Compared to rust, it has a ton of severe flaws. It doesn't have mutex guards, so manually releasing mutexes is the norm. Every error is returned as the 'error' interface, so your compiler can't tell you what types to check for (not that there are sum types anyway), nor can the function signature, and error handling in general is a total mess as a result. It's easy to mess up ownership with a few things, especially channels. It has its share of footguns and warts. It made the million dollar null mistake. It feels like writing code while hobbled with its intentional avoidance of macros, type-level abstractions, and a number of other "features".

But, compared to rust, it has a vibrant ecosystem of fairly mature libraries. If you want to, for example, speak some slightly obscure protocol, or interact with a somewhat unusual API, you'll usually find a pure go library that some pour soul has hacked out, one line at a time. In rust, you might find a half-completed nom parser some college kid wrote started and abandoned, if that.

There are other languages with this property (of libraries already existing for anything you might want to talk to), but these other languages are javascript and python, so if you want some semblance of speed and some semblance of a static type-system, Go's a good compromise between library support and static typing.

It's because of this reason that I begrudgingly use Go, even though as a language, I find rust, (modern) C++, and haskell to all be generally more pleasant to write and read.


People who prefer golang use it _because_ it is verbose. Nothing is implicit and that makes good easy to read.


> People who prefer golang use it _because_ it is verbose.

I don't prefer Go because it is verbose, error as values without constructs to manage them is a pain in the ass.

I'd use go over language X,Y or Z because of its standard library and ease of deployment. The language itself has great things but is verbosity, mainly when it comes to dealing with error C style, its absolutely not a feature.

Now that we have generics, things will get more interesting.

> Nothing is implicit and that makes good easy to read.

I mean this is a language with garbage collection, that thing certainly is implicit.


For me, it is a feature.

Coming from languages with exceptions, where you have no idea if a function call is going to explode without praying that the library has its exceptions documented or reading the source code, having error values that you cannot (easily) ignore is a blessing. It's not as good as something like Rust's Result sum type, but it's pretty close. It makes you explicitly handle, or bubble up, every error in your program which in my experience is a huge cause of unexpected exits in other languages.


This is me. I've been programming for decades, and the simplicity of Go appeals.

Projects like OP's just tell me that OP doesn't understand Go, and why Go was designed the way it was. They're trying to make this cool thing from another language (that they understand and know how to use) work in Go. But if they really understood the philosophy behind Go they wouldn't be doing this (and their life as a Go programmer would be a lot easier).

But then, I had the same experience coding in Rails. I hated all the magic, the "if you do this then Rails will automagically do that". I want my program to do something because I have specifically written it to do that thing. No surprises, no hidden layers of abstraction. Nothing implicit. I spent a lot of time fighting the magic and trying to write simple, verbose code in Rails. It wasn't fun.


I share your opinion overall especially the rails part

but let's not assume the intentions of the OP here. It's perfectly okay for someone to try and implement collections API using generics in go just to see if they could


25 years ago....

"People who prefer Java use it _because_ it is verbose. Nothing is implicit and that makes good easy to read."

The irony.


Well yes, I was one of the haters back then because it was too verbose (for me). IMHO Go is less verbose than early Java.

It could be less verbose without loosing readability. My two top picks would be to add a ternary expression and a real while loop so that you could write better iterators.

while next := some.Next(); next != nil { // ... }

Such a while loop would be consistent with the if statement support for an assignment and following check.


There are definitely people who prefer to use Go who think the verbosity is a downside. Someone had to craft the proposal for generics after all.


I am no fan of Go's design, other than the small Oberon-2 influence it got.

However, I will definitly advocate its use against C for userspace applications.


> Are there people with experience in a wide variety of languages that prefer Go?

I used to love Go, but that my "wide variety" of language experience was mostly all dynamic languages.

My current opinion of Go in short is:

> Gives micro-level simplicity at the cost of macro-level complexity


> Are there people with experience in a wide variety of languages that prefer Go?

I have a great deal of experience (decades) with assembly, C, Lisp, ML-style derivatives (Standard ML, OCaml, Haskell), shell script. Since Go is already 12 years old now and I've been doing it since day one, I guess it also belongs in this category.

I am very fluent in Rust, Prolog, Coq, Agda, Idris, Lean, Ada, Erlang.

I even have some professional experience with C++, Java, C#, Python, Objective C, Swift, APL, R, Julia, Matlab, Fortran.

And yes, Go is my language of choice in the domains where that makes sense, like servers and command-line tools.

> For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code.

I don't know anything about JavaScript (except that I don't want anything to do with it), but the code in question is a far cry from idiomatic Go code.

> Do you find Go preferable to other languages for solo projects?

Absolutely, when that makes sense. For example, Go is a lousy language for writing a compiler in, I'd prefer to use OCaml for that. That being said, I wrote a couple of compiler targets for Go in Go. Even though ML-derivatives are a better fit for writing compilers, there's a much greater value in having the Go compiler itself written in Go.


I’m fluent in JS/TS, Python, C++ and Go. Despite that I legitimately love all these languages for different sets of reasons, Go is, by a fair margin, my go-to.

After 15 some-off years of writing code, to me, the most key component to a language is the ability to grok a new codebase written in it quickly and be able to contribute with as little fuss as possible. The potential network effects to this are huge, and Go’s level of simplicity is central to that: there’s generally only a small handful of ways to implement a solution in Go, and the built-in testing, benchmarking (and now fuzzing) are really the icing on the cake for me. That the standard library is mostly comprehensive doesn’t hurt either.

TypeScript is probably a close second for me, but there’s a fair amount of complexity involved in testing and in choosing libraries to fill certain gaps. Feels like that’ll get more and more solved over time, but with Go there are really few arguments about it.


I think this speaks to why I’ve decided to pick it up despite it lacks of offerings to something like Python. I know anywhere I go (haha) the syntax won’t be too convoluted with syntactical sugar seen in other languages (for example I’m using Swift as the base language of reference due to the project and damn does that language have a lot of sugar).


Yes! I don’t “like” it in the same way I enjoy Crystal or Ruby syntax, but I like it as a pragmatic choice: it’s fast, it’s simple, it’s predictable and boring. I have extensive experience with node.js and would choose Go over it most of the time. It reduces the mental and maintenance overhead significantly.


I prefer Go to Python, PHP, Elixir, Rust, Swift, Lua, JS, TS etc. Ease of deployment (I use embed so single binary no external assets) No red/blue async functions pains Decent tooling Easy to read Good balance of performance and ergonomics Decent industry uptake


A bad programmer can make a mess in any language, including Go.

A great example is this .map().filter().reduce() joke we're looking at today.

Well-written Go is absolutely sublime.


I would argue that one of Go's strengths is its resistance to bad code.

Sure, you can still write gross Go. But I'll take gross Go over gross C++, gross Haskell, or gross Python any day.


I agree with the sentiment. Go is definitely more resistant to bad code than any other language I know. I'll take bad Go over bad Rust or bad C++ any day, but unfortunately bad Go is pretty prevalent these days...


C, C++, Tcl, Python, Ruby, ES6, Common Lisp, Go, Rust. Go's my favorite of all of these, though not the best tool for every job.

I'd be unlikely to use a library like this in my Go code.


Never, but it’s steadily harder to avoid at my day job, so I’m in favor of at least having a dialect that is less obstinately broken.


I started my programming career with Perl, Assembly, C, and C++. Over the years I've touched Java, PHP, Lisp, Lua, TCL, Ruby, Python, FORTH, and others to a greater or lesser extent.

In all my current projects I default to starting with golang. Some things annoy me, but on the whole the benefits outweigh the annoyance.

I am very much looking forward to the pending 1.18 release - not for the generics, but for the fuzz-testing support. I've been fuzz-testing my applications/libraries for the past few years, and I think this is extraordinarily useful and I hope with a wider audience using such techniques we'll all benefit.

(golang developers already have a good culture of writing test-cases, much like Perl did back in the day with the TAP format. But fuzz testing is magical and often catches things developers didn't think about.)


My very anecdotal experience has been that all (most?) of the Go fans I know have a heavy Python background and most (all?) of them also have some amount of C experience as well.

From what I've seen of Go, this makes sense to me.


I definitely see the C aspect. Go is like a cleaned up C with a tracing garbage collector.

However I'm surprised that many python fans like Go. It seems like just enough static typing to make the patterns that you are used to painful, without enough to actually describe your program.


Go is follows Python's original principles more closely than modern Python does. Take a look at the Zen of Python [1]. Go fits it far better than Python, sadly.

[1]: https://www.python.org/dev/peps/pep-0020/#the-zen-of-python


Thinking about it more, I suspect the pattern I've noticed is "C programmers who switched to Python along the way" and less "Python programmers who happened to dabble in C". I'm of a certain age where most everyone started as a C programmer, so there's a bias towards that background in my circle.


Of of the best things I like about is how all the source files are always following the same formatting rules. Also the idioms make it easy to read.


Some time ago, I wrote this in JS. The function has an understandable name, and this line is commented, so the implementation isn't that important.

    return String(this.path).split(".").reduce((obj, attr) => obj[attr], this.object);
Compare that to

    let attrs = String(this.path).split(".");
    let obj = this.object;
    for (let i = 0; i < attrs.length; i++) {
        obj = obj[attr[i]];
    }
    return obj;
Yes, I'm a fan of Go. I've got ample experience in C, C++, Java, JS/TS, Python, and I've seriously tried Rust, and minor experience in a bunch of others (Scheme, Prolog, Perl, etc.), but I prefer Go for back-end software. The balance between abstraction and simply saying what you're doing is great. It may not be optimal, but it gets the work done, reliably and pretty fast.

The other day, there were some complaints about Celery; the OP noted that just starting the application took 280MB, after which it grows to 14.5GB. I've inherited two Python web apps, and I understand the pain. My Python apps grow to about 4GB each. The Go app I've written powers two other web apps, one of which has a higher usage than the Python apps, and it hasn't grown beyond 40MB, and barely uses CPU, on a low-powered virtual server.


I personally can’t stand the language itself but after using it a bit more I do see the appeal. A single install gets you a whole environment to compile / run / test / format backend code without dependencies. Compiler is really fast too (at least compared to Rust or Swift).


yes go is my preferred language for solo projects. Mostly because of tooling. I easily get binaries for all platforms, lot of errors are caught by its compiler. It packages everything in single binary which means less operational overhead. Its standard library contains most of the things that you need for web services so it doesn't require lot of dependencies. It is widespread enough to have a rich ecosystem if i do need something outside standard library.

There are some drawbacks too but i guess more experienced you are in any language more issues you will discover with it but you also learn to workaround them.


This actually became a problem in Java after the streams API was introduced. A bunch of people in my company started writing Java code that looked nothing like Java with a bunch of chained functional style functions with liberal use of the Optional API where trying to decipher what was actually being done required stopping in your tracks and using the help of the IDE to figure out what type was being returned by each level of the chain. I myself fell victim to this and wrote code which made me feel clever in the moment but looking at it a couple of days later even I couldn't understand my own code.

If people want to write functional code they should use functional languages rather than writing non-idiomatic code in other languages.

Thankfully this style is unlikely to gain popularity in Go because they make the syntax for doing this verbose enough and ugly enough that most people aren't gonna bother with it except a top level map or filter.


I work with Java day to day. And I definitely fall victim to overuse of a functional style. Starting to think I shouldn't try to fit everything into a stream, but it's hard because that seems to be the general direction my company is going with their SDK.

I do heavily prefer Optionals however when dealing with Values that can be null. Dealing with nullity checks is annoying.


I hate this. I don't know why people insist on forcing this programming style into every language. Loops exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap.

Not every program needs to be golfed down to one line.


The point of these functions is to give names to common operations that happen in loops so that you can read the code more quickly. Map: we're transforming values. Filter: we're dropping values. Reduce: we're accumulating. Yes you can write these out explicitly in a loop but then you actually have to read all of the associated noise to grok what's happening.

In addition, the method-based approach scopes the variables to each individual call so there's no risk of, say, transforming the loop variable but then accidentally filtering based on the original instead of the transformed.


You could very easily say the same thing about goto vs structured loops, for example:

  "I hate this. I don't know why people insist on forcing this programming style into every language. Goto and labels exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap."
It's just a more structured way to express common operations that we use loops for, and it does has objective technological advantages over raw loops when implemented and used correctly.

As for performance, that is going to vary language to language, but in general iterator processing pipelines are not slower and can even be faster than raw loops. Whether this is the case in Go, I have no idea.

With all of that being said, I am not sure if I would use this style of programming in Go since it doesn't feel idiomatic (yet?).


Goto was simple and fast and did anything. We replaced that with if/then/else/while/foreach/return/throw because each of those clearly expresses what we’re currently doing and especially what we aren’t.


Then don’t use it? Bad code is written in every language, you don’t need a library to make that possible.


Now that generics are in beta, here's a library for those who want ergonomic slice/map wrappers that make chainable operations easy.

Or, a gateway drug for ECMAscripters who can't drop their `.map(x => y).filter(x => y).reduce(x => y)` habit.


Yep, nobody pipes data through any processing in say Haskell or Clojure.


That habit makes zero sense. Reduce already is a map and filter and one, why do you need to loop it 3 times? Reduce itself is awful to begin with.

https://twitter.com/jaffathecake/status/1213077702300852224


it's a readable, maintainable composable functional pattern that unfortunately scales badly in JS because the methods are all eager methods; there are libraries with lazy versions that return generators so that chaining them produces a pipeline that does one loop rather than one per chained operation, though, which most sensible languages do, or at least support, out of the box.

Still, if you know you’ll have a small working set, and you don't what the extra deependency, there's lots of cases where bet readability, composability, and maintainability wins over efficiency.


> Reduce already is a map and filter and one, why do you need to loop it 3 times?

Because being explicit about intent is important. Map means "transforming each element of the collection", filter means "keeping only some elements of the collection". With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. You don't have to spend energy to understand if the loop is only a transformation, or also filtering. Since you chain them, you can easily isolate each transformation, which makes code easier to read and easier to test.

> Reduce itself is awful to begin with.

I don't really understand what's supposed to be awful about reduce. It's just another form of for loop. I don't see people arguing that for loops are awful. The thread you linked is just someone that's biased against reduce for some reason that I don't quite understand. Reduce makes explicit that when using a for loop and accumulating a result, you need both a base case, and what to do when going from n to n+1. Again, this allows you to isolate the "going from n to n+1" part easily. Recursion itself is great for some problems.


> what to do when going from n to n+1

You mention the only worthwhile exception: sums and other 3-character operation.

If you’re using reduce to construct objects you’re just better off with a for loop. Having a random awkward named function that only makes sense when plugged into reduce is not any better than just have a function that includes the loop itself:

    - newArray = array.reduce(operation, [])
    + newArray = operation(array)
If you’re used to having infinite chains of loops then of course that’s not going to work.

The problem is that reduce can do anything and most people use it to do anything. It has no advantage over regular for loops unless you’re proficient with real functional programming (if you use forEach() in JavaScript, you’re not)

> With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read.

Filter and map are ok. Filter, map and reduce are not, at least not in JavaScript. Try writing the equivalent for loop and you’ll find that it’s just as simple to follow.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: