Hacker News new | past | comments | ask | show | jobs | submit login

That's a classic deadlock bug - two producer/consumer relationships in opposite directions.

Have another goroutine receive and output the "found" results, rather than doing it in the first loop. Now you have a proper pipeline.

The big win with goroutines is that they're green threads. They can block, and you can have lots of them. With regular threads, too many will use up resources. With "async", anything that blocks or just takes too long locks up the whole system. Go has the best of both worlds, which is very useful for the case of a server holding open connections to a huge number of human-paced clients.




I've never worked with a green threads system in any real sort of environment where I was actually solving real problems at scale, but I always wonder, does the underlying runtime ever push them into actual real threads that can actually run on another core, or are they always the imaginary concurrency of using the gaps where another green thread needs to wait the comparative eternity for some I/O?


By default, a go program runs computations on as many threads as there are CPUs in your system (this can be adjusted via gomaxprocs). The execution of all goroutines is distributed amongst these threads. For maximum efficiency you want to avoid spawning much more threads than there are CPUs, as scheduling has an overhead and each thread consumes valuable memory for the stack space. The Go model of running many goroutines distributed across threads is a very efficient solution and also very flexible, as the program writer does not have to make any assumptions about the CPU count of the machine the program runs on.


Yes, the runtime pushes them to actual threads as needed.


know nothing about go, but surely the solution is using something like select?


That only solves work blocked on I/O, and it only solves a subset of that because you can't move all I/O to select/poll/epoll/kqueue. One example is regular files, which cannot be made non-blocking.

There are some async APIs for regular files and we've got io_submit coming in, but it's still not a complete solution and never will be.

On the other hand, spawning threads works everywhere and is cheap enough.


You may also be blocked for non I/O reasons. This is a typical problem in user interfaces. Code designed to deal with 10 items suddenly has to deal with 10,000, and that takes seconds. During this period, your "async" code is stalled. Typical example - text edit box implemented in Javascript trying to deal with a large block of text.


> One example is regular files, which cannot be made non-blocking.

AFAIK this is true on Linux but it’s not true on Windows, which allows “overlapped I/O” on files. Not sure about other operating systems.


With io_uring Linux now also has async file I/O. Though async file I/O in both Linux and Windows is still implemented using a pool of kernel threads, in contrast to network I/O.


I mean go's select or wathever the channel multiplexer is called.


Ah, you're saying that you would,

    select {
        case send <- x:
            ...
        case y := <-recv:
            ...
    }
Ambiguous because the select() syscall is also relevant to the discussion.


Yes something like that. Me sayings i know nothing about go didn't help. I at least knew about select.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: