

Show HN: Golang Throttler – smart WaitGroups to easily manage worker goroutines - derekperkins
https://github.com/nozzle/throttler

======
derekperkins
This is my first time posting code to Hacker News. I'd love to get any
feedback on the quality and usefulness of the code as a package.

Throttler fills the gap between sync.WaitGroup and manually monitoring your
goroutines with channels. The API is almost identical to Wait Groups, but it
allows you to set a max number of workers that can be running simultaneously.
It uses channels internally to block until a job completes by calling Done()
or until all jobs have been completed.

See a fully functional example on the playground at [http://bit.ly/throttler-
docs](http://bit.ly/throttler-docs)

------
rakoo
camlistore has a syncutil package (godoc.org/camlistore.org/pkg/syncutil) that
has similar features, albeit in different structures:

\- Gate is used to limit the number of parallel workers

\- Group is used where sync.WaitGroup would be used. The most important thing:
you have access to the error that happened in your goroutines.

Here's how to use it:

    
    
      var g = syncutil.NewGate(num_parallel_workers)
    
      for _, job := range jobs {
        wg.Go(func() error {
          defer g.Done()
          // do some long-running job
    
          // the job may fail
          return errJob
        })
      }
    
      return wg.Err()
    
      // You also have access to failing goroutines through wg.Errs()

~~~
derekperkins
Interesting, I hadn't seen that package. Gate does look very similar to
Throttler, though it isn't very well documented.

The main difference appears to be that Throttler wants to know the total
number of jobs up front and so doesn't need to be told that a new job is
starting, where Gate more closely follows the WaitGroup concept of
incrementing and decrementing the counters manually.

Group is also interesting, but I don't mind running my own error channels and
executing my own routines so that I have a little more control.

------
leftrightupdown
looks useful, just my 2 cents

