pond

Minimalistic and High-performance goroutine worker pool written in Go

Features:

• Zero dependencies
• Create pools with fixed or dynamic size
• Worker goroutines are only created when needed (backpressure detection) and automatically purged after being idle for some time (configurable)
• Minimalistic APIs for:
  • Creating worker pools with fixed or dynamic size
  • Submitting tasks to a pool in a fire-and-forget fashion
  • Submitting tasks to a pool and waiting for them to complete
  • Submitting tasks to a pool with a deadline
  • Submitting a group of related tasks and waiting for them to complete
  • Getting the number of running workers (goroutines)
  • Stopping a worker pool
• Task panics are handled gracefully (configurable panic handler)
• Supports Non-blocking and Blocking task submission modes (buffered / unbuffered)
• Efficient memory usage

How to install

go get -u github.com/alitto/pond

How to use

Worker pool with dynamic size

package main

import (
	"fmt"

	"github.com/alitto/pond"
)

func main() {

	// Create a buffered (non-blocking) pool that can scale up to 100 workers
	// and has a buffer capacity of 1000 tasks
	pool := pond.New(100, 1000)

	// Submit 1000 tasks
	for i := 0; i < 1000; i++ {
		n := i
		pool.Submit(func() {
			fmt.Printf("Running task #%d\n", n)
		})
	}

	// Stop the pool and wait for all submitted tasks to complete
	pool.StopAndWait()
}

Worker pool with fixed size

package main

import (
	"fmt"

	"github.com/alitto/pond"
)

func main() {

	// Create an unbuffered (blocking) pool with a fixed 
	// number of workers
	pool := pond.New(10, 0, pond.MinWorkers(10))

	// Submit 1000 tasks
	for i := 0; i < 1000; i++ {
		n := i
		pool.Submit(func() {
			fmt.Printf("Running task #%d\n", n)
		})
	}

	// Stop the pool and wait for all submitted tasks to complete
	pool.StopAndWait()
}

Submitting groups of related tasks

package main

import (
	"fmt"

	"github.com/alitto/pond"
)

func main() {

	// Create a pool
	pool := pond.New(10, 1000)
	defer pool.StopAndWait()

	// Create a task group
	group := pool.Group()

	// Submit a group of related tasks
	for i := 0; i < 20; i++ {
		n := i
		group.Submit(func() {
			fmt.Printf("Running group task #%d\n", n)
		})
	}

	// Wait for all tasks in the group to complete
	group.Wait()
}