Concurrency Patterns in Go

June 18, 2019

Concurrency Patterns in Go
Concurrency Patterns in Go

Goroutines are cheap. Channels are how they talk. Everything else is a pattern built on top of those two ideas.

Estimated Reading Time : 12m

Goroutines

A goroutine is a lightweight thread managed by the Go runtime. You start one with the go keyword:

go func() {
    fmt.Println("running concurrently")
}()

Goroutines are cheap — you can spawn thousands of them. The runtime schedules them across available OS threads automatically.

The catch: if main returns before your goroutines finish, they’re killed. You need a way to wait for them.

WaitGroup

sync.WaitGroup is the standard way to wait for a group of goroutines to finish:

var wg sync.WaitGroup

for i := range 5 {
    wg.Add(1)
    go func(n int) {
        defer wg.Done()
        fmt.Println("worker", n)
    }(i)
}

wg.Wait() // blocks until all goroutines call Done

Call Add before launching the goroutine, not inside it. If you call Add after go, there’s a race between the goroutine starting and Wait being called.

Channels

Channels are typed conduits for passing values between goroutines:

ch := make(chan int)    // unbuffered
ch := make(chan int, 5) // buffered, capacity 5

An unbuffered channel blocks the sender until the receiver is ready, and vice versa. It’s a synchronization point.

A buffered channel allows the sender to proceed without a receiver, up to the buffer capacity. Once full, sends block.

ch := make(chan string, 2)
ch <- "first"
ch <- "second"

fmt.Println(<-ch) // first
fmt.Println(<-ch) // second

Select

select lets a goroutine wait on multiple channel operations at once:

select {
case msg := <-ch1:
    fmt.Println("from ch1:", msg)
case msg := <-ch2:
    fmt.Println("from ch2:", msg)
case <-time.After(1 * time.Second):
    fmt.Println("timed out")
}

If multiple cases are ready simultaneously, Go picks one at random. The default case runs immediately if no other case is ready — useful for non-blocking sends and receives.

Worker pool

The worker pool is one of the most useful patterns in Go. A fixed number of goroutines pull from a jobs channel and send results to a results channel:

func workerPool(numWorkers int, jobs <-chan int, results chan<- int) {
    var wg sync.WaitGroup
    for range numWorkers {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                results <- process(job)
            }
        }()
    }
    wg.Wait()
    close(results)
}

func main() {
    jobs := make(chan int, 100)
    results := make(chan int, 100)

    go workerPool(5, jobs, results)

    for i := range 20 {
        jobs <- i
    }
    close(jobs)

    for result := range results {
        fmt.Println(result)
    }
}

Workers range over jobs — when the channel is closed and drained, the loop exits and the goroutine terminates cleanly.

Fan-out

Fan-out spawns a goroutine per item, processing all of them concurrently:

func fanOut(items []string) []string {
    results := make([]string, len(items))
    var wg sync.WaitGroup

    for i, item := range items {
        wg.Add(1)
        go func(i int, item string) {
            defer wg.Done()
            results[i] = process(item)
        }(i, item)
    }

    wg.Wait()
    return results
}

Each goroutine writes to its own index in results, so no mutex is needed. This only works safely when goroutines write to distinct positions — if they share positions, you need synchronization.

Fan-out is appropriate when you have a bounded, known set of items and want maximum parallelism. For unbounded or high-volume work, a worker pool gives you better control over resource usage.

Cancellation with context

context.Context is the standard way to propagate cancellation across goroutines:

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

var wg sync.WaitGroup
for range 3 {
    wg.Add(1)
    go func() {
        defer wg.Done()
        select {
        case <-ctx.Done():
            fmt.Println("cancelled:", ctx.Err())
        case result := <-doWork(ctx):
            fmt.Println("done:", result)
        }
    }()
}
wg.Wait()

Pass the context down to every function that does I/O or long-running work. Check ctx.Done() or pass the context directly to library calls that accept one.

Always call cancel() — even if the context expires on its own. It releases the resources associated with the context.

Gotchas

Goroutine leaks. A goroutine blocked on a channel read or write with no way to unblock will run forever. Always give goroutines a way out — a done channel, a context, or a closed input channel.

// leaks if nothing ever sends to ch
go func() {
    v := <-ch // blocks forever
    _ = v
}()

// safe — goroutine exits when ctx is cancelled
go func() {
    select {
    case v := <-ch:
        _ = v
    case <-ctx.Done():
        return
    }
}()

Closing a channel twice panics. Only close a channel once, and only from the sender side. If multiple goroutines might close the same channel, coordinate with a sync.Once.

Ranging over a channel that’s never closed blocks forever. If you range over a channel in a goroutine, make sure the sender closes the channel when it’s done. Otherwise the range loop never terminates.

Don’t copy a mutex. sync.Mutex and sync.WaitGroup must not be copied after first use. Pass them by pointer or embed them in a struct that’s accessed by pointer.