Go Concurrency Series: Concurrency Patterns(II)
Understand how you could ensure that all goroutines stop working even if one of them fails, using Tomb and Context. Before that, understand how you could implement Fan In/Fan Out Concurrency Patterns
In our last post, we talked about the Worker Pool and Pipeline concurrency patterns, that we can use while designing concurrent applications in Golang. In this article, let’s continue and look at some more concurrency patterns in Golang.
Fan In Pattern
Fan-In is the process of combining multiple results from parallel tasks into a single stream or result. This can be visualized as collecting outputs from various workers and assembling them into a final product. Think of it like what “Reduce” does in Map-Reduce, where it collects data from different workers and aggregates them.
Fan Out Pattern
Fan-Out pattern involves dividing a task into multiple smaller, parallel tasks to be executed concurrently. It's akin to distributing parts of a task across multiple workers in an assembly line, where each worker focuses on a specific part of the job, enhancing overall efficiency and speed.
The above are sample implementations and fan-in fan-out patterns are generally used in conjunction with worker pool and even pipeline patterns.
Handling Errors Across Multiple Goroutines
While working with Go, we generally spin up multiple goroutines and it’s important to handle lifecycle of the goroutines esp in cases of failure. We’ll look at multiple approaches on how we can coordinate errors across multiple goroutines.
Tomb
The tomb package offers a structured way to manage goroutine lifecycles and handle errors, providing a higher-level abstraction over the native Go concurrency primitives like channels and waitgroups. It's particularly useful in applications where clean shutdown and error handling across multiple goroutines are critical. Lets dive into an example on how Tomb helps, when we simulate an error in one of the goroutines -