Go has popularized the mantra don't communicate by sharing memory; share memory by communicating. The language does have the traditional mutex (mutual exclusion construct) to coordinate access to shared memory, but it favors the use of channels to share information among goroutines.
In this article, a short look at goroutines, threads, and race conditions sets the scene for a look at two Go programs. In the first program, goroutines communicate through synchronized shared memory, and the second uses channels for the same purpose. The code is available from my website in a .zip file with a README.
Threads and race conditions
A thread is a sequence of executable instructions, and threads within the same process share an address space: Every thread in a multi-threaded process has read/write access to the very same memory locations. A memory-based race condition occurs if two or more threads (at least one of which performs a write operation) have uncoordinated access to the same memory location.
Consider this depiction of integer variable n, whose value is 777, and two threads trying to alter its contents:
n = n + 10 +-----+ n = n - 10
Thread1------------>| 777 |<------------Thread2
+-----+
nOn a multiprocessor machine, the two threads could execute literally at the same time. The impact on variable n is then indeterminate. It's critical to note that each attempted update consists of two machine-level operations: an arithmetic operation on n's current value (either adding or subtracting 10), and a subsequent assignment operation that sets n to a new value (either 787 or 767).
The paired operations executed in the two threads could interleave in various inappropriate ways. Consider the following scenario, with each numbered item as a single operation at the machine level. For simplicity, assume that each operation takes one tick of the system clock:
- Thread1 does the addition to compute 787, which is saved in a temporary location (on the stack or in a CPU register).
- Thread2 does the subtraction to compute 767, also saved in a temporary location.
- Thread2 performs the assignment; the value of
nis now 767. - Thread1 performs the assignment; the value of
nis now 787.
By coming in last, Thread1 has won the race against Thread2. It's clear that improper interleaving has occurred. Thread1 performs an addition operation, is delayed for two ticks, and then performs the assignment. By contrast, Thread2 performs the subtraction and subsequent assignment operations without interruption. The fix is clear: The arithmetic and assignment operations should occur as if they were a single, atomic operation. A construct such as a mutex provides the required fix, and Go has the mutex.
Go programs are typically multi-threaded, although the threading occurs beneath the surface. On the surface are goroutines. A goroutine is a green thread—a thread under the Go runtime control. By contrast, a native thread is directly under OS control. But goroutines multiplex onto native threads that the OS schedules, which means that memory-based race conditions are possible in Go. The first of two sample programs illustrates this.
MiserSpendthrift1
The MiserSpendthrift1 program simulates shared access to a bank account. In addition to main, there are two other goroutines:
- The miser goroutine repeatedly adds to the balance, one currency unit at a time.
- The spendthrift goroutine repeatedly subtracts from the balance, also one currency unit at a time.
The number of times each goroutine performs its operation depends on a command-line argument, which should be large enough to be interesting (e.g., 100,000 to a few million). The account balance is initialized to zero and should wind up as zero because the deposits and withdrawals are for the same amount and are the same in number.
Example 1. Using a mutex to coordinate access to shared memory
package main
import (
"os"
"fmt"
"runtime"
"strconv"
"sync"
)
var accountBalance = 0 // balance for shared bank account
var mutex = &sync.Mutex{} // mutual-exclusion lock
// critical-section code with explicit locking/unlocking
func updateBalance(amt int) {
mutex.Lock()
accountBalance += amt // two operations: update and assignment
mutex.Unlock()
}
func reportAndExit(msg string) {
fmt.Println(msg)
os.Exit(-1) // all 1s in binary
}
func main() {
if len(os.Args) < 2 {
reportAndExit("\nUsage: go ms1.go <number of updates per thread>")
}
iterations, err := strconv.Atoi(os.Args[1])
if err != nil {
reportAndExit("Bad command-line argument: " + os.Args[1]);
}
var wg sync.WaitGroup // wait group to ensure goroutine coordination
// miser increments the balance
wg.Add(1) // increment WaitGroup counter
go func() {
defer wg.Done() // invoke Done on the WaitGroup when finished
for i := 0; i < iterations ; i++ {
updateBalance(1)
runtime.Gosched() // yield to another goroutine
}
}()
// spendthrift decrements the balance
wg.Add(1) // increment WaitGroup counter
go func() {
defer wg.Done()
for i := 0; i < iterations; i++ {
updateBalance(-1)
runtime.Gosched() // be nice--yield
}
}()
wg.Wait() // await completion of miser and spendthrift
fmt.Println("Final balance: ", accountBalance) // confirm final balance is zero
}Flow-of-control in the MiserSpendthrift1 program (see above) can be described as follows:
- The program begins by attempting to read and verify a command-line argument that specifies how many times (e.g., a million) the miser and the spendthrift should each update the account balance.
- The
maingoroutine starts two others with the call:go func() { // either the miser or the spendthriftThe first of the two started goroutines represents the miser, and the second represents the spendthrift.
- The program uses a
sync.WaitGroupto ensure that themaingoroutine does not print the final balance until the miser and the spendthrift goroutines have finished their work and terminated.
The MiserSpendthrift1 program declares two global variables, one an integer variable to represent the shared bank account and the other a mutex to ensure coordinated goroutine access to the account:
var accountBalance = 0 // balance for shared bank account
var mutex = &sync.Mutex{} // mutual-exclusion lockThe mutex code occurs in the updateBalance function to safeguard a critical section, which is a code segment that must be executed in single-threaded fashion for the program to behave correctly:
func updateBalance(amt int) {
mutex.Lock()
accountBalance += amt // critical section
mutex.Unlock()
}The critical section is the statement between the Lock() and Unlock() calls. Although a single line in Go source code, this statement involves two distinct operations: an arithmetic operation followed by an assignment. These two operations must be executed together, one thread at a time, which the mutex code ensures. With the locking code in place, the accountBalance is zero at the end because the number of additions by 1 and subtractions by 1 is the same.
If the mutex code is removed, then the final value of the accountBalance is unpredictable. On two sample runs with the lock code removed, the final balance was 249 on the first run and -87 on the second, thereby confirming that a memory-based race condition occurred.
The mutex code's behavior deserves a closer look:
- To execute the critical section code, a goroutine must first grab the lock by executing the
mutex.Lock()call. If the lock is held already, then the goroutine blocks until the lock becomes available; otherwise, the goroutine executes the mutex-protected critical section. - The mutex guarantees mutual exclusion in that only one goroutine at a time can execute the locked code segment. The mutex ensures single-threaded execution of the critical section: the arithmetic operation followed by the assignment operation.
- The call to
Unlock()releases a held lock so that some goroutine (perhaps the one that just released the lock) can grab the lock anew.
In the MiserSpendthrift1 program, three goroutines (the miser, the spendthrift, and main) communicate through the shared memory location named accountBalance. A mutex coordinates access to this variable by the miser and the spendthrift, and main tries to access the variable only after both the miser and the spendthrift have terminated. Even with a relatively large command-line argument (e.g., five to 10 million), the program runs relatively fast and yields the expected final value of zero for the accountBalance.
The package sync/atomic has functions such as AddInt32 with synchronization baked in. For example, if the accountBalance type were changed from int to int32, then the updateBalance function could be simplified as follows:
func updateBalance(amt int32) { // argument must be int32 as well
atomic.AddInt32(&accountBalance, amt) // no explicit locking required
}The MiserSpendthrift1 program uses explicit locking to highlight the critical-section code and to underscore the need for thread synchronization to prevent a race condition. In a production-grade example, a critical section might comprise several lines of source code. In any case, a critical section should be as short as possible to keep the program as concurrent as possible.
MiserSpendthrift2
The MiserSpendthrift2 program again has a global variable accountBalance initialized to zero, and again there are miser and spendthrift goroutines contending to update the balance. However, this program does not use a mutex to prevent a race condition. Instead, there is now a banker goroutine that accesses the accountBalance in response to requests from the miser and the spendthrift. These two goroutines no longer update the accountBalance directly. Here is a sketch of the architecture:
requests updates
miser/spendthrift---------->banker--------->balanceThis architecture, with support from a thread-safe Go channel to serialize requests from the miser and the spendthrift, prevents a race condition on the accountBalance.
Example 2. Using a thread-safe channel to coordinate access to shared memory
package main
import (
"os"
"fmt"
"runtime"
"strconv"
"sync"
)
type bankOp struct { // bank operation: deposit or withdraw
howMuch int // amount
confirm chan int // confirmation channel
}
var accountBalance = 0 // shared account
var bankRequests chan *bankOp // channel to banker
func updateBalance(amt int) int {
update := &bankOp{howMuch: amt, confirm: make(chan int)}
bankRequests <- update
newBalance := <-update.confirm
return newBalance
}
// For now a no-op, but could save balance to a file with a timestamp.
func logBalance(current int) { }
func reportAndExit(msg string) {
fmt.Println(msg)
os.Exit(-1) // all 1s in binary
}
func main() {
if len(os.Args) < 2 {
reportAndExit("\nUsage: go ms1.go <number of updates per thread>")
}
iterations, err := strconv.Atoi(os.Args[1])
if err != nil {
reportAndExit("Bad command-line argument: " + os.Args[1]);
}
bankRequests = make(chan *bankOp, 8) // 8 is channel buffer size
var wg sync.WaitGroup
// The banker: handles all requests for deposits and withdrawals through a channel.
go func() {
for {
/* The select construct is non-blocking:
-- if there's something to read from a channel, do so
-- otherwise, fall through to the next case, if any */
select {
case request := <-bankRequests:
accountBalance += request.howMuch // update account
request.confirm <- accountBalance // confirm with current balance
}
}
}()
// miser increments the balance
wg.Add(1) // increment WaitGroup counter
go func() {
defer wg.Done() // invoke Done on the WaitGroup when finished
for i := 0; i < iterations ; i++ {
newBalance := updateBalance(1)
logBalance(newBalance)
runtime.Gosched() // yield to another goroutine
}
}()
// spendthrift decrements the balance
wg.Add(1) // increment WaitGroup counter
go func() {
defer wg.Done()
for i := 0; i < iterations; i++ {
newBalance := updateBalance(-1)
logBalance(newBalance)
runtime.Gosched() // be nice--yield
}
}()
wg.Wait() // await completion of miser and spendthrift
fmt.Println("Final balance: ", accountBalance) // confirm the balance is zero
}The changes in the MiserSpendthrift2 program can be summarized as follows. There is a BankOp structure:
type bankOp struct { // bank operation: deposit or withdraw
howMuch int // amount
confirm chan int // confirmation channel
}that the miser and the spendthrift goroutines use to make update requests. The howMuch field is the update amount, either 1 (miser) or -1 (spendthrift). The confirm field is a channel that the banker goroutine uses in responding to a miser or a spendthrift request; this channel carries the new balance back to the requester as confirmation. For efficiency, the address of a bankOp structure, rather than a copy of it, is sent over the bankRequests channel, which is declared as follows:
var bankRequests chan *bankOp // channel of pointers to a bankOpChannels are synchronized—that is, thread-safe—by default.
The miser and the spendthrift again call the updateBalance function in order to change the account balance. This function no longer has any explicit thread synchronization:
func updateBalance(amt int) int { // request structure
update := &bankOp{howMuch: amt,
confirm: make(chan int)}
bankRequests <- update // send request
newBalance := <-update.confirm // await confirmation
return newBalance // perhaps to be logged
}The bankRequests channel has a buffer size of eight to minimize blocking. The channel can hold up to eight unread requests before further attempts to add another bankOp pointer are blocked. In the meantime, the banker goroutine should be processing the requests as they arrive; a request is removed automatically from the channel when the banker reads it. The confirm channel is not buffered, however. The requester blocks until the confirmation message—the updated balance stored locally in the newBalanace variable—arrives from the banker.
Local variables and parameters in the updateBalance function (update, newBalance, and amt) are thereby thread-safe because every goroutine gets its own copies of them. The channels, too, are thread-safe so that the body of the updateBalance function no longer requires explicit locking. What a relief for the programmer!
The banker goroutine loops indefinitely, awaiting requests from the miser and spendthrift goroutines:
for {
select {
case request := <-bankRequests: // Is there a request?
accountBalance += request.howMuch // If so, update balance and
request.confirm <- accountBalance // confirm to requester
}
// other cases could be added (e.g., golf outings)
}While the miser and spendthrift goroutines are still active, only the banker goroutine has access to the accountBalance, which means that a race condition on this memory location cannot arise. Only after the miser and spendthrift finish their work and terminate does the main goroutine print the final value of the accountBalance and exit. When main terminates, so does the banker goroutine.
Locks or channels?
The MiserSpendthrift2 program adheres to the Go mantra by favoring channels over synchronized shared memory. To be sure, locked memory can be tricky. The mutex API is low-level and thus prone to mistakes such as locking but forgetting to unlock—with deadlock as a possible result. More subtle mistakes include locking only part of a critical section (underlocking) and locking code that does not belong to a critical section (overlocking). Thread-safe functions such as atomic.AddInt32 reduce these risks because the locking and unlocking occur automatically. Yet the challenge remains of how to reason about low-level memory locking in complicated programs.
The Go mantra brings challenges of its own. If the two miser/spendthrift programs are run with a sufficiently large command-line argument, the contrast in performance is noteworthy. The mutex may be low-level, but it performs well. Go channels are appealing because they provide built-in thread safety and encourage single-threaded access to shared critical resources such as the accountBalance in the two sample programs. Channels, however, incur a performance penalty compared to mutexes.
It's rare in programming that one tool fits all tasks. Go accordingly comes with options for thread safety, ranging from low-level locking through high-level channels.
2 Comments