8 posts tagged with "go"

Go has a built-in race detector, succinctly described in this Go article. I’ve found it to be a useful tool—often the first thing I reach for when debugging flaky Go tests and suspecting a race condition. I thought I’d write down some lessons I’ve learned from using this tool.

1. Go Race Detector is Your Friend​

Running tests in Go with the -race flag enables race detection and can give you an immediate first-pass indication of whether the test has a race condition. Since the race detector works by "instrumenting/monitoring" the code during execution, the slower runtime compared to running the same test without the flag can actually be beneficial.

The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.

This slowdown can make elusive race conditions more reproducible. Once you learn how to interpret the race detector’s logs, you’ll have a direct way to identify which code paths need further scrutiny. An example of how to interpret race detector logs can be found in my other post.

• In essence: find the concurrent read and write that are causing the race.

Using the race detector can give you a quick indicator of potential race conditions.

2. No Race Condition Detected != No Race Condition​

The race detector only finds races that happen at runtime, so it can't find races in code paths that are not executed.

Because the race detector works at runtime, it can only detect race conditions that are actually triggered during execution. This is important to acknowledge, as some race conditions are highly timing-sensitive—they may only occur when certain operations are unusually fast or slow.

3. Race Detected != Production Code is Impacted by Race Condition​

Again, since the race detector works at runtime, detecting a race condition doesn’t necessarily mean your production code is affected. Sometimes, the test code itself introduces the race.

For example, tests often try to hit edge scenarios, which can unintentionally induce race conditions. This is common in tests that use multiple goroutines to test concurrent functions, especially when results are not communicated through channels. Another pattern I’ve seen is tests that intentionally misuse the function under test—such as skipping synchronization or blocking on a done channel—to simulate concurrent processes.

One telltale sign is tweaking sleep durations across goroutines to simulate the handling of many incoming requests.

Lastly, test code is often less strict with shared variables (e.g., using a map or counter without a lock) and then making assertions across goroutines. This can introduce race conditions that the race detector rightly reports—but in such cases, it's more a sign of poorly written tests than a serious concern about the production code.

4. AI Can't Reliably Help You Detect Race Conditions​

On the topic of race detection, I want to share a quick note: after experimenting with AI tools (specifically GitHub Copilot) to analyze and detect race conditions, I’ve concluded that while they can be helpful, they aren’t 100% reliable. AI can sometimes spot issues and suggest fixes, but it may also miss certain race conditions or propose solutions that aren’t suitable.

That said, this is just my observation as of April 2025—AI is improving fast, and I hope it proves me wrong soon...

Usefulness of defer​

defer is a Go feature that defers execution until after the function exits. It is not simply a way to move a statement from the current line to the last line of the enclosing function. defer can become problematic when multiple defer statements are involved (making execution order tricky to determine) or when it is used to enforce a logical execution order.

Two Common Pitfalls When Using defer​

Misusing defer can lead to unintended consequences. A common mistake is invoking a function call without wrapping it in an anonymous function when capturing a dynamically changing value. For example:

package main

import (
	"fmt"
)

func world(val string) {
	fmt.Printf("%s from world", val)
}

func main() {
	val := "hi"
	defer world(val)
	val = "hello"
	fmt.Println("hello from main")
}

At the point of invoking defer world(val), the value of val is captured as "hi". Later changes to val do not affect this deferred function call, which can be undesirable.

One such undesirable scenario is passing an error object. If we declare var err error and attempt to defer funcName(err) or channelName <- err, the parameter err is immediately evaluated, but execution is delayed. This can result in sending an outdated error value.

To fix this, we can use an anonymous function:

package main

import (
	"fmt"
)

func world(val string) {
	fmt.Printf("%s from world", val)
}

func main() {
	val := "hi"
	defer func() {
		world(val)
	}()

	ch := make(chan string, 1)
	defer func() {
		v := <-ch
		fmt.Printf("%s from channel\n", v)
	}()

	defer func() {
		ch <- val
	}()

	val = "hello"

	fmt.Println("hello from main")
}

This produces:

hello from main
hello from channel
hello from world

Here, we deferred the channel send operation. If the channel is used to signal the completion of an entire operation, this ensures it triggers at the correct time.

Another example:

func lastOperation() {
	fmt.Println("Doing something")
}

func main() {
	ch := make(chan string, 1)
	defer lastOperation()
	ch <- "done"
}

The channel is notified before lastOperation() executes, making the logic incorrect. The last operation should be done before notifying completion, not the other way around.

One more noteworthy example on Reddit highlights how defer delays evaluation:

type A struct {
	text string
}

func (a *A) Do() {
	_ = a.text
}

func DoSomething() {
	var a *A

	defer a.Do()
	// vs
	// defer func() { a.Do() }()

	a = &A{}
}

defer a.Do() causes a runtime panic because a is nil at the time of defer evaluation. However, using defer func() { a.Do() }() delays evaluation, allowing a to be assigned a valid value before execution.

Contrast it with:

type A struct {}

func (a *A) Do() {}

func DoSomething() {
	var a *A

	defer a.Do()

	a = &A{}
}

Here, a is still a nil pointer at the time of defer evaluation, but since Do() does not dereference a, the call is safe.

To summarize, if function parameter evaluation is irrelevant, using defer funcName() is fine. Otherwise, wrap it in an anonymous function to delay evaluation.

The second common pitfall is to register defer statements too late in the function. This can result in them never executing if the function exits early (e.g., due to error handling).

Best practices:

• Wrap the defer call in an anonymous function if necessary to prevent immediate parameter evaluation.
• Place defer statements as early as possible (and logical) in the function to ensure they are registered before any early return logic.

More on Multiple defer​

When multiple defer statements are used, their execution follows a stack-based order—Last In, First Out (LIFO). Deferred executions occur in reverse order from their placement in the function. Understanding this order is critical in cases like:

• Ensuring consistent mutex unlocking sequences.
• Correctly signaling completion in operations that depend on ordered execution.

Consider a structure where A is an operation and A.a is a sub-operation. Without defer, the correct order would be:

• Send done to A.a's channel.
• Send done to A's channel.

But, with a single defer:

• (defer) Send done to A.a's channel.
• Send done to A's channel.

This could lead to incorrect order (A is marked done before A.a).

A similar issue arises when both are deferred incorrectly:

• (defer) Send done to A.a's channel.
• (defer) Send done to A's channel.

Since defer follows LIFO, A is marked done before A.a.

Correcting defer order:

• (defer) Send done to A's channel.
• (defer) Send done to A.a's channel.

Now, A.a completes before A, ensuring the correct sequence.

The Problem​

A nil pointer dereference is a well-known runtime error to avoid. The cause is simple to explain: a pointer to a struct is passed to or returned from a function. Accessing the struct's fields or methods can cause a runtime panic if the pointer turns out to be nil.

The FindXXX Pattern​

In our code logic, there are often cases where an identifier maps to an in-memory representation of a struct or object. These objects are frequently stored in a map, but they can also be retrieved or reconstructed from a file or an endpoint. When this object needs to be passed to relevant functions, instead of passing the object directly, some methods may rely on passing the identifier instead.

For example, in the context of a library system:

package main

import "fmt"

type Book struct {
	Title  string
	Author string
}

type Library struct {
	books map[string]*Book
}

func (l *Library) AddBook(title, author string) {
	l.books[title] = &Book{Title: title, Author: author}
}

func (l *Library) FindBook(title string) *Book {
	if book, ok := l.books[title]; !ok {
		return nil
	} else {
		return book
	}
}

func main() {
	library := Library{books: make(map[string]*Book)}
	library.AddBook("Your Code as a Crime Scene", "Adam Tornhill")

	// exist and well
	book := library.FindBook("Your Code as a Crime Scene")

	fmt.Printf("Book: %s, Author: %s\n", book.Title, book.Author)

	// not exist and panic
	book = library.FindBook("The Phoenix Project")

	fmt.Printf("Book: %s, Author: %s\n", book.Title, book.Author)
}

In the code above, FindBook provides a way to retrieve a book representation using the book title. The problem with this design is that nil checks are not enforced by the compiler, which can lead to carelessness in validating the returned object before accessing its fields.

The runtime panic:

Book: Your Code as a Crime Scene, Author: Adam Tornhill
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x100e81e0c]

goroutine 1 [running]:
main.main()
        /Users/yong/Documents/GitHub/learn-go/library/main.go:46 +0x1bc
exit status 2

The Fix​

In other languages, one might simply throw an exception when the requested object does not exist. However, in Go, returning an error is generally preferred as it provides a clearer indication of how the method should be used.

Of course, documenting the function with a comment to indicate the need for nil checks is better than nothing, but a more robust solution is to return an error explicitly:

func (l *Library) GetBook(title string) (*Book, error) {
	if book, ok := l.books[title]; !ok {
		return nil, fmt.Errorf("Book not found")
	} else {
		return book, nil
	}
}

Summary​

While this issue may seem trivial, it is more widespread and insidious than one might think. There are many scenarios where a FindXXX pattern (if such a term exists) can lead to the slippery slope of hidden nil pointer dereferences in the codebase.

Not returning an error and instead relying on a nil pointer is one part of the problem. The other issue is the practice of passing around identifiers, which leads to a loss of type safety—but that’s a topic for another day.