Testing an os.exit scenario in Golang

Today, I ran into an issue. I wanted to test that a function logged a fatal error when something bad happened. The problem with a fatal log message is that it calls os.Exit(1) after logging the message. As a result, if you try to test this by calling your function with the required arguments to make it fail, your test suite is just going to exit.

Suppose you want to test something like this:

package foo

import (
  "log"
)

func Crashes(i int) {
  if i == 42 {
    log.Fatal("It crashes because you gave the answer")
  }
}

Well, this is not so easy as explained before. It turns out that the solution is to start a subprocess to test that the function crashes. The subprocess will exit, but not the main test suite. This is explained in a talk about testing techniques given in 2014 by Andrew Gerrand. If you want to check that the fatal message is something specific, you can inspect the standard error by using the os.exec package. Finally, the code to test the crashing part of the previous function would be the following:

package foo

import (
  "io/ioutil"
  "os"
  "os/exec"
  "strings"
  "testing"
)

func TestCrashes(t *testing.T) {
  // Only run the failing part when a specific env variable is set
  if os.Getenv("BE_CRASHER") == "1" {
    Crashes(42)
    return
  }

  // Start the actual test in a different subprocess
  cmd := exec.Command(os.Args[0], "-test.run=TestCrashes")
  cmd.Env = append(os.Environ(), "BE_CRASHER=1")
  stdout, _ := cmd.StderrPipe()
  if err := cmd.Start(); err != nil {
    t.Fatal(err)
  }

  // Check that the log fatal message is what we expected
  gotBytes, _ := ioutil.ReadAll(stdout)
  got := string(gotBytes)
  expected := "It crashes because you gave the answer"
  if !strings.HasSuffix(got[:len(got)-1], expected) {
    t.Fatalf("Unexpected log message. Got %s but should contain %s", got[:len(got)-1], expected)
  }

  // Check that the program exited
  err := cmd.Wait()
  if e, ok := err.(*exec.ExitError); !ok || e.Success() {
    t.Fatalf("Process ran with err %v, want exit status 1", err)
  }
}

Not so readable, definitely feels like a hack, but it does the job.

Limit the number of goroutines running at the same time

Recently, I was working on package that was doing network requests inside goroutines and I encountered an issue: the program was really fast to finish, but the results were awful. This was because the number of goroutines running at the same time was too high. As a result, the network was congested, too many sockets were opened on my laptop and the final performance was degraded: requests were slow or failing.

In order to keep the network healthy while maintaining some concurrency, I wanted to limit the number of goroutines making requests at the same time. Here is a sample main file to illustrate how you can control the maximum number of goroutines that are allowed to run concurrently.

package main

import (
	"flag"
	"fmt"
	"time"
)

// Fake a long and difficult work.
func DoWork() {
	time.Sleep(500 * time.Millisecond)
}

func main() {
	maxNbConcurrentGoroutines := flag.Int("maxNbConcurrentGoroutines", 5, "the number of goroutines that are allowed to run concurrently")
	nbJobs := flag.Int("nbJobs", 100, "the number of jobs that we need to do")
	flag.Parse()

	// Dummy channel to coordinate the number of concurrent goroutines.
	// This channel should be buffered otherwise we will be immediately blocked
	// when trying to fill it.
	concurrentGoroutines := make(chan struct{}, *maxNbConcurrentGoroutines)
	// Fill the dummy channel with maxNbConcurrentGoroutines empty struct.
	for i := 0; i < *maxNbConcurrentGoroutines; i++ {
		concurrentGoroutines <- struct{}{}
	}

	// The done channel indicates when a single goroutine has
	// finished its job.
	done := make(chan bool)
	// The waitForAllJobs channel allows the main program
	// to wait until we have indeed done all the jobs.
	waitForAllJobs := make(chan bool)

	// Collect all the jobs, and since the job is finished, we can
	// release another spot for a goroutine.
	go func() {
		for i := 0; i < *nbJobs; i++ {
			<-done
			// Say that another goroutine can now start.
			concurrentGoroutines <- struct{}{}
		}
		// We have collected all the jobs, the program
		// can now terminate
		waitForAllJobs <- true
	}()

	// Try to start nbJobs jobs
	for i := 1; i <= *nbJobs; i++ {
		fmt.Printf("ID: %v: waiting to launch!\n", i)
		// Try to receive from the concurrentGoroutines channel. When we have something,
		// it means we can start a new goroutine because another one finished.
		// Otherwise, it will block the execution until an execution
		// spot is available.
		<-concurrentGoroutines
		fmt.Printf("ID: %v: it's my turn!\n", i)
		go func(id int) {
			DoWork()
			fmt.Printf("ID: %v: all done!\n", id)
			done <- true
		}(i)
	}

	// Wait for all jobs to finish
	<-waitForAllJobs
}

This file is available as a gist on GitHub if you find it more convenient.

Sample runs

For the command time go run concurrent.go -nbJobs 25 -maxNbConcurrentGoroutines 10:

ID: 1: waiting to launch!
ID: 1: it's my turn!
ID: 2: waiting to launch!
ID: 2: it's my turn!
ID: 3: waiting to launch!
ID: 3: it's my turn!
ID: 4: waiting to launch!
ID: 4: it's my turn!
ID: 5: waiting to launch!
ID: 5: it's my turn!
ID: 6: waiting to launch!
ID: 6: it's my turn!
ID: 7: waiting to launch!
ID: 7: it's my turn!
ID: 8: waiting to launch!
ID: 8: it's my turn!
ID: 9: waiting to launch!
ID: 9: it's my turn!
ID: 10: waiting to launch!
ID: 10: it's my turn!
ID: 11: waiting to launch!
ID: 1: all done!
ID: 9: all done!
ID: 11: it's my turn!
ID: 12: waiting to launch!
ID: 12: it's my turn!
ID: 7: all done!
ID: 13: waiting to launch!
ID: 5: all done!
ID: 13: it's my turn!
ID: 14: waiting to launch!
ID: 4: all done!
ID: 14: it's my turn!
ID: 8: all done!
ID: 15: waiting to launch!
ID: 15: it's my turn!
ID: 16: waiting to launch!
ID: 16: it's my turn!
ID: 10: all done!
ID: 17: waiting to launch!
ID: 2: all done!
ID: 17: it's my turn!
ID: 18: waiting to launch!
ID: 18: it's my turn!
ID: 3: all done!
ID: 19: waiting to launch!
ID: 6: all done!
ID: 19: it's my turn!
ID: 20: waiting to launch!
ID: 20: it's my turn!
ID: 21: waiting to launch!
ID: 20: all done!
ID: 16: all done!
ID: 17: all done!
ID: 12: all done!
ID: 21: it's my turn!
ID: 19: all done!
ID: 11: all done!
ID: 14: all done!
ID: 18: all done!
ID: 15: all done!
ID: 13: all done!
ID: 22: waiting to launch!
ID: 22: it's my turn!
ID: 23: waiting to launch!
ID: 23: it's my turn!
ID: 24: waiting to launch!
ID: 24: it's my turn!
ID: 25: waiting to launch!
ID: 25: it's my turn!
ID: 24: all done!
ID: 21: all done!
ID: 22: all done!
ID: 25: all done!
ID: 23: all done!
0,28s user 0,05s system 18% cpu 1,762 total

For the command time go run concurrent.go -nbJobs 10 -maxNbConcurrentGoroutines 1:

ID: 1: waiting to launch!
ID: 1: it's my turn!
ID: 2: waiting to launch!
ID: 1: all done!
ID: 2: it's my turn!
ID: 3: waiting to launch!
ID: 2: all done!
ID: 3: it's my turn!
ID: 4: waiting to launch!
ID: 3: all done!
ID: 4: it's my turn!
ID: 5: waiting to launch!
ID: 4: all done!
ID: 5: it's my turn!
ID: 6: waiting to launch!
ID: 5: all done!
ID: 6: it's my turn!
ID: 7: waiting to launch!
ID: 6: all done!
ID: 7: it's my turn!
ID: 8: waiting to launch!
ID: 7: all done!
ID: 8: it's my turn!
ID: 9: waiting to launch!
ID: 8: all done!
ID: 9: it's my turn!
ID: 10: waiting to launch!
ID: 9: all done!
ID: 10: it's my turn!
ID: 10: all done!
0,32s user 0,03s system 6% cpu 5,274 total

Questions? Feedback? Hit me on Twitter @AntoineAugusti

Openness for engineering teams

As a student, I am quite often looking at companies to see what they are doing, to understand the market and discover trends. As an engineering student, I am on the lookout for technical content, written by engineers. I discovered recently that I value a lot openness for engineering teams. Being open can be done in different ways:

Having a technical blog. You can understand this in multiple ways. First, you can have a blog where you talk about new features, new releases of your API / SDK. This one is quite common. The second one is really rare and very valuable to me: you talk about your engineering process, your hiring process, you share reports of outages. If you have open source projects, you have a blog post to let the technical community know about it.
Involvement in communities. You can be involved in communities in multiple ways: regularly sending members of your team to local meetups (not just attending if you can. Presenting and volunteering are awesome), being visible in conferences, giving explicit credit to open source solutions you are using (or giving money to them if you can afford to), host hackathons or hack days at your office. Be explicit about causes you care about and defend them.
Open source. Whether you contribute to open source projects or you open source some of your projects, involvement in the community is a great way to gain some exposure, let people know which technologies you are using and giving back to the community.

An update to the Joel Test?

Maybe some of these points will be in an updated “Joel Test” in the future, even if some people already say that it is partially antiquated. Personally, I would add the following questions to an updated version of the Joel Test:

Do you support developer education by attending conferences, purchasing books (or something equivalent)?
Do you have a simple, documented process to adopt new tools your team uses?
Do you have an engineering blog where you talk about your processes, ideas, beliefs and failures?

You can’t have it all

Being able to answer “Yes” to every questions above seems fairly difficult, and really impossible for small engineering teams. If your company is 1 year old and you are 2 engineers, you cannot put all these things in place. But as they say, “practice makes perfect”, so try to keep these goals in mind. Giving an awesome work environment to your engineers will make them productive, happy to work and so much more! Great engineering teams attract great engineers.

Antoine Augusti