During my path learning go so far I have come across some amazing libraries and utilites, one of my favourite for integration testing is dockertest.
Whenever I am using a service backed by postgres, mongo, mysql or other services that are not part of my codebase, I generally create a docker-compose file so that my development environments are isolated. Then when I’m working in a particular project, all I have to do is docker-compose up -d to get going, and docker-compose down when I’m finished for the day.
For example if I just need a postgres database I would usually have something like
services:
postgres:
image: postgres:15
ports:
- 5432:5432
volumes:
- pg-db:/var/lib/postgresql/data
environment:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_DB: example
volumes:
pg-data:
This is great for development, but when running integration tests, if I want a clean database is means I need some additional steps. I can either:
- Manually drop and create the database
- Drop and create the database as part of the integration test
- Many other options
Instead of creating some automation to drop and create databases or tables, dockertest allows us to create a completley clean isolated instance of the service, so we know every time we are starting from nothing. This follows the cattle not pets idealology from devops.
We are just discussing postgres here, but some other docker services might require a LOT manual steps to get into a good condition for test runs.
Example Link to heading
I created an example project here to show everything in action, but I will work through each step I took.
The project I created is very simple and the main work is in the following code:
package database
import (
"fmt"
"log"
"os"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type (
Person struct {
gorm.Model
Name string
Age uint
}
)
var db *gorm.DB
func Connect() {
log.Println("Setting up the database")
pgUrl := fmt.Sprintf("postgresql://postgres@127.0.0.1:%s/example", os.Getenv("POSTGRES_PORT"))
log.Printf("Connecting to %s\n", pgUrl)
var err error
db, err = gorm.Open(postgres.Open(pgUrl), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
db.AutoMigrate(&Person{})
}
func CreatePerson() {
log.Println("Creating a new person in the database")
person := Person{Name: "Danny", Age: 42}
db.Create(&person)
log.Println("Trying to write a new person to the database")
}
func CountPeople() int {
var count int64
db.Model(&Person{}).Count(&count)
return int(count)
}
So we have a method to connect, a method to create a new person record and a method to count the records. These are all called from the main entry point
func main() {
database.Connect()
database.CreatePerson()
count := database.CountPeople()
log.Printf("Database has %d people", count)
}
If I run this from the command line (once docker compose is up) the result increments the count on every run (as it should)
go-dockertest-example λ git main → go run main.go
2023/09/03 13:41:06 Setting up the database
2023/09/03 13:41:06 Connecting to postgresql://postgres@127.0.0.1:5432/example
2023/09/03 13:41:06 Creating a new person in the database
2023/09/03 13:41:06 Trying to write a new person to the database
2023/09/03 13:41:06 Database has 3 people
go-dockertest-example λ git main → go run main.go
2023/09/03 13:41:07 Setting up the database
2023/09/03 13:41:07 Connecting to postgresql://postgres@127.0.0.1:5432/example
2023/09/03 13:41:08 Creating a new person in the database
2023/09/03 13:41:08 Trying to write a new person to the database
2023/09/03 13:41:08 Database has 4 people
The Test Link to heading
Now lets look at the test. First there is the setup of dockertest:
func TestMain(m *testing.M) {
// Start a new docker pool
pool, err := dockertest.NewPool("")
if err != nil {
log.Fatalf("Could not construct pool: %s", err)
}
// Uses pool to try to connect to Docker
err = pool.Client.Ping()
if err != nil {
log.Fatalf("Could not connect to Docker: %s", err)
}
pg, err := pool.RunWithOptions(&dockertest.RunOptions{
Repository: "postgres",
Tag: "15",
Env: []string{
"POSTGRES_DB=example",
"POSTGRES_HOST_AUTH_METHOD=trust",
"listen_addresses = '*'",
},
}, func(config *docker.HostConfig) {
// set AutoRemove to true so that stopped container goes away by itself
config.AutoRemove = true
config.RestartPolicy = docker.RestartPolicy{
Name: "no",
}
})
if err != nil {
log.Fatalf("Could not start resource: %s", err)
}
pg.Expire(10)
// Set this so our app can use it
postgresPort := pg.GetPort("5432/tcp")
os.Setenv("POSTGRES_PORT", postgresPort)
// Wait for the Postgres to be ready
if err := pool.Retry(func() error {
_, connErr := gorm.Open(postgres.Open(fmt.Sprintf("postgresql://postgres@localhost:%s/example", postgresPort)), &gorm.Config{})
if connErr != nil {
return connErr
}
return nil
}); err != nil {
panic("Could not connect to postgres: " + err.Error())
}
code := m.Run()
os.Exit(code)
}
So first we create a new docker pool, we make sure the pool responds to a ping then start a postgres instance. When you start a new instance it will grab a random port, so you need a way to pass this to the service, this is why we have the POSTGRES_PORT env var. At the very end of the module setup we set that env so that test will use it.
// Set this so our app can use it
postgresPort := pg.GetPort("5432/tcp")
os.Setenv("POSTGRES_PORT", postgresPort)
There is a section for waiting for postgres to be ready, you can use this in different ways, but basically you are using some condition to test the instance is ready. This could be anything from checking a port is addressable, to calling a http healthcheck.
// Wait for the Postgres to be ready
if err := pool.Retry(func() error {
_, connErr := gorm.Open(postgres.Open(fmt.Sprintf("postgresql://postgres@localhost:%s/example", postgresPort)), &gorm.Config{})
if connErr != nil {
return connErr
}
return nil
}); err != nil {
panic("Could not connect to postgres: " + err.Error())
}
Then there is:
// Make sure we expire the instance after 10 seconds
postgres.Expire(10)
this ensures if cleanup does not succeed we will remove the docker instance after 10 seconds regardless.
Finally we have the actual test
func TestCreatePerson(t *testing.T) {
// Connect to the database
database.Connect()
// Create a person in the database
database.CreatePerson()
// Check that the person was created
count := database.CountPeople()
if count != 1 {
t.Errorf("Expected 1 person to be in the database, got %d", count)
}
}
Each time the test runs there will only be one record in the database.