Running a Go test with a Transaction

Using FoundationDB
1

I have Go code using FoundationDB and I want to test it.

I have literally copied the Go test from Apple’s Github. When I uncomment the line that actually runs Set within the transaction, the test hangs and then times out. (With the line still commented out it works as expected.)

func TestExampleTransactor(t *testing.T) {
	fdb.MustAPIVersion(400)  // Note: test behaves the same with MustAPIVersion(600)
	db := fdb.MustOpenDefault()

	setOne := func(t fdb.Transactor, key fdb.Key, value []byte) error {
		fmt.Printf("setOne called with:  %T\n", t)
		_, e := t.Transact(func(tr fdb.Transaction) (interface{}, error) {
			// We don't actually call tr.Set here to avoid mutating a real database.
			tr.Set(key, value) // **NOTE** this is the line I uncommented
			return nil, nil
		})
		return e
	}

	setMany := func(t fdb.Transactor, value []byte, keys ...fdb.Key) error {
		fmt.Printf("setMany called with: %T\n", t)
		_, e := t.Transact(func(tr fdb.Transaction) (interface{}, error) {
			for _, key := range keys {
				setOne(tr, key, value)
			}
			return nil, nil
		})
		return e
	}

	var e error

	fmt.Println("Calling setOne with a database:")
	e = setOne(db, []byte("foo"), []byte("bar"))
	if e != nil {
		fmt.Println(e)
		return
	}
	fmt.Println("\nCalling setMany with a database:")
	e = setMany(db, []byte("bar"), fdb.Key("foo1"), fdb.Key("foo2"), fdb.Key("foo3"))
	if e != nil {
		fmt.Println(e)
		return
	}
}

I am running MacOS 10.13.6. I have FoundationDB installed and I’m able to connect to it using fdbcli, as well as read from and write to it.

Here are the foundation processes:

ps aux | grep foundation
root             12942   1.0  0.3  4488980  46608   ??  S    12:35PM   2:00.06 /usr/local/libexec/fdbserver --cluster_file /usr/local/etc/foundationdb/fdb.cluster --datadir /usr/local/foundationdb/data/4689 --listen_address
root             12517   0.3  0.1  4382548  12204   ??  S    12:32PM   0:29.63 /usr/local/foundationdb/backup_agent/backup_agent --cluster_file /usr/local/etc/foundationdb/fdb.cluster
bancron          25399   0.0  0.0  4258468    200 s005  R+    2:55PM   0:00.00 grep foundation
root             12515   0.0  0.0  4297540    584   ??  Ss   12:32PM   0:00.01 /usr/local/libexec/fdbmonitor --conffile /usr/local/etc/foundationdb/foundationdb.conf --lockfile /var/run/FoundationDB.pid

In case it is relevant, this is the contents of the default cluster file found at /usr/local/etc/foundationdb/fdb.cluster:

JCKqKWc6:ku0VRske@127.0.0.1:4689

And this is the contents of /usr/local/etc/foundationdb/foundationdb.conf:

[general]
restart_delay = 60
cluster_file = /usr/local/etc/foundationdb/fdb.cluster

## Default parameters for individual fdbserver processes
[fdbserver]
command = /usr/local/libexec/fdbserver
public_address = auto:$ID
listen_address = public
datadir = /usr/local/foundationdb/data/$ID
logdir = /usr/local/foundationdb/logs

## An individual fdbserver process with id 4689
## Parameters set here override defaults from the [fdbserver] section
[fdbserver.4689]

[backup_agent]
command = /usr/local/foundationdb/backup_agent/backup_agent
logdir = /usr/local/foundationdb/logs

[backup_agent.1]
2

I have this working now by running some fdbserver and fdbcli commands first.

func createDB() *exec.Cmd {
	port := 4692
	testFileContents := []byte(fmt.Sprintf("test:test%d@127.0.0.1:%d", port, port))
	err := ioutil.WriteFile("fdb_test.cluster", testFileContents, 0755)
	if err != nil {
		log.Fatalf("unable to write fdb_test.cluster file %s", err)
	}
	fdbServerLoc := "/usr/local/libexec/fdbserver"
	fdbServerCmd := exec.Command(fdbServerLoc,
		"-p", fmt.Sprintf("127.0.0.1:%d", port),
		"--datadir", fmt.Sprintf("/tmp/fdb%d", port),
		"-C", "fdb_test.cluster",
	)
	fdbServerCmd.Start()
	configureLoc := "/usr/local/bin/fdbcli"
	configureCmd := exec.Command(configureLoc,
		"-C", "fdb_test.cluster", "--exec", "configure new single memory")
	// This may error if the DB already exists (from previous test runs), so ignore the error.
	configureCmd.Run()
	return fdbServerCmd
}

func TestMain(m *testing.M) {
	port := 4692
	testFileContents := []byte(fmt.Sprintf("test:test%d@127.0.0.1:%d", port, port))
	err := ioutil.WriteFile("fdb_test.cluster", testFileContents, 0755)
	db := fdb.MustOpen("fdb_test.cluster", []byte("DB"))
	mErr := m.Run()
	// ... tests run now

	err = fdbServerCmd.Process.Kill()
	if err != nil {
		log.Fatalf("error killing fdbserver process %s", err)
	}
	os.Exit(mErr)
}
3

Yeah, this seems like something that could have been caused by a general problem connecting to FDB. In particular, with the “set” line commented like that, I think the tests don’t actually do any I/O (neither reading nor writing), so the tests never need to connect to FDB. (In particular, they don’t do any reads, and committing a transaction without any writes is a no-op.)

4

Yeah. I’m still not sure why my production code was able to connect to the default FDB instance with fdb.MustOpenDefault() but the test code wasn’t. I’ve replaced it entirely with a custom .cluster file and port.

5

@bancron I can’t tell why the MustOpenDefault isn’t able to connect. But I just want to let you know that I’ve created a Go package to ease running tests against an transient foundationdb node. The package takes care of the node life cycle and cluster file generation.

Here is a little example:

func TestRoundtrip(t *testing.T) {
	// start foundationdb node
	node := fdbtest.MustStart()
	
	// destroy node at the end of this test
	defer node.Destroy()

	// open fdb.Database
	db := node.MustOpenDB()

	// set foo key to bar
	_, err := db.Transact(func(tx fdb.Transaction) (interface{}, error) {
		tx.Set(fdb.Key("foo"), []byte("bar"))
		return nil, nil
	})
	if err != nil {
		t.Fatalf("set foo key failed: %v", err.Error())
	}

	// get foo key
	value, err := db.Transact(func(tx fdb.Transaction) (interface{}, error) {
		return tx.Get(fdb.Key("foo")).Get()
	})
	if err != nil {
		t.Fatalf("get foo key failed: %v", err.Error())
	}

	// assert foo value
	if expected := "bar"; string(value.([]byte)) != expected {
		t.Fatalf("expected %v, got %v", expected, string(value.([]byte)))
	}
}

Learn more: https://github.com/pjvds/fdbtest

6

@pjvds, would you mind submitting a PR to add this to awesome-foundationdb? It looks like something generally useful for anyone writing FDB Go code.