harrigan.xyz

Testing Go gRPC Server Using an in-memory Buffer with `bufconn`

· 3 min
go development testing

It can be cumbersome to setup a testing environment targeting a live server to implement full API testing against your gRPC server. Even spinning up a server from your test file can lead to unintended consequences that require you to allocate a TCP port (parallel runs, multiple runs under same CI server).

bufconn is a package which provides a Listener object that implements net.Conn. We can substitute this listener in a gRPC server - allowing us to spin up a server that acts as a full-fledged server that can be used for testing that talks over an in-memory buffer instead of a real port.

Goals

  • Spin up a gRPC server using an in-memory buffer
  • Use the server in a standard testing.Test test function

Setup

Same as my other post, I created a simple gRPC service that implements Ping and StreamPring. Calling Ping replies with (you guessed it!) a pong message. StreamPing is similar, but you can specify how many times it pong’s back.

syntax = "proto3";

package ping;

option go_package = "protos";

service Pinger {
    rpc Ping(PingRequest) returns (PingResponse) {}
    rpc PingStream (PingRequest) returns (stream PingResponse) {}
}

message PingRequest {
    int32 count = 1;
}


message PingResponse {
    bytes payload = 1;
}

Using bufconn

To easily spin up a server and a client that talk over a bufconn buffer, I created a function that is called by the test file:

func server(ctx context.Context) (pb.PingerClient, func()) {
	buffer := 1024 * 1024
	listener := bufconn.Listen(buffer)

	s := grpc.NewServer()
	pb.RegisterPingerServer(s, &Pinger{})
	go func() {
		if err := s.Serve(listener); err != nil {
			panic(err)
		}
	}()

	conn, _ := grpc.DialContext(ctx, "", grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) {
		return listener.Dial()
	}), grpc.WithInsecure(), grpc.WithBlock())

	client := pb.NewPingerClient(conn)

	return client, s.Stop
}

Important thing to note here are, this function:

  • Spins up gRPC server using the bufconn buffer
  • Creates a client that talks over the buffer using gRPC.WithContextDialer
  • Returns function to terminate the listener and the server

Using the bufconn client

With the above function, the rest are easy:

ctx := context.Background()
assert := assert.New(t)

client, closer := server(ctx)
defer closer()

out, err := client.Ping(ctx, tc.in)
assert.Nil(err)

if tc.expected.err == nil {
    assert.Nil(err)
    assert.Equal(tc.expected.out, out)
} else {
    assert.Nil(out)
    assert.Equal(tc.expected.err, err)
}

That is the meat of my test. It uses the server function, after making sure the closer is defered, it uses the client to make a request. The response is asserted against expectation. This all happens against a gRPC server, except it uses bufconn instead of a TCP socket.

Source code for the full setup is available here.

Conclusion

bufconn allows you to spin up a gRPC server that talk over an in-memory buffer. Tests are fast and reliable, and allows you to easily setup integration style tests that uses a full server.