Testing Go gRPC Server Using an in-memory Buffer with `bufconn`
It can be cumbersome to setup a testing environment targeting a live server to implement full API testing against your gRPC server. Even spinning up a server from your test file can lead to unintended consequences that require you to allocate a TCP port (parallel runs, multiple runs under same CI server).
bufconn is a package
which provides a Listener
object that implements net.Conn
. We can
substitute this listener in a gRPC server - allowing us to spin up a server
that acts as a full-fledged server that can be used for testing that talks
over an in-memory buffer instead of a real port.
Goals
- Spin up a gRPC server using an in-memory buffer
- Use the server in a standard
testing.Test
test function
Setup
Same as my other post, I created a simple gRPC service that implements Ping
and StreamPring
. Calling Ping
replies with (you guessed it!) a
pong
message. StreamPing
is similar, but you can specify how many times
it pong
’s back.
syntax = "proto3";
package ping;
option go_package = "protos";
service Pinger {
rpc Ping(PingRequest) returns (PingResponse) {}
rpc PingStream (PingRequest) returns (stream PingResponse) {}
}
message PingRequest {
int32 count = 1;
}
message PingResponse {
bytes payload = 1;
}
Using bufconn
To easily spin up a server and a client that talk over a bufconn
buffer,
I created a function that is called by the test file:
func server(ctx context.Context) (pb.PingerClient, func()) {
buffer := 1024 * 1024
listener := bufconn.Listen(buffer)
s := grpc.NewServer()
pb.RegisterPingerServer(s, &Pinger{})
go func() {
if err := s.Serve(listener); err != nil {
panic(err)
}
}()
conn, _ := grpc.DialContext(ctx, "", grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) {
return listener.Dial()
}), grpc.WithInsecure(), grpc.WithBlock())
client := pb.NewPingerClient(conn)
return client, s.Stop
}
Important thing to note here are, this function:
- Spins up gRPC server using the bufconn buffer
- Creates a client that talks over the buffer using
gRPC.WithContextDialer
- Returns function to terminate the listener and the server
Using the bufconn
client
With the above function, the rest are easy:
ctx := context.Background()
assert := assert.New(t)
client, closer := server(ctx)
defer closer()
out, err := client.Ping(ctx, tc.in)
assert.Nil(err)
if tc.expected.err == nil {
assert.Nil(err)
assert.Equal(tc.expected.out, out)
} else {
assert.Nil(out)
assert.Equal(tc.expected.err, err)
}
That is the meat of my test. It uses the server
function, after making sure
the closer
is defered, it uses the client to make a request. The response
is asserted against expectation. This all happens against a gRPC server,
except it uses bufconn
instead of a TCP socket.
Source code for the full setup is available here.
Conclusion
bufconn
allows you to spin up a gRPC server that talk over an in-memory
buffer. Tests are fast and reliable, and allows you to easily setup
integration style tests that uses a full server.