Testing Go gRPC Server Using an in-memory Buffer with `bufconn`
Jan 17, 2020
3 minute read

    It can be cumbersome to setup a testing environment targeting a live server to implement full API testing against your gRPC server. Even spinning up a server from your test file can lead to unintended consequences that require you to allocate a TCP port (parallel runs, multiple runs under same CI server).

    bufconn is a package which provides a Listener object that implements net.Conn. We can substitute this listener in a gRPC server - allowing us to spin up a server that acts as a full-fledged server that can be used for testing that talks over an in-memory buffer instead of a real port.

    Goals

    • Spin up a gRPC server using an in-memory buffer
    • Use the server in a standard testing.Test test function

    Setup

    Same as my other post, I created a simple gRPC service that implements Ping and StreamPring. Calling Ping replies with (you guessed it!) a pong message. StreamPing is similar, but you can specify how many times it pong’s back.

    syntax = "proto3";
    
    package ping;
    
    option go_package = "protos";
    
    service Pinger {
        rpc Ping(PingRequest) returns (PingResponse) {}
        rpc PingStream (PingRequest) returns (stream PingResponse) {}
    }
    
    message PingRequest {
        int32 count = 1;
    }
    
    
    message PingResponse {
        bytes payload = 1;
    }
    

    Using bufconn

    To easily spin up a server and a client that talk over a bufconn buffer, I created a function that is called by the test file:

    func server(ctx context.Context) (pb.PingerClient, func()) {
    	buffer := 1024 * 1024
    	listener := bufconn.Listen(buffer)
    
    	s := grpc.NewServer()
    	pb.RegisterPingerServer(s, &Pinger{})
    	go func() {
    		if err := s.Serve(listener); err != nil {
    			panic(err)
    		}
    	}()
    
    	conn, _ := grpc.DialContext(ctx, "", grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) {
    		return listener.Dial()
    	}), grpc.WithInsecure(), grpc.WithBlock())
    
    	client := pb.NewPingerClient(conn)
    
    	return client, s.Stop
    }
    

    Important thing to note here are, this function:

    • Spins up gRPC server using the bufconn buffer
    • Creates a client that talks over the buffer using gRPC.WithContextDialer
    • Returns function to terminate the listener and the server

    Using the bufconn client

    With the above function, the rest are easy:

    ctx := context.Background()
    assert := assert.New(t)
    
    client, closer := server(ctx)
    defer closer()
    
    out, err := client.Ping(ctx, tc.in)
    assert.Nil(err)
    
    if tc.expected.err == nil {
        assert.Nil(err)
        assert.Equal(tc.expected.out, out)
    } else {
        assert.Nil(out)
        assert.Equal(tc.expected.err, err)
    }
    

    That is the meat of my test. It uses the server function, after making sure the closer is defered, it uses the client to make a request. The response is asserted against expectation. This all happens against a gRPC server, except it uses bufconn instead of a TCP socket.

    Source code for the full setup is available here.

    Conclusion

    bufconn allows you to spin up a gRPC server that talk over an in-memory buffer. Tests are fast and reliable, and allows you to easily setup integration style tests that uses a full server.



    comments powered by Disqus