Fastpb

A faster Protobuf serializer & deserializer.

Attention

• only proto3 is supported now.
• known-types(any, api, duration…) is not supported now.

Install

go install github.com/cloudwego/fastpb/protoc-gen-fastpb@latest

Usage

Refer to examples, use in two steps:

1. use fastpb to generate code. (refer here)
2. use fastpb API to marshal/unmarshal. (refer here)

Step 1: Generate Code

Using the command line tool to generate code:

protoc --go_out=. \
  --fastpb_out=. \
  ${your_idl}.proto

or

protoc --go_opt=paths=source_relative --go_out=. \
  --fastpb_opt=paths=source_relative --fastpb_out=. \
   ${your_idl}.proto

Step 2: Codec Message

Encoding and Decoding must use fastpb’s API, shown as demo:

package examples

import (
    "github.com/cloudwego/fastpb"
)

// Marshal .
func Marshal(msg fastpb.Writer) []byte {
    // TODO: buffer can be reused.
    buf := make([]byte, msg.Size())

    msg.FastWrite(buf)
    return buf
}

// Unmarshal .
func Unmarshal(buf []byte, msg fastpb.Reader) error {
    _, err := fastpb.ReadMessage(buf, int8(fastpb.SkipTypeCheck), msg)
    return err
}

Performance

goos: linux
goarch: amd64
pkg: github.com/cloudwego/fastpb/benchmark
cpu: Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz

Benchmarks have compared golang/protobuf (referred to as _golang) and fastpb here.

Marshal

Benchmark_marshal_number_golang-48	375.2 ns/op	~	96 B/op	~	1 allocs/op
Benchmark_marshal_number_fastpb-48	145.7 ns/op	-61.17%	0 B/op	-100.00%	0 allocs/op
Benchmark_marshal_string_golang-48	1010 ns/op	~	2304 B/op	~	1 allocs/op
Benchmark_marshal_string_fastpb-48	58.57 ns/op	-94.20%	0 B/op	-100.00%	0 allocs/op
Benchmark_marshal_list_golang-48	8788 ns/op	~	18432 B/op	~	1 allocs/op
Benchmark_marshal_list_fastpb-48	3430 ns/op	-60.97%	0 B/op	-100.00%	0 allocs/op
Benchmark_marshal_map_golang-48	43497 ns/op	~	21680 B/op	~	393 allocs/op
Benchmark_marshal_map_fastpb-48	5951 ns/op	-86.32%	0 B/op	-100.00%	0 allocs/op

Unmarshal

Benchmark_unmarshal_number_golang-48	497.1 ns/op	~	144 B/op	1 allocs/op
Benchmark_unmarshal_number_fastpb-48	431.6 ns/op	-13.18%	144 B/op	1 allocs/op
Benchmark_unmarshal_string_golang-48	939.7 ns/op	~	2128 B/op	3 allocs/op
Benchmark_unmarshal_string_fastpb-48	668.4 ns/op	-28.87%	2128 B/op	3 allocs/op
Benchmark_unmarshal_list_golang-48	12527 ns/op	~	20296 B/op	99 allocs/op
Benchmark_unmarshal_list_fastpb-48	12593 ns/op	+0.53%	20296 B/op	99 allocs/op
Benchmark_unmarshal_map_golang-48	49868 ns/op	~	24226 B/op	426 allocs/op
Benchmark_unmarshal_map_fastpb-48	21213 ns/op	-57.46%	21467 B/op	61 allocs/op