Certificate Transparency: Go Code

This repository holds Go code related to Certificate Transparency (CT). The repository requires Go version 1.24.

• Repository Structure
• Trillian CT Personality
• Working on the Code
  • Running Codebase Checks
  • Rebuilding Generated Code

Support

• Slack: https://transparency-dev.slack.com/ (invitation)

Repository Structure

The main parts of the repository are:

• Encoding libraries:
  • asn1/ and x509/ are forks of the upstream Go encoding/asn1 and crypto/x509 libraries. We maintain separate forks of these packages because CT is intended to act as an observatory of certificates across the ecosystem; as such, we need to be able to process somewhat-malformed certificates that the stricter upstream code would (correctly) reject. Our x509 fork also includes code for working with the pre-certificates defined in RFC 6962.
  • tls holds a library for processing TLS-encoded data as described in RFC 5246.
  • x509util/ provides additional utilities for dealing with x509.Certificates.
• CT client libraries:
  • The top-level ct package (in .) holds types and utilities for working with CT data structures defined in RFC 6962.
  • client/ and jsonclient/ hold libraries that allow access to CT Logs via HTTP entrypoints described in section 4 of RFC 6962.
  • dnsclient/ has a library that allows access to CT Logs over DNS.
  • scanner/ holds a library for scanning the entire contents of an existing CT Log.
• CT Personality for Trillian:
  • trillian/ holds code that allows a Certificate Transparency Log to be run using a Trillian Log as its back-end – see below.
• Command line tools:
  • ./client/ctclient allows interaction with a CT Log.
  • ./ctutil/sctcheck allows SCTs (signed certificate timestamps) from a CT Log to be verified.
  • ./scanner/scanlog allows an existing CT Log to be scanned for certificates of interest; please be polite when running this tool against a Log.
  • ./x509util/certcheck allows display and verification of certificates
  • ./x509util/crlcheck allows display and verification of certificate revocation lists (CRLs).
• Other libraries related to CT:
  • ctutil/ holds utility functions for validating and verifying CT data structures.
  • loglist3/ has a library for reading v3 JSON lists of CT Logs.

Trillian CT Personality

The trillian/ subdirectory holds code and scripts for running a CT Log based on the Trillian general transparency Log, and is documented separately.

Working on the Code

Developers who want to make changes to the codebase need some additional dependencies and tools, described in the following sections.

Running Codebase Checks

The scripts/presubmit.sh script runs various tools and tests over the codebase; please ensure this script passes before sending pull requests for review.

# Install golangci-lint
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.1.6

# Run code generation, build, test and linters
./scripts/presubmit.sh

# Run build, test and linters but skip code generation
./scripts/presubmit.sh  --no-generate

# Or just run the linters alone:
golangci-lint run

Rebuilding Generated Code

Some of the CT Go code is autogenerated from other files:

• Protocol buffer message definitions are converted to .pb.go implementations.
• A mock implementation of the Trillian gRPC API (in trillian/mockclient) is created with GoMock.
• Some enums have string-conversion methods (satisfying the fmt.Stringer interface) created using the stringer tool (go get golang.org/x/tools/cmd/stringer).

Re-generating mock or protobuffer files is only needed if you’re changing the original files. The recommended way to do this is by using the Docker image used by the Cloud Build:

docker build -f ./integration/Dockerfile -t ctgo-builder .
docker run -it --mount type=bind,src="$(pwd)",target=/src ctgo-builder /bin/bash -c "cd /src; ./scripts/install_deps.sh; go generate -x ./..."

These commands first create a docker image from the Dockerfile in this repo, and then launch a container based on this image with the local directory mounted. The correct versions of the tools are determined using the go.mod file in this repo, and these tools are installed. Finally, all of the generated files are regenerated and Docker exits.

Alternatively, you can install the prerequisites locally:

• a series of tools, using go install to ensure that the versions are compatible and tested:

  cd $(go list -f '{{ .Dir }}' github.com/google/certificate-transparency-go); \
  go install github.com/golang/mock/mockgen; \
  go install google.golang.org/protobuf/proto; \
  go install google.golang.org/protobuf/cmd/protoc-gen-go; \
  go install google.golang.org/grpc/cmd/protoc-gen-go-grpc; \
  go install github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc; \
  go install golang.org/x/tools/cmd/stringer

• protoc tool: you’ll need version 3.20.1 installed, and PATH updated to include its bin/ directory.

and run the following:

go generate -x ./...  # hunts for //go:generate comments and runs them