Protobuf and gRPC rules for Bazel

Bazel rules for building Protocol Buffers ± gRPC code and libraries from proto_library targets

Announcements 📣

2021/02/21 - Version 3.0.0

Version 3.0.0 has been released with updated Protobuf and gRPC versions and a number of major improvements. For some languages this may not be a drop-in replacement and it may be necessary to update your WORKSPACE file due to changes in dependencies; please see the above linked release notes for details or the language specific rules pages. If you discover any problems with the new release, please open a new issue, create a discussion or otherwise check in the #proto channel on Bazel Slack.

2021/02/09 - Bazel Version 4

Version 4.0 of Bazel has been released recently and is supported without update for most languages. Changes for full Bazel 4.0 compatibility and updated dependencies are in progress.

Contents

• Overview
• Installation
• Rules
  • Android
  • C
  • C++
  • C#
  • D
  • Documentation
  • Go
  • grpc-gateway
  • Java
  • JavaScript
  • Objective-C
  • PHP
  • Python
  • Ruby
  • Rust
  • Scala
  • Swift
• Example Usage
• Developers
  • Code Layout
  • Rule Generation
  • Developing Custom Plugins
• License
• Contributing

Overview

These rules provide Protocol Buffers (Protobuf) and gRPC rules for a range of languages and services.

Each supported language ({lang} below) is generally split into four rule flavours:

• {lang}_proto_compile: Provides generated files from the Protobuf protoc plugin for the language. e.g for C++ this provides the generated *.pb.cc and *.pb.h files.

• {lang}_proto_library: Provides a language-specific library from the generated Protobuf protoc plugin outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc and *pb.h files, with the Protobuf library linked. For languages that do not have a 'library' concept, this rule may not exist.

• {lang}_grpc_compile: Provides generated files from both the Protobuf and gRPC protoc plugins for the language. e.g for C++ this provides the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files.

• {lang}_grpc_library: Provides a language-specific library from the generated Protobuf and gRPC protoc plugins outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files, with the Protobuf and gRPC libraries linked. For languages that do not have a 'library' concept, this rule may not exist.

Therefore, if you are solely interested in the generated source code files, use the {lang}_{proto|grpc}_compile rules. Otherwise, if you want a ready-to-go library, use the {lang}_{proto|grpc}_library rules.

Installation

Add rules_proto_grpc to your WORKSPACE file and then look at the language specific examples linked below:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_proto_grpc",
    urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/3.0.0.tar.gz"],
    sha256 = "f9b50e672870fe5d60b8b2f3cd1731ceb89a9edef3513d81ba7f7c0d2991b51f",
    strip_prefix = "rules_proto_grpc-3.0.0",
)

load("@rules_proto_grpc//:repositories.bzl", "rules_proto_grpc_toolchains", "rules_proto_grpc_repos")
rules_proto_grpc_toolchains()
rules_proto_grpc_repos()

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()

It is recommended that you use the tagged releases for stable rules. Master is intended to be 'ready-to-use', but may be unstable at certain periods. To be notified of new releases, you can use GitHub's 'Watch Releases Only' on the repository.

Important: You will also need to follow instructions in the language-specific README.md for additional workspace dependencies that may be required.

Rules

Language	Rule	Description
Android	android_proto_compile	Generates an Android protobuf .jar file (example)
Android	android_grpc_compile	Generates Android protobuf and gRPC .jar files (example)
Android	android_proto_library	Generates an Android protobuf library using android_library from rules_android (example)
Android	android_grpc_library	Generates Android protobuf and gRPC library using android_library from rules_android (example)
C	c_proto_compile	Generates C protobuf .h & .c files (example)
C	c_proto_library	Generates a C protobuf library using cc_library, with dependencies linked (example)
C++	cpp_proto_compile	Generates C++ protobuf .h & .cc files (example)
C++	cpp_grpc_compile	Generates C++ protobuf and gRPC .h & .cc files (example)
C++	cpp_proto_library	Generates a C++ protobuf library using cc_library, with dependencies linked (example)
C++	cpp_grpc_library	Generates a C++ protobuf and gRPC library using cc_library, with dependencies linked (example)
C#	csharp_proto_compile	Generates C# protobuf .cs files (example)
C#	csharp_grpc_compile	Generates C# protobuf and gRPC .cs files (example)
C#	csharp_proto_library	Generates a C# protobuf library using csharp_library from rules_dotnet. Note that the library name must end in .dll (example)
C#	csharp_grpc_library	Generates a C# protobuf and gRPC library using csharp_library from rules_dotnet. Note that the library name must end in .dll (example)
D	d_proto_compile	Generates D protobuf .d files (example)
D	d_proto_library	Generates a D protobuf library using d_library from rules_d (example)
Documentation	doc_docbook_compile	Generates DocBook .xml documentation file (example)
Documentation	doc_html_compile	Generates .html documentation file (example)
Documentation	doc_json_compile	Generates .json documentation file (example)
Documentation	doc_markdown_compile	Generates Markdown .md documentation file (example)
Go	go_proto_compile	Generates Go protobuf .go files (example)
Go	go_grpc_compile	Generates Go protobuf and gRPC .go files (example)
Go	go_proto_library	Generates a Go protobuf library using go_library from rules_go (example)
Go	go_grpc_library	Generates a Go protobuf and gRPC library using go_library from rules_go (example)
grpc-gateway	gateway_grpc_compile	Generates grpc-gateway .go files (example)
grpc-gateway	gateway_openapiv2_compile	Generates grpc-gateway OpenAPI v2 .json files (example)
grpc-gateway	gateway_grpc_library	Generates grpc-gateway library files (example)
Java	java_proto_compile	Generates a Java protobuf srcjar file (example)
Java	java_grpc_compile	Generates a Java protobuf and gRPC srcjar file (example)
Java	java_proto_library	Generates a Java protobuf library using java_library (example)
Java	java_grpc_library	Generates a Java protobuf and gRPC library using java_library (example)
JavaScript	js_proto_compile	Generates JavaScript protobuf .js and .d.ts files (example)
JavaScript	js_grpc_node_compile	Generates JavaScript protobuf and gRPC-node .js and .d.ts files (example)
JavaScript	js_grpc_web_compile	Generates JavaScript protobuf and gRPC-Web .js and .d.ts files (example)
JavaScript	js_proto_library	Generates a JavaScript protobuf library using js_library from rules_nodejs (example)
JavaScript	js_grpc_node_library	Generates a Node.js protobuf + gRPC-node library using js_library from rules_nodejs (example)
JavaScript	js_grpc_web_library	Generates a JavaScript protobuf + gRPC-Web library using js_library from rules_nodejs (example)
Objective-C	objc_proto_compile	Generates Objective-C protobuf .m & .h files (example)
Objective-C	objc_grpc_compile	Generates Objective-C protobuf and gRPC .m & .h files (example)
Objective-C	objc_proto_library	Generates an Objective-C protobuf library using objc_library (example)
Objective-C	objc_grpc_library	Generates an Objective-C protobuf and gRPC library using objc_library (example)
PHP	php_proto_compile	Generates PHP protobuf .php files (example)
PHP	php_grpc_compile	Generates PHP protobuf and gRPC .php files (example)
Python	python_proto_compile	Generates Python protobuf .py files (example)
Python	python_grpc_compile	Generates Python protobuf and gRPC .py files (example)
Python	python_grpclib_compile	Generates Python protobuf and grpclib .py files (supports Python 3 only) (example)
Python	python_proto_library	Generates a Python protobuf library using py_library from rules_python (example)
Python	python_grpc_library	Generates a Python protobuf and gRPC library using py_library from rules_python (example)
Python	python_grpclib_library	Generates a Python protobuf and grpclib library using py_library from rules_python (supports Python 3 only) (example)
Ruby	ruby_proto_compile	Generates Ruby protobuf .rb files (example)
Ruby	ruby_grpc_compile	Generates Ruby protobuf and gRPC .rb files (example)
Ruby	ruby_proto_library	Generates a Ruby protobuf library using ruby_library from rules_ruby (example)
Ruby	ruby_grpc_library	Generates a Ruby protobuf and gRPC library using ruby_library from rules_ruby (example)
Rust	rust_proto_compile	Generates Rust protobuf .rs files (example)
Rust	rust_grpc_compile	Generates Rust protobuf and gRPC .rs files (example)
Rust	rust_proto_library	Generates a Rust protobuf library using rust_library from rules_rust (example)
Rust	rust_grpc_library	Generates a Rust protobuf and gRPC library using rust_library from rules_rust (example)
Scala	scala_proto_compile	Generates a Scala protobuf .jar file (example)
Scala	scala_grpc_compile	Generates Scala protobuf and gRPC .jar file (example)
Scala	scala_proto_library	Generates a Scala protobuf library using scala_library from rules_scala (example)
Scala	scala_grpc_library	Generates a Scala protobuf and gRPC library using scala_library from rules_scala (example)
Swift	swift_proto_compile	Generates Swift protobuf .swift files (example)
Swift	swift_grpc_compile	Generates Swift protobuf and gRPC .swift files (example)
Swift	swift_proto_library	Generates a Swift protobuf library using swift_library from rules_swift (example)
Swift	swift_grpc_library	Generates a Swift protobuf and gRPC library using swift_library from rules_swift (example)

Example Usage

These steps walk through the actions required to go from a raw .proto file to a C++ library. Other languages will have a similar high-level layout.

Step 1: Write a Protocol Buffer .proto file (example: thing.proto):

syntax = "proto3";

package example;

import "google/protobuf/any.proto";

message Thing {
    string name = 1;
    google.protobuf.Any payload = 2;
}

Step 2: Write a BAZEL.build file with a proto_library target:

proto_library(
    name = "thing_proto",
    srcs = ["thing.proto"],
    deps = ["@com_google_protobuf//:any_proto"],
)

In this example we have a dependency on a well-known type any.proto, hence the proto_library to proto_library dependency ("@com_google_protobuf//:any_proto")

Step 3: Add a cpp_proto_compile target

NOTE: In this example thing.proto does not include service definitions (gRPC). For protos with services, use the cpp_grpc_compile rule instead.

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_compile")

cpp_proto_compile(
    name = "cpp_thing_proto",
    protos = [":thing_proto"],
)

But wait, before we can build this, we need to load the dependencies necessary for this rule (see cpp/README.md):

Step 4: Load the workspace macro corresponding to the build rule.

# WORKSPACE
load("@rules_proto_grpc//cpp:repositories.bzl", "cpp_repos")

cpp_repos()

We're now ready to build the target:

Step 5: Build it!

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

If we were only interested in the generated files, the cpp_grpc_compile rule would be fine. However, for convenience we'd rather have the outputs compiled into a C++ library. To do that, let's change the rule from cpp_proto_compile to cpp_proto_library:

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_library")

cpp_proto_library(
    name = "cpp_thing_proto",
    protos = [":thing_proto"],
)

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-bin/example/proto/libcpp_thing_proto.a
  bazel-bin/example/proto/libcpp_thing_proto.so  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

This way, we can use //example/proto:cpp_thing_proto as a dependency of any other cc_library or cc_binary target as per normal.

NOTE: The cpp_proto_library target implicitly calls cpp_proto_compile, and we can access that rule's by adding _pb at the end of the target name, like bazel build //example/proto:cpp_thing_proto_pb

Developers

Code Layout

Each language {lang} has a top-level subdirectory that contains:

1. {lang}/README.md: Generated documentation for the language rules

2. {lang}/repositories.bzl: Macro functions that declare repository rule dependencies for that language

3. {lang}/{rule}.bzl: Rule implementations of the form {lang}_{kind}_{type}, where kind is one of proto|grpc and type is one of compile|library

4. {lang}/BUILD.bazel: proto_plugin() declarations for the available plugins for the language

5. example/{lang}/{rule}/: Generated WORKSPACE and BUILD.bazel demonstrating standalone usage of the rules

6. {lang}/example/routeguide/: Example routeguide example implementation, if possible

Rule Generation

To help maintain consistency of the rule implementations and documentation, all of the rule implementations are generated by the tool //tools/rulegen. Changes in the main README.md should be placed in tools/rulegen/README.header.md or tools/rulegen/README.footer.md. Changes to generated rules should be put in the source files (example: tools/rulegen/java.go).

Developing Custom Plugins

Generally, follow the pattern seen in the multiple language examples in this repository. The basic idea is:

1. Load the plugin rule: load("@rules_proto_grpc//:defs.bzl", "proto_plugin")
2. Define the rule, giving it a name, options (not mandatory), tool and outputs. tool is a label that refers to the binary executable for the plugin itself
3. Choose your output type (pick one!):
   • outputs: A list of strings patterns that predicts the pattern of files generated by the plugin. For plugins that produce one output file per input proto file
   • out: The name of a single output file generated by the plugin
   • output_directory: Set to true if your plugin generates files in a non-predictable way. e.g. if the output paths depend on the service names within the files
4. Create a compilation rule and aspect using the following template:

load("@rules_proto//proto:defs.bzl", "ProtoInfo")
load(
    "@rules_proto_grpc//:defs.bzl",
    "ProtoLibraryAspectNodeInfo",
    "ProtoPluginInfo",
    "proto_compile_aspect_attrs",
    "proto_compile_aspect_impl",
    "proto_compile_attrs",
    "proto_compile_impl",
)

# Create aspect
example_aspect = aspect(
    implementation = proto_compile_aspect_impl,
    provides = [ProtoLibraryAspectNodeInfo],
    attr_aspects = ["deps"],
    attrs = dict(
        proto_compile_aspect_attrs,
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
        _prefix = attr.string(
            doc = "String used to disambiguate aspects when generating outputs",
            default = "example_aspect",
        ),
    ),
    toolchains = ["@rules_proto_grpc//protobuf:toolchain_type"],
)

# Create compile rule to apply aspect
_rule = rule(
    implementation = proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        protos = attr.label_list(
            mandatory = False,  # TODO: set to true in 4.0.0 when deps removed below
            providers = [ProtoInfo],
            doc = "List of labels that provide a ProtoInfo (such as rules_proto proto_library)",
        ),
        deps = attr.label_list(
            mandatory = False,
            providers = [ProtoInfo, ProtoLibraryAspectNodeInfo],
            aspects = [example_aspect],
            doc = "DEPRECATED: Use protos attr",
        ),
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
    ),
    toolchains = [str(Label("//protobuf:toolchain_type"))],
)

# Create macro for converting attrs and passing to compile
def example_compile(**kwargs):
    _rule(
        verbose_string = "{}".format(kwargs.get("verbose", 0)),
        **kwargs
    )

License

This project is derived from stackb/rules_proto under the Apache 2.0 license and this project therefore maintains the terms of that license

Contributing

Contributions are very welcome. Please see CONTRIBUTING.md for further details.