If you are using Bazel and one of the dependencies you need is not in the Bazel Central Registry, but you have the source code, you can still integrate that dependency into Bazel as a module.
The source code can be locally (inside your project) or in any remote location you can access.
I’ll provide an example of integrating Boost.Container version 1.86.0, which at the time of writing is not present in the Bazel Central Registry.
Jump to the end if you want to skip the reading and see the final result.
The concept I’m using is present in other package managers, too, and it’s called overriding. I can declare I want to use a Bazel module, and then I can override Bazel’s behavior when it wants to fetch that module. By default, Bazel searches in its central registry when it sees a dependency declared in MODULE.bazel:
bazel_dep(name = "boost.container", version="1.86.0")
I will tell Bazel to stop looking in its registry and instead use the location I provide for the source code.
There are several ways to do this depending on where the source code is located:
-
- archive_override: Uses a URL to an archive, downloads it, and extracts the files from it.
- git_override: Fetches code from a remote git repository.
- local_path_override: Uses a directory on the local disk.
I chose archive_override because the code is on a remote server and, as opposed to git_override, the module’s version is visible inside the URL, thus making it clearer.
The test application
Currently, the test application below uses Boost.Container version 1.83.0, which is present inside Bazel Registry.
Run bazel run //:app and everything works.
# MODULE.bazel bazel_dep(name = "boost.container", version="1.83.0")
# BUILD.bazel
load("@rules_cc//cc:defs.bzl", "cc_binary")
cc_binary(
name = "app",
srcs= [
"main.cpp",
],
deps = [
"@boost.container",
],
)
// main.cpp
#include <boost/container/vector.hpp>
#include <cassert>
boost::container::vector<int> f() { return {1, 2, 3}; }
int main()
{
auto numbers = f();
assert(numbers.size() == 3);
}
I want to use Boost.Container 1.86.0, which is not yet inside the central registry.
Remove version
The first step is to remove the version from bazel_dep because it will no longer have any effect once I tell Bazel where to fetch the source code from.
ERROR: Error computing the main repository mapping: bad bazel_dep on module ‘boost.container’ with no version. Did you forget to specify a version, or a non-registry override?
archive_override
Now I need to tell Bazel where to get the source code from:
archive_override(
module_name = "boost.container",
urls = ["https://github.com/boostorg/container/archive/refs/tags/boost-1.86.0.zip"],
strip_prefix = "container-boost-1.86.0",
integrity = "sha256-InQd64oqlm3cqCbghCbck7dl+z+FdRT9HNA/2VA2u/A=",
)
ERROR: Error computing the main repository mapping: MODULE.bazel expected but not found at /home/user/.cache/bazel/_bazel_user/…/external/boost.container~/MODULE.bazel: /home/user/.cache/bazel/_bazel_user/…/external/boost.container~/MODULE.bazel (No such file or directory)
The problem is that no Bazel module is declared inside the source code. The source code that Bazel fetched from the archive is not a Bazel module.
There are two ways to dynamically declare a Bazel module inside the source code:
-
- patches: List of files to add to the source code.
- patch_cmds: Bash commands to run on the source code.
I chose patch_cmds because I prefer to pollute only the MODULE.bazel file and not to add other files inside my repository.
archive_override(
module_name = "boost.container",
patch_cmds = [
"echo '" + boost_container_module + "' > MODULE.bazel",
"echo '" + boost_container_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/container/archive/refs/tags/boost-1.86.0.zip"],
strip_prefix = "container-boost-1.86.0",
integrity = "sha256-InQd64oqlm3cqCbghCbck7dl+z+FdRT9HNA/2VA2u/A=",
)
The two patch commands will write the files that will create a Bazel module of the Boost.Container library.
MODULE.bazel
I set the name of the module and its dependencies. The dependencies will follow the same pattern of overriding that I am using for Boost.Container.
boost_container_module = """module(
name = "boost.container",
)
bazel_dep(name = "boost.assert")
bazel_dep(name = "boost.config")
bazel_dep(name = "boost.move")
bazel_dep(name = "boost.intrusive")
"""
BUILD.bazel
The library target needs to be created like any other library does.
boost_container_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.container",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
deps = [
"@boost.assert",
"@boost.config",
"@boost.move",
"@boost.intrusive",
],
)
"""
Module dependencies
I must use the same approach for all the module’s dependencies.
The first dependency is Boost.Assert.
boost_assert_module = """module(
name = "boost.assert",
)"""
boost_assert_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.assert",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
)
"""
archive_override(
module_name = "boost.assert",
patch_cmds = [
"echo '" + boost_assert_module + "' > MODULE.bazel",
"echo '" + boost_assert_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/assert/archive/refs/tags/boost-1.86.0.tar.gz"],
strip_prefix = "assert-boost-1.86.0",
integrity = "sha256-pi7CB0zDid2jJZXsep9K9o5ljHDtnBrDK0Z6SLF2EW0=",
)
Same for the other ones.
Final result
It’s verbose and repetitive.
Some function(s) can be created, but I would need a bzl file because functions may not be defined in MODULE.bazel files.
The module and build contents can be extracted to files, and thus, patches will be used instead of patch_cmds.
It’s all about the context. For the packages that I needed, I knew this was a temporary solution until they were present in the registry, so I preferred to keep everything inside MODULE.bazel. When the packages were published in the registry, I just cleaned up MODULE.bazel and I was done. If I were to use this overriding approach permanently, I would consider implementing in a cleaner way.
# MODULE.bazel
bazel_dep(name = "boost.container")
# boost.container bazel module setup
boost_container_module = """module(
name = "boost.container",
)
bazel_dep(name = "boost.assert")
bazel_dep(name = "boost.config")
bazel_dep(name = "boost.move")
bazel_dep(name = "boost.intrusive")
"""
boost_container_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.container",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
deps = [
"@boost.assert",
"@boost.config",
"@boost.move",
"@boost.intrusive",
],
)
"""
archive_override(
module_name = "boost.container",
patch_cmds = [
"echo '" + boost_container_module + "' > MODULE.bazel",
"echo '" + boost_container_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/container/archive/refs/tags/boost-1.86.0.zip"],
strip_prefix = "container-boost-1.86.0",
integrity = "sha256-InQd64oqlm3cqCbghCbck7dl+z+FdRT9HNA/2VA2u/A=",
)
# boost.assert bazel module setup
boost_assert_module = """module(
name = "boost.assert",
)"""
boost_assert_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.assert",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
)
"""
archive_override(
module_name = "boost.assert",
patch_cmds = [
"echo '" + boost_assert_module + "' > MODULE.bazel",
"echo '" + boost_assert_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/assert/archive/refs/tags/boost-1.86.0.tar.gz"],
strip_prefix = "assert-boost-1.86.0",
integrity = "sha256-pi7CB0zDid2jJZXsep9K9o5ljHDtnBrDK0Z6SLF2EW0=",
)
# boost.config bazel module setup
boost_config_module = """module(
name = "boost.config",
)"""
boost_config_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.config",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
)
"""
archive_override(
module_name = "boost.config",
patch_cmds = [
"echo '" + boost_config_module + "' > MODULE.bazel",
"echo '" + boost_config_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/config/archive/refs/tags/boost-1.86.0.tar.gz"],
strip_prefix = "config-boost-1.86.0",
integrity = "sha256-v+0XvsA4uOlPXxjROvnnIAVI+4Vfoz8C5B0iD1z1tBo=",
)
# boost.move bazel module setup
boost_move_module = """module(
name = "boost.move",
)"""
boost_move_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.move",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
)
"""
archive_override(
module_name = "boost.move",
patch_cmds = [
"echo '" + boost_move_module + "' > MODULE.bazel",
"echo '" + boost_move_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/move/archive/refs/tags/boost-1.86.0.tar.gz"],
strip_prefix = "move-boost-1.86.0",
integrity = "sha256-YTsZGBKGXLdw87EuhAAR6ce/r7OKUIari/K6t4yCOBw=",
)
# boost.intrusive bazel module setup
boost_intrusive_module = """module(
name = "boost.intrusive",
)"""
boost_intrusive_build = """package(default_visibility = ["//visibility:public"])
cc_library(
name = "boost.intrusive",
includes = ["include"],
hdrs = glob(["include/**/*.hpp"]),
)
"""
archive_override(
module_name = "boost.intrusive",
patch_cmds = [
"echo '" + boost_intrusive_module + "' > MODULE.bazel",
"echo '" + boost_intrusive_build + "' > BUILD.bazel",
],
urls = ["https://github.com/boostorg/intrusive/archive/refs/tags/boost-1.86.0.tar.gz"],
strip_prefix = "intrusive-boost-1.86.0",
integrity = "sha256-cAtAd9Rv0+H/NY+3ez9iDGZAx4eSAIo/nWTmGiMNKkc=",
)
$ bazel run //:app
INFO: Analyzed target //:app (92 packages loaded, 756 targets configured).
INFO: Found 1 target…
Target //:app up-to-date:
bazel-bin/app
INFO: Elapsed time: 3.868s, Critical Path: 0.24s
INFO: 7 processes: 5 internal, 2 processwrapper-sandbox.
INFO: Build completed successfully, 7 total actions
INFO: Running command line: bazel-bin/app