One of the core use-cases for modern container systems is to run networked workloads, often across a group of machines deployed in a cluster. A variety of different networking models exist, but until now no networking at all was possible with runj.

Now, after this change, runj has its first networking capability! The functionality that pull request enable jails to share the IPv4 network stack with the underlying FreeBSD host, similar to the “host networking” model common for Linux containers.

Without further ado, here’s how you can use it:

$ cat <<EOF >runj.ext.json
{"network":{"ipv4":{"mode":"inherit"}}}
EOF
$ sudo ctr run \
    --runtime wtf.sbk.runj.v1 \
    --rm \
    --tty \
    --runtime-config-path $(pwd)/runj.ext.json \
    public.ecr.aws/samuelkarp/freebsd:13.1-RELEASE \
    my-container \
    sh

Once inside your container, you might also need to create your own /etc/resolv.conf. Here’s a simple example using Google’s public DNS resolver:

nameserver 8.8.8.8

And you’ve got network! Similar to host networking on Linux, you can make network requests from the jail to the Internet (assuming your host has access to it), from the jail to the host, from the host to the jail, and from the Internet to the jail (again, assuming you’ve configured your host to accept those connections).

So what did we do? First we created a small JSON file called runj.ext.json. This file had the following contents:

{
    "network": {
        "ipv4": {
            "mode":"inherit"
        }
    }
}

This file is a new file that runj accepts (and can be passed through the containerd shim) modeling FreeBSD-specific configuration settings for containers. It’s really intended to be a subset of the OCI bundle config.json which contains the rest of the container configuration (and is normally generated by a container manager like containerd). runj can also accept this content directly as part of the config.json embedded inside a "freebsd" field.

In the file, we’ve set the IPv4 network mode to “inherit”. This is the same setting that can be configured in jail.conf (or on the command line) using jail(8); it’s called ip4 there.

Once we had the file, we invoked ctr to run a container and passed it the argument --runtime-config-path $(pwd)/runj.ext.json. ctr is the development- and debugging-focused CLI for containerd and makes it easy to experiment with runj and containerd together. ctr communicates with containerd using protobuf messages, containerd communicates with the shim using protobuf messages, and the shim ends up invoking the underlying runtime (runj in this case). Under the hood, ctr creates a runtimeoptions.Options protobuf message and packs that inside a types.Any field in the protobuf message sent to containerd. containerd then treats this inner message as a pass-through and includes it when it calls the shim without attempting to interpret the value. The shim is then able to deserialize that message (checking to make sure it actually is a runtimeoptions.Options) and interpret it how it pleases. The runtimeoptions.Options type is fairly generic and is usually used to configure the shim in a somewhat global sense (in other words, all copies of the shim would generally receive the same configuration), but we’re slightly abusing it here to configure the actual container/jail instead of the shim itself. (In the future, the runj shim might define its own protobuf type and accept that rather than abusing this one.)

When the shim is invoked and deserializes the message, it copies the file into the bundle directory prepared by containerd for the actual container, and then invokes runj. runj starts up, reads both the config.json and the runj.ext.json, merges them together, and follows the directions specified in those files.

This mechanism — a new "freebsd" field, a separate runj.ext.json file, and a simple way to use ctr to hook it all together — provides a good way to experiment and play around with the FreeBSD technologies that should really be available for running jails in the container ecosystem. The ultimate goal here is to come up with reasonable proposals to extend the official OCI runtime specification. One of the founding values of the OCI is “rough consensus and working code”; this mechanism provides a real way to get there. And with this in place, I’d like to welcome you to work together with me on what an OCI-compatible FreeBSD jail runtime should really be.