runc network

Today we’ll be continuing our containerization blog series with a discussion about runC, a tool for launching containers according to Open Container Initiative (OCI) specifications. The initiative’s mission is to develop a single standard for containerization technology and is supported by such companies as Facebook, Google, Microsoft, Oracle, EMC, and Docker. The OCI Runtime Specifications were published in the summer of 2015.

Modern containerization tools already implement runC. The latest versions of Docker (starting with version 1.11) have been made according to OCI specifications and are built on runC. The libcontainer library, which is essentially a part of runC, has replaced LXC in Docker as of version 1.8.

In this article, we’ll show you how you can create and manage containers using runC.

Installation

The installation process described in this post is for Ubuntu 16.04. This operating system contains the latest stable version of Go (1.6) in the official repository, which can be installed by running:

$ sudo apt-get install golang-go

The Ubuntu 16.04 repository also includes runC, but not the latest version. The latest version (1.0.0) needs to be compiled from the source code. To do this, you first have to install the necessary dependencies:

sudo apt-get install build-essential make libseccomp-dev

That’s it. Now we can start compiling runC:

$ git clone https://github.com/opencontainers/runc
$ cd runc
$ make
$ sudo make install

runC will be installed to the directory /usr/local/bin/runc.

Creating Our First Container

Now we’re ready to create our first container.

The first thing we have to do is create a separate directory for the new container and then a rootfs directory inside of that.

$ mkdir /mycontainer
$ cd /mycontainer
$ mkdir rootfs

We’ll start with a simple example. We’ll load a memcached Docker image, convert it to a *.tar archive, and extract it to the rootfs directory:

$ docker export $(docker create memcached) | tar -C rootfs -xvf -

Afterwards, all of the system files for our future container will be found in the rootfs directory.

$ ls rootfs
bin   dev            etc   lib    media  opt   root  sbin  sys  usr
boot  entrypoint.sh  home  lib64  mnt    proc  run   srv   tmp  var

Now we can launch and manage our container without referring to Docker. We’ll make a configuration where we can write the settings for our new container.

$ sudo runc spec

Afterwards, we’ll find a new file in the rootfs directory: config.json. Now we’re ready to launch our new container. We run

$ sudo runc run mycontainer

The command shell will be launched in the new container.

This was an extremely simple example, where the container was launched with automatically generated settings. For custom container settings, we’ll have to edit the aforementioned config.json file manually. Let’s take a closer look at its structure.

The config.json Configuration File

In the first part of the configuration file, we have the container’s general settings: OCI version, operating system and architecture, and terminal settings:

"ociVersion": "1.0.0-rc1",
        "platform": {
                "os": "linux",
                "arch": "amd64"
        },
"process": {
                "terminal": true,
                "user": {
                        "uid": 0,
                        "gid": 0
                },
                "args": [
                        "sh"
                ],
                "env": [
                        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                        "TERM=xterm"
                ],

The next section contains settings for the directory where the container will run from:

 
"cwd": "/",
                "capabilities": [
                        "CAP_AUDIT_WRITE",
                        "CAP_KILL",
                        "CAP_NET_BIND_SERVICE"
                ],

CWD stands for current working directory. In our case, this is “/”. Next we have capabilities, which are permissions for executable files in given subsystems that lack root permissions. CAP_AUDIT_WRITE lets them write to the audit log, CP_KILL lets signals be sent to processes, and CAP_NET_BIND_SERVICE allows binding sockets to privileged ports (i.e. ports below 1024).

The next section is rlimits:

 
"rlimits": [
                        {
                                "type": "RLIMIT_NOFILE",
                                "hard": 1024,
                                "soft": 1024
                        }
                ],
                "noNewPrivileges": true
        },

Here we set resource limits for the container, namely the maximum number of files that can be open at a given time (RLIMIT_NOFILE), which is set to 1024.

Next is a description of the root file system:

 
"root": {
                "path": "rootfs",
                "readonly": true
}

Under mounts we have a description of the directories mounted in the container:

 
"mounts": [
    {
        "destination": "/tmp",
        "type": "tmpfs",
        "source": "tmpfs",
        "options": ["nosuid","strictatime","mode=755","size=65536k"]
    },
    {
        "destination": "/data",
        "type": "bind",
        "source": "/volumes/testing",
        "options": ["rbind","rw"]
    }
]

These are only the main sections of the config.json file. We’ll look at a few other sections below. A more detailed look at this file can be found here.

Hooks

Another interesting feature of runC is the ability to configure hooks: we can write specific actions to the configuration file, which will be run before launching a user process in the container (prestart), after launching a user process (poststart), and after it stops (poststop).

We’ll look at a few examples to better understand why we need hooks and how to make the appropriate changes to the configuration file. Let’s imagine we have to set up a network before a program is launched in a container. To do this, we’ll add a hook to the configuration file (this example comes from the official documentation):

"hooks": {
        "prestart": [
              {
                "path": "/path/to/script"
              }

Next to path, we indicate the path to the network setup program. You can find these kinds of tools on Github (see here).

Now let’s look at an example of a poststart hook:

 
"poststart": [
            {
                "path": "/usr/bin/notify-start",
                "timeout": 5
            }

When the hook activates, a script will be executed (in our example this is notify-start), which logs information on container launch events.

Poststop hooks initialize actions that occur when a user process terminates in a container. These may come in handy if we have to delete logs and session files, which are left by the container in the system, as well as the container itself. For example:

 
"poststop": [
            {
                "path": "/usr/sbin/cleanup.sh",
                "args": ["cleanup.sh", "-f"]
            }

When this hook activates, the cleanup.sh script will be launched, which performs the abovementioned actions.

Managing Containers: Basic Commands

Containers in runC can be managed with easy-to-use commands. Here is a short list:

#view a list of containers and their status
runc list
 
 
#launch a process in a container
runc start mycontainerid
 
 
#halt a process in a container
runc stop mycontainerid
 
 
#delete a container
runc delete mycontainerid

Configuring a Network

We’ve already looked at some basic operations for managing containers. Now we’ll try to configure a container’s network. This isn’t necessarily the easiest thing to do. All operations need to be run manually.

To start with, we run the following sequence of commands (taken from this article):

$ sudo brctl addbr runc0
$ sudo ip link set runc0 up
$ sudo ip addr add 192.168.10.1/24 dev runc0
$ sudo ip link add name veth-host type veth peer name veth-guest
$ sudo ip link set veth-host up
$ sudo brctl addif runc0 veth-host
$ sudo ip netns add runc
$ sudo ip link set veth-guest netns runc
$ sudo ip netns exec runc ip link set veth-guest name eth1
$ sudo ip netns exec runc ip addr add 192.168.10.101/24 dev eth1
$ sudo ip netns exec runc ip link set eth1 up
$ sudo ip netns exec runc ip route add default via 192.168.10.1

These commands speak for themself. First we create a bridge for connections between the container and interface on the main host. Then we deploy the virtual interface and add it to the bridge. Afterwards, we create a network namespace with the name runc and assign an IP address to the interface eth1.
To set up the network in the container, we should associate it with the runc namespace. We do this by making a few changes to the config.json file:

......
 
 "root": {
                "path": "rootfs",
                "readonly": false
}

In the namespaces section, we enter a path to the runc namespace:

{
     "type": "network",
     "path": "/var/run/netns/runc"
                },

That’s it. All of the necessary settings have been added. We’ll save these changes and relaunch the container.
On the main host, we run the command:

$  ping 192.168.10.101
 
PING 192.168.10.101 (192.168.10.101) 56(84) bytes of data.
64 bytes from 192.168.10.101: icmp_seq=2 ttl=64 time=0.070 ms
64 bytes from 192.168.10.101: icmp_seq=3 ttl=64 time=0.090 ms
64 bytes from 192.168.10.101: icmp_seq=4 ttl=64 time=0.106 ms
64 bytes from 192.168.10.101: icmp_seq=5 ttl=64 time=0.091 ms
64 bytes from 192.168.10.101: icmp_seq=6 ttl=64 time=0.097 ms

This output shows us that the container has received a ping from the main host.