Get started with OSv unikernels as a Docker-alternative

Container technologies and especially Docker has become the go-to standard for how to package and deploy microservices. It makes for a lightweight, flexible, easy-to-use and well-performing way to run single-purpose units of computing.

In this post I will introduce unikernels as an alternative to Docker-containers and show you how to get started with the OSv framework and how to build a simple Spring Boot based application inside of a OSv image and run it on your local machine and in the cloud on Amazon EC2.

Docker recap

Docker works by using a number of existing technologies in the Linux kernel and other open source projects. Simplified, it creates an isolated process on an existing Linux OS with its own isolated file system which can have a software stack installed which is completely independent of the host OS.

At least when running containers natively on Linux, virtualization is not involved. The main benefits of this are high performance and fast startup-times. But one of the disadvantages is that you get a lesser degree of isolation between containers than you would get between virtual machines. Containers also present a new unit that needs to be managed in addition to the existing layers in the system.

A diagram of the architecture of a system running on Docker containers (three apps on two VMs in this example):

Unikernels intro

Enter unikernels, which is basically a technology whereby you integrate a minimal operating system into your application itself. The result can be run directly on a dedicated machine, without starting a traditional operating system first. In modern solutions, this machine is usually a virtual machine, running either locally or in the cloud. The virtual machine images are reminiscent of docker images, they are lightweight disk images that can be started quickly. The image can be very compact if it does not need to take differing underlying hardware into account, since there are only so many major virtualization platforms with virtual devices that need to be supported.

A diagram of the architecture of a system running on a unikernel platform (three apps in this example):

The startup times can be very fast and can even compete with Docker containers depending on the tools and frameworks used. Performance can also be very good, but you are dependent upon the virtualization technology (hypervisor) used. Perhaps one of the main advantages is the possibility of running very lightweight VMs on IaaS (Infrastructure as a Service) such as Amazon EC2.

If you want an overview of unikernel-frameworks, see http://unikernel.org/projects. There are quite different approaches to building unikernel-based applications out there. Some are merely research projects, others are closer to being production ready. Some are tied to one specific programming language such as C++, Javascript or OCaml, others are more agnostic to what will be running on the system. The more specialized systems can have performance and size benefits, but can be more difficult to get started with if you are not already familiar with the language.

OSv intro

One of the more established frameworks out there which allows for applications created with quite a wide array of programming languages, is OSv. Its command line tools (mainly Capstan) are reminiscent of the Docker, and you can create images with Capstan in a similar fashion as with Docker. The project provides a number of base images for different platforms that can be extended based on what your application is implemented in, for instance Node.js, Clojure, Java and native (C/C++) applications.

It also provides a REST-API and a web-UI for management for managing the VMs once they are running.

The underlying technologies (hypervisors) that OSv use to run virtual machines are currently QEMU/KVM, VirtualBox, Xen and VMWare.

Installing OSv

Note: These instructions assume that you are running some Linux-variant on the host system. This will not work inside a Linux VM, but it can be easily adapted to Mac OSX. The author used Arch Linux.

The easiest was to get started with OSv is simply by installing Capstan, this can be done with (run as a normal user):

$ curl https://raw.githubusercontent.com/cloudius-systems/capstan/master/scripts/download | bash

For security reasons, I would recommend you to read the contents of the script before you run it, but it basically checks what type of system you are on and downloads the correct binary.

When the binary is downloaded, it is placed inside the folder /home/username/bin/capstan by default. From there you can copy it to a directory in your path variable for convenience. An example:

# cp /home/username/bin/capstan /usr/bin/capstan
# chmod a+x /usr/bin/capstan

Hello world with OSv

The default hello world application provided by OSv simply prints "Hello world" to console and exits. But before you can run it, you need to check that prerequisites are installed on your host machine first. The virtual machine needs somewhere to run, and that is on QEMU and KVM by default. How to install QEMU will depend on your Linux distro, but usually there is a package called qemu that can be installed using apt, pacman, yum or similar.

You can run a simple hello world example with:

$ capstan run cloudius/osv

This downloads the image "cloudius/osv" and runs it. The output looks similar to the following:

In the screenshot you can see that it prints the IP-address of the virtual machine when it starts up. Similar to Docker, ports from the virtual machine can be forwarded on the host machine with a command line argument (-f).

Packaging a Spring Boot application as an OSv image

While the hello world application demonstrates that the virtual image works, it does not show how this technology can be applied to a real world application.

For a slightly more realistic scenario, I have chosen to package a hello world application based on Spring Boot as an image. The Spring Java-framework is used in many enterprise applications and constitutes a more realistic proof of concept for the OSv framework.

The Spring Boot example

The example application I use is basically the default example used by Spring Boot itself in their "Getting Started guide".

The only class in the application looks like this and responds to HTTP requests with a plain text hello world message (which can be viewed in a web browser):

@Controller
@EnableAutoConfiguration
public class BootController {

    @RequestMapping("/")
    @ResponseBody
    String home() {
        return "Hello Spring Boot World!";
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(BootController.class, args);
    }
}

The application can be built using Maven as normal and be started as a standalone server on the host machine by simply running the resulting JAR-file. When the JAR-file is run, it starts an embedded Tomcat servlet-container that hosts the application.

Creating the image with Capstan

OSv already have an example of how to package Java-applications on their site. It basically extends an existing image called "cloudius/osv-openjdk" that they have made available which already contains an OpenJDK installation. As long as your application can be packaged as a runnable JAR-file, it can be plugged straight into this example.

So to package the application which I decided to name capstan-spring-boot, simply create the following Capstanfile in the root folder of the Maven project for the application:

base: cloudius/osv-openjdk
cmdline: /java.so -jar /capstan-spring-boot.jar
build: mvn package
files:
  /capstan-spring-boot.jar: target/capstan-spring-boot-1.0-SNAPSHOT.jar

Base specifies the base image on which to base the new image. cmdline specifies what command should be run when the virtual machine is started, in this case the JAR-file should be executed. build specifies the command that will be used to prepare the application before it is packaged into the image (in this case build it with Maven). Finally, files simply specifies files that should be copied into the image when it is being built.

Now simply run capstan to build and start the image with QEMU/KVM:

capstan run -f 8080:8080

You will see that wonderful ASCII art logo from Spring Boot being output to the console and finally a confirmation similar to "Started BootController in 13.591 seconds (JVM running for 17.874)".

The command above also forwarded port 8080 to the host, so if you open up the URL http://localhost:8080 in a web browser, you will see the hello world message.

If you tried to start the application by simply running the JAR file on the host earlier, you likely noticed that the startup time was slower, but more on that later.

Bonus entry: Export the image to VirtualBox

When running the virtual machine with Capstan locally all the details of configuring and starting the virtual machine are hidden. But once built, the image can be used independently of Capstan on all supported virtualization platforms.

To demonstrate this, the image can be imported into VirtualBox and started from the UI like any other VM. It is assumed here that VirtualBox is already installed and working on the host machine (it is open source software that can be installed using most package managers).

Since the images that Capstan creates are in a QEMU-format, they need to be converted to one of the disk formats that VirtualBox supports. Luckily, OSv has created a shell script that both converts the disk a creates a new VM in VirtualBox using its command line tools.

To use the script to package our image, a copy of it can be made and the file path modified so that it points to our image file. The line in question is the following which does the conversion of the disk image:

qemu-img convert -O vdi build/release/usr.img "$vmdir/$name.vdi"

By default, all images are stored under ~/.capstan/repository, so it can for instance be modified to:

qemu-img convert -O vdi ~/.capstan/repository/capstan-spring-boot/capstan-spring-boot.qemu "$vmdir/$name.vdi"

The rest of the script just creates the VM in VirtualBox and could be performed in the UI as well. The resulting VM after it has been booted will look similar to this:

The application can be reached on the IP shown when it is booting up, in the example above it would be http://192.168.56.101:8080. If you run into problems with insufficient memory (OOM error messages), simply adjust it in the settings window.

Running the image on Amazon EC2

While running virtual machines locally with Capstan is useful for development, it also contains tools to deploy the image to servers in the cloud, more specifically Amazon EC2 and Openstack-based cloud platforms. In this example I am going to run the image on Amazon EC2.

Before we get started a few prerequisites need to be installed, these are:

• The "Amazon EC2 API Tools" which can be downloaded as a ZIP-file and extracted, an installation guide is provided by Amazon
• AWS Command Line Interface which can be installed with pip install awscli or in my case I installed the aws-cli package with pacman (in Arch Linux).

Just like the conversion to a VirtualBox-image, OSv has provided a shell script for converting its images into an EC2 image.

But before we run the shell script, we need to convert our image into raw format:

qemu-img convert -O raw ~/.capstan/repository/capstan-spring-boot/capstan-spring-boot.qemu ~/.capstan/repository/capstan-spring-boot/capstan-spring-boot.raw

If the image was built using the OSv build scripts instead of Capstan, it could be output in this format directly.

Before we can run the script we need to set environment variables with the credentials for EC2. See the Amazon documentation for more information on how you retrieve these.

export AWS_ACCESS_KEY_ID=ENTER_ACCESS_KEY
export AWS_SECRET_ACCESS_KEY=ENTER_SECRET_ACCESS_KEY

Now we are ready to run the conversion script:

./release-ec2.sh 
--private-ami-only 
--override-image=~/.capstan/repository/capstan-spring-boot/capstan-spring-boot.raw 
--region=eu-west1

As of june 2017 this script does not work out of the box, most likely due to changes in the tools and APIs used for AWS. But as long as the image itself is uploaded as a volume, it is fairly easy to use that to launch a new instance from the web interface, which I was able to do. Someone who would use these tools as part of their workflow could probably fairly easily update the scripts so that they work without problems, since the standard AWS-tools are being used.

Comparison between OSv and Docker

As you might have noticed by now, there are a lot of similarities between OSv and Docker from a developer perspective. During my short evaluation of OSv, these are some of the advantages and disadvantages of unikernels compared to Docker-containers:

Advantages

• The possibility of having a really small memory footprint, especially when using IaaS services.
• Better isolation from other VMs than what can be achieved between containers. This can improve the security of the system and make if less likely that applications will affect each other or the host system.
• Less dependent on the underlying OS than Docker. That being said, most solutions seems to best support Linus as a host.
• Existing tools for managing virtual machines can be used to manage application scaling.

Disadvantages

• Several of the frameworks are not really production ready. Many are research projects or short-lived open source projects created as experiments.
• Virtualization can impact performance negatively, but it depends on many factors, among others the unikernel framework used, the virtualization technology and the type of application.
• Has a smaller ecosystem than Docker which makes it harder to find other images to reuse and find help if you encounter problems.
• Some frameworks are closely tied to implementation technologies for the application that is going to run on it. They do not all support running existing applications without a complete rewrite in a new language.

All in all, unikernels are an interesting technology that can be useful for some highly scalable applications. But just for sheer versatility and the availability of documentation only, Docker is likely still going to be the easiest way for developers to work on their applications.

Disclaimer: The opinions included above are my own, and not those of my employer, Accenture.