On Tuesday I attended the inaugural New Directions in Operating Systems conference in London which was excellently organised by Justin Cormack, and sponsored by :ByteMark (no relation to me!) and RedHat.
I attended with the attitude that since things seem to be moving so fast in this space (e.g. Docker), there would be a high chance that I would get glimpses of the not-too-distant future. I was not disappointed.
I’m not going to cover every talk in detail. For that, I recommend reading this which someone from Cambridge somehow managed to live blog. Also most of the presentation links are now up here and I expect videos will follow.
Instead, here two main highlights that I took away.
If we are aspiring to treat our servers as cattle (as per the popular metaphor from CERN), a number of talks were (to me) about how to make better cows.
The foundation of all solutions was unanimously either bare metal or a XEN hypervisor. As per the name, this wasn’t the conference for talking about the Open Compute Project or advances like AWS have made recently with C4. We can think about the hypervisor as our “field” in the cow metaphor.
For the sake of my vegetarian and Indian friends, let’s say the reason for owning cows is to get milk. Cows (like the servers we use today) have evolved significantly (under our influence) and cows are now very good at producing milk. But they still have a lot of original “features” not all of which may directly serve our requirements. For example they can run, they can moo, they can hear etc.
A parallel can be drawn to a web server which similarly may possess its own “redundant features”. Most web servers can talk the language required by printers, support multiple locales and languages, and can be connected to using a number of different protocols. There could even be redundancy in traditional “core features” for example supporting multiple users, multiple threads, or even virtual memory.
The downside of all of this redundancy is not just efficiency of storage, processing and maintenance, it’s also the impact on security. All good sysadmins will understand that unnecessary daemons running on a box (e.g. hopefully even sshd) expand your attack surface by exposing additional attack vectors. This principle can be taken further. As Antti Kantee said, drivers can add millions of lines of code to your server. Every one of these presents the potential to a security defect.
Robert Watson was amongst others to quote Bruce Schneier:
Defenders have to protect against every possible vulnerability, but an attacker only has to find one security flaw to compromise the whole system.
With HeartBleed, Shellshock and Poodle all in the last few months, this is clearly needs some serious attention (for the good of all of us!).
To address this we saw demonstrations of:
- Rump Kernels which are stripped down to only include filesystems, TCP/IP, device calls and system calls, device drivers. But no threads, locking, scheduling etc. So this is more of a basis from which to build a cow but not a working one.
- Unikernels where all software layers are from the same language framework. So this is building a whole working cow from bespoke parts with no redundant parts (ears etc!).
- RumpRun for building Unikernels for running a POSIX application (mathopd a http server in the example), i.e. taking a Rump Kernel and building it into a single application kernel for one single job application. So another way to build a bespoke cow.
- MirageOS a programming language for building type-safe Unikernels. So another way to build a bespoke very safe cow.
- GenodeOS a completely new operating system taking a new approach to delegating trust through the application stack. So to some extent a new animal that produces milk with a completely re-conceived anatomy.
Use cases range from the “traditional” like building normal (but much more secure) servers to completely novel such as very light-weight and short-lived servers to start up almost instantaneously, do something quickly and disappear.
Docker and CoreOs with its multi virtual machine-aware and now stripped down container ready functionality were mentioned. However, whilst CoreOs is as smaller attack surface, if you are running a potentially quite big Docker container on it, you may be adding back lots security vectors. Possibly as the dependency resolution algorithm for Docker improves, this will progressively reduce the size of Docker containers and hence the number of lines of code and potential vulnerabilities included.
Two presentations of the day stood out for generally focussing on a different (and to me more recognisable) level of problem. First was Gareth Rushgrove’s about Configuration Management. He covered a very wide range of concepts and tools focused on the management of the configuration of single and fleets of servers over time rather than novel ways to construct operating systems. He made the statement:
If servers are cattle not pets, we need to talk about fields and farms
Which inspired the title of this blog and led to some discussion (during the presentation and on Twitter) about using active runtime Configuration Management tools like Puppet to manage the adding and removing of infrastructure resources over time. Even if most of your servers are immutable, it’s quite appealing to think Puppet or Chef could manage both what servers exist and the state of those those more pet-like creatures that do have change over time (in the applications that most good “horse” organisations run).
Whilst if you are using AWS CloudFormation it can provision and update your environment (so called Stack), resultant changes may be heavy-handed and this is clearly a single cloud provider solution. Terraform is multi cloud provider solution to consider and supports a good preview mode, but doesn’t evolve the configuration on your servers over time.
Gareth also mentioned:
- OSv which at first I thought was just a operating system query engine like Facebook’s osquery. But it appears to be a fully API driven operating system.
- Atomic and OSTree which Michael Scherer covered in the next talk. These look like very interesting solutions for providing confidence and integrity in those bits of the application and operating system that aren’t controlled by Chef, Puppet or DockerFile.
I really feel like I’ve barely done justice to describing even 20% of this excellent conference. Look out for the videos and look out for the next event.
No animals were harmed during the making of the conference or this blog.