Create Kubernetes cluster with multiple runtimes-runc/crun, kata & gvisor

Check the first installment of this series and the and the github repo
Remember objective is to unbox Kubernetes to its source and for that you need a good test bench, I hope k8s-unboxing would provide some help in that journey.

I was asking myself where do we start? what is that one part in this machine which is absolutely not replaceable and keep the sanctity intact, that is OCI -Open Container Initiative, specifically image-spec & runtime-spec these are two repositories to watch out.

Any high-level container runtime interface like containerd (or cri-o) which is responsible to manage container lifecycle for the Kubernetes or rather I would say abstracting the linux primitive, need to respect these OCI specifications.

Like for an example following go.mod will show containerd reference to those standards, but containerd is replaceable by cri-o or the runtime these interfaces use, there are plenty of options for low level runtimes which are the first candidate in the food chain, following are few notable ones with unique use cases

1. Most original and popular is runc
2. A C version of the same with low memory footprint crun (UC: IOT)
3. Kata — lightweight virtual machines that feel and perform like containers, but provide stronger workload isolation using hardware virtualization technology as a second layer of defense (UC: multitenant cluster for absolute security)
4. Gvisor provides an isolation boundary between the application and the host kernel, It limits the host kernel surface accessible to the application while still giving the application access to all the features it expects (UC: multitenant cluster for more security)

Now how all these different runtimes supposed to work with containerd or cri-o? OCI is that normalizer which provides the standards, which all of these stakeholder’s supposed to follow.

Remember only Linux containers rely on namespaces and cgroups. However, not all standard containers are Linux.

Refer to this reading, VM based containers is thing now and it will shape the industry concerns around security of multi-tenant cluster workload.

But I do not want to convert this article towards a full-blown discussion, in fact it is not even possible to cut it short in one page. I do have plan for some podcast or video blog around it, where we can go through the source codes of these low-level runtime.

But for a first step towards that objective find the latest interim release (1.2.1) for k8s-unboxing , which support multiple runtimes (based on containerd interface),

following YouTube video instruction shows how can you use this release to create a cluster where you can run multiple runtimes in different nodes and deploy specific workload by using runtime-class.

This release support following, please go through the readme for this specific release re further details

provider: only libvirt tested and supported for this release

runtime: support for both runc | crun | kata | gvisor

cni: “default” Note: For this release “calico” & “cilium” not tested

Kubernetes version: 1.25 (1:24–1:25)

cri tool version: 1.25

runc version: 1.1 | crun version: 1.7 Note: kata & gvisor runtime version has no effect

containerd: 1.6

cni plugin version: 1.1

os: Ubuntu 1804 | Ubuntu 2004 | Ubuntu 2204