hypervisor

a distributed operating system

properties

• Total determinism: Every node is exactly its manifest. Nothing else.
• Clean slate on boot: Reboot resets to definition. Nothing persists unless you said it should.
• Single source of truth: The manifest is the system. Running state is a pure function of it.
• Auditable: You can enumerate everything on every node, because you defined it.
• No drift: There's no gap between "intended state" and "actual state" to discover.
• Disposable infrastructure: Lose a node, boot a new one. Identical.

design

Xen virtualization backed by inter-host L2 VXLAN mesh. More specifically: in-RAM netbooted Xen hypervisor hosts attached to automatic VXLAN meshnets transparently spanning individually-and-centrally configured diskless x86_64 hardware nodes booted with iPXE. Managed from a central orchestration node so that any Xen host can be disposed of and replaced with minor centralized host identification changes.

The orchestration node distributes the individualized configuration of every machine as an APKOVL. It auto-detects the network topology and hosts everything required to network boot the Xen hypervisors. It anchors the VXLAN meshnet and serves as the management node and cluster observability terminal. The configuration and security keys of the network are exclusively stored on the orchestration node. The xen hypervisor nodes and the orchestration node itself can be restored trivially.

The hypervisor nodes boot from the network with nothing stored locally. After iPXE chainloading and booting a kernel and initramfs, the local hardware is fingerprinted and that fingerprint is used to download a unique APKOVL compiled for that specific host. The machine boots fully configured into an in-RAM OS.

The nodes join the mesh after initialization. If storage is installed locally, it can be mounted as a supplementary filesystem. If VXLANs are specified, they are joined automatically. The machine periodically uploads telemetry to the orchestration node. Xen virtual machines can be started and joined to the VXLAN mesh, remaining agnostic to the interchangeable node hosting them.

installation

requirements

• A Linux-based system with minimum 10GB available and L2 network connectivity to every Xen host
• GNU make, version 3.81 or later
• Networked x86_64 machines with hardware virtualization support and PXE enabled

licensing

Hypervisor is available under the AGPL. Commercial licensing is available for organizations that prefer not to comply with AGPL terms — contact licensing@hypervisor.dev.

download

• The current released version of hypervisor is 0.26.0. You can access its checksum here.
• Alternatively, you can checkout a copy of the hypervisor git repository:

git clone git://git.hypervisor.dev

compilation

• See the enclosed INSTALL file for installation details.

philosophy

A program is generally exponentially complicated by the number of notions that it invents for itself. To reduce this complication to a minimum, you have to make the number of notions zero or one. —Dennis Ritchie

The graph of interactions between programs is itself exponentially complicated by the number of interactions. To minimize this complication, linearize the graph into a sequence of steps, so that every step does one thing and emits one thing.

A notion ought to specify a domain: not omitting anything properly within the domain, and not reaching outside that domain. Fitting a curve using too many parameters introduces noise, while too few are imprecise. When several curves together best model the domain the notion should be decomposed.

A correctly modeled domain may be subdivided without remainder into constituent notions, and the resulting notions will not exceed the domain so decomposed unless the curve is badly fitted and the domain badly modeled.