Network Config

Network configs in Mesh Hypervisor define custom VXLAN meshes for node connectivity, stored in /host0/network/ on the central orchestration node. These files (e.g., manage.conf) are installed to remote nodes via manifests and used to build VXLAN bridges like br456. This section explains the keys and how they work, assuming you know basics like ssh and cat. For setup steps, see Network Configuration.

Overview

Each config file sets up a VXLAN mesh—a virtual network linking nodes over your physical Ethernet. On each node, a script reads the config, creates a vxlan<vni> interface (the tunnel) and a br<vni> bridge (where workloads connect), and assigns IPv6 addresses. The central node, called host0, runs an HTTP API inside each VXLAN (e.g., on port 8000) to list all nodes in that mesh. Nodes use this API to find and connect to each other, updating their neighbor lists dynamically.

Here’s the tricky part: nodes need host0’s address to join the VXLAN, but they’re not in it yet. Mesh Hypervisor solves this by giving host0 a fixed IPv6 address—always ending in 0001 (e.g., fd42:1234::1). Nodes start by connecting to that, fetch the API data, then link up with everyone else. If a node disappears, the API updates, and others drop it. Simple, right?

Structure and Keys

Configs are plain text files with key=value lines. Here’s what each key does:

  • name:
    • Format: Any word (e.g., manage).
    • Purpose: Names the VXLAN mesh—helps generate unique addresses and IDs.
    • Example: name=manage—just a label you pick.
    • Must Have: Yes—if missing, the script fails with an error.
  • prefix:
    • Format: IPv6 address with /64 (e.g., fd42:2345:1234:9abc::/64).
    • Purpose: Sets the IPv6 range for the mesh—like a big address pool starting with fd42:2345:1234:9abc:. Every node gets a unique address from this.
    • Example: prefix=fd42:2345:1234:9abc::/64—host0 gets fd42:2345:1234:9abc::1, others get random endings.
    • Must Have: Yes—needs to be /64 (64-bit network part), or the script chokes.
  • vni:
    • Format: A number (e.g., 456).
    • Purpose: Virtual Network Identifier—makes vxlan456 and br456. Keeps meshes separate.
    • Example: vni=456—creates br456 on nodes for workloads to join.
    • Must Have: Yes—duplicate VNIs crash the script; each mesh needs its own.
  • key:
    • Format: A number (e.g., 456).
    • Purpose: A seed number—feeds into genid to make unique IPv6 and MAC addresses for each node.
    • Example: key=456—ensures addresses like fd42:2345:1234:9abc:1234:5678:9abc:def0 are predictable.
    • Must Have: Yes—if missing, addressing fails. Same key across meshes might overlap, so mix it up.

Example Config

/host0/network/manage.conf:

name=manage
prefix=fd42:2345:1234:9abc::/64
vni=456
key=456
  • Sets up a mesh called manage with bridge br456, IPv6 starting fd42:2345:1234:9abc:, and key=456 for address generation.

How It Works

When a node boots, it copies this config to /var/pxen/networks/ (via a manifest) and runs a script. Here’s what happens, step-by-step:

  1. VXLAN Interface: Creates vxlan<vni> (e.g., vxlan456)—a tunnel over your Ethernet.
    • Uses port 4789, MTU 1380 (hardcoded).
    • Gets a MAC like 02:12:34:56:78:9a from genid(name+vni).
  2. Bridge Interface: Creates br<vni> (e.g., br456)—a virtual switch.
    • Gets a MAC like 02:ab:cd:ef:01:23 from genid(bridge+name+vni).
    • Links vxlan456 to br456 so traffic flows through.
  3. IPv6 Address: Assigns the node an address like fd42:2345:1234:9abc:1234:5678:9abc:def0.
    • Uses prefix plus a genid(name+vni) suffix—unique per node.
    • host0 always gets prefix:0000:0000:0000:0001 (e.g., fd42:2345:1234:9abc::1).
  4. Connect to host0: Adds host0’s IPv4 (from PXE boot URL) and MAC to the VXLAN’s neighbor list.
    • Starts talking to host0 at fd42:2345:1234:9abc::1:8000.
  5. Fetch Neighbors: Grabs a list of other nodes from host0’s HTTP API.
    • Format: hostname ipv4 mac ipv6 per line.
    • Updates every 3 seconds—adds new nodes, drops missing ones.
  6. Stay Alive: Pings host0’s IPv6 to keep the mesh active.

Workloads (e.g., VMs) plug into br<vni>—like a virtual LAN cable.

Notes

Install configs with a manifest (e.g., O MODE=root:root:0644 SRC=/host0/network/manage.conf TGT=/var/pxen/networks/manage.conf). The HTTP API runs only inside the VXLAN—nodes bootstrap via host0’s 0001 address, not external access. Overlapping prefix or vni values break the mesh—check logs (mesh system logview) if nodes don’t connect. For workload bridges, see Workload Config; for node control, see Managing Nodes.

Next, explore Manifest Syntax.