Skip to content

Requirements

Before you install, prepare your workstation, the servers, the network, and a few accounts. The provisioner checks most of these at the start of every run, before it changes anything.

Your workstation

You run the provisioner from a control machine, usually your own workstation or a CI runner. It needs:

  • ansible-core 2.19 or newer.
  • The Python cryptography library. Some certificate checks run on the control machine rather than on the servers.
  • dig, used to check your load balancer DNS records before the install starts. It's in the dnsutils package (Debian/Ubuntu) or bind (macOS with Homebrew).
  • git and make.
  • SSH access to every server as root, by key.

The Ansible roles and collections the provisioner depends on are pinned, and installed into the project directory in a later step. You don't install them globally.

Servers

See the Architecture Overview for sizing. Every server must be:

  • Ubuntu 26.04 LTS on amd64, freshly installed. Other distributions, releases, and architectures are rejected.
  • Reachable over SSH as root with a key. The provisioner doesn't use sudo, and the controller also uses root SSH to the nodes and registry at runtime.
  • Named with a single lowercase word, such as node101, not node101.example.com. The hostname on the server must match the name you give it in the inventory.

    hostnamectl set-hostname node101
    

Each server needs a private address for the control plane and a public address. They may be the same.

One disk is enough for every server except a container node, which needs a second disk; see Container node disks.

Ubuntu cloud images block root login

Ubuntu cloud images install your SSH key for root behind a forced command that says "Please login as the user ubuntu". Add disable_root: false to the server's cloud-init user data.

The provisioner installs everything else it needs. It disables password SSH login, and it removes ufw and ntp if they are present, replacing them with its own nftables firewall and chrony.

Container node disks

Every container node needs two disks: the root disk the operating system is installed on, and a second, blank data disk for Docker and containerd. Docker 29 keeps images and container layers in containerd's store under /var/lib/containerd, not /var/lib/docker, so a node with one disk, or with a data disk mounted only at /var/lib/docker, fills its root disk.

The provisioner sets up the data disk before it installs Docker:

/dev/vda   root disk, any layout
  └─ /                       operating system, logs, packages

/dev/vdb   data disk, whole device: XFS, label dockerdata, project quotas
  └─ /var/lib/docker-data
       ├─ docker/       bind-mounted on  /var/lib/docker
       └─ containerd/   bind-mounted on  /var/lib/containerd

It formats the disk as XFS with project quotas enforced, mounts it at /var/lib/docker-data by UUID, bind-mounts it over /var/lib/docker and /var/lib/containerd, and makes Docker and containerd require those mounts, so neither can start without them. Re-running the provisioner on a finished node changes nothing.

Hand the data disk over blank: a whole device with no partition table, no filesystem, no mount, and nothing in /etc/fstab.

  • Don't partition or format it in the operating system installer or your hypervisor template.
  • Make sure cloud-init leaves it alone: no disk_setup, fs_setup, or mounts entry for it, and no image default that formats a second disk as ephemeral storage.
  • Don't run Docker on the node before the first install. A node where /var/lib/docker or /var/lib/containerd already has contents is refused, because mounting over them would hide that data.

You name the disk in the inventory with docker_data_device; see Servers.

Size the data disk for every image, container layer, and volume the node will hold. With the data disk in place, the root disk holds only the operating system, logs, and packages, so it stays small however large the node grows. The smallest sizes the provisioner has been tested on end to end are a 20 GB root disk and a 30 GB data disk, which is a floor for a test environment, not a production recommendation.

To grow the data disk later, enlarge the virtual disk in your hypervisor, then run this on the node. There's no partition to grow first. XFS can grow online but can never shrink.

xfs_growfs /var/lib/docker-data

A node won't boot without its data disk

The data disk's mounts are deliberately required at boot. If the disk is missing, the node stops in emergency mode rather than letting Docker write to the root disk. Keep console access to your nodes.

Container networks

Each container node needs a private network range for its containers:

  • RFC 1918 private addresses.
  • At least a /28. A /22 is typical.
  • It must not overlap any other network in your environment, including other nodes' container networks.

Network

The provisioner configures each server's firewall. Your provider's network, and any firewall in front of your servers, must allow the following.

Traffic Ports To
Customer web traffic 80, 443 TCP Container nodes
Customer TCP/UDP services 10000–50000 TCP and UDP Container nodes
Portal, API, and certificate issuance 80, 443 TCP Controller
Certificate issuance (HTTP-01) 80 TCP Metrics and registry servers
Customer container registries 443 TCP, 10000–50000 TCP Registry server
DNS 53 TCP and UDP Nameservers
SSH 22 TCP Every server, from your control machine and the controller
Tailscale (optional) 41641 UDP Every server

Traffic between servers (Docker TLS, the agent, metrics scrapes, PostgreSQL replication, and so on) is opened by the provisioner's own firewall rules, limited to the servers that need it. If a region reaches the other servers over the public internet rather than a private network or Tailscale, your provider's firewall must also allow:

Traffic Ports From To
Docker TLS and the agent 2376, 8500 TCP Controller Container nodes
Metrics scrapes 9100, 8080, 81 TCP Metrics server Container nodes
Node metrics scrapes 9100 TCP Metrics server Every server
Certificate service 3000 TCP Container nodes Controller
Log shipping 3102 TCP Container nodes Metrics server
Metrics and log queries 3101, 3102 TCP Controller Metrics servers
Backups (borg over SSH) 22 TCP Container nodes Backup server

Every server also needs outbound HTTPS to download packages and images, including from download.docker.com, apt.postgresql.org, repo.computestacks.com, ghcr.io, Docker Hub, and GitHub.

Remote regions and cleartext traffic

Without a private network or Tailscale, the controller's requests to each node's agent and Prometheus's scrapes of each node cross the public internet unencrypted. That is a supported choice, but make it deliberately. See Tailscale.

If port 80 can't be opened to the internet on the controller, metrics, or registry servers, use a DNS-01 challenge for their certificates instead. See TLS certificates.

Accounts

Required:

  • An email address for your ACME certificate authority account. The default CA is ZeroSSL; Let's Encrypt, Google Trust Services, Buypass, and SSL.com are also supported.
  • Control of a public DNS domain for the portal, metrics, registry, and customer container hostnames. See Prepare DNS.

Optional:

  • A Tailscale auth key, to encrypt control-plane traffic between sites.
  • An Ubuntu Pro token, for Livepatch.
  • API credentials for your DNS provider, if you use DNS-01 certificate challenges.
  • A Sentry DSN, for error reporting from the controller.
  • S3-compatible storage keys, for exports.

The ComputeStacks controller image is published publicly at ghcr.io/computestacks/controller, so no registry credentials are needed to install it.

Next steps

Continue to Prepare DNS.