Requirements¶
Before you install, prepare your workstation, the servers, the network, and a few accounts. The provisioner checks most of these at the start of every run, before it changes anything.
Your workstation¶
You run the provisioner from a control machine, usually your own workstation or a CI runner. It needs:
- ansible-core 2.19 or newer.
- The Python
cryptographylibrary. Some certificate checks run on the control machine rather than on the servers. dig, used to check your load balancer DNS records before the install starts. It's in thednsutilspackage (Debian/Ubuntu) orbind(macOS with Homebrew).gitandmake.- SSH access to every server as
root, by key.
The Ansible roles and collections the provisioner depends on are pinned, and installed into the project directory in a later step. You don't install them globally.
Servers¶
See the Architecture Overview for sizing. Every server must be:
- Ubuntu 26.04 LTS on amd64, freshly installed. Other distributions, releases, and architectures are rejected.
- Reachable over SSH as
rootwith a key. The provisioner doesn't usesudo, and the controller also uses root SSH to the nodes and registry at runtime. -
Named with a single lowercase word, such as
node101, notnode101.example.com. The hostname on the server must match the name you give it in the inventory.
Each server needs a private address for the control plane and a public address. They may be the same.
One disk is enough for every server except a container node, which needs a second disk; see Container node disks.
Ubuntu cloud images block root login
Ubuntu cloud images install your SSH key for root behind a forced command that says "Please login as the user ubuntu". Add disable_root: false to the server's cloud-init user data.
The provisioner installs everything else it needs. It disables password SSH login, and it removes ufw and ntp if they are present, replacing them with its own nftables firewall and chrony.
Container node disks¶
Every container node needs two disks: the root disk the operating system is installed on, and a second, blank data disk for Docker and containerd. Docker 29 keeps images and container layers in containerd's store under /var/lib/containerd, not /var/lib/docker, so a node with one disk, or with a data disk mounted only at /var/lib/docker, fills its root disk.
The provisioner sets up the data disk before it installs Docker:
/dev/vda root disk, any layout
└─ / operating system, logs, packages
/dev/vdb data disk, whole device: XFS, label dockerdata, project quotas
└─ /var/lib/docker-data
├─ docker/ bind-mounted on /var/lib/docker
└─ containerd/ bind-mounted on /var/lib/containerd
It formats the disk as XFS with project quotas enforced, mounts it at /var/lib/docker-data by UUID, bind-mounts it over /var/lib/docker and /var/lib/containerd, and makes Docker and containerd require those mounts, so neither can start without them. Re-running the provisioner on a finished node changes nothing.
Hand the data disk over blank: a whole device with no partition table, no filesystem, no mount, and nothing in /etc/fstab.
- Don't partition or format it in the operating system installer or your hypervisor template.
- Make sure cloud-init leaves it alone: no
disk_setup,fs_setup, ormountsentry for it, and no image default that formats a second disk as ephemeral storage. - Don't run Docker on the node before the first install. A node where
/var/lib/dockeror/var/lib/containerdalready has contents is refused, because mounting over them would hide that data.
You name the disk in the inventory with docker_data_device; see Servers.
Size the data disk for every image, container layer, and volume the node will hold. With the data disk in place, the root disk holds only the operating system, logs, and packages, so it stays small however large the node grows. The smallest sizes the provisioner has been tested on end to end are a 20 GB root disk and a 30 GB data disk, which is a floor for a test environment, not a production recommendation.
To grow the data disk later, enlarge the virtual disk in your hypervisor, then run this on the node. There's no partition to grow first. XFS can grow online but can never shrink.
A node won't boot without its data disk
The data disk's mounts are deliberately required at boot. If the disk is missing, the node stops in emergency mode rather than letting Docker write to the root disk. Keep console access to your nodes.
Container networks¶
Each container node needs a private network range for its containers:
- RFC 1918 private addresses.
- At least a
/28. A/22is typical. - It must not overlap any other network in your environment, including other nodes' container networks.
Network¶
The provisioner configures each server's firewall. Your provider's network, and any firewall in front of your servers, must allow the following.
| Traffic | Ports | To |
|---|---|---|
| Customer web traffic | 80, 443 TCP | Container nodes |
| Customer TCP/UDP services | 10000–50000 TCP and UDP | Container nodes |
| Portal, API, and certificate issuance | 80, 443 TCP | Controller |
| Certificate issuance (HTTP-01) | 80 TCP | Metrics and registry servers |
| Customer container registries | 443 TCP, 10000–50000 TCP | Registry server |
| DNS | 53 TCP and UDP | Nameservers |
| SSH | 22 TCP | Every server, from your control machine and the controller |
| Tailscale (optional) | 41641 UDP | Every server |
Traffic between servers (Docker TLS, the agent, metrics scrapes, PostgreSQL replication, and so on) is opened by the provisioner's own firewall rules, limited to the servers that need it. If a region reaches the other servers over the public internet rather than a private network or Tailscale, your provider's firewall must also allow:
| Traffic | Ports | From | To |
|---|---|---|---|
| Docker TLS and the agent | 2376, 8500 TCP | Controller | Container nodes |
| Metrics scrapes | 9100, 8080, 81 TCP | Metrics server | Container nodes |
| Node metrics scrapes | 9100 TCP | Metrics server | Every server |
| Certificate service | 3000 TCP | Container nodes | Controller |
| Log shipping | 3102 TCP | Container nodes | Metrics server |
| Metrics and log queries | 3101, 3102 TCP | Controller | Metrics servers |
| Backups (borg over SSH) | 22 TCP | Container nodes | Backup server |
Every server also needs outbound HTTPS to download packages and images, including from download.docker.com, apt.postgresql.org, repo.computestacks.com, ghcr.io, Docker Hub, and GitHub.
Remote regions and cleartext traffic
Without a private network or Tailscale, the controller's requests to each node's agent and Prometheus's scrapes of each node cross the public internet unencrypted. That is a supported choice, but make it deliberately. See Tailscale.
If port 80 can't be opened to the internet on the controller, metrics, or registry servers, use a DNS-01 challenge for their certificates instead. See TLS certificates.
Accounts¶
Required:
- An email address for your ACME certificate authority account. The default CA is ZeroSSL; Let's Encrypt, Google Trust Services, Buypass, and SSL.com are also supported.
- Control of a public DNS domain for the portal, metrics, registry, and customer container hostnames. See Prepare DNS.
Optional:
- A Tailscale auth key, to encrypt control-plane traffic between sites.
- An Ubuntu Pro token, for Livepatch.
- API credentials for your DNS provider, if you use DNS-01 certificate challenges.
- A Sentry DSN, for error reporting from the controller.
- S3-compatible storage keys, for exports.
The ComputeStacks controller image is published publicly at ghcr.io/computestacks/controller, so no registry credentials are needed to install it.
Next steps¶
Continue to Prepare DNS.