Skip to content

Planning Your Installation

Before proceeding, we recommend you first review our Architecture Overview page.

Our installation process requires ansible to be installed either locally on your computer, or remotely on a linux machine. Please ensure this has been setup prior to continuing with this installation process.

Supported Linux Distribution

Our supported linux distribution is Debian 11 Bullseye.

Domain Names

Having your domain names setup is a critical part of the installation process.

The following roles will require domain names:

Controller this is the main URL for the service. Examples include: portal, dashboard, or cp.
Container Registry Examples: cr., or registry..
Metrics For our log aggregation service and container metrics. This should be specific to the region it's in, such as: metrics.region.example.com.
App URL This is the default URL we use to generate domains for each service (container). This can be something like a.region.example.com.

Each Availability zone will need to have their own domain, which is why I recommend including the region and AZ in the URL. For example, if you have regions in Amsterdam and Frankfurt, you could use something like this:
a.ams.example.net <-- Region AMS, Availability Zone 1
b.ams.example.net <-- Region AMS, Availability Zone 2
a.fra.example.net <-- Region FRA, Availability Zone 1
b.fra.example.net <-- Region FRA, Availability Zone 2
            
PowerDNS ns1/ns2
Example DNS Records
a.dev.cmptstks.net. IN A %{floating_ip}
metrics.dev.cmptstks.net. IN A %{PUBLIC IP OF METRICS SERVER}
*.a.dev.cmptstks.net. IN CNAME a.dev.cmptstks.net.
portal.dev.cmptstks.net. IN A %{controller ip address}
cr.dev.cmptstks.net. IN CNAME portal.dev.cmptstks.net.

Linux Users and Groups

Our container nodes depend on having UID & GID 1001 available for our use. This is generally not a problem on most cloud and virtual machine images, however if you performed some pre-installation steps that included creating a user, this UID/GID may be already taken.

Please change the UID and GID of the user who took that ID before proceeding. Here is a guide to help you accomplish this.

You can verify that this is available by running the following commands on the container nodes:

cat /etc/passwd | grep 1001
cat /etc/group | grep 1001

Network Setup

Our recommended ip setup is: 1 Public, 1 Private, per server. Please see our example configurations for more details.

Please ensure that the container node does not have any kind of source/destination filtering on it’s private network interface. If this is not possible, you will need to set calico_network_ipip to true in your inventory.yml file.

Typically if you have your own L2 network, this won’t be a problem. Where this typically is an issue is in cloud environments, at which case you will need to check if you can disable that. Falling back to ip-in-ip tunneling is possible, but not recommended for production environments. Most of the larger cloud providers will allow you to do this. Please contact us if you need assistance.

For clusters (3+ container nodes), we will also require a Floating IP Address. We will use arp to announce that ip on one of the 3 nodes, and handle failover automatically through corosync and pacemaker.

If you do not want a floating IP, or are unable to allocate one, then please set enable_floating_ip: false and floating_ip: to the IP of first node, in your inventory.yml file.

Disk Setup

If you have the option to select the partition layout for the container nodes, please place most of your disk storage at /var/lib/docker. You can skip a dedicated /home partition, as this is not used in our environment.

Alternatively, create a single / partition with all available disk space.

Example Disk Layout
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda        100G  3.5G  96GB   1% /
/dev/sdb        1.5T   14G  1.5T   1% /var/lib/docker
/dev/sdc        487M   94M  368M  21% /boot

Minimum Server Requirements

Here are our minimum requirements. If you plan to have customers in your environment running production workloads, we recommend you review our example configurations for better guidance on what we recommend for your cluster.

Tip

All CPU Cores should be newer x86 2Ghz+ Intel or AMD processors.

Name CPU Memory Disk
Controller 4 Cores 8 GB 50 GB
Container Node 4 Cores 8 GB 100 GB

If you plan to use the default PowerDNS Servers, you will also need:

Name CPU Memory Disk
NS1 1 Core 512 MB 15 GB
NS2 1 Core 512 MB 15 GB

DNS Server Requirements

AutoDNS

Our default installation is to provision two powerdns services in active passive. We also support directly integrating with AutoDNS, which would allow you to skip provisioning the two powerdns naem servers. In order to configure that, please add the following to the controller section of the inventory file:

controller:
  hosts:
    # ... existing parameters ...
    dns_driver: autodns
    autodns_endpoint: "gateway.autodns.com" # https:// <-- should not be changed.
    autodns_username: ""
    autodns_password: ""
    autodns_context: "4" # The default for AutoDNS is `4`; please only change if necessary.
    autodns_nameservers: # You can omit this section if you will use the default `a/b/c/d.ns14.net` servers.
      - a.ns14.net
      - b.ns14.net
      - c.ns14.net
      - d.ns14.net
    autodns_master_ns: a.ns14.net # Generally the first in the list of nameservers.
    autodns_ns_ttl: 86400
    autodns_soa_email: 'dns@example.org'
    autodns_soa_level: 2

We suggest creating a dedicated AutoDNS user with limited permissions to just create and manage their own zone files, and withuot two-factor authentication. We also recommend enabling IP access restriction for that user to just the controller’s public IP.

PowerDNS

As part of the ansible installer, we will automatically provision two powerdns nameservers using Postgresql streaming replication. If you wish to use your own powerdns server, please adjust your inventory.yml file and add the following under the controller host:

controller:
  hosts:
    # ... existing parameters ...
    pdns_skip_provisioning: true
    pdns_endpoint: "http://ns1.example.com:8081/api/v1/servers"
    pdns_api_key: "" # PowerDNS API Key
    pdns_web_key: "" # WebServer Password
    pdns_nslist:
        - "ns1.example.com"
        - "ns2.example.com"
    pdns_zone_type: Native
    pdns_masters: [] # Leave empty for Native replication

You would then ommit the nameserver hosts from your inventory file. It should look like this:

primary_nameserver:
  hosts:
follower_nameservers:
  hosts:
Example PowerDNS Configuration File
api=yes
api-key=CHANGEME # pwgen -s 32 1
webserver=yes
webserver-address=0.0.0.0
webserver-port=8081
webserver-allow-from=0.0.0.0/0
webserver-password=CHANGEME # pwgen -s 32 1
local-port=53
local-address=0.0.0.0, ::
include-dir=/etc/powerdns/pdns.d
launch=gpgsql
config-dir=/etc/powerdns
default-soa-edit=inception-increment
default-ttl=14400
default-soa-content=ns1.example.com hostmaster.@ 0 10800 3600 604800 3600
query-cache-ttl=20
dnsupdate=yes
allow-dnsupdate-from=

Next Step: Preparing Installation Resources


Last update: 2022-05-18