Exploring talos.dev: My First Experiences

I've always wanted to experiment with my own Kubernetes cluster. When I discovered Talos Linux, I believed it would be the perfect toy for such a cluster.

Talos Linux is a "Kubernetes Operating System." Once Talos is installed, there won't be an SSH server running; instead, you'll only have access through the API using talosctl.

The hardware

I recently purchased three cloud servers for 20 EUR/month each during Black Friday from PHP-FRIENDS, a German server hosting company.

The specifications of the servers are:

  • 8 vCPU cores (Intel Xeon)

  • 32 GB RAM

  • 320 GB NVMe running on Ceph (I have yet to determine if etcd will perform well enough)

  • 10 GBit/s shared network

PHP-Friends Cloud Control Panel

The installation

They offer the option to upload your own ISOs, making it possible to install Talos. Currently, you can only request their support to mount the ISO. After sending an email, they mounted the metal-amd64 ISO and booted into it. Upon opening their VNC-based web console, I saw the Talos interface displaying logs, configuration options, and more. After tinkering with some settings and sending a Ctrl+Alt+Delete command, the server rebooted and displayed Proxmox. With that clue, I was able to easily follow the Proxmox tutorial on the Talos website.

Talos begins in maintenance mode, during which no actual bootstrapping occurs. For some reason, the network connection failed to function. To resolve this issue, I had to manually assign an IP address to each server using the web console.

The cluster

I went with a 3 control plane setup without any workers. I'll schedule all my workloads on them. As this isn't a production cluster, this should be fine.

After the successful setup, I see this via talosctl (real IPs and names are replaced):

❯ talosctl get members
NODE             NAMESPACE   TYPE     ID              VERSION   HOSTNAME        MACHINE TYPE   OS               ADDRESSES
100.200.300.40   cluster     Member   talos-foo-foo   1         talos-foo-foo   controlplane   Talos (v1.5.5)   ["100.200.300.40"]
100.200.300.40   cluster     Member   talos-foo-bar   1         talos-foo-bar   controlplane   Talos (v1.5.5)   ["100.200.300.41"]
100.200.300.40   cluster     Member   talos-foo-baz   1         talos-foo-baz   controlplane   Talos (v1.5.5)   ["100.200.300.42"]

Running kubectl also works:

❯ KUBECONFIG=kubeconfig kubectl get pods -A
NAMESPACE     NAME                                    READY   STATUS    RESTARTS        AGE
kube-system   coredns-78f679c54d-c9sxm                1/1     Running   0               3h15m
kube-system   coredns-78f679c54d-m5q59                1/1     Running   0               3h15m
kube-system   kube-apiserver-talos-foo-foo            1/1     Running   0               3h14m
kube-system   kube-apiserver-talos-foo-bar            1/1     Running   0               3h13m
kube-system   kube-apiserver-talos-foo-baz            1/1     Running   0               3h13m
kube-system   kube-controller-manager-talos-foo-foo   1/1     Running   1 (3h15m ago)   3h14m
kube-system   kube-controller-manager-talos-foo-bar   1/1     Running   1 (3h15m ago)   3h13m
kube-system   kube-controller-manager-talos-foo-baz   1/1     Running   1 (3h14m ago)   3h13m
kube-system   kube-flannel-4gh9z                      1/1     Running   0               3h15m
kube-system   kube-flannel-r45hl                      1/1     Running   0               3h14m
kube-system   kube-flannel-tl7vw                      1/1     Running   0               3h14m
kube-system   kube-proxy-29s6v                        1/1     Running   0               3h14m
kube-system   kube-proxy-8h2vm                        1/1     Running   0               3h15m
kube-system   kube-proxy-cmbfd                        1/1     Running   0               3h14m
kube-system   kube-scheduler-talos-foo-foo            1/1     Running   1 (3h15m ago)   3h14m
kube-system   kube-scheduler-talos-foo-bar            1/1     Running   2 (3h15m ago)   3h13m
kube-system   kube-scheduler-talos-foo-baz            1/1     Running   2 (3h14m ago)   3h13m

I am the proud owner of my own Talos-based Kubernetes cluster. The next chapter of my journey will involve setting up components such as cert-manager, external-dns, and others using kluctl.

Cheers