K3s – Kalman's Tech Page

K3s

This guide will help you to deploy a 3 node K3S cluster, with ArgoCD and Cert-Manager on the top of it, using Let’s encrypt certificates if you have a valid domain managed by Cloudflare

This guide will deploy the below architecture, an HA 3 node k3s cluster with embeded db, using etcd. You will require a front load-balancer to monitor the node’s healt, internal traffic will be load-balanced between different pods by the built-in traefik load-balancer in k3s:

How to install the 3 node k3s cluster

To create the first server on the first node

curl -sfL https://get.k3s.io | K3S_TOKEN=V3rryS3cr3t sh -s - server \
--cluster-init \
--tls-san=k3s-vm01.bokors.net \
--tls-san=k3s-vm01 \
--tls-san=k3s.bokors.net \
--tls-san=192.168.10.200 \
--tls-san=k3s

To verify the k3s node, use the below command:

kubectl get nodes

To join the first node to the previously created cluster

curl -sfL https://get.k3s.io | K3S_TOKEN=V3rryS3cr3t sh -s - server \
--server https://192.168.10.201:6443 \
--tls-san=k3s-vm02.bokors.net \
--tls-san=k3s-vm02 \
--tls-san=192.168.10.200

To join the second node to the previously created cluster


curl -sfL https://get.k3s.io | K3S_TOKEN=V3rryS3cr3t sh -s - server \
--server https://192.168.10.201:6443 \
--tls-san=k3s-vm03.bokors.net \
--tls-san=k3s-vm03 \
--tls-san=192.168.10.200

To get another traefik instance on a secondery node simply edit the config file

sudo kubectl edit deploy traefik --namespace kube-system

and change the replicas number from 1 to 2.

To add this K3S cluster to an already exising Portainer (running on Docker on a different node)

First enable the docker agent on the server node with below command:

kubectl apply -f https://downloads.portainer.io/ce2-21/portainer-agent-k8s-lb.yaml

Then simply use the wizard in portainer under the Administration -> Environment-related -> Environments -> Add environment

Install cert-manager using Let’s Encrypt as Root CA and Cloudflare as DNS registrar

Install cert-meneger

helm repo add jetstack https://charts.jetstack.io --force-update

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.16.2 \
  --set crds.enabled=true

Create the API secret of Cloudflare in the secret.yaml file

---
apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-key-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-key: <your-global-api-comes-here>

Apply the secret

kubectl apply -f secret.yaml -n cert-manager

Apply the clusterissuer to your namespace

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: cloudflare-clusterissuer
spec:
  acme:

    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: cloudflare-clusterissuer-account-key
    solvers:
      - dns01:
          cloudflare:
            email: <your-email-address-comes-here>
            apiKeySecretRef:
              name: cloudflare-api-key-secret
              key: api-key

kubectl apply -f clusterissuer.yaml

To test the certificate issuing progress, use the below commands
To change the namespace to cert-manager

kubectl config set-context --current --namespace=cert-manager

To see the status of the issued certificate

kubectl describe certificate

To create a test certificate, let’s say for ArgoCD
Create the namespace for ArgoCD

kubectl create ns argocd

Apply the wildcard certificate for this namespace

kubectl apply -f ./wc_certificate.yaml -n argocd

kubectl apply -f ./argocd_cert.yaml -n argocd

Install ArgoCD with Traefik Ingress

Install ArgoCD with the below command

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

To get the dafult admin user’s password, use the below command:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Create the Ingress for ArgoCD with the below file

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: argocd-server
  namespace: argocd
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`argocd.k3s.bokors.net`)
      priority: 10
      services:
        - name: argocd-server
          port: 80
    - kind: Rule
      match: Host(`argocd.k3s.bokors.net.`) && Header(`Content-Type`, `application')
      priority: 11
      services:
        - name: argocd-server
          port: 80
          scheme: h2c
  tls:
    secretName: argocd-k3s-bokors-net-tls
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: argocd-cert
spec:
  secretName: argocd-k3s-bokors-net-tls
  renewBefore: 240h
  dnsNames:
  - 'argocd.k3s.bokors.net'
  issuerRef:
    name: cloudflare-clusterissuer
    kind: ClusterIssuer

Apply the above manifest to the k3s cluster (Make sure your CertManager is properly configure prior to this step)

kubectl apply -f ./<your-file-name>.yml

To fix the too many redirect WebUI error, please add the below value to the argocd-cmd-params-cm file and redeploy the argocd-server application

server.insecure: true

Install Longhorn

Download the environment check sript and run it for testing your kubernetes environment

wget https://raw.githubusercontent.com/longhorn/longhorn/refs/tags/v1.7.2/scripts/environment_check.sh
chmod +x environment_check.sh

./environment_check.sh

To fix all of the issues which were present using a 3 node k3s cluster with Ubuntu 22.04 LTS follow below steps
Install nfs-common on all your nodes

sudo apt install nfs-common

Enable iscsi deamon

sudo apt install open-iscsi
sudo modprobe iscsi_tcp
sudo systemctl enable iscsid

Disable multipathd

systemctl stop multipathd
systemctl disable multipathd

You can run the test script again, which should not have any issues
Add the lonhorn repo to your k3s cluster

helm repo add longhorn https://charts.longhorn.io

Install longhorn using the helm chart

helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.7.2

If you are having issues connecting to you cluster (Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get “http://localhost:8080/version”: dial tcp 127.0.0.1:8080: connect: connection refused), just apply the below fix:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Once lonhorn was successfully deployed on your k3s cluster, create an ingress route using Traefik and CertManager with the below code

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: longhorn-ui
  namespace: longhorn-system
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`longhorn.k3s.bokors.net`)
      services:
        - name: longhorn-frontend
          port: 80
  tls:
    secretName: longhorn-k3s-bokors-net-tls
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: longhorn-cert
spec:
  secretName: longhorn-k3s-bokors-net-tls
  renewBefore: 240h
  dnsNames:
  - 'longhorn.k3s.bokors.net'
  issuerRef:
    name: cloudflare-clusterissuer
    kind: ClusterIssuer

Apply the above creted yaml file

kubectl apply -f <your-file-name>.yaml

Install an external NFS storage

Add helm repo

helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/

Install helm chart with your NFS storage settings:

helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=192.168.10.10 \
    --set nfs.path=/volume1/MEDIA

To verify the newly installed NFS storage class

kubectl get storageclasses

Last updated on July 30, 2025