How to deploy Nginx instead of Traefik as your ingress controller on K3s

This document (000020082) is provided subject to the disclaimer at the end of this document.

Environment

K3s 1.27+ (may apply to other versions)

Situation

Task

This knowledge base article will provide the directions for deploying NGINX instead of Traefik as your Kubernetes ingress controller on K3s. Please note that Traefik is the support ingress controller for K3s and NGINX is not officially supported by SUSE Rancher support.

Background

By default, K3s uses Traefik as the ingress controller for your cluster. The decision to use Traefik over NGINX was based on multi-architecture support across x86 and ARM based platforms. Normally Traefik meets the needs of most Kubernetes clusters. However, there are unique use cases where NGINX may be required or preferred. If you don't think you need NGINX, it's recommended to stick with Traefik.

Resolution

The first step to using NGINX or any alternative ingress controller is to tell K3s that you do not want to deploy Traefik. When installing K3s add the following --no-deploy traefik flag to the INSTALL_K3S_EXEC environment variable. Or in some cases, you may need a --disable traefik option instead:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy traefik" sh -s -
or
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik" sh -s -

If you have already downloaded the k3s.sh install script, you can run the following:

INSTALL_K3S_EXEC="--no-deploy traefik" k3s.sh
or
INSTALL_K3S_EXEC="--disable traefik" k3s.sh

This will install the K3s server and form a single node cluster. You can confirm the cluster is operational ("Ready") by running:

$ kubectl get nodes
NAME                         STATUS   ROLES                  AGE     VERSION
ubuntu-s-2vcpu-4gb-nyc1-01   Ready    control-plane,master   3h35m   v1.30.4+k3s1

Note, if you already had the kubectl binary installed on your host and it is not configured correctly, you may need to run k3s kubectl instead of kubectl.

Next, confirm your out-of-box pods are running and Traefik is not running:

$ kubectl get pods -A
NAMESPACE       NAME                                      
READY   STATUS      RESTARTS   AGE
kube-system     coredns-576bfc4dc7-n4k72                  
1/1     Running     0          3h47m
kube-system     local-path-provisioner-6795b5f9d8-mzwb8   
1/1     Running     0          3h47m
kube-system     metrics-server-557ff575fb-ntkh6           
1/1     Running     0          3h47m

K3s has a nice feature that allows you to deploy Helm Charts by placing a HelmChart YAML in /var/lib/rancher/k3s/server/manifests. Create this file by running:

cat >/var/lib/rancher/k3s/server/manifests/ingress-nginx.yaml <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: ingress-nginx
  namespace: kube-system
spec:
  chart: ingress-nginx
  repo: https://kubernetes.github.io/ingress-nginx
  targetNamespace: ingress-nginx
  version: v4.11.2
  set:
  valuesContent: |-
    fullnameOverride: ingress-nginx
    controller:
      kind: DaemonSet
      hostNetwork: true
      hostPort:
        enabled: true
      service:
        enabled: false
      publishService:
        enabled: false
      metrics:
        enabled: true
        serviceMonitor:
          enabled: false
      config:
        use-forwarded-headers: "true"
EOF

K3s periodically polls the manifests folder and applies the YAML in these files. After about a minute, you should see new pods running, including the NGINX Ingress Controller and default backend:

$ kubectl get pods -A
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
ingress-nginx   ingress-nginx-controller-t25nm            1/1     Running     0          34m
kube-system     coredns-576bfc4dc7-n4k72                  1/1     Running     0          3h49m
kube-system     helm-install-ingress-nginx-bfcgs          0/1     Completed   0          35m
kube-system     local-path-provisioner-6795b5f9d8-mzwb8   1/1     Running     0          3h49m
kube-system     metrics-server-557ff575fb-ntkh6           1/1     Running     0          3h49m

You'll also see a helm-install-ingress-nginx pod in your environment. K3s uses this pod to deploy the Helm Chart and it's normal for it to be in a READY=0/1 and STATUS=Completed state once the Helm Chart has been successfully deployed. In the event your Helm Chart failed to deploy, you can view the logs of this pod to troubleshoot further.

Additional Information

Reference

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.