How to enable NGINX support for HTTP headers with underscores

This document (000020119) is provided subject to the disclaimer at the end of this document.

Environment

A Kubernetes cluster provisioned by the Rancher Kubernetes Engine (RKE) CLI or Rancher.
For RKE provisioned clusters, you will require the RKE binary and access to the cluster configuration YAML, rkestate file and kubectl access with the kubeconfig for the cluster sourced.
For Rancher provisioned clusters, you will require cluster owner or global admin permissions in Rancher

Situation

This article details how to enable HTTP headers with underscores on the nginx-ingress controller in Rancher Kubernetes Engine (RKE) CLI or Rancher v2.x provisioned Kubernetes clusters.

Resolution

Configuration for RKE provisioned clusters

Edit the cluster configuration YAML file to include the "enable-underscores-in-headers: true" option for the ingress, as follows:
```
ingress:
  provider: nginx
  options:
    enable-underscores-in-headers: true
```

Apply the changes to the cluster, by invoking "rke up":
```
rke up --config <cluster configuration yaml file>
```

Recycle the nginx pods in-order to pick up new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name); do kubectl delete $pod -n ingress-nginx; echo "Sleeping for 5 seconds"; sleep 5; done

Verify the new configuration:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name | awk -F '/' '{print $2}'); do echo -n "Checking $pod .... "; kubectl -n ingress-nginx exec "$pod" -- bash -c "cat /etc/nginx/nginx.conf | grep underscores_in_headers | grep on > /dev/null 2>&1 && echo 'Good' || echo 'Bad'"; done

Configuration for Rancher provisioned clusters

Login into the Rancher UI.
Go to Global -> Clusters -> Cluster Name
From the Cluster Dashboard edit the cluster by Clicking on "⋮" then select Edit.
Click "Edit as YAML".

Include the "enable-underscores-in-headers" option for the ingress, as follows:

ingress:
  provider: nginx
  options:
    enable-underscores-in-headers: true

Click "Save" at the bottom of the page.
Wait for cluster to finish upgrading.
Go back to the Cluster Dashboard and click "Launch kubectl".
Run the following inside the kubectl CLI to recycle the nginx pods in-order to pick up new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name); do kubectl delete $pod -n ingress-nginx; echo "Sleeping for 5 seconds"; sleep 5; done

Run the following inside the kubectl CLI to verify the new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name | awk -F '/' '{print $2}'); do echo -n "Checking $pod .... "; kubectl -n ingress-nginx exec "$pod" -- bash -c "cat /etc/nginx/nginx.conf | grep underscores_in_headers | grep on > /dev/null 2>&1 && echo 'Good' || echo 'Bad'"; done

Cause

Underscore in HTTP headers is not allowed by the Nginx Ingress by default. Refer to enable-underscores-in-headers.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.