Update Enforcer daemonset tolerations

By default, the NeuVector Enforcer daemonset YAML includes the default Kubernetes master node toleration.  

 tolerations:
   - effect: NoSchedule
     key: node-role.kubernetes.io/master

In many cases, nodes may have other taints that block the NeuVector Enforcer daemonset from deploying.  Below are ways to add additional tolerations to the Enforcer daemonset.

To learn the taints in place, describe the node.

❯ kubectl describe node gtk8s-master
Name: gtk8s-master
...
Taints:  node-role.kubernetes.io/control-plane:NoSchedule
         node-role.kubernetes.io/master:NoSchedule

As we see from above taints, node-role.kubernetes.io/control-plane:NoScheule will block the enforcer from deploying to gtk8s-master.

Example 1: kubectl

Edit the neuvector-enforcer-pod daemonset and add the additional toleration for node-role.kubernetes.io/control-plane:NoSchedule.

❯ kubectl edit daemonset neuvector-enforcer-pod -n neuvector
 tolerations:
 - effect: NoSchedule
   key: node-role.kubernetes.io/master
 - effect: NoSchedule
   key: node-role.kubernetes.io/control-plane

Note: Similar to kubectl edit, you may have deployed using manifest which you should be able to edit and use `kubectl apply ...`.

Example 2: Helm Upgrade

If you use helm chart for the deployment, you can update the deployment with the following.  The change will redeploy the enforcer pods.

❯ helm upgrade nvlab --reuse-values -f tolerations.yaml --namespace=neuvector neuvector/core

Contents of the tolerations.YAML:

enforcer:
  tolerations:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
  - effect: NoSchedule
    key: node-role.kubernetes.io/control-plane

Alternatively, you can use --set instead of supplying a file.

❯ helm upgrade nvlab --reuse-values --namespace=neuvector neuvector/core --set enforcer.tolerations[0].effect=NoSchedule,enforcer.tolerations[0].key=node-role\.kubernetes\.io/control-plane,enforcer.tolerations[1].effect=NoSchedule,enforcer.tolerations[1].key=node-role\.kubernetes\.io/master

Note: You can always use --dry-run to review the resulting manifest before actual run.

helm upgrade --dry-run nvlab --reuse-values -f tolerations.yaml --namespace=neuvector neuvector/core