RKE errors connecting to the Docker socket whilst updating clusters with the Aqua Enforcer deployed

Issue

During invocations of rke up via the RKE CLI or whilst modifying Rancher provisioned Kubernetes clusters, the process fails upon attempted creation of a Kubernetes component container with an error of the following format:

2019-04-30T15:19:17.9826528Z time="2019-04-30T15:19:17Z" level=fatal msg="[etcd] Failed to bring up Etcd Plane: Failed to create [etcd] container on host [rancher.example.com]: Failed to create [etcd] container on host [rancher.example.com]: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create?name=etcd: EOF

Pre-requisites

• A Kubernetes cluster provisioned via the RKE CLI or Rancher
• The Aqua Enforcer workload deployed in the cluster, with AQUA_RUNC_INTERCEPTION environment variable set to 0

Root cause

The issue is caused by Aqua Enforcer's use of the Docker socket to perform runtime enforcement operations preventing RKE from successfully connecting to the Docker socket upon some requests.

Resolution

To resolve this issue set the AQUA_RUNC_INTERCEPTION environment variable on the Aqua Enforcer daemonset to 1. With this setting the Aqua Enforcer will interact directly with runC to perform runtime enforcement operations, and not with the Docker daemon via the Docker socket. This is the default behaviour in new versions of the Aqua Enforcer, as it brings stability and performance benefits. More information on this setting can be found at https://docs.aquasec.com/docs/40-ga#section-new-aqua-enforcer-architecture-for-enhanced-stability-and-performance