Using Admission Control to Prevent Unauthorized or Vulnerable Image Deployments in Kubernetes

Kubernetes Admission Control is a Critical Link in a Container CI/CD Pipeline

An important security enforcement point to build into the container CI/CD pipeline is to prevent unauthorized or vulnerable images from being deployed into production Kubernetes clusters. While basic Kubernetes admission control provides some capabilities, preventing vulnerable images from being deployed requires extensions to the built-in Kubernetes admission control features. This is where a full-lifecycle container security platform like NeuVector, with tight integration to Kubernetes admission control webhooks, can bridge the gap between development and operational security.

Enforcement of admission control policies is a recommended best practice for automating container security into a pipeline, but what is most important is to be able to have flexible criteria for creating rules so that the policies can map into various enterprise work flows and use cases. For example, some companies enforce deployment only from a production registry which is locked down. Others allow vulnerable images to be stored into a registry but enforce vulnerability policies based on the namespace in which the containers are being deployed. And some want a combination of criteria such as user, namespace, vulnerability profile, and labels in their admission control policies.

NeuVector Admission Control Integrates with Kubernetes, OpenShift

NeuVector integrates into and extends Kubernetes’ own Admission Control features, ensuring completely seamless operation between the two. These additions will further prevent vulnerable images – scanned and discovered by NeuVector or one of its security solution partners – from deploying into Kubernetes production environments. NeuVector’s also ensures that only authorized users and service accounts are capable of deploying containers into production.

This integration is just one of the many integration points with Kubernetes and other orchestration tools to provide seamless security from build to ship to run. For example, there is integration with Kubernetes namespaces and OpenShift RBACs to provide access control in NeuVector, and with service meshes such as Istio to provide a Layer 7 network firewall.

Admission Control Use Cases

There are many ways to enforce Kubernetes admission control, depending on the desired work flow and control points in the pipeline. Here are a few examples:

• Trusted Registry. Images are only allowed to be deployed from a trusted ‘production’ registry. Appropriate access controls are required to ensure that malicious images can’t be pushed to the registry. Images with vulnerabilities may or may not be allowed depending on the vulnerability management policies and what tools exist in deployment or run-time to mitigate vulnerabilities.
• Vulnerability Profile. Images are scanned for vulnerabilities and security policy created which allows or blocks deployments based on vulnerability scores, levels, criticality (high, medium etc) or presence of specific vulnerabilities (e.g. CVE’s).
• Namespace Restrictions. Policies are created based on the target namespace of the deployment, often in combination with other criteria.
• User or Service Account. Allow only certain users or service accounts to deploy containers.
• Special Image Labels for Security. This is a catch-all criteria which can be used to control admission based on virtually any policy or condition. During the build process, the code and image can be scanned and even tested and a security tool can add an image label to signal the downstream admission controller how to handle this image.
• Container Run-Time Profile. The run-time profile of a container can include many criteria which are useful for admission control, such as:
  • Root. Containers may not be allowed to run as root.
  • Privileged. Containers may only be allowed to run as privileged in certain namespaces, or only for certain images.
  • Mounted volumes. Sensitive volumes may not be allowed to be mounted by containers.
  • Deployment yaml label. The deployment yaml for a service/container can be inspected for run-time labels applied to pods to determine admission control results.
• Image Signatures. The signature of an image can be verified prior to deployment to ensure it is not tampered with prior to deployment.

Any of the above can be used independently or in combination to enforce security in a pipeline, and there may be several variations of policies required in a large enterprise with several divisions, each with distinct work flows.

How to Use NeuVector Admission Control

Kubernetes admission control is supported in Kubernetes 1.9+ and Red Hat OpenShift 3.9+. The appropriate admission control webhooks are required, which are activated by default on Kubernetes but not on OpenShift. To enable Admission Control in NeuVector (disabled by default), go to Security Risks -> Admission Control in the console to enable it. All actions are also supported by the REST API. Select Monitor mode, which will allow deployments with violations but alert only, or Protect mode, which will block deployments with violations.

Rules can be created by selecting the Criterion, then the matching condition (Operator and Value). Multiple criteria can be used in a single rule. Typically, Deny rules are added to block certain conditions, such as vulnerable images.

When a deployment is attempted which violates a rule, and when in Protect mode, the deployment will fail, as shown below.

The failed deployment will be logged into notifications as a security violation and appropriate alerts issued. The event can also be used in the NeuVector Response Rules to trigger custom webhooks or other actions.

What About Kubernetes Pod Security Policies?

There are some built-in admission control capabilities of Kubernetes which can be configured using pod security policies, however these have some limitations. Pod security policies enable admission control on criteria such as namespaces, volumes, privileged use, SElinux or AppArmor profiles and more. The main limitations are the inability to factor vulnerability scan results into rules and the lack of an organized management framework for rule creation and event monitoring.

We recommend that Kubernetes admission control rules should be created as much as possible in the NeuVector platform so they can be centrally managed and monitored, and the benefits of integration upstream to vulnerability scanning as well as downstream to run-time scanning and other security functions can be realized. There may be certain pod security policies such as enforcement of certain SElinux or AppArmor settings that could be set directly in Kubernetes.

Admission Control Integrates with Leading CI/CD Security Partners

NeuVector Kubernetes Admission Control can be integrated with other code analysis, open source management, and image scanners such as Black Duck by Synopsys, Sonatype, OpenShift, and Jfrog to enforce admission based on scan results or policy decisions earlier in the pipeline. For example, special labels can be added to the container image for NeuVector to use in the admission control decision. Or a security policy go/no-go decision may have already been made by the partner solution, and NeuVector can query the image status during deployment to relay it’s decision to the Kubernetes admission controller.

“Considering the dynamic, ever-changing nature of modern containerized applications, and that in 2018 an average of 47 vulnerabilities are disclosed each day, a continuous approach to preventing vulnerable images from being deployed into production systems is critical,” said Tim Mackey, senior technical evangelist, Synopsys. “By integrating Black Duck OpsSight, our open source vulnerability detection solution for containers, with NeuVector Admission Control and run-time security, our customers are able to deploy Kubernetes with end-to-end security across their full container pipeline.”

The NeuVector Full Life-Cycle Container Security Platform

Beyond Kubernetes Admission Control, NeuVector provides a complete container security platform to automate container security from build to ship to run. The vulnerability scanning in the build and registry phases of the CI/CD pipeline are tied to the run-time environment with admission control. Complete run-time security is delivered by combining container and host security with the industry’s only Layer 7 container firewall, giving enterprises unprecedented visibility and protection for attacks, network threats, vulnerability exploits in containers, and host privilege escalations.