How to Secure Containers in a Service Mesh such as Istio and Linkerd2

Visualize and Protect Service Mesh System and Application Containers with NeuVector

By Chip Hwang

We recently announced NeuVector integration with service meshes which enables powerful Layer 7 network inspection and protection of container traffic, even with pod to pod encryption on. This is an exciting technology leading capability from NeuVector which allows enterprises to protect Kubernetes in production by extending service mesh security features of Istio and linkerd2.

Before I get into the details of this integration and show how it helps simplify a complex service mesh, let me provide a quick recap of what a service mesh and the NeuVector Kubernetes Security Solution are.

What is a Service Mesh?

The success of Kubernetes has been a major enabler for the “service mesh” concept to become a reality, as a “sidecar” container is the ideal form-factor for service mesh functions to be placed together with the service itself.

Two key aspects of the service mesh are the data plane and the control plane. For the data plane, there are solutions such as Envoy, NGINX and Linkerd. For the control plane, there are solutions such as Istio and Conduit.

Service meshes can deliver functions such as:

• Routing
• Load balancing
• Blue-green
• Authentication
• Encryption

And benefits such as:

• Reliability
• Scalability
• Security
• Performance

You can read more about the concepts of a service mesh here.

The important thing to remember is that, although a service mesh has security features, it is NOT a security solution. It is not designed to provide the type of network, endpoint and host security required for defense in depth.

NeuVector – A Security Mesh for the Service Mesh

The NeuVector container security platform enhances the built-in security features of Kubernetes and service meshes such as Istio to provide true defense in depth. It does this through an end-to-end solution that provides:

• True layer 7 container firewall with deep packet inspection between containers
• Behavioral learning and automated creation of whitelist rules to segment containers
• Container and host process and file system security
• Vulnerability management for the entire CI/CD pipeline
• and many more cool things…

NeuVector is compatible with any service mesh, and you can think of it as a ‘security mesh‘ for your service mesh. However, the tight integration with service meshes now enables NeuVector to visualize, monitor and protect service mesh system connections and containers in addition to application workloads managed by a service mesh.

NeuVector Integration with Istio and Linkerd2

Istio as part of its security features provides authentication and authorization between services within its service mesh. Istio provides authentication between services using mutual TLS and traffic between services are encrypted. You can watch the demo video of this integration at the end of this post.

NeuVector can work alongside Istio authentication and provide deep packet inspection for network threats and the ability to segment services at the application layer. This ability is unique to NeuVector as it is the only container security platform that can interrogate network traffic within Istio’s encrypted tunnels.

Istio also has the ability to define authorization between services. The process of creating authorization requires several steps. The following is an example taken directly from the Istio documentation to allow access to following the services.

“details.default.svc.cluster.local”,
“reviews.default.svc.cluster.local”

Step 1. Create a Service Role to Allow Access to Services

apiVersion: "rbac.istio.io/v1alpha1"

kind: ServiceRole
metadata:
  name: details-reviews-viewer
  namespace: default
spec:
  rules:
  - services: ["details.default.svc.cluster.local", "reviews.default.svc.cluster.local"]

Step 2. Assigning the Access to Allowed Service

apiVersion: "rbac.istio.io/v1alpha1"
kind: ServiceRoleBinding
metadata:
  name: bind-details-reviews
  namespace: default
spec:
  subjects:
  - user: "cluster.local/ns/default/sa/bookinfo-productpage"
  roleRef:
    kind: ServiceRole
    name: "details-reviews-viewer"

These rules need to be defined for each set of services that is allowed to communicate, which can be time consuming, prone to error, and present a management nightmare.

With NeuVector, layer 7 segmentation rules are generated automatically using behavioral learning. NeuVector defines the rules on the protocol level by examining the network traffic with deep packet inspection even if the network traffic is encrypted by Istio. NeuVector does not rely on port numbers to determine the protocol type and uses the learned rules to secure communication between services.

In addition, NeuVector protects Istio system components from network based attacks by automatically creating a baseline of network traffic and processes for each Istio system service and can take action if any anomalous behavior is detected.

Using Istio’s authentication and NeuVector provides secure encrypted traffic between services and the ease of segmenting services at the application layer plus threat detection. As a side note, if you do choose to use Istio authorization together with NeuVector, they are compatible and will co-exist fine.

NeuVector Istio View

This is a view of the Istio System traffic. With the addition of Istio to any Kubernetes Cluster there is an explosion of east/west traffic which makes monitoring of network traffic within the cluster difficult.

FIGURE 1. ISTIO SYSTEM TRAFFIC

NeuVector simplifies this by providing the ability to remove Istio control Traffic from view so you can focus on traffic between your application services.

FIGURE 2. APPLICATION ONLY TRAFFIC

With NeuVector we can see both the unencrypted traffic between each application container within a pod as well as the encrypted TLS traffic between the envoy containers.

FIGURE 3. APPLICATION TRAFFIC AND ENCRYPTED TLS TRAFFIC

NeuVector automatically creates whitelist rules based on application traffic going across each service. NeuVector uses deep packet inspection to determine the application protocol. It does not rely on port numbers to determine the application. Even if the traffic is in encrypted with Istio’s service mesh.

FIGURE 4. WHITE LIST RULES

NeuVector will also monitor for threats that are outside of the Istio service mesh such as attacks that are based on ICMP or UDP traffic.

FIGURE 5. ATTACKS OUTSIDE OF SERVICE MESH

Once the whitelist rules are in place and a baseline for each service is created the network traffic that is outside of the baseline will be considered anomalous and will be alerted.

FIGURE 6. VIOLATIONS

Within the service mesh NeuVector will be able to detect embedded attacks even if there is a whitelist rule in place to allow the application traffic between the services. This embedded threat detection works even if you are using Istio’s authorization features to control access between pods.

FIGURE 7. EMBEDDED ATTACKS

NeuVector can also provide custom responses. NeuVector can quarantine an offending container. This will block all network connections in and out of the container, but the container will be kept alive so forensic analysis can be done on it. NeuVector can also perform automatic packet captures when a threat or violation is detected. In the following example a pod is using a DNS Tunnel to exfiltrate data. NeuVector using deep packet inspection is able to determine that the traffic is not DNS and quarantine the container. The traffic is going to a system service which is often overlooked by security tools and teams, even though it could be a potential exploit technique.

FIGURE 8. QUARANTINE OF CONTAINER

NeuVector can also do automated packet capture in response to suspicious activity even if traffic is encrypted by the service mesh.

FIGURE 9. AUTOMATED NETWORK CAPTURE

Watch the NeuVector – Istio Integration Demo Video

A Complete CI/CD Container Security Solution

Istio is a great traffic management tool for a Kubernetes environment. It does not, however, provide all the tools required to secure a containerized environment. NeuVector is a full multi-vector security platform that secures your containerized environment and provides advance container network security features such as packet capture and quarantine . It also provides system security at the process and file system level.

Together Istio and NeuVector deliver a complete application layer service mesh and security mesh for Kubernetes.