Using Network Policy in Concert with Istio — Part 2

Welcome to Part 2 of our series on using Network Policy in concert with Istio. In this installment, we will recommend what policy controls to put in place if you are experimenting with Istio for your applications today.

2FA for Microservices

One great feature of Istio today is the ability to encrypt traffic in your service mesh with TLS. Your applications don’t have to manage certificates or even enable TLS themselves: the sidecar intercepts and encrypts traffic before it leaves the pod. The Envoy proxies mutually authenticate connections and this makes the service mesh recognize “self” versus connections from outside the mesh. This can be enabled globally at present. Combined with Network Policy this is like 2-factor authentication: an incoming connection has to have the right cryptographic identity and originate from the right IP.

Fine-Grained Isolation

For more fine-grained policy, it’s worth noting that Istio’s support is currently very limited: if you want to control access to your services you are limited to a blacklist or whitelist on a single attribute. Configuring Istio’s Mixer component for this kind of access control is complex and only a subset of the config is even available via the REST API. Full configuration is only possible by modifying files in the Mixer container. However, Istio is a new project and making rapid progress. The Envoy proxy Istio uses is itself very mature even though Istio only uses a small subset of its function at present.

For fine-grained isolation, I would strongly recommend you use Kubernetes Network Policy (implemented by Calico or some other plug-in) rather than Istio, even for HTTP services. In practice, it’s just very difficult to get Istio to do this today.

Also remember that while in Istio you apply policy to service versions, in Network Policy you apply to pods. To keep things sane you should consider having at least one unique label per service to make it easy to write Network Policy that selects all versions of your service.

Let’s walk through a few examples of what you might want to do with Kubernetes Network Policy for an Istio-enabled application. Consider the BookInfo sample application. We’re going to cover the following use cases for Network Policy:

• Reduce attack surface of the application ingress
• Enforce fine-grained isolation within the application

We will also consider use cases for egress policy, which is not available at present in the Kubernetes Network Policy API, but can be configured directly in Calico.

Reduce Attack Surface of the Application Ingress

Our application ingress controller is the main entry-point to our application from the outside world. A quick peek at istio.yaml (used to install Istio) defines the Istio ingress like this:

apiVersion: v1
kind: Service
metadata:
  name: istio-ingress
  labels:
    istio: ingress
spec:
  type: LoadBalancer
  ports:
  - port: 80
#   nodePort: 32000
    name: http
  - port: 443
    name: https
  selector:
    istio: ingress

The istio-ingress exposes ports 80 and 443. Let’s limit incoming traffic to just these two ports. Envoy has a built-in administrative interface, and we don’t want a misconfigured istio-ingress image to accidentally expose our admin interface to the outside world. This is an example of defense in depth: a properly configured image should not expose the interface, and a properly configured Network Policy will prevent anyone from connecting to it. Either can fail or be misconfigured and we are still protected.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: istio-ingress-lockdown
  namespace: default
spec:
  podSelector:
    matchLabels:
      istio: igress
  ingress:
  - ports:
    - protocol: TCP
      port: 80
    - protocol: TCP
      port: 443

Enforce fine-grained isolation within the application

Here is the service graph for the BookInfo application.

This graph shows every connection that a correctly functioning application should be allowed to make. All other connections, say from the Istio Ingress directly to the Rating service, are not part of the application. Let’s lock out those extraneous connections so they cannot be used by an attacker. Imagine, for example, that the Ingress pod is compromised by an exploit that allows an attacker to run arbitrary code. If we only allow connections to the Product Page pods using Network Policy, the attacker has gained no more access to our application backends even though they have compromised a member of the service mesh.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: product-page-ingress
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: productpage
  ingress:
  - ports:
    - protocol: TCP
      port: 9080
    from:
    - podSelector:
        matchLabels:
          istio: ingress

You can and should write a similar policy for each service to enforce which other pods are allowed to access it.

Egress Policy

As we discussed in Part 1 of this post, anyone with control of a pod in the Kubernetes cluster, whether a malicious user or an attacker, can use that pod to launch an attack — at other pods in and out of the service mesh, your infrastructure, or services outside Kubernetes like legacy databases. That’s one of the main reasons we also enforce egress policy with Calico. While Kubernetes Network Policy doesn’t yet support it, you can apply it with calicoctl, the Calico CLI tool.

For example, the Reviews service only accesses the Ratings service in our application. We can enforce this at the network layer with the following Calico policy.

apiVersion: v1
kind: policy
metadata:
 name: netpolicy-reviews-ew
spec:
 selector: app == 'reviews' &&  calico/k8s_ns == 'default'
egress:
 - action: allow
   destination:
      selector: app == 'ratings'

Also consider that many types of malware make connections to external servers to exfiltrate data or receive commands from their operators — Calico egress policy disallows these connections at the host.

Summary

Istio Auth + Network Policy is the one-two punch of policy layers working in concert to secure against attackers.

With Istio Auth and correctly configured Network Policy as above you won’t see any difference in your Istio-enabled application, even when using Istio’s advanced service routing to different versions of your service. That’s exactly what we want: for policy to stay out of the way of a correctly operating application!

While Istio is working fast to establish support for some of the use cases described in this post, at present we strongly recommend you use Network Policy. Where support does exist in Istio, it is hard to configure and uses very rough, alpha APIs. In the final post of this series we will look to the future for both Network Policy and Istio policy.

Next: Read about what the future development of Istio and Network Policy might look like, in Part 3 of this series.