While getting stuff to just work is fun, I decided I couldn’t set up a cluster without at least giving some thought to security. Here’s my small attempt at a nominally useful security strategy.

By default, anything is allowed in Kubernetes. No, noone is stopping you. If you are on the node or in the cluster, you can contact *anything* in the cluster. This might be fine in a small home setup, but it doesn’t need to be advanced to be moderately useful.

Security rules comes in form of Network Policies. Since I am using Calico, I can choose between either stock network policies or the more powerful calico network policies. I decided to use the calico network policies.

Default Deny

The goal of any security policy in the end is denying anything that’s not specifically allowed. If you’re just starting your Kubernetes Cluster, it might in fact be a good idea to do that immediately. You can always override them with more relaxed rules while configuring your services’ network policies.

Default deny as in totally default deny might also be a bit harsh. Here’s a recommended version I found, that just excempts kubernetes system and core networking stuff from the network policy, but denying everything else. Oh, except DNS. We’ll allow DNS for everyone, because that’s such an integral part of Kubernetes.

So, here’s my current default deny policy:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: global-default-deny
spec:
  order: 1000
  namespaceSelector: kubernetes.io/metadata.name not in {"calico-system", "kube-public", "kube-system", "tigera-operator"}
  types:
  - Ingress
  - Egress
  egress:
   # allow all namespaces to communicate to DNS pods
  - action: Allow
    protocol: UDP
    destination:
      selector: 'k8s-app == "kube-dns"'
      ports:
      - 53
  - action: Allow
    protocol: TCP
    destination:
      selector: 'k8s-app == "kube-dns"'
      ports:
      - 53

Calico has the concept of a globalnetworkpolicy, that applies to your whole workload. Whatever isn’t matched with a specific match in the network policies you define will end up in this rule. It is generally a good idea of explicitly denying also in your workloads network policy , to try to make sure nothing ends in this policy. This is a last resort policy.

Structure your network policies

I’ve been a firewall admin at times. If it’s one thing you’ll regret, it’s allowing your security rules to become a mess. It will be hard to maintain, and the mess tends to grow. By no means will I suggest this is the only way, but it works for me.

For documenting and getting and overview, I try to keep my Kubernetes configuration in a file hierarchy where I have one directory per namespace. I try to avoid just creating something without saving the configuration here. So, I decided to keep my network policies in a file calicopolicy.yaml in each directory

Again using this blog as an example, I will outline my network policy

I will start off my namespace policy with another default deny rule

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
name: default-deny
namespace: wordpress
spec:
ingress:
- action: Deny
egress:
- action: Deny

This just denies everything, and is probably not particularly useful.

A tradeoff I have done so far it to allow anything in a namespace to talk to each other. I might improve on this later, but for me this is good enough. Here’s the policy for that:

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-all-in-namespace
  namespace: wordpress  # Change this to your namespace
spec:
  ingress:
    - action: Allow
      source:
        namespaceSelector: kubernetes.io/metadata.name == "wordpress"  # Allows traffic from any pod in the same namespace
  egress:
    - action: Allow
      destination:
        namespaceSelector: kubernetes.io/metadata.name == "wordpress"  # Allows traffic to any pod in the same namespace
  selector: all()  # Applies this policy to all pods in the namespace

Here, make sure the label kubernetes.io/metadata.name exists and equals wordpress

Then, we come to egress. I’m not restricting that either, so far:

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-egress
  namespace: wordpress
spec:
  order: 200
  egress:
    - action: Allow

Finally, we come to things I am restricting more: What enters the namespace. I want ingress traffic:

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-ingress-to-wordpress-service
  namespace: wordpress
spec:
  order: 300
  egress:
  ingress:
    - action: Allow
      protocol: TCP
      source:
        namespaceSelector: role == "ingress"  # Only allow from ingress namespace
      destination:
        selector: app == "wordpress"
        ports: [80]  # Allow traffic to the Service (which forwards to 3000 internally)
  selector: app == "wordpress"

I have labeled the traefik namespace with role = ingress, so we can filter on that.

Also, make sure the label app = «wordpress» exists on the wordpress pod(s).

Finally, metrics traffic, from the namespace with label role = «metrics»

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-metrics
  namespace: wordpress
spec:
  order: 400
  ingress:
    - action: Allow
      protocol: TCP
      source:
        namespaceSelector: role == "metrics"  # Only allow from ingress namespace
      destination:
        selector: app == "wordpress"
        ports: [11011]
  selector: app == "wordpress"

Note: The order here is the order in which they are executed. Generally I want more specific before more general rules, and make sure earlier rules doesn’t block for later rules.

Summary: Start off with something simple, just to get the applications to work. Once you are familiar with them, you can refine the policies. Sometimes, for example with wordpress, it’s obvious that the only intra-namespace traffic will be the app talking to the database, so you can replace the «allow-all-in-namespace» with

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-wordpress-to-database
  namespace: wordpress  # Change this to your namespace
spec:
  ingress:
    - action: Allow
      source:
        selector: app == "wordpress"  
  egress:
    - action: Allow
      destination:
        selector: app == "mariadb"
        ports: [3306]
  selector: all()  # Applies this policy to all pods in the namespace

Summary: Focus on the ingress, that will give you the most protection. The more knowledge you have on the application, the better policy can you make. But even a simple policy will give you security benefits.

And that’s all for now. There’s tons of possibilities with the security policies, so it will eventually come down to your own level of paraonia. For my personal setup, I’m not too worried, but it’s always good to catch the low hanging fruits with a simple network policy.