Network policy that someone can debug

An empty cluster is a flat network: every pod can reach every other pod, in every namespace. NetworkPolicy is how you stop that, and the reason most teams never finish rolling it out is that the first attempt breaks something at 9am.

The model, which is unlike a firewall

• Policies are allow-only. There is no deny rule.
• A pod with no policy selecting it allows everything.
• A pod with any policy selecting it denies everything not explicitly allowed, for that direction.
• Ingress and egress are independent. A policy with only ingress rules leaves egress wide open.

Default-deny, and why it bites

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: team-a
spec:
  podSelector: {}            # every pod in the namespace
  policyTypes: [Ingress, Egress]

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: team-a
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }

The selector trap that silently widens a rule

In a single to or from block, listing namespaceSelector and podSelector as two array items is an OR. Putting them under one item, as above, is an AND. The YAML difference is a single dash:

# AND - pods matching app=api, in namespaces matching team=a
from:
  - namespaceSelector: { matchLabels: { team: a } }
    podSelector: { matchLabels: { app: api } }

# OR - every pod in those namespaces, PLUS every app=api pod anywhere
from:
  - namespaceSelector: { matchLabels: { team: a } }
  - podSelector: { matchLabels: { app: api } }

The second form is almost never what anyone means, and it reviews as correct. It is the most common real defect in policy I have seen.

Roll it out without an incident

1. Start in a non-production namespace and get DNS right there first.
2. Write the allow policies before the deny, and apply them first. They do nothing on their own, because nothing is denied yet.
3. Apply default-deny per namespace, never cluster-wide in one go.
4. Watch for drops rather than waiting for complaints.

Proving it, instead of hoping

# the direct test, from a pod that should be refused
kubectl run probe --rm -it --image=nicolaka/netshoot -n team-b -- \
  curl -m 3 http://api.team-a.svc.cluster.local

# with Cilium: watch the verdicts live
cilium hubble observe --verdict DROPPED --namespace team-a

# with Cilium: ask, rather than test
cilium connectivity test

Plain upstream NetworkPolicy has no observability at all — a dropped packet is just a timeout. That absence is the strongest practical argument for Cilium or Calico over a minimal CNI: not the policy features, but being able to see the drop and name the rule that caused it.

What to actually do with this

• Apply default-deny plus DNS to one non-production namespace this week.
• Grep your existing policies for the OR-versus-AND mistake. Check every from block with two dashes.
• Pick a CNI that can show you drops before you need to debug one.