Every pod in a Kubernetes cluster receives its own IP address, routable within the cluster without NAT for pod-to-pod communication. The Container Network Interface specification defines how container runtimes request network connectivity from pluggable implementations. CNI plugins handle IP address management, interface creation, routing, and increasingly network policy enforcement and observability. Choosing a CNI is one of the most consequential infrastructure decisions in a Kubernetes deployment — migration after production workloads land is painful and risky.
The CNI model executes a binary with JSON configuration whenever a pod is created or destroyed. The plugin allocates an IP from a configured range, attaches the container to a virtual network — bridge, VXLAN tunnel, or direct routing — and configures host routing so other nodes reach the new pod. Different plugins make different architectural choices about overlay versus underlay networking, BGP versus static routes, and where policy enforcement occurs. Understanding these tradeoffs prevents selecting a plugin that works in development but fails at production scale.
Flannel: Simplicity First
Flannel provides a basic overlay network using VXLAN or host-gw backends. VXLAN encapsulates pod traffic between nodes, requiring no routing protocol on the underlay — only IP connectivity between node addresses. Host-gw mode installs static routes on each node pointing pod CIDRs to peer node IPs, offering better performance when the underlay supports direct routing without encapsulation overhead. Flannel does not implement Kubernetes NetworkPolicy — clusters requiring policy must add a separate component like Calico policy controller or Cilium in policy-only mode.
Flannel suits small clusters, development environments, and teams prioritizing operational simplicity over advanced features. Configuration is minimal: define a pod CIDR, select a backend, deploy the DaemonSet. Limitations appear at scale: VXLAN overhead on high-throughput workloads, no built-in encryption, and no native integration with enterprise network policy frameworks. For production clusters without NetworkPolicy requirements and moderate scale, Flannel remains a valid choice that stays out of the way.
Calico: Policy and BGP
Calico supports multiple data plane modes including BGP-routed pod IPs without overlay, VXLAN or WireGuard overlays, and eBPF data planes on Linux. In BGP mode, each node peers with top-of-rack switches or route reflectors, advertising its pod CIDR as a host route. The physical network carries pod traffic natively without encapsulation — ideal for data centers where network teams accept /32 or /26 routes from Kubernetes nodes. NetworkPolicy enforcement uses iptables or eBPF depending on configuration.
Calico NetworkPolicy extends Kubernetes policy with Calico-specific resources supporting broader match criteria: host endpoints, global network sets, and hierarchical tiers. Integration with enterprise firewalls through BGP advertisement of service IPs and policy export satisfies security teams requiring consistent controls between VMs and containers. Calico's operational complexity scales with features enabled — BGP peering with physical fabric requires coordination with network engineering that Flannel avoids entirely. Windows node support and dual-stack IPv4/IPv6 are production-ready in current releases.
Cilium: eBPF-Native Networking
Cilium programs the Linux kernel using eBPF for networking, security, and observability without iptables chains that grow unwieldy at thousands of rules. Pod networking uses VXLAN, direct routing, or BGP modes similar to Calico but with eBPF replacing iptables for forwarding and policy. CiliumNetworkPolicy supports L3/L4 rules plus DNS-based and HTTP path-based policies — allowing rules like permit pods labeled frontend to call API paths matching /v1/users on backend services identified by DNS name rather than IP.
Cilium's service mesh capabilities through Envoy integration or eBPF-based load balancing reduce sidecar overhead compared to traditional Istio deployments. Hubble provides flow visibility built into the CNI — packet-level observability without deploying a separate monitoring agent on every node. For large clusters running hundreds of NetworkPolicies, eBPF scales more predictably than iptables. The learning curve is steeper and kernel version requirements are stricter — verify eBPF feature support on your node OS before committing.
Comparison and Selection Criteria
Evaluate CNIs against your team's skills, cloud provider managed offerings, Windows workload requirements, and policy complexity. Managed Kubernetes services often default to a specific CNI — GKE uses Dataplane V2 based on Cilium, EKS defaults to VPC CNI with optional add-ons, AKS offers Azure CNI or kubenet. Deviating from managed defaults is possible but increases support burden when issues span cloud provider and CNI vendor.
Performance testing with your actual workload profiles matters more than benchmark blog posts. Test pod-to-pod latency, throughput across nodes, NetworkPolicy rule scaling, and DNS resolution time under load. Measure control plane impact during large deployments — some CNIs generate significant API server load when thousands of pods start simultaneously. Document the selected CNI's IP address management model to prevent overlapping pod CIDRs during cluster upgrades or multi-cluster federation.
Migration and Operations
CNI migration in production requires draining nodes, reconfiguring networking, and validating policy equivalence — plan for maintenance windows and rollback paths. Run CNI components as DaemonSets with resource limits and health checks. Monitor IP address pool exhaustion; Calico and Flannel support configurable block sizes and aggregation. For multi-cluster connectivity, evaluate CNI-specific cluster mesh features in Cilium or service mesh overlays that abstract underlying CNI differences.
Kubernetes networking is not one-size-fits-all. Flannel for simplicity, Calico for BGP-integrated policy-rich environments, Cilium for eBPF performance and advanced observability — the right choice aligns with your network team's capabilities and your security requirements, not whichever plugin appears most frequently in tutorial documentation.
IPv6 and Dual-Stack Clusters
CNI dual-stack configuration assigns both IPv4 and IPv6 addresses to pods with varying maturity across plugins. Calico and Cilium support dual-stack with BGP advertisement of both address families. Verify cloud provider load balancer integration supports dual-stack Services before enabling — some cloud integrations create IPv4-only frontends despite dual-stack cluster configuration.
Network policy enforcement must cover both address families identically. A policy permitting TCP 443 on IPv4 but omitting IPv6 rules creates a bypass path for workloads that prefer IPv6. Audit NetworkPolicy resources for address family coverage during security reviews. Cilium supports FQDN-based policies that abstract address family concerns by targeting DNS names rather than IP literals.
Service mesh integration varies — Istio sidecar injection interacts with CNI chaining on some platforms, requiring CNI configuration that leaves room for istio-init container network setup. Cilium's service mesh mode reduces sidecar dependency but changes troubleshooting workflows. Document the interaction between your chosen CNI and service mesh before production deployment to avoid pod startup failures during rollout.