K8s Cluster Network Topology

proxmox-k8s · Talos + Cilium on 192.168.20.0/24

LAN Physical topology

192.168.20.0/24 Gateway .1 AdGuard DNS .2 (LXC) Proxmox .45 (host) k8s-cp-1 .50 (VM 800) 4 CPU · 8G RAM k8s-worker-1 .51 (VM 801) RTX 3090 24G k8s-worker-2 .52 (VM 802) RTX 4070 16G Cilium L2 LoadBalancer Pool 192.168.20.60 – 80 · ARP announced on LAN by worker nodes VXLAN tunnels (Cilium) External traffic → e.g. browser → .60 (ArgoCD)

All VMs run on a single Proxmox host (pve). Each K8s node VM gets a static IP on the LAN via cloud-init. GPUs are passed through via VFIO-PCI.

IPs Address allocation

IP / RangeAssignmentManaged by
192.168.20.1 Gateway / router static
192.168.20.2 AdGuard Home DNS (Proxmox LXC) OpenTofu
192.168.20.10 Proxmox VE host (pve) static
192.168.20.50 k8s-cp-1 — control plane OpenTofu
192.168.20.51 k8s-worker-1 — RTX 3090 (LLM inference) OpenTofu
192.168.20.52 k8s-worker-2 — RTX 4070 (Immich / ML) OpenTofu
192.168.20.60 – 80 Cilium L2 LoadBalancer pool Cilium

Node IPs are set via Proxmox cloud-init. LB IPs are virtual — Cilium responds to ARP on behalf of worker nodes, so external clients reach Services at those IPs on the LAN.

K8s Internal IP spaces

CIDRPurposeAssigned byExample
10.244.0.0/16 Pod IPs — every pod gets a unique IP Cilium 10.244.1.15 (an ArgoCD pod)
10.96.0.0/12 Service ClusterIPs — stable virtual IPs for services kube-api 10.96.0.10 (CoreDNS)
192.168.20.50-52 Node IPs — used by kubelet, Cilium tunnels cloud-init 192.168.20.51 (worker-1)

Pod and Service IPs are virtual — they exist only inside the cluster. Cilium's eBPF programs route packets between all three spaces. Node IPs must be unique LAN addresses for VXLAN tunnels to work.

DNS How pods find each other

Any Pod nameserver: 10.96.0.10 search: svc.cluster.local 1 DNS query → 10.96.0.10 CoreDNS pod: 10.244.0.8 svc: 10.96.0.10 (kube-dns) watches kube-api for endpoints 2 Reply → ClusterIP Service (ClusterIP) virtual IP — no interface, no process 3 Connect to ClusterIP Cilium eBPF DNAT: ClusterIP → real pod IP 4 Rewrite dst to pod IP Target Pod pod IP: 10.244.x.y may be on a different node 5 VXLAN encap if cross-node VXLAN tunnel uses node IPs (192.168.20.50↔51↔52)
DNS resolution
Service (virtual IP)
Cilium eBPF + VXLAN
Target pod

Kubelet injects nameserver 10.96.0.10 into every pod's /etc/resolv.conf. CoreDNS watches the kube-api for Service/Endpoint changes and serves name.namespace.svc.cluster.local → ClusterIP records. Cilium replaces kube-proxy entirely — Service routing is done in eBPF, not iptables.

L2 External access (LoadBalancer)

Browser / client on your LAN ARP: who has .60? Cilium L2 Announcer responds with worker MAC TCP to 192.168.20.60:80 Service type: LoadBalancer externalIP: 192.168.20.60 ArgoCD 10.244.x.y Pool: .60–.80 (CiliumLBIPPool)

Cilium L2 Announcements replace MetalLB. When a type: LoadBalancer Service is created, Cilium assigns an IP from the pool and responds to ARP on the LAN. External clients reach the Service at that IP without any router config.

L7 Gateway API — HTTP routing

Browser → 192.168.20.60 LB IP Gateway homelab listener: HTTP :80 allowedRoutes: All namespaces gatewayClassName: nginx ← switch GatewayClass nginx GatewayClass cilium NGF controller NGINX data plane Cilium gateway Envoy data plane ACTIVE idle HTTPRoute resources define path/host → backend mapping HTTPRoute host: argocd.home.lab parentRef: gateway/homelab → argocd-server:443 HTTPRoute host: ollama.home.lab parentRef: gateway/homelab → ollama:11434 HTTPRoute host: immich.home.lab parentRef: gateway/homelab → immich-server:2283 Each route forwards to a backend K8s Service → Cilium routes to pods Full path (external request → pod): Browser → LB IP (.60) → Cilium L2 ARP → NGINX pod → HTTPRoute match → backend Service ClusterIP → Cilium eBPF DNAT → pod Cilium handles L3/L4 throughout. The Gateway controller (NGINX or Envoy) only handles L7 (HTTP parsing, host/path matching, TLS).

Gateway API is the successor to Ingress. A single Gateway can serve many HTTPRoute resources across namespaces. The gatewayClassName field selects which controller (NGINX or Cilium/Envoy) handles the traffic — change it, push, and ArgoCD switches the L7 provider. Both controllers are installed; only the selected one processes the Gateway.

Stack Networking layers

L3/L4 — Cilium

CNI (pod networking), kube-proxy replacement (eBPF Service routing), VXLAN tunnels between nodes, L2 LoadBalancer announcements, network policy enforcement.

DNS — CoreDNS

Cluster DNS at 10.96.0.10. Resolves svc.cluster.local names to ClusterIPs. Injected into every pod by kubelet. Forwards non-cluster queries to 192.168.20.2 (AdGuard).

L7 — Gateway API

Two providers installed: NGINX Gateway Fabric (default) and Cilium/Envoy. Switch via gatewayClassName in the Gateway resource. HTTP host/path routing, TLS termination, cross-namespace routes.

TLS — cert-manager

Automated Let's Encrypt certificates via DNS-01 challenge. Provisions wildcard certs for Gateway listeners.