Services & networking in K8s

Services: one stable address in front of disposable pods

Section 1 of 5 · ~3 min

Yesterday’s Deployment taught you the uncomfortable truth about pods: they are cattle, not pets. The scheduler kills and recreates them on a whim — a rolling update, a crashed node, a scale-up — and every new pod gets a brand-new IP. So the moment one pod wants to reach another (your web pod calling your API pod), it can’t hard-code an IP: that IP is gone by lunchtime. A Service solves exactly this. It’s a stable, long-lived virtual IP and DNS name that sits in front of a set of pods and load-balances traffic across whichever ones are alive right now.

The glue is labels. A Service carries a selector — say app: linkstash — and Kubernetes continuously watches for every pod whose labels match, recording their pod IP and port as endpoints. Scale the Deployment from 3 to 5 pods and the endpoint list grows automatically; a pod crashes and it drops off. You never touch the Service. The selector is a live query, not one-time wiring.

There are three Service types, each a superset of the last:

• ClusterIP (the default) — a virtual IP reachable only inside the cluster. Perfect for pod-to-pod traffic like a web tier calling a database.
• NodePort — everything ClusterIP does, plus it opens the same high port (30000–32767) on every node, so traffic hitting <node-ip>:<nodeport> is forwarded in. The crude way to reach a service from outside.
• LoadBalancer — everything NodePort does, plus it asks the cloud provider for a real external load balancer with a public IP. On a local kind cluster there’s no cloud, so this one sits at EXTERNAL-IP: <pending> forever — expected, not broken.

How does traffic actually reach a pod? kube-proxy runs on every node and turns each Service into real packet-forwarding rules (iptables or IPVS). When something sends a packet to the Service’s ClusterIP, those rules rewrite the destination to one of the healthy endpoint pod IPs — that’s the load balancing, done in the kernel, no proxy process in the hot path.

And how do pods find a Service by name? Cluster DNS. Every cluster runs CoreDNS, and every Service automatically gets a record: <service>.<namespace>.svc.cluster.local. From a pod in the same namespace you can use the short name linkstash; from another namespace you use the fully-qualified linkstash.default.svc.cluster.local. That name resolves to the Service’s ClusterIP, kube-proxy takes it from there, and your app never learns which pod answered.

Real world: A Service is the single hotline number for a pizza chain. You always dial the same number; it quietly routes your call to whichever branch is open and staffed tonight. Branches (pods) open, close, and get renovated; the number on the fridge magnet — the ClusterIP and its DNS name — never changes.

CoreDNS makes this concrete. It’s the CNCF graduated project that ships as the DNS server in virtually every modern cluster, including the one kind just handed you. It watches the Kubernetes API for Services and answers *.svc.cluster.local queries from your pods. Learn to read kubectl get endpoints and resolve a name from inside a pod, and you can debug most “why can’t my app reach the other service” problems on sight.

Hands-On Lab

Section 2 of 5 · ~4 min

Budget about 25 minutes. This runs entirely on your local kind cluster from Day 67 — Kubernetes in Docker on WSL2, no cloud and no bill. Type each command yourself and read every line. Pod name suffixes, pod IPs, the ClusterIP, node names and ages are assigned at runtime, so yours will differ from the samples below.

# 1. Confirm your local kind cluster is up (Day 67). Everything today runs on kind — no cloud.
kubectl get nodes
# Output (node name and AGE will differ; kind names the node <cluster>-control-plane):
# NAME                 STATUS   ROLES           AGE   VERSION
# kind-control-plane   Ready    control-plane   2d    v1.31.0

# 2. linkstash.yaml — a 3-replica Deployment. Every pod carries the label app: linkstash.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: linkstash
spec:
  replicas: 3
  selector:
    matchLabels:
      app: linkstash
  template:
    metadata:
      labels:
        app: linkstash
    spec:
      containers:
        - name: linkstash
          image: gcr.io/google-samples/hello-app:1.0
          ports:
            - containerPort: 8080

Heads-up: These Kubernetes labs use a public stand-in image (hello-app) in place of the linkstash image you built in Project 1, so every command runs even if you skipped that project. The Deployment and labels still read linkstash so the walk matches your real app.

# 3. Create the Deployment, then list pods with their IPs. Note each pod has its OWN IP.
kubectl apply -f linkstash.yaml
kubectl get pods -o wide
# Output (pod name suffixes, IPs and NODE are assigned at runtime — yours will differ):
# deployment.apps/linkstash created
# NAME                        READY   STATUS    RESTARTS   AGE   IP           NODE
# linkstash-7c9f8b6d4-2xk9p   1/1     Running   0          20s   10.244.0.6   kind-control-plane
# linkstash-7c9f8b6d4-8mtqr   1/1     Running   0          20s   10.244.0.7   kind-control-plane
# linkstash-7c9f8b6d4-lp4wz   1/1     Running   0          20s   10.244.0.8   kind-control-plane

# 4. svc-clusterip.yaml — a ClusterIP Service. Its selector must match the pods' labels.
apiVersion: v1
kind: Service
metadata:
  name: linkstash
spec:
  type: ClusterIP
  selector:
    app: linkstash        # matches the Deployment's pod label
  ports:
    - port: 80            # the Service's virtual port
      targetPort: 8080    # the container's listening port

# 5. Create the Service and list it. CLUSTER-IP is the stable virtual IP; EXTERNAL-IP stays <none>.
kubectl apply -f svc-clusterip.yaml
kubectl get svc linkstash
# Output (CLUSTER-IP is assigned from the service range — yours will differ):
# service/linkstash created
# NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
# linkstash   ClusterIP   10.96.132.47   <none>        80/TCP    5s

# 6. The proof the selector worked: the Service has 3 endpoints — one per matching pod IP:port.
kubectl get endpoints linkstash
# Output (these are the pod IPs from step 3, on the container's 8080 — yours will differ):
# NAME        ENDPOINTS                                           AGE
# linkstash   10.244.0.6:8080,10.244.0.7:8080,10.244.0.8:8080     30s

# 7. Reach the Service from your host. port-forward holds one terminal; curl from a SECOND terminal.
kubectl port-forward svc/linkstash 8080:80
# Output (leave this running):
# Forwarding from 127.0.0.1:8080 -> 8080
#
# In a second terminal:  curl -s http://localhost:8080/
# Output (hello-app's response — the Hostname is the pod that answered, so yours will differ):
# Hello, world!
# Version: 1.0.0
# Hostname: linkstash-<pod-suffix>

# 8. svc-nodeport.yaml — same selector, but type NodePort opens a fixed port on every node.
apiVersion: v1
kind: Service
metadata:
  name: linkstash-np
spec:
  type: NodePort
  selector:
    app: linkstash
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 30080     # must be 30000–32767, or omit to auto-assign

# 9. Create it, see the node port, then curl the NodePort from a throwaway pod.
#    (On kind the host can't reach a NodePort unless you set extraPortMappings at cluster create.)
kubectl apply -f svc-nodeport.yaml
kubectl get svc linkstash-np
# Get the node's internal IP, then curl the NodePort from a throwaway pod:
NODE_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
kubectl run tmp --image=curlimages/curl:8.9.1 --rm -it --restart=Never -- curl -s $NODE_IP:30080/
# Output (PORT(S) shows the 80:30080 mapping; the page is served through the node port):
# service/linkstash-np created
# NAME           TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
# linkstash-np   NodePort   10.96.44.10   <none>        80:30080/TCP   3s
# Hello, world!
# Version: 1.0.0
# Hostname: linkstash-<pod-suffix>

# 10. Resolve the Service name from inside a throwaway pod — this is cluster DNS (CoreDNS) at work.
kubectl run dnstest --image=busybox:1.36 --rm -it --restart=Never -- nslookup linkstash
# Output (the ClusterIP matches step 5; the FQDN is the full cluster-DNS name — IPs yours will differ):
# Server:    10.96.0.10
# Address:   10.96.0.10:53
# Name:      linkstash.default.svc.cluster.local
# Address:   10.96.132.47
# pod "dnstest" deleted

# 11. Clean up everything you created. The kind cluster itself stays for tomorrow.
kubectl delete -f svc-nodeport.yaml -f svc-clusterip.yaml -f linkstash.yaml
# Output:
# service "linkstash-np" deleted
# service "linkstash" deleted
# deployment.apps "linkstash" deleted

Read step 6 and step 10 back to yourself: the selector turned a set of disposable pods into a stable endpoint list, and a single DNS name resolved to one virtual IP in front of all three. That is the whole Service model — a name and an IP that outlive the pods behind them.

Common Errors & Fixes

Section 3 of 5 · ~3 min

These are the mistakes that trip people up the first time they wire Services on Ubuntu 24.04 with kind. Read the error text slowly; parsing it is the skill.

Common error: A Service whose selector doesn’t match any pod’s labels — a typo like app: linkstash-web instead of app: linkstash:

$ kubectl get endpoints linkstash
NAME        ENDPOINTS   AGE
linkstash   <none>      10s

Why: The selector is a live label query. If no running pod carries labels matching it, the endpoint list is empty and the Service has nowhere to send traffic — every request to its ClusterIP times out even though the pods are perfectly healthy.

Fix: Compare the two sides — kubectl get pods --show-labels against the Service’s selector (kubectl describe svc linkstash). Edit whichever is wrong so they match exactly; endpoints populate within a second.

How you’d spot it in prod: A service returning connection timeouts while its pods are Running and Ready almost always means empty endpoints. kubectl get endpoints <svc> showing <none> is the instant tell — check the selector-versus-label match before touching anything else.

Common error: A Service pointing targetPort at a port the container isn’t listening on — e.g. targetPort: 80 when linkstash serves on 8080:

$ curl -s http://localhost:8080/
curl: (52) Empty reply from server

Why: The Service has endpoints and DNS resolves fine, but kube-proxy forwards the connection to port 80 inside the pod, where nothing is listening. The pod resets the connection, so curl gets an empty reply (or “connection refused”) — a routing problem, not a DNS one.

Fix: Make targetPort equal the container’s real containerPort. Check what the app listens on (kubectl exec <pod> -- ss -tlnp, or the image’s docs), set targetPort: 8080, and re-apply.

How you’d spot it in prod: DNS resolves and endpoints exist, yet connections are refused or empty — suspect a mismatch between the Service’s targetPort and the container’s port. This is the classic “the Service is up but returns nothing” bug.

Common error: Creating a type: LoadBalancer Service on a local kind cluster and waiting for an external IP that never arrives:

$ kubectl get svc linkstash-lb
NAME           TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
linkstash-lb   LoadBalancer   10.96.51.9    <pending>     80:31567/TCP   2m

Why: A LoadBalancer Service asks the cloud provider to provision a real external load balancer. kind runs locally in Docker with no cloud controller, so nothing fulfils the request and EXTERNAL-IP stays <pending> — this is expected, not a failure.

Fix: Locally, reach the app with kubectl port-forward or a NodePort instead. If you genuinely want LoadBalancer semantics on kind, install a bare-metal controller like MetalLB (or Cloud Provider KIND) to hand out addresses. On real cloud clusters the provider fills EXTERNAL-IP within a minute.

How you’d spot it in prod: An EXTERNAL-IP stuck at <pending> on a real cluster means the cloud controller can’t provision the LB — wrong IAM permissions, a subnet or quota limit, or a missing controller. On kind it simply means there’s no cloud to ask.

Services & Networking Interview Questions

Section 4 of 5 · ~1 min

Cover the answers below and say your own version out loud first — define what a Service is and name the three types, then explain how a pod finds one by DNS, before you reveal each answer. Recalling before revealing is what makes these stick when an interviewer asks them cold. The four questions and answers render right after this note.

Go Deeper

Section 5 of 5 · ~1 min

Optional extras if you have ~40 more minutes:

• 5 min — Run kubectl get endpointslices -l kubernetes.io/service-name=linkstash, then kubectl scale deploy/linkstash --replicas=5 and watch the endpoint list grow — the live selector query in action.
• 10 min — Peek at how kube-proxy wires it: docker exec kind-control-plane iptables-save | grep linkstash to see the ClusterIP rewritten to pod IPs in the kernel (or read up on the newer IPVS mode).
• 10 min — Make a headless Service (clusterIP: None) and resolve it from a pod: DNS returns every pod IP instead of one virtual IP — the pattern StatefulSets rely on.
• 15 min — Read the official Kubernetes Service docs on service types and EndpointSlices to see how what you just ran maps back to the API.