flowchart TB
subgraph cluster["Kubernetes Cluster"]
direction TB
subgraph controlPlane["Control Plane"]
api["kube-apiserver"]
controller["kube-controller-manager"]
scheduler["kube-scheduler"]
etcd["etcd"]
end
subgraph node["Node"]
kubelet["kubelet"]
proxy["kube-proxy"]
runtime["Container Runtime"]
cni["Container Network Interface (CNI)"]
subgraph pod["Pod"]
container["Container"]
end
end
end
classDef control fill:#DBEAFE,stroke:#2563EB,color:#1E3A8A
classDef worker fill:#CCFBF1,stroke:#0D9488,color:#134E4A
classDef workload fill:#EDE9FE,stroke:#7C3AED,color:#4C1D95
class api,controller,scheduler,etcd control
class kubelet,proxy,runtime,cni worker
class container workload
style cluster fill:#F8FAFC,stroke:#64748B,color:#0F172A
style controlPlane fill:#EFF6FF,stroke:#2563EB,color:#1E3A8A
style node fill:#F0FDFA,stroke:#0D9488,color:#134E4A
style pod fill:#F5F3FF,stroke:#7C3AED,color:#4C1D95
While working on Orion, my home Kubernetes cluster, I wanted to document the steps I took to get it running and the errors I ran into so my future self could refer back to them. This post starts with a simple overview of the components and how they work together.
The main parts
There are two main parts to start with, the control plane and the nodes. The control plane manages the cluster, and the nodes run the applications.
Control plane
etcd
The cluster needs somewhere to keep its data. This is where etcd comes in. It is a distributed key-value store that holds the configuration and recorded status of Kubernetes objects. The API server reads and writes this data for the other components.
Saving a change in etcd is only part of the work. A Pod can already be recorded there while its containers are still waiting to start.
kube-apiserver
When I run a kubectl command, it talks to the API server. The API server checks who is making the request, whether they have permission, and whether the request is valid.
kube-controller-manager
Once the cluster knows what should be running, something needs to keep checking that it is. The controller manager runs several controllers that do this work. They watch the cluster and make changes when its current state differs from what was requested. This is called reconciliation.
kube-scheduler
The controllers make sure the required Pods exist, and the scheduler decides which nodes those Pods should run on. It checks each Pod’s CPU and memory requests along with any rules about where it can run.
If ten new Pods need to run, the scheduler finds a place for each one. Starting the containers is then the node’s job. A Pod waits if no suitable node is available.
Nodes
Once a Pod has been assigned to a node, the components on that machine take care of running it.
kubelet
The kubelet runs on each node and watches for Pods assigned to it. It works with the container runtime to keep their containers running as specified. It also reports back to the API server so the cluster knows how the node and its Pods are doing.
Container runtime
The container runtime handles downloading images and starting and stopping containers. containerd and CRI-O are common choices. The kubelet talks to the runtime through the Container Runtime Interface, usually shortened to CRI.
Pods and containers
Kubernetes runs containers inside Pods. A Pod is the smallest unit it schedules onto a node. Usually there is one application container inside, though several containers can share a Pod when they need to run together.
For a small web server, a Pod could contain a single Nginx container. To run three copies, I would normally use three Pods. Containers within the same Pod stay together on one node, share networking and can share storage.
Pod networking and CNI
Pods also need a way to communicate. On Linux, the container runtime uses Container Network Interface plugins, or CNI plugins, to set up their networking. The cluster’s networking software lets Pods reach each other, even when they are on different nodes.
kube-proxy and Services
When a Pod is replaced, its IP address can change. A Service gives other applications a stable address to use, so they do not have to keep track of individual Pods. Cluster DNS lets applications find a Service by name. This is one way Kubernetes provides service discovery.
kube-proxy sets up the network rules that send traffic from a Service to its Pods. It usually runs on each node, though some networking tools handle this themselves and do not need kube-proxy.