Run a multi-node Kubernetes Cluster locally using minikube
Run a multi-node Kubernetes Cluster locally using minikube
Minikube is a fantastic tool for running Kubernetes clusters locally, but out of the box it only creates a single-node cluster. What if you want to simulate a real multi-node environment, including persistent storage and volume snapshots? This guide will walk you through setting up a local multi-node Minikube cluster using the CSI Hostpath driver, enabling volume snapshots, and deploying a simple app across the nodes.
Why Multi-Node?
Using a multi-node Minikube cluster lets you:
- Test pod scheduling and anti-affinity rules.
- Simulate real-world distributed workloads.
- Experiment with persistent volumes, snapshots, and failover scenarios.
Prerequisites
Step 1: Configure Minikube for Docker
Set Docker as your default Minikube driver for best compatibility:
minikube config set driver dockerminikube config view
Step 2: Start a Multi-Node Cluster
Start a new Minikube cluster with 3 nodes and enable essential addons:
minikube start \ --nodes 3 \ --addons=metrics-server,volumesnapshots,csi-hostpath-driver \ --profile minikube-multinode
Set your shell to use this profile for convenience:
export MINIKUBE_PROFILE=minikube-multinodeminikube status
For a graphical interface, launch:
minikube dashboard
Step 3: Make CSI Hostpath the Default StorageClass (Optional)
By default, Minikube installs its own storage provisioner. To use the CSI Hostpath driver as the default storage class (optional but recommended for snapshotting):
minikube addons disable storage-provisionerminikube addons disable default-storageclasskubectl patch storageclass csi-hostpath-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
Verify the available snapshot classes:
kubectl get volumesnapshotclasses
Step 4: Create and Snapshot a Volume
Create a PersistentVolumeClaim
(PVC) that will use the CSI Hostpath provisioner:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: PersistentVolumeClaimmetadata: name: csi-pvcspec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: csi-hostpath-scEOF
Check the resulting PersistentVolume:
kubectl get pv
Create a snapshot of the PVC:
cat <<EOF | kubectl apply -f -apiVersion: snapshot.storage.k8s.io/v1kind: VolumeSnapshotmetadata: name: snapshot-demospec: volumeSnapshotClassName: csi-hostpath-snapclass source: persistentVolumeClaimName: csi-pvcEOF
Verify the snapshot:
kubectl get volumesnapshot
Step 5: Restore a PVC from a Snapshot
Create a new PVC that restores data from the earlier snapshot:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: PersistentVolumeClaimmetadata: name: csi-pvc-restorespec: storageClassName: csi-hostpath-sc dataSource: name: snapshot-demo kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io accessModes: - ReadWriteOnce resources: requests: storage: 1GiEOF
Check the PVCs:
kubectl get pvc
To remove the restored PVC:
kubectl delete pvc csi-pvc-restore
Step 6: Deploy an App Across Nodes
Let’s deploy a simple app (hello-from
) with two replicas and anti-affinity rules so each pod lands on a different Minikube node:
cat <<EOF | kubectl apply -f -apiVersion: apps/v1kind: Deploymentmetadata: name: hellospec: replicas: 2 strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 100% selector: matchLabels: app: hello template: metadata: labels: app: hello spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: [{ key: app, operator: In, values: [hello] }] topologyKey: "kubernetes.io/hostname" containers: - name: hello-from image: pbitty/hello-from:latest ports: - name: http containerPort: 80 terminationGracePeriodSeconds: 1EOF
Check rollout status:
kubectl rollout status deployment/hello
Step 7: Expose and Access the App
Expose the deployment using a NodePort service:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: Servicemetadata: name: hellospec: type: NodePort selector: app: hello ports: - protocol: TCP nodePort: 31000 port: 80 targetPort: httpEOF
Get pod IPs and node locations:
kubectl get pods -o wide
Open the service in your browser:
minikube service hello
Or, curl
the service directly (the port will be printed by the previous command):
curl http://127.0.0.1:<node-port>
Step 8: Cleanup
Remove your test deployment and service:
kubectl delete svc hellokubectl delete deployment hello
Cluster Lifecycle Management
You can pause and unpause the Minikube cluster to save resources:
minikube pauseminikube unpause
Stop the cluster when done:
minikube stop
References
With this setup, you now have a realistic multi-node Kubernetes playground on your laptop, complete with dynamic volumes, snapshot/restore capabilities, and real pod distribution. Happy hacking!