Run a multi-node Kubernetes Cluster locally using minikube
Run a multi-node Kubernetes Cluster locally using minikube
Minikube is a fantastic tool for running Kubernetes clusters locally, but out of the box it only creates a single-node cluster. What if you want to simulate a real multi-node environment, including persistent storage and volume snapshots? This guide will walk you through setting up a local multi-node Minikube cluster using the CSI Hostpath driver, enabling volume snapshots, and deploying a simple app across the nodes.
Why Multi-Node?
Using a multi-node Minikube cluster lets you:
- Test pod scheduling and anti-affinity rules.
- Simulate real-world distributed workloads.
- Experiment with persistent volumes, snapshots, and failover scenarios.
Prerequisites
Step 1: Configure Minikube for Docker
Set Docker as your default Minikube driver for best compatibility:
minikube config set driver dockerminikube config viewStep 2: Start a Multi-Node Cluster
Start a new Minikube cluster with 3 nodes and enable essential addons:
minikube start \ --nodes 3 \ --addons=metrics-server,volumesnapshots,csi-hostpath-driver \ --profile minikube-multinodeSet your shell to use this profile for convenience:
export MINIKUBE_PROFILE=minikube-multinodeminikube statusFor a graphical interface, launch:
minikube dashboardStep 3: Make CSI Hostpath the Default StorageClass (Optional)
By default, Minikube installs its own storage provisioner. To use the CSI Hostpath driver as the default storage class (optional but recommended for snapshotting):
minikube addons disable storage-provisionerminikube addons disable default-storageclasskubectl patch storageclass csi-hostpath-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'Verify the available snapshot classes:
kubectl get volumesnapshotclassesStep 4: Create and Snapshot a Volume
Create a PersistentVolumeClaim (PVC) that will use the CSI Hostpath provisioner:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: PersistentVolumeClaimmetadata: name: csi-pvcspec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: csi-hostpath-scEOFCheck the resulting PersistentVolume:
kubectl get pvCreate a snapshot of the PVC:
cat <<EOF | kubectl apply -f -apiVersion: snapshot.storage.k8s.io/v1kind: VolumeSnapshotmetadata: name: snapshot-demospec: volumeSnapshotClassName: csi-hostpath-snapclass source: persistentVolumeClaimName: csi-pvcEOFVerify the snapshot:
kubectl get volumesnapshotStep 5: Restore a PVC from a Snapshot
Create a new PVC that restores data from the earlier snapshot:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: PersistentVolumeClaimmetadata: name: csi-pvc-restorespec: storageClassName: csi-hostpath-sc dataSource: name: snapshot-demo kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io accessModes: - ReadWriteOnce resources: requests: storage: 1GiEOFCheck the PVCs:
kubectl get pvcTo remove the restored PVC:
kubectl delete pvc csi-pvc-restoreStep 6: Deploy an App Across Nodes
Let’s deploy a simple app (hello-from) with two replicas and anti-affinity rules so each pod lands on a different Minikube node:
cat <<EOF | kubectl apply -f -apiVersion: apps/v1kind: Deploymentmetadata: name: hellospec: replicas: 2 strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 100% selector: matchLabels: app: hello template: metadata: labels: app: hello spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: [{ key: app, operator: In, values: [hello] }] topologyKey: "kubernetes.io/hostname" containers: - name: hello-from image: pbitty/hello-from:latest ports: - name: http containerPort: 80 terminationGracePeriodSeconds: 1EOFCheck rollout status:
kubectl rollout status deployment/helloStep 7: Expose and Access the App
Expose the deployment using a NodePort service:
cat <<EOF | kubectl apply -f -apiVersion: v1kind: Servicemetadata: name: hellospec: type: NodePort selector: app: hello ports: - protocol: TCP nodePort: 31000 port: 80 targetPort: httpEOFGet pod IPs and node locations:
kubectl get pods -o wideOpen the service in your browser:
minikube service helloOr, curl the service directly (the port will be printed by the previous command):
curl http://127.0.0.1:<node-port>Step 8: Cleanup
Remove your test deployment and service:
kubectl delete svc hellokubectl delete deployment helloCluster Lifecycle Management
You can pause and unpause the Minikube cluster to save resources:
minikube pauseminikube unpauseStop the cluster when done:
minikube stopReferences
With this setup, you now have a realistic multi-node Kubernetes playground on your laptop, complete with dynamic volumes, snapshot/restore capabilities, and real pod distribution. Happy hacking!