Velero-Based Disaster Recovery for Amazon EKS: A Complete Hands-On Guide

Disaster Recovery isn't optional anymore — it's table stakes. In this guide, I walk through building a production-grade, multi-region DR setup for Amazon EKS using Velero. Everything is installed via Helm, secured with IRSA (no static credentials), and backed by S3 with Cross-Region Replication. I've documented every step including the real errors I hit and how I fixed them.

Architecture & DR Flow

The setup follows an Active–Passive pattern. The primary cluster in us-east-1 runs all workloads. Velero backs up cluster state to S3, which replicates to a DR bucket in ap-south-1. A standby EKS cluster in Mumbai reads from the replicated bucket and restores workloads on demand.

Creating EKS Clusters

Two clusters — primary in US East, DR in Mumbai. eksctl handles VPC, subnets, IAM roles, and node groups automatically.

# Primary Cluster
eksctl create cluster --name primary-cluster --region us-east-1 \
  --version 1.35 --nodegroup-name primary-nodes \
  --node-type t3.medium --nodes 1 --managed

eksctl provisioning primary-cluster in us-east-1 via CloudFormation

# DR Cluster
eksctl create cluster --name dr-cluster --region ap-south-1 \
  --version 1.35 --nodegroup-name dr-nodes \
  --node-type t3.medium --nodes 1 --managed

DR cluster creation in ap-south-1 (Mumbai)

Velero Installation via Helm (GitOps)

Helm values stored in GitHub for version-controlled, repeatable deployments.

values-primary.yaml and values-dr.yaml created and stored in Git

configuration:
  backupStorageLocation:
    - bucket: velero-primary-bucket
      provider: aws
      config:
        region: us-east-1
  volumeSnapshotLocation:
    - provider: aws
      config:
        region: us-east-1
serviceAccount:
  server:
    name: velero
    create: false
initContainers:
  - name: velero-plugin-for-aws
    image: velero/velero-plugin-for-aws:v1.9.0

IRSA + Helm Install

IRSA creation on primary cluster

helm install velero — STATUS: deployed

Fix: IRSA Credential Issue

helm upgrade velero vmware-tanzu/velero --namespace velero \
  -f values-primary.yaml \
  --set credentials.useSecret=false \
  --set serviceAccount.server.create=false \
  --set serviceAccount.server.name=velero \
  --set serviceAccount.server.annotations."eks\.amazonaws\.com/role-arn"="<ROLE_ARN>"

After helm upgrade: BSL shows Available — Velero reaches S3 via IRSA

DR Cluster: Same Process

IRSA + Velero Running on DR cluster

Helm upgrade on DR — deployed, revision 2

S3 Buckets & Cross-Region Replication

CRR is the heart of DR. Every backup Velero writes to the primary bucket gets automatically replicated to Mumbai.

aws s3 mb s3://velero-primary-bucket --region us-east-1
aws s3 mb s3://velero-dr-bucket --region ap-south-1
aws s3api put-bucket-versioning --bucket velero-primary-bucket \
  --versioning-configuration Status=Enabled
aws s3api put-bucket-versioning --bucket velero-dr-bucket \
  --versioning-configuration Status=Enabled

Replication rule: primary → dr-bucket (ap-south-1)

CRR rule active — entire bucket, enabled

Backup, Restore & DR Simulation

Deploy App & Backup

Velero Running + nginx deployment created in demo namespace

velero backup create test-backup --include-namespaces demo

test-backup: Completed, 0 errors, 0 warnings

9 backup objects in velero-primary-bucket/backups/test-backup/

Scheduled Backups

daily-backup schedule: 2 AM daily, 30-day retention (720h TTL)

Backup for DR

Take a fresh backup on primary (after CRR is enabled) so it replicates to the DR bucket:

# On primary cluster
velero backup create dr-backup --include-namespaces demo
velero backup get

velero-primary-bucket — dr-backup objects (9 files)

velero-dr-bucket — same dr-backup replicated via CRR

Restore in DR Region

# Switch to DR cluster
kubectl config use-context iam-root-account@dr-cluster.ap-south-1.eksctl.io

# Verify backup visible via CRR
velero backup get

# Restore
velero restore create --from-backup dr-backup

Switched to DR context → dr-backup visible → restore submitted

Restore Phase: Completed — 9 items restored in 2 seconds

Verify Workloads in DR

nginx pod Running + LoadBalancer service with ap-south-1 ELB endpoint

nginx welcome page served from DR region (ap-south-1 ELB) — DR is live!

Restore Metadata in DR Bucket

velero-dr-bucket/restores/ — restore logs, resource list, and results

Troubleshooting & Lessons Learned

Issue 1: ServiceAccount Not Found

FailedCreate: serviceaccount "velero" not found

Fix: Create IRSA before Helm install, or use --override-existing-serviceaccounts then restart.

Velero recovers: 0/1 → 1/1 after IRSA fix

Issue 2: Credentials File Missing

Issue 3: Backup FailedValidation

Key Takeaways