AWS (EKS)

Peer Pods Helm Chart using Cloud API Adaptor (CAA) on AWS

Before you begin

Complete the common prerequisites, including the AWS-specific tools. Then return here to prepare AWS.

EKS Preparation

Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY (or AWS_PROFILE) and AWS_REGION for AWS CLI access

Note: As an alternative to static credentials, you can use IRSA (IAM Roles for Service Accounts) on EKS. With IRSA, the CAA pods authenticate via OIDC — no static AWS keys are stored in Kubernetes secrets. You still need AWS_REGION and temporary credentials for the cluster setup steps below.

Set the region:

export AWS_REGION="us-east-2"

Note: We have chose region us-east-2 as it has AMD SEV-SNP instances as well as prebuilt pod VM images readily available.

export AWS_REGION="us-east-2"

Note: We have chose region us-east-2 because it has prebuilt pod VM images readily available.

Setup cluster

Make changes to the following environment variable as you see fit:

export CLUSTER_NAME="caa-$(date '+%Y%m%b%d%H%M%S')"
export CLUSTER_NODE_TYPE="m5.xlarge"
export CLUSTER_NODE_FAMILY_TYPE="Ubuntu2204"
export SSH_KEY=~/.ssh/id_rsa.pub

Example EKS cluster creation using the default AWS VPC-CNI

eksctl create cluster --name "$CLUSTER_NAME" \
    --node-type "$CLUSTER_NODE_TYPE" \
    --node-ami-family "$CLUSTER_NODE_FAMILY_TYPE" \
    --nodes 1 \
    --nodes-min 0 \
    --nodes-max 2 \
    --node-private-networking \
    --kubeconfig "$CLUSTER_NAME"-kubeconfig

Wait for the cluster to be created.

Label the cluster nodes with node.kubernetes.io/worker=

for NODE_NAME in $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'); do
  kubectl label node $NODE_NAME node.kubernetes.io/worker=
done

Allow required network ports

EKS_VPC_ID=$(aws eks describe-cluster --name "$CLUSTER_NAME" \
--query "cluster.resourcesVpcConfig.vpcId" \
--output text)
echo $EKS_VPC_ID

EKS_CLUSTER_SG=$(aws eks describe-cluster --name "$CLUSTER_NAME" \
  --query "cluster.resourcesVpcConfig.clusterSecurityGroupId" \
  --output text)
echo $EKS_CLUSTER_SG

EKS_VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids "$EKS_VPC_ID" \
--query 'Vpcs[0].CidrBlock' --output text)
echo $EKS_VPC_CIDR

# agent-protocol-forwarder port
aws ec2 authorize-security-group-ingress --group-id "$EKS_CLUSTER_SG" --protocol tcp --port 15150 --cidr "$EKS_VPC_CIDR"

# vxlan port
aws ec2 authorize-security-group-ingress --group-id "$EKS_CLUSTER_SG" --protocol tcp --port 9000 --cidr "$EKS_VPC_CIDR"
aws ec2 authorize-security-group-ingress --group-id "$EKS_CLUSTER_SG" --protocol udp --port 9000 --cidr "$EKS_VPC_CIDR"

Note:

Port 15150 is the default port for CAA to connect to the agent-protocol-forwarder running inside the pod VM.

Port 9000 is the VXLAN port used by CAA. Ensure it doesn’t conflict with the VXLAN port used by the Kubernetes CNI.

Configure Authentication

Choose how CAA authenticates with AWS. Static credentials use a Kubernetes secret with access keys. IRSA uses OIDC federation so the pods assume an IAM role directly — no static keys needed.

Ensure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (or AWS_PROFILE) are set in your environment. These will be stored in a Kubernetes secret and used by the CAA pods.

No additional setup is needed here — the secret is created during the Helm chart deployment step.

IRSA eliminates the need for long-lived AWS access keys stored in Kubernetes secrets, and is the recommended authentication method for EKS.

Enable OIDC Provider

Check if the IAM OIDC provider is already registered:

OIDC_ID=$(aws eks describe-cluster \
  --name ${CLUSTER_NAME} \
  --region ${AWS_REGION} \
  --query "cluster.identity.oidc.issuer" \
  --output text | awk -F'/' '{print $NF}')

aws iam list-open-id-connect-providers | grep ${OIDC_ID}

If the command returns empty, create the OIDC provider:

eksctl utils associate-iam-oidc-provider \
  --cluster ${CLUSTER_NAME} \
  --region ${AWS_REGION} \
  --approve

Export the account ID and OIDC provider for use in the following steps:

export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

export OIDC_PROVIDER=$(aws eks describe-cluster \
  --name ${CLUSTER_NAME} \
  --region ${AWS_REGION} \
  --query "cluster.identity.oidc.issuer" \
  --output text | sed 's|https://||')

Create IAM Role for cloud-api-adaptor

export NAMESPACE="confidential-containers-system"
export CAA_SERVICE_ACCOUNT="cloud-api-adaptor"
export CAA_ROLE_NAME="CAA-IRSA-Role"

Create the trust policy:

cat > /tmp/caa-trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:${NAMESPACE}:${CAA_SERVICE_ACCOUNT}",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

Create the IAM role and attach the AmazonEC2FullAccess managed policy:

aws iam create-role \
  --role-name ${CAA_ROLE_NAME} \
  --assume-role-policy-document file:///tmp/caa-trust-policy.json \
  --description "IRSA role for Cloud API Adaptor on EKS"

aws iam attach-role-policy \
  --role-name ${CAA_ROLE_NAME} \
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess

Note: AmazonEC2FullAccess grants broad EC2 permissions. For production workloads, it is strongly recommended to replace it with a custom least-privilege policy scoped to the specific EC2 actions CAA requires. See the AWS IRSA documentation for more configuration options.

Export the role ARN for later use:

export CAA_ROLE_ARN=$(aws iam get-role \
  --role-name ${CAA_ROLE_NAME} \
  --query 'Role.Arn' \
  --output text)

echo "CAA Role ARN: ${CAA_ROLE_ARN}"

Create IAM Role for Peerpod-ctrl (optional)

This step is only required if you are deploying Peerpod-ctrl.

export CTRL_SERVICE_ACCOUNT="peerpodctrl-controller-manager"
export CTRL_ROLE_NAME="PeerpodCtrl-IRSA-Role"

Create the trust policy:

cat > /tmp/peerpod-ctrl-trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:${NAMESPACE}:${CTRL_SERVICE_ACCOUNT}",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

Create the role and attach permissions:

aws iam create-role \
  --role-name ${CTRL_ROLE_NAME} \
  --assume-role-policy-document file:///tmp/peerpod-ctrl-trust-policy.json \
  --description "IRSA role for PeerPod Controller on EKS"

aws iam attach-role-policy \
  --role-name ${CTRL_ROLE_NAME} \
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess

Note: AmazonEC2FullAccess grants broad EC2 permissions. For production workloads, it is strongly recommended to replace it with a custom least-privilege policy scoped to the specific EC2 actions CAA requires. See the AWS IRSA documentation for more configuration options.

Export the role ARN:

export CTRL_ROLE_ARN=$(aws iam get-role \
  --role-name ${CTRL_ROLE_NAME} \
  --query 'Role.Arn' \
  --output text)

echo "Controller Role ARN: ${CTRL_ROLE_ARN}"

Deploy

Preparation

Complete Prepare the CAA deployment.

This downloads the Helm artifacts, selects the CAA image, installs cert-manager, and creates the installation namespace.

Note: Keep the resulting CAA_VERSION, CAA_IMAGE, and CAA_TAG values in your terminal, then return here.

Export PodVM image version

Exports the PodVM image ID used by peer pods. This variable tells the deployment tooling which PodVM image version to use when creating peer pod virtual machines in AWS.

The image is pulled from the Coco community gallery (or manually built) and must match the current CAA release version.

We have a pre-built debug pod VM image available in us-east-2 for PoCs. You can find the AMI id for the release specific image by running the following CLI:

export PODVM_AMI_ID=$(aws ec2 describe-images \
    --filters Name=name,Values="podvm-ubuntu-amd64-${CAA_VERSION//./-}" \
    --query 'Images[*].[ImageId]' --output text)

echo $PODVM_AMI_ID

There are no pre-built pod VM AMI for latest builds. You’ll need to build your own pod VM image and then create the AMI by following the instructions here.

Remember to set TEE_PLATFORM=amd before building the pod VM image for AWS.

Once image build is finished, export image id to the environment variable PODVM_AMI_ID.

You can build your custom pod VM image by following the instructions here.

Remember to set TEE_PLATFORM=amd before building the pod VM image for AWS.

Once image build is finished, export image id to the environment variable PODVM_AMI_ID.

Select peer-pods machine type

export PODVM_INSTANCE_TYPE="m6a.large"
export DISABLECVM="false"

Find more AMD SEV-SNP machine types on this AWS documentation.

export PODVM_INSTANCE_TYPE="t3.large"
export DISABLECVM="true"

Populate the provider file

List of all available configuration options can be found in two places:

Run the following command to update the providers/aws.yaml file:

cat <<EOF > providers/aws.yaml
provider: aws
image:
  name: "${CAA_IMAGE}"
  tag: "${CAA_TAG}"
providerConfigs:
   aws:
      DISABLECVM: ${DISABLECVM}
      PODVM_AMI_ID: "${PODVM_AMI_ID}"
      PODVM_INSTANCE_TYPE: "${PODVM_INSTANCE_TYPE}"
      VXLAN_PORT: 9000
EOF

Deploy helm chart

Create credentials and install the Helm chart. The -f and --set customization options are described in advanced configuration.

Create the secret using kubectl. See providers/aws-secrets.yaml.template for required keys.

kubectl create secret generic my-provider-creds \
  -n confidential-containers-system \
  --from-literal=AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
  --from-literal=AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
  --from-file=id_rsa.pub=${SSH_KEY}

Note: --from-file=id_rsa.pub=${SSH_KEY} is optional. It allows user to SSH into the pod VMs for troubleshooting purposes. This option works only for custom debug enabled pod VM images. The prebuilt pod VM images do not have SSH connection enabled.

Install the Helm chart:

helm install peerpods . \
  -f providers/aws.yaml \
  --set secrets.mode=reference \
  --set secrets.existingSecretName=my-provider-creds \
  --dependency-update \
  -n confidential-containers-system

When using IRSA, no AWS credentials secret is needed. The CAA pods authenticate via the IAM role configured in the Configure Authentication section.

Install the Helm chart with IRSA annotations:

helm install peerpods . \
  -f providers/aws.yaml \
  --set "daemonset.serviceAccount.annotations.eks\.amazonaws\.com/role-arn=${CAA_ROLE_ARN}" \
  --set "resourceCtrl.serviceAccount.annotations.eks\.amazonaws\.com/role-arn=${CTRL_ROLE_ARN}" \
  --dependency-update \
  -n confidential-containers-system

Note: The resourceCtrl.serviceAccount.annotations line is only required if you are deploying Peerpod-ctrl. If not, omit that --set flag and skip the optional Peerpod-ctrl IAM role step above.

To verify IRSA is working, check the service account annotation and pod environment:

kubectl get serviceaccount cloud-api-adaptor \
  -n confidential-containers-system \
  -o jsonpath='{.metadata.annotations.eks\.amazonaws\.com/role-arn}'

CAA_POD=$(kubectl get pods -n confidential-containers-system \
  -l app=cloud-api-adaptor \
  -o jsonpath='{.items[0].metadata.name}')

kubectl exec -n confidential-containers-system ${CAA_POD} -- env | grep AWS

The output should include AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN variables.

Generic Peer pods Helm charts deployment instructions are also described here.

Verify deployment

Follow Verify the installation and run a sample workload, then confirm that AWS created the PodVM:

You can verify that the peer pod VM was created by running the following command:

aws ec2 describe-instances --filters "Name=tag:Name,Values=podvm*" \
   --query 'Reservations[*].Instances[*].[InstanceId, Tags[?Key==`Name`].Value | [0]]' --output table

The output should include the VM associated with the nginx pod.

Note: If you run into problems, see the troubleshooting guide.

Uninstall

First remove the workloads and uninstall CAA.

Verify that all peer-pod VMs are deleted. This command lists VMs whose name starts with podvm and their status:

aws ec2 describe-instances --filters "Name=tag:Name,Values=podvm*" \
--query 'Reservations[*].Instances[*].[InstanceId, Tags[?Key==`Name`].Value | [0], State.Name]' --output table

Delete the EKS cluster by running the following command:

eksctl delete cluster --name=$CLUSTER_NAME

Last modified October 15, 2026: docs: moved providers to subgroup (f270caf)