This is the multi-page printable view of this section.
Click here to print.
Return to the regular view of this page.
Trustee
Trusted Components for Attestation and Secret Management
Trustee contains tools and components for attesting confidential guests and providing secrets to them. Collectively, these components are known as Trustee. Trustee typically operates on behalf of the “workload provider” / “data owner” and interacts remotely with guest components.
Trustee is developed for the Confidential Containers project, but can be used with a wide variety of applications and hardware platforms.
Architecture
Trustee is flexible and can be deployed in several different configurations. This figure shows one common way to deploy these components in conjunction with certain guest components.
flowchart LR
AA -- attests guest ----> KBS
CDH -- requests resource --> KBS
subgraph Guest
CDH <.-> AA
end
subgraph Trustee
KBS -- validates evidence --> AS
RVPS -- provides reference values--> AS
end
client-tool -- configures --> KBS
Legend
CDH
: Confidential Data Hub
AA
: Attestation Agent
KBS
: Key Broker Service
RVPS
: Reference Value Provider Service
AS
: Attestation Service
1 - Key Broker Service (KBS)
This service facilitates remote attestation and secret delivery
The Confidential Containers Key Broker Service (KBS) facilitates remote attestation and secret delivery. The KBS is an implementation of a Relying Party from the Remote ATtestation ProcedureS (RATS) Architecture. The KBS itself does not validate attestation evidence. Instead, it relies on the Attestation-Service (AS) to verify TEE evidence.
In conjunction with the AS or Intel Trust Authority (ITA), the KBS supports the following TEEs:
- AMD SEV-SNP
- AMD SEV-SNP on Azure with vTPM
- Intel TDX
- Intel TDX on Azure with vTPM
- Intel SGX
- ARM CCA
- Hygon CSV
Deployment Configurations
The KBS can be deployed in several different environments, including as part of a docker compose cluster, part of a Kubernetes cluster or without any containerization. Additionally, the KBS can interact with other attestation components in different ways. This section focuses on the different ways the KBS can interact with other components.
Background Check Mode
Background check mode is a more straightforward and simple way to configure the Key Broker Service (KBS) and Attestation-Service (AS). The term “Background Check” is from the RATS architecture. In background check mode, the KBS directly forwards the hardware evidence of a confidential guest to the AS to validate. Once the validation passes, the KBS will release secrets to the confidential guest.
flowchart LR
AA -- attests guest --> KBS
CDH -- requests resource ----> KBS
subgraph Guest
AA <.-> CDH
end
subgraph Trustee
KBS -- validates evidence --> AS
end
In background check mode, the KBS is the relying party and the AS is the verifier.
Passport Mode
Passport mode decouples the provisioning of resources from the validation of evidence. In background check mode these tasks are already handled by separate components, but in passport mode they are decoupled even more. The term “Passport” is from the RATS architecture.
In passport mode, there are two Key Broker Services (KBSes), one that uses a KBS to verify the evidence and a second to provision resources.
flowchart LR
CDH -- requests resource ----> KBS2
AA -- attests guest --> KBS1
subgraph Guest
CDH <.-> AA
end
subgraph Trustee 1
KBS1 -- validates evidence --> AS
end
subgraph Trustee 2
KBS2
end
In the RATS passport model the client typically connects directly to the verifier to get an attestation token (a passport). In CoCo we do not support direct connections to the AS, so KBS1 serves as an intermediary. Together KBS1 and the AS represent the verifier. KBS2 is the relying party.
Passport mode is good for use cases when resource provisioning and attestation are handled by separate entities.
1.1 - KBS backed by AKV
This documentation describes how to mount secrets stored in Azure Key Vault into a KBS deployment
Premise
AKS
We assume an AKS cluster configured with Workload Identity and Key Vault Secrets Provider. The former provides a KBS pod with the privileges to access an Azure Key Vault (AKV) instance. The latter is an implementation of Kubernetes’ Secret Store CSI Driver, mapping secrets from external key vaults into pods. The guides below provide instructions on how to configure a cluster accordingly:
AKV
There should be an AKV instance that has been configured with role based access control (RBAC), containing two secrets named coco_one
coco_two
for the purpose of the example. Find out how to configure your instance for RBAC in the guide below.
Provide access to Key Vault keys, certificates, and secrets with an Azure role-based access control
Note: You might have to toggle between Access Policy and RBAC modes to create your secrets on the CLI or via the Portal if your user doesn’t have the necessary role assignments.
CoCo
While the steps describe a deployment of KBS, the configuration of a Confidential Containers environment is out of scope for this document. CoCo should be configured with KBS as a Key Broker Client (KBC) and the resulting KBS deployment should be available and configured for confidential pods.
Azure environment
Configure your Resource group, Subscription and AKS cluster name. Adjust accordingly:
export SUBSCRIPTION_ID="$(az account show --query id -o tsv)"
export RESOURCE_GROUP=my-group
export KEYVAULT_NAME=kbs-secrets
export CLUSTER_NAME=coco
Instructions
Create Identity
Create a User managed identity for KBS:
az identity create --name kbs -g "$RESOURCE_GROUP"
export KBS_CLIENT_ID="$(az identity show -g "$RESOURCE_GROUP" --name kbs --query clientId -o tsv)"
export KBS_TENANT_ID=$(az aks show --name "$CLUSTER_NAME" --resource-group "$RESOURCE_GROUP" --query identity.tenantId -o tsv)
Assign a role to access secrets:
export KEYVAULT_SCOPE=$(az keyvault show --name "$KEYVAULT_NAME" --query id -o tsv)
az role assignment create --role "Key Vault Administrator" --assignee "$KBS_CLIENT_ID" --scope "$KEYVAULT_SCOPE"
Namespace
By default KBS is deployed into a coco-tenant
Namespace:
export NAMESPACE=coco-tenant
kubectl create namespace $NAMESPACE
KBS identity and Service Account
Workload Identity provides individual pods with IAM privileges to access Azure infrastructure resources. An azure identity is bridged to a Service Account using OIDC and Federated Credentials. Those are scoped to a Namespace, we assume we deploy the Service Account and KBS into the default
Namespace, adjust accordingly if necessary.
export AKS_OIDC_ISSUER="$(az aks show --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" --query "oidcIssuerProfile.issuerUrl" -o tsv)"
az identity federated-credential create \
--name kbsfederatedidentity \
--identity-name kbs \
--resource-group "$RESOURCE_GROUP" \
--issuer "$AKS_OIDC_ISSUER" \
--subject "system:serviceaccount:${NAMESPACE}:kbs"
Create a Service Account object and annotate it with the identity’s client id.
cat <<EOF> service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: ${KBS_CLIENT_ID}
name: kbs
namespace: ${NAMESPACE}
EOF
kubectl apply -f service-account.yaml
Secret Provider Class
A Secret Provider Class specifies a set of secrets that should be made available to k8s workloads.
cat <<EOF> secret-provider-class.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: ${KEYVAULT_NAME}
namespace: ${NAMESPACE}
spec:
provider: azure
parameters:
usePodIdentity: "false"
clientID: ${KBS_CLIENT_ID}
keyvaultName: ${KEYVAULT_NAME}
objects: |
array:
- |
objectName: coco_one
objectType: secret
- |
objectName: coco_two
objectType: secret
tenantId: ${KBS_TENANT_ID}
EOF
kubectl create -f secret-provider-class.yaml
Deploy KBS
The default KBS deployment needs to be extended with label annotations and CSI volume. The secrets are mounted into the storage hierarchy default/akv
.
git clone https://github.com/confidential-containers/kbs.git
cd kbs
git checkout v0.8.2
cd kbs/config/kubernetes
mkdir akv
cat <<EOF> akv/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: coco-tenant
resources:
- ../base
patches:
- path: patch.yaml
target:
group: apps
kind: Deployment
name: kbs
version: v1
EOF
cat <<EOF> akv/patch.yaml
- op: add
path: /spec/template/metadata/labels/azure.workload.identity~1use
value: "true"
- op: add
path: /spec/template/spec/serviceAccountName
value: kbs
- op: add
path: /spec/template/spec/containers/0/volumeMounts/-
value:
name: secrets
mountPath: /opt/confidential-containers/kbs/repository/default/akv
readOnly: true
- op: add
path: /spec/template/spec/volumes/-
value:
name: secrets
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: ${KEYVAULT_NAME}
EOF
kubectl apply -k akv/
Test
The KBS pod should be running, the pod events should give indication of possible errors. From a confidential pod the AKV secrets should be retrievable via Confidential Data Hub:
$ kubectl exec -it deploy/nginx-coco -- curl http://127.0.0.1:8006/cdh/resource/default/akv/coco_one
a secret
2 - Attestation Service (AS)
This service verifies TEE evidence
The Attestation Service (AS or CoCo-AS) verifies hardware evidence. The AS was designed to be used with the Key Broker Service (KBS) for Confidential Containers, but it can be used in a wide variety of situations. The AS can be used anytime TEE evidence needs to be validated.
Today, the AS can validate evidence from the following TEEs:
- Intel TDX
- Intel SGX
- AMD SEV-SNP
- ARM CCA
- Hygon CSV
- Intel TDX with vTPM on Azure
- AMD SEV-SNP with vTPM on Azure
Overview
┌───────────────────────┐
┌───────────────────────┐ Evidence │ Attestation Service │
│ ├────────────►│ │
│ Verification Demander │ │ ┌────────┐ ┌──────────┴───────┐
│ (Such as KBS) │ │ │ Policy │ │ Reference Value │◄───Reference Value
│ │◄────────────┤ │ Engine │ │ Provider Service │
└───────────────────────┘ Attestation │ └────────┘ └──────────┬───────┘
Results Token │ │
│ ┌───────────────────┐ │
│ │ Verifier Drivers │ │
│ └───────────────────┘ │
│ │
└───────────────────────┘
The Attestation Service (AS) has a simple API. It receives attestation evidence and returns an attestation token containing the results of a two-step verification process. The AS can be consumed directly as a Rust crate (library) or built as a standalone service, exposing a REST or gRPC API. In Confidential Containers, the client of the AS is the Key Broker Service (KBS), but the evidence originates from the Attestation Agent inside the guest.
The AS has a two-step verification process.
- Verify the format and provenance of evidence itself (e.g. check the signature of the evidence).
- Appraise the claims presented in the evidence (e.g. check that measurements match reference values).
The first step is accomplished by one of the platform-specific Verifier Drivers.
The second step is driven by the Policy Engine with help from the Reference Value Provider Service (RVPS).
3 - Reference Value Provider Service (RVPS)
This service manages reference values used to verify TEE evidence
Reference Value Provider Service (RVPS) is a component to receive software supply chain provenances / metadata, verify them and extract the reference values. All the reference values are stored inside RVPS. When Attestation Service (AS) queries specific software claims, RVPS will response with related reference values.
Architecture
RVPS contains the following components:
-
Pre-Processor: Pre-Processor contains a set of *wares (like middleware). These wares can process the input Message and then deliver it to the Extractors.
-
Extractors: Extractors has sub-modules to process different type of provenance. Each sub-module will consume the input Message, and then generate an output Reference Value.
-
Store: Store is a trait object, which can provide key-value like API. All verified reference values will be stored in the Store. When requested by Attestation Service (AS), related reference value will be provided.
Message Flow
The message flow of RVPS is like the following figure:
![](https://raw.githubusercontent.com/confidential-containers/trustee/main/attestation-service/rvps/diagrams/rvps.svg)
Message
A protocol helps to distribute provenance of binaries. It will be received and processed by RVPS, then RVPS will generate Reference Value if working correctly.
{
"version": <VERSION-NUMBER-STRING>,
"type": <TYPE-OF-THE-PROVENANCE-STRING>,
"provenance": #provenance,
}
"version"
: This field is the version of this message, making extensibility possible.
"type"
: This field specifies the concrete type of the provenance the message carries.
"provenance"
: This field is the main content passed to RVPS. This field contains the payload to be decrypted by RVPS. The meaning of the provenance depends on the type and concrete Extractor which process this.
Trust Digests
It is the reference values really requested and used by Attestation Service to compare with the gathered evidence generated from HW TEE. They are usually digests. To avoid ambiguity, they are named trust digests
rather than reference values
.
4 - KBS Client Tool
Simple tool to test or configure Key Broker Service and Attestation Service
This is a simple client for the Key Broker Client (KBS) that facilitates testing of the KBS and other basic attestation flows.
You can run this tool inside of a TEE to make a request with real attestation evidence. You can also provide pre-existing evidence or use the sample attester as a fallback.
The client tool can also be used to provision the KBS/AS with resources and policies.