feat: add keyrotation feature

.github/workflows/e2e-tests.yaml

@@ -79,7 +79,7 @@ jobs:
           kubectl -n traefik port-forward svc/traefik 8080:80 &
 
           # install Gateway API CRDs
-          kubectl apply --server-side --force-conflicts -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.1/experimental-install.yaml
+          kubectl apply --server-side --force-conflicts -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml
 
           sleep 5 # to be safe
 

README.md

@@ -416,10 +416,13 @@ For example, if a participant onboarding went only through half-way, we recommen
 
 In some cases, even deleting and re-creating the KinD cluster may be required.
 
-## JAD's APIs – A single pane of glass
+## JAD's APIs: GlassAPI - A single pane of glass
 
 All JAD services are exposed through a single Traefik gateway (`edcv-gateway`) on `jad.localhost`, acting as a single
-pane of glass. Each service is reachable via a path prefix that is rewritten before forwarding to the backend.
+pane of glass. Each service is reachable via a path prefix rewritten before forwarding to the backend.
+
+This single pane of glass is called the `GlassAPI`. and is protected by an auth backend called `clearglass`, details
+are [here](#clearglass).
 
 Authentication is enforced at the gateway level using Traefik `ForwardAuth` middlewares. Each middleware forwards the
 `Authorization` header to the `clearglass` service, which validates the Bearer token against Keycloak via RFC 7662
@@ -486,13 +489,63 @@ This design keeps authentication logic out of the individual services and centra
 to add or modify access rules by updating the middleware definitions in
 [`k8s/base/jwt-middleware.yaml`](k8s/base/jwt-middleware.yaml).
 
-## Deploying JAD on a bare-metal/cloud-hosted Kubernetes
+## Advanced topics
+
+### Rotate a participant's key material
+
+Keys should be rotated periodically to reduce the chance of all sorts of nasty attacks such as Lattice attacks or
+side-channel attacks. In practical applications, this would be done by a management application that uses the Glass
+API, like an admin dashboard, that periodically invokes these APIs here to perform the action.
+
+Participant keys are managed by IdentityHub and stored in a secure vault.
+
+Rotating keys is not a single operation; rather, several individual steps are performed in IdentityHub, such as deleting
+the old private key, updating the DID document, etc. However, all of this is abstracted by a single API call.
+
+#### Requests
+
+The general sequence of operations necessary to collect the required information is as follows.
+
+If the `participantContextId` is not known, perform these steps to obtain it:
+
+- query the participant profile using the TenantManager API
+- record participant `participantProfileId`, its `tenantId` and the `participantContextId`. The latter is how we
+  establish a correlation between CFM and dataspace components such as IdentityHub and ControlPlane.
+
+After that, use the `participantContextId` to perform the following steps:
+
+- get the client secret from Vault. We need this to authenticate REST calls against IdentityHub
+- query all available KeyPairs owned by the `participantContextId`, select the (first) active one (`state=200`) and
+  record the ID of the key pair.
+- call IdentityHub's `/keypairs/rotate` endpoint with the information provided. In that call, please provide in the body
+  a `keyId` and a `privateKeyAlias`. The `keyId` should be the participant's identifier (web:DID) plus the `"#"` sign,
+  followed by a random string:
+  ```json
+  {
+    "keyGeneratorParams": {
+      "algorithm": "eddsa",
+      "curve": "ed25519"    
+     },
+    "keyId": "{{participantIdentifier}}#{{$randomUUID}}",
+    "privateKeyAlias": "{{$randomUUID}}" 
+  }
+  ```
+- verify the correct execution of the rotation, either by checking the participant context's DID document and see if the
+  new key is there, or call the Query-Keypair API again
+
+Programmatically, this sequence is demonstrated in an E2E test in
+[KeyRotationEndToEndTest.java](./tests/end2end/src/test/java/org/eclipse/edc/jad/tests/KeyRotationEndToEndTest.java).
+
+The REST calls are demonstrated in the "Rotate Participant Key" folder in
+the [Bruno collection](./requests/EDC-V%20Onboarding/Rotate%20Participant%20Key).
+
+### Deploying JAD on a bare-metal/cloud-hosted Kubernetes
 
 KinD is geared towards local development and testing. For example, it comes with a bunch of useful defaults, such as
 storage classes, load balancers, network plugins, etc. If you want to deploy JAD on a bare-metal or cloud-hosted
 Kubernetes cluster, then there are some caveats to keep in mind.
 
-### Configure network access and DNS
+#### Configure network access and DNS
 
 EDC-V, Keycloak and Vault will need to be accessible from outside the cluster. For this, your cluster needs a network
 plugin and an external load balancer. For bare-metal installations, consider using [MetalLB](https://metallb.io).
@@ -518,7 +571,7 @@ Where `194.178.218.88` is the IP address of the Kubernetes host running MetalLB
 _In actual production deployments, these individual hostnames, and potentially also individual IP addresses, would be
 necessary to isolate security domains and prevent unauthorized access or privilege escalation._
 
-### Tune Traefik gateway controller
+#### Tune Traefik gateway controller
 
 By default, Traefik binds to port 80 and 443 on the host machine. This is useful to enable clean URLs like
 `http://tm.yourdomain.com/api/v1alpha1/cells` etc. without any ports. However, some Linux distributions don't allow
@@ -538,7 +591,7 @@ securityContext:
   runAsUser: 0
 ```
 
-### Create Bruno Environment
+#### Create Bruno Environment
 
 Some of the URL paths used in Bruno are hard coded to `localhost` in a Bruno environment named `KinD Local`. This is
 tailored to running JAD on a local KinD cluster. To make the collection usable for a remote deployment, we recommend
@@ -548,7 +601,7 @@ Create another environment to suit your setup:
 
 ![img.png](docs/images/bruno_custom_env.png)
 
-### Update deployment manifests
+#### Update deployment manifests
 
 in [keycloak.yaml](k8s/base/keycloak.yaml) and [vault.yaml](k8s/base/vault.yaml), update the `hostnames` fields in the
 `HTTPRoute` resources to match your DNS:
@@ -567,7 +620,7 @@ spec:
 Do this for all `HTTPRoute` declarations in all components' manifests. The `hostnames` field should contain entries
 matching your DNS subdomains that you have also used to create the new Bruno environment.
 
-### Update the Keycloak realm
+#### Update the Keycloak realm
 
 In `k8s/base/keycloak.yaml`, find the line that says:
 
@@ -583,7 +636,7 @@ and replace with
 
 This is crucial for Vault authentication to work properly.
 
-### Tune readiness probes
+#### Tune readiness probes
 
 We've set up the readiness probes fairly tight, to avoid long wait times on local KinD clusters. However, in some
 Kubernetes

gradle/libs.versions.toml

@@ -48,6 +48,7 @@ edc-monitor-console = { module = "org.eclipse.edc:console-monitor", version.ref
 edc-spi-web = { module = "org.eclipse.edc:web-spi", version.ref = "edc" }
 edc-spi-http = { module = "org.eclipse.edc:http-spi", version.ref = "edc" }
 edc-spi-catalog = { module = "org.eclipse.edc:catalog-spi", version.ref = "edc" }
+edc-spi-keypair = { module = "org.eclipse.edc:keypair-spi", version.ref = "edc" }
 edc-spi-transaction = { module = "org.eclipse.edc:transaction-spi", version.ref = "edc" }
 edc-spi-jwt = { module = "org.eclipse.edc:jwt-spi", version.ref = "edc" }
 

k8s/apps/cfm-agents.yaml

@@ -119,4 +119,7 @@ spec:
         - name: onboarding-agent-config
           configMap:
             name: ob-agent-config
+        - name: ih-keyrotate-agent-config
+          configMap:
+            name: ih-keyrotate-agent-config
       restartPolicy: Always

k8s/base/clearglass.yaml

@@ -36,7 +36,7 @@ spec:
       containers:
         - name: clearglass
           image: ghcr.io/metaform/jad/clearglass:latest
-          imagePullPolicy: Never
+          imagePullPolicy: Always
           ports:
             - containerPort: 8080
               name: http

requests/EDC-V Onboarding/Rotate Participant Key/Obtain Client Secret from Vault.bru

@@ -0,0 +1,41 @@
+meta {
+  name: Obtain Client Secret from Vault
+  type: http
+  seq: 6
+}
+
+get {
+  url: {{VAULT_HOST}}/v1/secret/data/{{participantContextId}}
+  body: none
+  auth: apikey
+}
+
+auth:apikey {
+  key: X-Vault-Token
+  value: root
+  placement: header
+}
+
+vars:pre-request {
+  consumer_context_id: ee369bba6e7f431197fe0262893acb11-api
+}
+
+script:post-response {
+  let body = res.getBody()
+  test("Response contains Secret", function () {
+      expect(body).to.have.property("data")
+      expect(body.data).to.have.property("data")
+      expect(body.data.data).to.have.property("content")
+  });
+
+  if (body && body.data) {
+      bru.setVar("participant_secret", body.data.data.content);
+  }
+}
+
+settings {
+  encodeUrl: false
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5
+}

requests/EDC-V Onboarding/Rotate Participant Key/Query DID documents.bru

@@ -0,0 +1,38 @@
+meta {
+  name: Query DID documents
+  type: http
+  seq: 6
+}
+
+post {
+  url: {{identityHubUrl}}/participants/{{participantContextId}}/dids/query
+  body: json
+  auth: oauth2
+}
+
+auth:oauth2 {
+  grant_type: client_credentials
+  access_token_url: http://jad.localhost/auth/realms/edcv/protocol/openid-connect/token
+  refresh_token_url: 
+  client_id: {{participantContextId}}
+  client_secret: {{participant_secret}}
+  scope: identity-api:read identity-api:write
+  credentials_placement: body
+  credentials_id: participant_credentials
+  token_source: access_token
+  token_placement: header
+  token_header_prefix: Bearer
+  auto_fetch_token: true
+  auto_refresh_token: false
+}
+
+body:json {
+  {}
+}
+
+settings {
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5
+}

requests/EDC-V Onboarding/Rotate Participant Key/Query KeyPairResource from IdentityHub.bru

@@ -0,0 +1,44 @@
+meta {
+  name: Query KeyPairResource from IdentityHub
+  type: http
+  seq: 6
+}
+
+get {
+  url: {{identityHubUrl}}/participants/{{participantContextId}}/keypairs
+  body: json
+  auth: oauth2
+}
+
+auth:oauth2 {
+  grant_type: client_credentials
+  access_token_url: http://jad.localhost/auth/realms/edcv/protocol/openid-connect/token
+  refresh_token_url: 
+  client_id: {{participantContextId}}
+  client_secret: {{participant_secret}}
+  scope: identity-api:read identity-api:write
+  credentials_placement: body
+  credentials_id: participant_credentials
+  token_source: access_token
+  token_placement: header
+  token_header_prefix: Bearer
+  auto_fetch_token: true
+  auto_refresh_token: false
+}
+
+script:post-response {
+  const body = res.getBody()
+
+  // body contains KeyPairResource objects. This script selects the (first) one where the state == 200 (active)
+
+  const match = body.find(item => item.state === 200);
+  expect(match).to.not.be.undefined;
+  bru.setVar("active_key_id", match.id)
+}
+
+settings {
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5
+}

requests/EDC-V Onboarding/Rotate Participant Key/Query Participant Profiles.bru

@@ -0,0 +1,63 @@
+meta {
+  name: Query Participant Profiles
+  type: http
+  seq: 6
+}
+
+post {
+  url: {{tmBaseUrl}}/participant-profiles/query
+  body: json
+  auth: inherit
+}
+
+body:json {
+  {
+    "predicate": "identifier like '%provider%'"
+  }
+
+}
+
+script:post-response {
+  let body = res.getBody()[0]
+
+  test("Response contains id", function () {
+      expect(body).to.have.property("id");
+  });
+
+  test("Response contains tenantId", function(){
+    expect(body).to.have.property("tenantId");
+  })
+
+  test("Response contains participantContextId", function(){
+    expect(body).to.have.nested.property("properties.cfm\\.vpa\\.state.participantContextId")
+  })
+
+  test("Response contains identifier", function(){
+    expect(body).to.have.property("identifier")
+
+  })
+
+  if (body && body.id) {
+      bru.setVar("participant_profile_id", body.id);
+  }
+
+  if(body && body.tenantId){
+    bru.setVar("tenant_id", body.tenantId)
+
+  }
+
+  if(body && body.properties["cfm.vpa.state"].participantContextId){
+    bru.setVar("participantContextId", body.properties["cfm.vpa.state"].participantContextId)
+  }
+
+  if(body && body.identifier){
+    bru.setVar("participantIdentifier", body.identifier)
+  }
+}
+
+settings {
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5
+}

requests/EDC-V Onboarding/Rotate Participant Key/Rotate key.bru

@@ -0,0 +1,30 @@
+meta {
+  name: Rotate key
+  type: http
+  seq: 6
+}
+
+post {
+  url: {{identityHubUrl}}/participants/{{participantContextId}}/keypairs/{{active_key_id}}/rotate
+  body: json
+  auth: inherit
+}
+
+body:json {
+  {
+    "keyGeneratorParams": {
+      "algorithm": "eddsa",
+      "curve": "ed25519"    
+    },
+    "keyId": "{{participantIdentifier}}#{{$randomUUID}}",
+    "privateKeyAlias": "{{$randomUUID}}" 
+  }
+
+}
+
+settings {
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5
+}

requests/EDC-V Onboarding/Rotate Participant Key/folder.bru

@@ -0,0 +1,8 @@
+meta {
+  name: Rotate Participant Key
+  seq: 5
+}
+
+auth {
+  mode: inherit
+}

requests/EDC-V Onboarding/collection.bru

@@ -30,6 +30,7 @@ vars:pre-request {
   tenant_clientSecret: 
   tenant_apiKey: 
   POLICY_ID: 
+  : 
 }
 
 script:post-response {