Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f4ff72d
Keep a new Health FHIR service in the Terraform state (#5147)
JayDoubleu Oct 26, 2026
52919cb
Add a private Health lifecycle check (#5147)
JayDoubleu Oct 23, 2026
38768da
Create an Automatic workspace for the Health check (#5147)
JayDoubleu Oct 23, 2026
f920cfd
Remove Health from the generic workspace service install case (#5147)
JayDoubleu Oct 23, 2026
61d9397
Replace the Health private endpoint when its services change (#5147)
JayDoubleu Oct 23, 2026
566c305
Turn off public network access on the Health DICOM service (#5147)
JayDoubleu Oct 23, 2026
52e72b2
Try the Health probe again after a failed FHIR name lookup (#5147)
JayDoubleu Oct 23, 2026
8de1c59
Check that the Health services reject public requests (#5147)
JayDoubleu Oct 24, 2026
947c051
Send FHIR data requests again while the probe role applies (#5147)
JayDoubleu Oct 24, 2026
c3f98b3
Keep the Health private endpoint when its services change (#5147)
JayDoubleu Oct 25, 2026
4c1af66
Create the Health services after the private endpoint (#5147)
JayDoubleu Oct 25, 2026
7df22df
Keep the Health data roles when the service changes (#5147)
JayDoubleu Oct 25, 2026
60a3f8e
Wrap a long CHANGELOG entry
JayDoubleu Oct 25, 2026
fa09cf3
Format the Health bundle contract test
JayDoubleu Oct 26, 2026
3d33076
Give the DICOM data roles after the endpoint link (#5147)
JayDoubleu Oct 25, 2026
fe7c11b
Use one Health version bump
JayDoubleu Oct 30, 2026
f4f5575
Use one owner token in the Health test, as the MySQL test does (#5147)
JayDoubleu Oct 31, 2026
bba7e0a
Create the Health DNS zone group again when FHIR changes (#5147)
JayDoubleu Oct 31, 2026
4ad9b95
Merge the PR bot marker sentences and remove a clean-up without effect
JayDoubleu Oct 31, 2026
88988b9
Delete the old Health zone group after the FHIR and DICOM changes
JayDoubleu Oct 31, 2026
5b3c7b4
Name azurerm 4.75.0 as the first version with the FHIR create fix
JayDoubleu Oct 31, 2026
31eb9ea
Create the Health private endpoint after FHIR and DICOM again (#5147)
JayDoubleu Oct 31, 2026
bab198b
Stop a Health service without FHIR and DICOM before a change (#5147)
JayDoubleu Oct 31, 2026
cb48fac
Use the bundle defaults in the Health form and FHIR R4 without fhir_k…
JayDoubleu Oct 31, 2026
d5b8adb
Give the Health repairs for a failed endpoint and a failed FHIR chang…
JayDoubleu Oct 31, 2026
8f36a52
Give the FHIR data roles after the Health private endpoint (#5147)
JayDoubleu Nov 1, 2026
df9797d
Set Health 1.0.0 for the stop of a service without FHIR and DICOM (#5…
JayDoubleu Nov 1, 2026
ff18c9d
Link the Health CHANGELOG entries to issue #5295
JayDoubleu Nov 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/deploy_tre_reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1083,6 +1083,9 @@ jobs:
- name: Reserve Gitea cleanup and reporting time
if: inputs.e2eBundle == 'tre-workspace-service-gitea'
run: echo "GITEA_VALIDATION_DEADLINE=$(( $(date +%s) + 270 * 60 ))" >> "$GITHUB_ENV"
- name: Reserve Health cleanup and reporting time
if: inputs.e2eBundle == 'tre-workspace-service-health'
run: echo "HEALTH_VALIDATION_DEADLINE=$(( $(date +%s) + 270 * 60 ))" >> "$GITHUB_ENV"

- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Expand Down Expand Up @@ -1113,6 +1116,7 @@ jobs:
SQL_VALIDATION_DEADLINE=${{ env.SQL_VALIDATION_DEADLINE }}
MYSQL_VALIDATION_DEADLINE=${{ env.MYSQL_VALIDATION_DEADLINE }}
GITEA_VALIDATION_DEADLINE=${{ env.GITEA_VALIDATION_DEADLINE }}
HEALTH_VALIDATION_DEADLINE=${{ env.HEALTH_VALIDATION_DEADLINE }}
TEST_BUNDLE_REQUEST_FILE=e2e_tests/bundle-validation-request.json make test-e2e-bundle
DEVCONTAINER_TAG: ${{ inputs.DEVCONTAINER_TAG }}
CI_CACHE_ACR_NAME: ${{ secrets.CI_CACHE_ACR_NAME}}
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<!-- markdownlint-disable MD041 -->
## (Unreleased)
**BREAKING CHANGES**
* `tre-workspace-service-health` 1.0.0 stops an install, update or upgrade of a Health service without FHIR and DICOM before Terraform changes a resource. Azure fails a private endpoint to a healthcare workspace without a FHIR or DICOM service.
If the upgrade of an earlier service without FHIR and DICOM stops, turn on FHIR or DICOM with an update, or delete the service. ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Existing resources with legacy Porter parameter overrides must complete a resource upgrade before custom actions or uninstall. The current template version can be reused. See [upgrading resources](docs/tre-admins/upgrading-resources.md#upgrade-existing-resources-for-document-based-porter-parameters). (`resource_processor` 0.13.10) ([#4904](https://github.com/microsoft/AzureTRE/pull/4904))
* Set `enable_legacy_airlock` explicitly to `true` in your `config.yaml`. It currently defaults to `true` but will default to `false` in a future release;
Setting to `false` will delete existing airlock storage accounts and must only be done once all workspaces use the v2 airlock. ([#5048](https://github.com/microsoft/AzureTRE/pull/5048))
Expand All @@ -9,6 +11,7 @@ Setting to `false` will delete existing airlock storage accounts and must only b

ENHANCEMENTS:
* Register the OpenAI bundle for branch validation, add GPT-5.1 as the default model and check regional model and quota prerequisites before its private lifecycle test. (`tre-workspace-service-openai` 1.1.0) ([#5150](https://github.com/microsoft/AzureTRE/issues/5150))
* Add a private Health lifecycle check. A workspace VM reads the FHIR metadata through private DNS, and gets HTTP 401 without a token and HTTP 403 without a role. Then the VM writes and reads a Patient across an update that adds DICOM. The generic workspace service install case no longer installs Health, because its CI workspace has no Entra ID groups. ([#5147](https://github.com/microsoft/AzureTRE/issues/5147))
* Add a private MySQL lifecycle check. A Nexus-backed Linux VM writes a row through private DNS, the SKU changes from Burstable to General Purpose, and the VM reads the row again. ([#5148](https://github.com/microsoft/AzureTRE/issues/5148))
* Run the `stop`, `start` and `reset_password` actions in the Guacamole Linux VM check, and make sure of the VM power state and the new Key Vault secret version through Azure. ([#5145](https://github.com/microsoft/AzureTRE/issues/5145))
* Add a private Gitea lifecycle check. A workspace VM uses the Gitea API through private DNS, gets HTTP 401 without credentials and with a wrong password, and commits a file to a private repository.
Expand Down Expand Up @@ -47,6 +50,14 @@ BUG FIXES:
* Clean up backup-test workspaces after failed provisioning. ([#5135](https://github.com/microsoft/AzureTRE/issues/5135))
* Create the OpenAI private endpoint after the model deployment, so that Azure does not reject it while the deployment updates the account. (`tre-workspace-service-openai` 1.1.0) ([#5249](https://github.com/microsoft/AzureTRE/issues/5249))
* Give the Gitea workspace service identity the Key Vault role only on the three secrets that its web app reads. It had the role on the whole workspace Key Vault. (`tre-workspace-service-gitea` 2.0.0) ([#5283](https://github.com/microsoft/AzureTRE/issues/5283))
* Add the DICOM zone to the DNS zone group of the Health private endpoint when an update adds DICOM. Azure wrote private DNS records only for the services that existed when it created the zone group. So a DICOM service that an update added resolved to a public address.
The zone group is now a separate resource, so the endpoint stays, and Terraform tries again while Azure links the new service to the endpoint. An upgrade creates the zone group again and writes a record that an earlier update did not get.
The FHIR record is in the zone of the workspace record, so an update that adds or removes FHIR also creates the zone group again. Private access to the Health services stops until the new group exists. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Turn off public network access on the Health DICOM service. The azurerm provider created DICOM with public network access on, and an upgrade turned it on again after Azure turned it off. An upgrade changes the setting in place. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Keep the Health data roles of the workspace groups when the service changes. The role assignments used a role definition ID without the subscription, but Azure returns the ID with the subscription. So each update deleted the role assignments and created them again. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Give the workspace groups their DICOM data role after Azure links the DICOM service to the Health private endpoint. Terraform gave the role immediately after the DICOM create. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Give FHIR R4 to a Health request that turns on FHIR without `fhir_kind`. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Give the workspace groups their FHIR data role after the Health private endpoint exists. Terraform gave the role before the endpoint, and an uninstall did not delete the role before the endpoint. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))
* Disable public network access on the MySQL flexible server, so that clients can connect only through the private endpoint. Move the MySQL bundle to azurerm 4.27.0, which has this setting. An upgrade changes the setting in place. (`tre-workspace-service-mysql` 1.0.16) ([#5148](https://github.com/microsoft/AzureTRE/issues/5148))
* Create the Gitea workspace service file share through Resource Manager. Terraform used the file endpoint of the storage account, which the TRE firewall blocks until the private endpoint and its DNS record exist. So an install can fail, and after a delete removes the private endpoint, each later update or delete fails.
Also disable public network access on the Gitea MySQL server. An upgrade keeps the existing share and changes the server setting in place. (`tre-workspace-service-gitea` 2.0.0) ([#5141](https://github.com/microsoft/AzureTRE/issues/5141))
Expand Down Expand Up @@ -103,6 +114,7 @@ BUG FIXES:
* Fix Guacamole login failures in large workspaces: raise the user-resources API timeout from 5s to 30s, batch template version lookups into one Cosmos query, and query VM power states concurrently. (`tre-service-guacamole` 0.14.6, `API` 0.27.31) ([#5099](https://github.com/microsoft/AzureTRE/issues/5099))
* Fix API resource update retries so a successful retry returns the patched resource to the original caller. (`API` 0.27.32) ([#5025](https://github.com/microsoft/AzureTRE/pull/5025))
* Fix the health check falsely returning OK when Cosmos is down or inaccessible. ([#4926](https://github.com/microsoft/AzureTRE/issues/4926))
* Keep a new Health FHIR service in the Terraform state when Azure reports it as `Accepted` after the create. Before version 4.75.0, the azurerm provider waited for `Succeeded` after the create and saved the service only after this wait. So the install failed, and each later update and delete of the service failed. The Health bundle now uses azurerm 5.11.0. (`tre-workspace-service-health` 1.0.0) ([#5295](https://github.com/microsoft/AzureTRE/issues/5295))

## (0.29.1)
**BREAKING CHANGES**
Expand Down
22 changes: 22 additions & 0 deletions docs/tre-developers/end-to-end-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,28 @@ The query returns counts only. The CI identity needs read access to the Log Anal
At the end, the case makes sure that the web app and MySQL have public network access disabled, and that the storage firewall denies by default.
The Gitea job uses the SQL time limits with its own `GITEA_VALIDATION_DEADLINE`.

### Health selection

Select `tre-workspace-service-health` to run `test_health.py::test_private_fhir_access_and_dicom_update`.
The case creates its own workspace, Guacamole service, Windows VM and Health service with FHIR R4.
The Health bundle needs the workspace owners group, and only an Automatic workspace gets Entra ID groups.
So the case always creates an Automatic workspace. It does not use `TEST_WORKSPACE_ID`, `TEST_WORKSPACE_SERVICE_ID` or `TEST_WORKSPACE_APP_ID`.
From the VM, the case makes sure that the FHIR host resolves to a private address and that the FHIR metadata is available.
If the FHIR name lookup fails, the case runs that probe again after 60 seconds, 3 attempts in total. Before this lookup, the probe changes no data.
It also makes sure that FHIR rejects a data request without a token (HTTP 401) and from an identity without a role (HTTP 403).
The case then gives the VM identity a temporary FHIR Data Contributor role on the FHIR service and writes a Patient.
The role can take some minutes to apply, so the probe waits up to 10 minutes for a first HTTP 200.
Until the end of this wait, the probe sends the create again after HTTP 403 or 429.
It sends the search again after HTTP 403, 429, 500, 502, 503 or 504.
If a probe fails, the test error gives the stage and the HTTP status of the failed request.
An update adds DICOM. The case makes sure that the update kept the private endpoint.
It then reads the Patient again and makes sure that the DICOM host is private and needs a token.
At the end, the case sends FHIR and DICOM requests without a token from the test runner, outside the workspace network.
Azure must reject each request with HTTP 403. The case tries again for up to 5 minutes, because Azure can take this time to block public traffic.
The case then reads the healthcare workspace and DICOM again, and makes sure that each has public network access disabled.
The case does not read this value on FHIR. Azure keeps `publicNetworkAccess` `Enabled` on FHIR, also after a request sets `Disabled`.
The Health job uses the SQL time limits with its own `HEALTH_VALIDATION_DEADLINE`.

### Run the branch workflow before merge

1. Push the reviewed changes to a branch in `microsoft/AzureTRE`.
Expand Down
2 changes: 1 addition & 1 deletion docs/tre-developers/github-pr-bot-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The table describes the selected assertions. A passing command does not prove ev
| `/test-airlock` | Import requests, access checks, the import review VM flow and export request storage routing | No export review VM or migration proof. |
| `/test-backups` | Base workspace deployment with backups enabled and disabled | No backup restore or retention proof. |

The `workspace_services`, `linux_vm`, `airlock_validation`, `sql_validation`, `mysql_validation` and `gitea_validation` markers have no dedicated slash command.
The `workspace_services`, `linux_vm`, `airlock_validation`, `sql_validation`, `mysql_validation`, `gitea_validation` and `health_validation` markers have no dedicated slash command.
Use the branch workflow or the local test runner for these selections.
The `airlock_validation` marker selects two Airlock cases. It does not select the full `airlock` suite.
See [bundle selection](end-to-end-tests.md#validate-one-bundle) for exact cases and declared coverage gaps.
Expand Down
48 changes: 48 additions & 0 deletions docs/tre-templates/workspace-services/health_services.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,54 @@ In addition, when creating this workspace service you can choose to deploy [FHIR

![Healthcare Service](images/hs_details.png)

## Install failures before version 1.0.0

Before version 1.0.0, an install can fail after Azure creates the FHIR service.
If Azure reports the FHIR service as `Accepted` after the create, the azurerm provider stops its wait, and Terraform does not save the FHIR service.
After this failure, each update and each delete of the Health service fails.
To repair the service, delete its FHIR service in the Azure portal, and then delete the Health service again.

Before version 1.0.0, an install without FHIR and DICOM can fail at the private endpoint.
Azure keeps the failed endpoint, but Terraform does not save it. After this failure, each update and each delete of the Health service fails.
To repair the service, delete the private endpoint `pe-hs<suffix>` of the service in the Azure portal. Its provisioning state is `Failed`.
Then delete the Health service, or upgrade it and turn on FHIR or DICOM with an update.

## Private network access

The service connects the healthcare workspace to the workspace network through one private endpoint.
Azure fails an endpoint to a healthcare workspace without a FHIR or DICOM service. So Terraform creates the endpoint after FHIR and DICOM, and each Health service must deploy FHIR, DICOM or both.
If an install, an update or an upgrade has neither FHIR nor DICOM, Terraform stops before it changes a resource, with the message "Deploy FHIR, DICOM or both".
This applies to an upgrade of an earlier Health service without FHIR and DICOM. The upgrade dialog of the TRE UI cannot change a property.
If the upgrade stops for this reason, update the service and turn on FHIR or DICOM, or delete the service.
A disable of such a service also stops at this check, but the delete then works.
If a request turns on FHIR without `fhir_kind`, the service gets FHIR R4. The update form of the TRE UI does not send `fhir_kind`.
Azure writes the private DNS records of a zone when the DNS zone group of this endpoint gets that zone.
The DNS zone group always has the zone for the healthcare workspace and FHIR. It has the DICOM zone only when the service has DICOM.
If an update adds DICOM, Terraform adds the DICOM zone to the group, and Azure writes the DICOM record. The endpoint stays.
The FHIR record is in the same zone as the healthcare workspace record. So if an update adds or removes FHIR, Terraform deletes the group before the FHIR change and creates it again after the change. Azure then writes the records again.
Private access to the healthcare workspace and DICOM stops until the new group exists. For an update that adds FHIR, this time includes the FHIR create. In tests, the whole update that adds FHIR took 12 to 18 minutes.
If the FHIR change fails, the new group does not exist, and private access stays stopped. To repair the service, run the update again.
If the update fails again, set `deploy_fhir` back to its earlier value. Terraform then creates the group for the other services.
If Azure has a FHIR service that Terraform did not save, delete that FHIR service in the Azure portal first.
Azure links a new service to the endpoint some minutes after Terraform creates the service.
During this time, Azure refuses a change to the DNS zone group with HTTP 409, and Terraform tries again.

Before version 1.0.0, the endpoint resource created the DNS zone group, and the group did not change when an update added a service.
If you added DICOM to an existing service before this version, the DICOM host can resolve to a public address.
An upgrade to version 1.0.0 deletes the old DNS zone group and creates a new one, so Azure writes the missing record.
Terraform deletes the old group after the changes to FHIR and DICOM, directly before it creates the new group.
Until the upgrade creates the new group, the private DNS records do not exist, and private access to the services stops.
Do not delete a working private endpoint to repair the service.

The DICOM service has public network access turned off.
Before version 1.0.0, Terraform created the DICOM service with public network access on.
An upgrade to version 1.0.0 turns it off.

The private endpoint on the healthcare workspace blocks public traffic to FHIR and DICOM.
A request from outside the workspace network gets HTTP 403.
The FHIR service reports `publicNetworkAccess` `Enabled`, but Azure does not use this value when the workspace has an approved private endpoint.
Azure also keeps this value when a request sets it to `Disabled`.

## Authentication

Learn more about authentication and application roles in [this doc](https://learn.microsoft.com/en-us/azure/healthcare-apis/authentication-authorization). Make sure to assign your users/apps with the required role and follow the guidelines to retrieve a token.
Expand Down
17 changes: 12 additions & 5 deletions e2e_tests/bundle_coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -372,19 +372,26 @@
"manifest": "templates/workspace_services/health-services/porter.yaml",
"family": "workspace_service",
"prerequisites": [
"tre-workspace-base"
"tre-workspace-base",
"tre-service-guacamole",
"tre-service-guacamole-windowsvm"
],
"tests": [
"test_workspace_services.py::test_install_workspace_service[tre-workspace-service-health]"
"test_health.py::test_private_fhir_access_and_dicom_update"
],
"related_tests": [],
"coverage": [
"install",
"install with FHIR R4",
"private FHIR metadata",
"anonymous and unassigned FHIR requests denied",
"private FHIR write/read with a temporary data role",
"update that adds DICOM with retained FHIR data",
"public network access disabled",
"uninstall"
],
"limitations": [
"Functional, upgrade and custom-action coverage must be assessed separately.",
"Default inputs do not enable FHIR or DICOM. Endpoint validation requires an explicit configuration."
"AzureCloud only. Requires permission to run commands on a test-created workspace VM and create a temporary service-scoped role assignment.",
"The case does not use DICOM data, FHIR STU3 or researcher identities. The update uses the current bundle version. Previous-version migration remains separate (#5128)."
],
"unavailable_reason": ""
},
Expand Down
1 change: 1 addition & 0 deletions e2e_tests/pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ markers =
sql_validation: Private SQL query and configuration upgrade lifecycle
mysql_validation: opt-in private MySQL query and SKU change lifecycle from a Nexus-backed Linux VM
gitea_validation: opt-in private Gitea repository write/read across an update and an app restart
health_validation: opt-in private FHIR access control, write/read and DICOM update lifecycle

asyncio_mode = auto
asyncio_default_fixture_loop_scope = session
Expand Down
Loading