Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/e2e_helper_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ on: # yamllint disable-line rule:truthy
paths:
- 'e2e_tests/**'
- 'templates/**/porter.yaml'
# test_azure_monitor_contract.py reads these Terraform files.
- 'core/terraform/azure-monitor/**'
- 'templates/workspaces/base/terraform/azure-monitor/**'
- '.github/workflows/e2e_helper_tests.yml'

permissions:
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ See [Legacy Airlock & migration](docs/azure-tre-overview/airlock.md#legacy-airlo

BUG FIXES:
* Update Nexus from 3.94.0 to 3.94.3. The Nexus 3.94.3 image stops trusting the eMudhra certificate chains that Sonatype reports as compromised. (`tre-shared-service-sonatype-nexus` 3.11.3) ([#5276](https://github.com/microsoft/AzureTRE/issues/5276))
* Change the Azure Monitor private link scope of the core and of each workspace only after its private endpoint exists. In workspaces, also wait 5 minutes after the scope changes before Terraform deletes the endpoint.
A scope change makes Azure update the endpoint, and during the update Azure rejects DNS zone group changes with 409 AnotherOperationInProgress, so some workspaces failed to deploy or delete.
(`core` 0.18.16, `tre-workspace-base` 2.12.3, `tre-workspace-unrestricted` 0.15.2, `tre-workspace-airlock-import-review` 0.18.2) ([#3193](https://github.com/microsoft/AzureTRE/issues/3193))
* Link App Insights to the Azure Monitor private link scope after the private endpoint exists. Before this fix, App Insights rejected all private ingestion with HTTP 403, so the core and workspace App Insights received no telemetry.
Rename the scoped service, so that existing deployments replace the link on upgrade. (`core` 0.18.16, `tre-workspace-base` 2.12.3, `tre-workspace-unrestricted` 0.15.2, `tre-workspace-airlock-import-review` 0.18.2) ([#5280](https://github.com/microsoft/AzureTRE/issues/5280))
* Update `vite` from 7.3.6 to 7.3.7 and the UI Vitest packages from 4.1.11 to 4.1.12 to address security advisories in the development server and the test mocker. (`ui` 0.10.4) ([#5274](https://github.com/microsoft/AzureTRE/pull/5274))
* Update `aiohttp` from 3.14.3 to 3.14.4 in `api_app`, `resource_processor`, and the CLI to address security advisories. (`API` 0.28.7, `resource_processor` 0.13.15, `CLI` 0.2.13) ([#5270](https://github.com/microsoft/AzureTRE/pull/5270))
* Rename a test variable in the E2E polling tests so that CodeQL stops reporting false clear-text logging alerts. ([#5266](https://github.com/microsoft/AzureTRE/pull/5266))
Expand Down
16 changes: 12 additions & 4 deletions core/terraform/azure-monitor/azure-monitor.tf
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,11 @@ resource "azurerm_monitor_private_link_scoped_service" "ampls_log_anaytics" {
resource_group_name = var.resource_group_name
scope_name = azurerm_monitor_private_link_scope.ampls_core.name
linked_resource_id = azurerm_log_analytics_workspace.core.id

# A link makes Azure update the private endpoint. During the update, Azure rejects a create of the DNS zone group.
depends_on = [
azurerm_private_endpoint.azure_monitor_private_endpoint,
]
}


Expand All @@ -104,10 +109,16 @@ resource "azurerm_application_insights" "core" {
}

resource "azurerm_monitor_private_link_scoped_service" "ampls_app_insights" {
name = "ampls-app-insights-service"
# The name changed from ampls-app-insights-service to replace the link in existing deployments.
name = "ampls-app-insights"
resource_group_name = var.resource_group_name
scope_name = azurerm_monitor_private_link_scope.ampls_core.name
linked_resource_id = azurerm_application_insights.core.id

# App Insights rejects ingestion through a private endpoint created after this link, with HTTP 403.
depends_on = [
azurerm_private_endpoint.azure_monitor_private_endpoint,
]
}

resource "azurerm_private_endpoint" "azure_monitor_private_endpoint" {
Expand All @@ -118,9 +129,6 @@ resource "azurerm_private_endpoint" "azure_monitor_private_endpoint" {
tags = var.tre_core_tags

lifecycle { ignore_changes = [tags] }
depends_on = [
azurerm_monitor_private_link_scoped_service.ampls_app_insights,
]

private_service_connection {
private_connection_resource_id = azurerm_monitor_private_link_scope.ampls_core.id
Expand Down
2 changes: 1 addition & 1 deletion core/version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.18.15"
__version__ = "0.18.16"
1 change: 1 addition & 0 deletions e2e_tests/unit_tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ These unit tests check the code used by the E2E test suite.
They cover authentication, operation polling, cleanup, timeouts, test selection and result reporting.

They run without Azure credentials or deployed resources.
`test_azure_monitor_contract.py` reads the Azure Monitor Terraform of the core and the base workspace. It makes sure of the order of the private link scope changes and the private endpoint.
Cases named after SQL, Nexus or certificates check the E2E setup, validation and cleanup for those cases.

The tests stay beside the E2E helpers they exercise.
Expand Down
67 changes: 67 additions & 0 deletions e2e_tests/unit_tests/test_azure_monitor_contract.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
"""Check the order of Azure Monitor private link scope changes and the private endpoint."""

from pathlib import Path
import re
import unittest

ROOT = Path(__file__).resolve().parents[2]
CORE = "core/terraform/azure-monitor/azure-monitor.tf"
WORKSPACE = "templates/workspaces/base/terraform/azure-monitor/azure-monitor.tf"
FILES = (CORE, WORKSPACE)
LINKS = (
"azurerm_monitor_private_link_scoped_service.ampls_app_insights",
"azurerm_monitor_private_link_scoped_service.ampls_log_anaytics",
)
ENDPOINT = "azurerm_private_endpoint.azure_monitor_private_endpoint"
WAIT = "terraform_data.ampls_endpoint_update_wait"


def read(path):
# A comment does not change the order, so a depends_on in a comment line must not count.
return re.sub(r"(?m)^\s*#.*\n", "", (ROOT / path).read_text())


def header(address):
kind, label = address.split(".")
return f'resource "{kind}" "{label}" {{'


def block(source, address):
body = source.split(header(address), 1)[1]
# terraform fmt puts the closing brace of a top-level block in the first column.
return body[: re.search(r"(?m)^}", body).start()]


def waits_for(source, address, target):
found = re.search(r"depends_on\s*=\s*\[([^\]]*)\]", block(source, address))
names = re.findall(r"\w+\.\w+", found.group(1)) if found else []
return target in names or any(waits_for(source, name, target) for name in names if header(name) in source)


class PrivateLinkScopeOrderTests(unittest.TestCase):
def test_links_wait_for_the_private_endpoint(self):
# App Insights rejects ingestion through a private endpoint that Azure creates after its link.
for path in FILES:
source = read(path)
for link in LINKS:
with self.subTest(path=path, link=link):
self.assertTrue(waits_for(source, link, ENDPOINT))
self.assertNotIn(link, block(source, ENDPOINT))


class WorkspaceEndpointUpdateTests(unittest.TestCase):
def test_scope_changes_wait_for_the_endpoint_update(self):
source = read(WORKSPACE)
for link in LINKS:
with self.subTest(link=link):
self.assertTrue(waits_for(source, link, WAIT))
self.assertTrue(waits_for(source, WAIT, ENDPOINT))

def test_destroy_waits_longer_than_the_measured_updates(self):
source = read(WORKSPACE)
wait = block(source, WAIT)
self.assertRegex(wait, r"when\s*=\s*destroy")
# Azure runs one endpoint update of up to 132 seconds for both unlinks, or one update of up to about
# 100 seconds for each unlink, one after the other. The longest measured total was 203 seconds.
unlinks = source.count('resource "azurerm_monitor_private_link_scoped_service"')
self.assertGreater(int(re.search(r'command\s*=\s*"sleep (\d+)"', wait).group(1)), unlinks * 100)
2 changes: 1 addition & 1 deletion templates/workspaces/airlock-import-review/porter.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
schemaVersion: 1.0.0
name: tre-workspace-airlock-import-review
version: 0.18.1
version: 0.18.2
description: "A workspace to do Airlock Data Import Reviews for Azure TRE"
dockerfile: Dockerfile.tmpl
registry: azuretre
Expand Down
2 changes: 1 addition & 1 deletion templates/workspaces/base/porter.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
schemaVersion: 1.0.0
name: tre-workspace-base
version: 2.12.2
version: 2.12.3
description: "A base Azure TRE workspace"
dockerfile: Dockerfile.tmpl
registry: azuretre
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,27 @@ resource "azurerm_monitor_private_link_scoped_service" "ampls_log_anaytics" {
resource_group_name = var.resource_group_name
scope_name = azapi_resource.ampls_workspace.name
linked_resource_id = azurerm_log_analytics_workspace.workspace.id

depends_on = [
terraform_data.ampls_endpoint_update_wait,
]
}

# A link or unlink in the scope makes Azure update the private endpoint. For the two unlinks, Azure runs one update
# of up to 132 seconds, or one update of up to about 100 seconds for each unlink, one after the other.
# In the measured deletes, the endpoint stayed busy for up to 3 minutes and 23 seconds.
# During the update, Azure rejects changes to the endpoint DNS zone group with 409 AnotherOperationInProgress.
# The scoped services depend on this resource, so they change only after the endpoint exists,
# and on destroy Terraform waits here before it deletes the endpoint.
resource "terraform_data" "ampls_endpoint_update_wait" {
provisioner "local-exec" {
when = destroy
command = "sleep 300"
}

depends_on = [
azurerm_private_endpoint.azure_monitor_private_endpoint,
]
}


Expand Down Expand Up @@ -157,11 +178,17 @@ resource "azapi_resource" "appinsights" {
}

resource "azurerm_monitor_private_link_scoped_service" "ampls_app_insights" {
name = "ampls-app-insights-service"
# The name changed from ampls-app-insights-service to replace the link in existing deployments.
name = "ampls-app-insights"
resource_group_name = var.resource_group_name
scope_name = azapi_resource.ampls_workspace.name

linked_resource_id = azapi_resource.appinsights.id

# App Insights rejects ingestion through a private endpoint created after this link, with HTTP 403.
depends_on = [
terraform_data.ampls_endpoint_update_wait,
]
}

resource "azurerm_private_endpoint" "azure_monitor_private_endpoint" {
Expand Down Expand Up @@ -191,10 +218,6 @@ resource "azurerm_private_endpoint" "azure_monitor_private_endpoint" {
var.blob_core_dns_zone_id,
]
}

depends_on = [
azurerm_monitor_private_link_scoped_service.ampls_app_insights,
]
}

# We don't really need this, but if not present the RG will not be empty and won't be destroyed
Expand Down
2 changes: 1 addition & 1 deletion templates/workspaces/unrestricted/porter.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
schemaVersion: 1.0.0
name: tre-workspace-unrestricted
version: 0.15.1
version: 0.15.2
description: "A base Azure TRE workspace"
dockerfile: Dockerfile.tmpl
registry: azuretre
Expand Down
Loading