Retire the Grafana, Prometheus and Alertmanager UIs from the Coroot profiles #202

🤖 Generated by the Agentic Engineer

Evidence

The two Coroot profiles (k8s/clusters/{local,prod}-coroot) already drop Loki, Alloy and OpenCost (#96, #81). They still ship kube-prometheus-stack in full, because the infrastructure-controllers-coroot overlays include ../infrastructure/controllers/ and no component removes it. scripts/validate-observability-option.sh asserts this on purpose. Watchdog, FluxHelmReleaseNotReady and the platform-critical metric rules still run on Prometheus and Alertmanager.

So a platform started from a Coroot profile gets two observability UIs today: Coroot, and Grafana plus the Prometheus and Alertmanager web UIs. The second set is published through HTTPRoutes, the auth proxy, a Dex redirect and a Grafana OIDC secret.

Audience and problem

People who create a platform from this template and pick Coroot. They get a second set of dashboards the reference platform no longer runs, and three extra sign-in-protected endpoints to secure and maintain.

Hypothesis

Removing only the UI and ingress surfaces from the Coroot profiles leaves one observability UI, with no change to alerting. The alert engines and CRDs stay in place until later slices move their rules.

Smallest useful change

A k8s/components/observability-coroot/remove-kps-ui/ component, wired into both infrastructure-controllers-coroot overlays and following the remove-controllers-loki-alloy pattern. It:

• disables Grafana in the kube-prometheus-stack HelmRelease;
• deletes the grafana, prometheus and alertmanager HTTPRoutes;
• removes the auth-proxy routers and services for prometheus and alertmanager, using test-then-remove;
• deletes the grafana-oidc ExternalSecret;
• removes the Dex redirect URI for Grafana.

Acceptance criteria

• The Coroot renders contain none of those routes, auth-proxy entries, the grafana-oidc ExternalSecret or the Dex Grafana redirect. The default renders still contain each of them exactly once.
• The kube-prometheus-stack HelmRelease still renders with Watchdog on the heartbeat receiver, and the FluxHelmReleaseNotReady rule is still kept (the existing assert_watchdog_enabled and assert_legacy_flux_alert_deduplicated checks pass).
• scripts/validate-observability-option.sh asserts the removals, and its doc-phrase checks match the updated docs/TEMPLATING.md and docs/dr/alerting.md.
• The default profiles render byte-identical, and validate-scaffold is green.

Later slices (not this issue)

1. Move FluxHelmReleaseNotReady to a Flux Alert.
2. Give the remaining platform-critical rules Coroot parity, or delete them.
3. Replace Watchdog with the cluster heartbeat alone.
4. Remove the Tetragon ServiceMonitors and the CRDs.
5. Delete the release and the monitoring namespace, together with its network-policy, policy-exception, secret and homepage references.

Size: M.