Emit ##vso[task.logissue] from output device on Azure DevOps#8777
Open
Evangelink wants to merge 7 commits into
Open
Emit ##vso[task.logissue] from output device on Azure DevOps#8777Evangelink wants to merge 7 commits into
Evangelink wants to merge 7 commits into
Conversation
When running on Azure DevOps (TF_BUILD=true) the output device now automatically prefixes ErrorMessageOutputDeviceData, ExceptionOutputDeviceData and WarningMessageOutputDeviceData with a `##vso[task.logissue type=error|warning]` line so that the message surfaces on the Azure Pipelines build summary instead of being just colored text in the agent log. The escaping rules (`;` -> `%3B`, `\r` -> `%0D`, `\n` -> `%0A`) match the ones used by the existing `Microsoft.Testing.Extensions.AzureDevOpsReport` extension and the behavior can be disabled per-run by setting `TESTINGPLATFORM_AZDO_OUTPUT=off|false|0`. Fixes microsoft#5979. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enhances Microsoft.Testing.Platform’s terminal output so that, when running on Azure DevOps agents (TF_BUILD=true), error/warning output emitted via TerminalOutputDevice also produces ##vso[task.logissue type=error|warning]... logging commands. This makes warnings/errors appear in the Azure Pipelines build summary (not just as colored log lines), and supports an opt-out via TESTINGPLATFORM_AZDO_OUTPUT=off|false|0.
Changes:
- Added an
AzureDevOpsLogIssueFormatterhelper to detect AzDO environment and format/escapetask.logissuecommands. - Updated
TerminalOutputDeviceto emittask.logissuelines before existing colored output for error/warning/exception output device data when in AzDO. - Added unit tests covering environment detection, formatting, and escaping.
Show a summary per file
| File | Description |
|---|---|
src/Platform/Microsoft.Testing.Platform/OutputDevice/AzureDevOpsLogIssueFormatter.cs |
New helper for AzDO detection and formatting/escaping ##vso[task.logissue] lines. |
src/Platform/Microsoft.Testing.Platform/OutputDevice/TerminalOutputDevice.cs |
Emits task.logissue commands for warnings/errors/exceptions when running under Azure DevOps. |
test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/AzureDevOpsLogIssueFormatterTests.cs |
New tests validating AzDO detection and task.logissue formatting/escaping behavior. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 3
Per the official Azure Pipelines logging-command spec, the message body must also escape '%' as '%25' (FIRST, so other replacements aren't double-encoded) and ']' as '%5D'. Updated Escape + added regression tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pipeline logging commands need stdout to be picked up by the AzDO agent, but JSON list mode must keep stdout clean. Drop the AzDO command in that mode and rely on the existing stderr warning/error text for human visibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Acceptance tests assert against literal stdout content. When tests run on Azure DevOps, TF_BUILD=true is inherited by child test hosts, causing the new TerminalOutputDevice to emit ##vso[task.logissue type=...] commands that break index-based line assertions (e.g. Help_WhenNoExtensionRegisteredAndUnknownOptionIsSpecified_OutputDefaultHelpContentAndUnknownOption). - TestHost.ExecuteAsync now injects TESTINGPLATFORM_AZDO_OUTPUT=off by default (opt-out parameter). - WellKnownEnvironmentVariables.ToSkipEnvironmentVariables now skips TESTINGPLATFORM_AZDO_OUTPUT so the parent value doesn't bleed in. - TerminalOutputDeviceTests sanitizes its assertion message so a failure does not itself contain a literal ##vso command (Copilot review feedback). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The new bool parameter shifted the CancellationToken positional argument; callers now pass cancellationToken as a named argument. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5979.
When running on Azure DevOps (
TF_BUILD=true)TerminalOutputDevicenow automatically prefixesErrorMessageOutputDeviceData,ExceptionOutputDeviceDataandWarningMessageOutputDeviceDatawith a##vso[task.logissue type=error|warning]line. The message therefore shows up on the Azure Pipelines build summary instead of being just colored text in the agent log.This is intentionally implemented in the platform (not in
Microsoft.Testing.Extensions.AzureDevOpsReport) because the AzureDevOpsReport extension only consumesTestNodeUpdateMessageand cannot intercept arbitraryIOutputDevicecalls. Any extension that emits error/warning output (e.g.HangDump,CrashDump,Retry,TrxReport,HtmlReport,ServerTestHost, ...) now benefits automatically.Escaping
The escaping rules (
;→%3B,\r→%0D,\n→%0A) match the ones used by the existing AzureDevOpsReport extension so the multi-line stack traces produced byException.ToString()collapse into a single##vso[...]command.Mapping
ErrorMessageOutputDeviceDataerrorExceptionOutputDeviceDataerrorWarningMessageOutputDeviceDatawarningOpt-out
The behavior can be disabled per-run by setting:
(
falseand0are also recognized.)Notes
ITerminalTestReporter.WriteMessage(normal mode) orWriteToStandardErrorAsync(--list-tests --output jsonmode) so that the AzDO line integrates with the active progress UI redraw logic rather than racing the raw console.Tests
AzureDevOpsLogIssueFormatterTestscovers detection (TF_BUILD, opt-out, case sensitivity), formatting, and escaping.Microsoft.Testing.Platform.UnitTestsrun: 1019 passed, 2 skipped, 0 failed.Microsoft.Testing.Extensions.UnitTestsrun: 362 passed, 4 skipped, 0 failed.