Skip to content

No Popups from test crashes#745

Open
johnml1135 wants to merge 3 commits intomainfrom
no-popups-from-test-crashes
Open

No Popups from test crashes#745
johnml1135 wants to merge 3 commits intomainfrom
no-popups-from-test-crashes

Conversation

@johnml1135
Copy link
Contributor

@johnml1135 johnml1135 commented Mar 6, 2026

Summary

  • suppress popup-based assertion behavior in managed and native test runs
  • mirror assertion and last-chance managed exception details to console output
  • ensure test failures fail fast instead of blocking or silently continuing
  • cover installer test projects that opt out of the shared test assembly bootstrap
  • fix TestGeneric debug include paths so native tests build with the new DebugProcs dependency

Validation

  • ./build.ps1
  • ./test.ps1
  • ./test.ps1 -Native

Notes

  • managed test suite passed across 43 test assemblies
  • native test executables passed: testGenericLib.exe and TestViews.exe

This change is Reviewable

Copilot AI review requested due to automatic review settings March 6, 2026 20:07
@johnml1135 johnml1135 changed the title No Popups from test branches No Popups from test crashes Mar 6, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to prevent modal popups during managed and native test runs by enforcing “test mode” assertion behavior, and by ensuring assertion/exception details are visible in console output so CI fails fast and diagnosably.

Changes:

  • Add FW_TEST_MODE=1 to managed/native test runners and runsettings to bypass assertion dialog behavior (including registry overrides).
  • Introduce NUnit assembly-level attributes to suppress assert dialogs and log last-chance exceptions to console for projects that don’t use the shared bootstrap.
  • Update native test project includes to build with the DebugProcs dependency and disable assertion dialogs in testGeneric.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test.ps1 Sets FW_TEST_MODE for managed test runs outside .runsettings.
Test.runsettings Adds FW_TEST_MODE to test environment variables.
Src/InstallValidator/InstallerArtifactsTests/TestAssemblyInfo.cs Adds assembly-level unhandled-exception logging for installer artifact tests.
Src/InstallValidator/InstallerArtifactsTests/InstallerArtifactsTests.csproj Links AppForTests.config and compiles the logging attribute for tests opting out of shared bootstrap.
Src/InstallValidator/InstallValidatorTests/TestAssemblyInfo.cs Adds assembly-level unhandled-exception logging for installer validator tests.
Src/InstallValidator/InstallValidatorTests/InstallValidatorTests.csproj Compiles the logging attribute for tests opting out of shared bootstrap.
Src/Generic/Test/testGeneric.cpp Disables native assert dialogs during global test setup.
Src/Generic/Test/TestGeneric.vcxproj Fixes include paths for DebugProcs dependency.
Src/DebugProcs/DebugProcs.cpp Adds FW_TEST_MODE override for assert-dialog behavior; mirrors asserts to stderr.
Src/Common/FwUtils/FwUtilsTests/Attributes/SuppressAssertDialogsAttribute.cs New NUnit attribute to suppress assert UI and mirror trace/assert output to console.
Src/Common/FwUtils/FwUtilsTests/Attributes/LogUnhandledExceptionsAttribute.cs New NUnit attribute to log last-chance exceptions to console.
Src/Common/FwUtils/FwUtilsTests/Attributes/HandleApplicationThreadExceptionAttribute.cs Writes WinForms thread exceptions to console before rethrowing.
Src/AssemblyInfoForUiIndependentTests.cs Enables last-chance exception logging for UI-independent test assemblies.
Src/AssemblyInfoForTests.cs Enables last-chance exception logging and assert-dialog suppression for most test assemblies.
Src/AppForTests.config Adds ConsoleTraceListener to mirror trace output to console.
Build/scripts/Invoke-CppTest.ps1 Sets FW_TEST_MODE for native test runner execution.

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

NUnit Tests

    1 files  ± 0      1 suites  ±0   5m 49s ⏱️ -29s
4 417 tests +10  4 330 ✅ +10  87 💤 ±0  0 ❌ ±0 
4 426 runs  +10  4 339 ✅ +10  87 💤 ±0  0 ❌ ±0 

Results for commit 5bfc3fe. ± Comparison against base commit 1fda1e5.

♻️ This comment has been updated with latest results.

@johnml1135 johnml1135 force-pushed the no-popups-from-test-crashes branch from 917324f to 88c7070 Compare March 13, 2026 13:55
@johnml1135
Copy link
Contributor Author

This fix changes LogUnhandledExceptionsAttribute so unobserved task exceptions no longer throw from TaskScheduler.UnobservedTaskException.

Why this was needed:

  • The attribute summary says it is a logging-only hook, but the previous implementation threw UnobservedTaskExceptionLoggedException from the event handler.
  • On .NET Framework 4.5+ / net48, unobserved task exceptions are raised during GC/finalization and are swallowed by default after the event fires.
  • Throwing from inside that handler creates a new unhandled exception on the finalizer thread, which can terminate the entire test host nondeterministically and without useful NUnit attribution.

What this change does instead:

  • Log the unobserved task exception when the runtime raises the event.
  • Capture the AggregateException in a queue and mark it observed.
  • During suite teardown, force GC/finalizer passes while the handler is still attached, then fail once with a deterministic NUnit AssertionException that includes the captured exception details.

That preserves the original goals:

  • no popup/assert UI,
  • no silent loss of background task failures,
  • no random process crash from the finalizer thread.

Also added focused tests in FwUtilsTests for the new collector/failure behavior.

@johnml1135
Copy link
Contributor Author

Follow-up review fixes are now in 5bfc3fedf.

What changed:

  • applied SuppressAssertDialogs to the UI-independent shared test assembly info so HCSynthByGlossTest and HCSynthByGlossDllTest get the same assert-dialog suppression/fallback behavior as the UI test assemblies
  • kept AppForTests.config on FwTraceListener only; the earlier ConsoleTraceListener concern is no longer applicable because that listener was removed
  • preserved the earlier cleanup in SuppressAssertDialogsAttribute:
    • restore AssertUiEnabled, AssertExceptionEnabled, and FW_TEST_MODE after the suite
    • only insert the custom trace listener when one is not already present
    • log only failure/assert paths instead of mirroring all trace output
  • preserved the native DebugProcs cleanup:
    • gate stderr mirroring on FW_TEST_MODE
    • use warning-clean environment variable helpers so native builds stay green under warnings-as-errors

Validation run:

  • ./test.ps1 -TestProject "Src/Common/FwUtils/FwUtilsTests"
  • ./test.ps1 -Native -TestProject TestGeneric
  • ./test.ps1 -TestProject "Src/Utilities/HCSynthByGloss/HCSynthByGloss/HCSynthByGlossTest"
  • ./test.ps1 -TestProject "Src/Utilities/HCSynthByGloss/HCSynthByGlossDll/HCSynthByGlossDllTest"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants