DisassemblyDiagnoserTests randomly failed on CI (macos(arm64))

On CI environment,
DisassemblyDiagnoserTests randomly failed with following exception.

StackTrace

CanDisassembleAllMethodCallsUsingFilters(jit: RyuJit, platform: X64, toolchain: .NET 8.0) [FAIL]
System.InvalidOperationException : Sequence contains no elements
Stack Trace:
     at System.Linq.ThrowHelper.ThrowNoElementsException()
     at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
  /Users/runner/work/BenchmarkDotNet/BenchmarkDotNet/tests/BenchmarkDotNet.IntegrationTests/DisassemblyDiagnoserTests.cs(114,0): at BenchmarkDotNet.IntegrationTests.DisassemblyDiagnoserTests.CanDisassembleAllMethodCallsUsingFilters(Jit jit, Platform platform, IToolchain toolchain)
     at InvokeStub_DisassemblyDiagnoserTests.CanDisassembleAllMethodCallsUsingFilters(Object, Span`1)
     at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

It seems disassembler silently failed on some conditions #2637

Currently this issue is observed on test-macos (x64) only.

When error occurred.
Following logs are outputted additionally.

It seems benchmark process is crashed with SIGSEGV signal (ExitCode: 139)

[createdump] Gathering state for process 21852 
[createdump] Writing full dump to file /var/folders/kk/699g20m50k56v2wnfvqxt3mm0000gn/T/tmpTzTOoa.tmp
[createdump] Written 5661995304 bytes (1382323 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 17504ms
ProcessData operation is interrupted by EndOfStream.
 // Benchmark Process 21852 has exited with code 139.

#3081 update ClrMD version and remove following temporary workaround code.
https://github.com/filzrev/BenchmarkDotNet/blob/415983103b3f6b63e201f966eabc21beb605559e/src/BenchmarkDotNet/Disassemblers/ClrMdDisassembler.cs#L59-L79

So this flaky test on test-macos (x64) issue might be resolved.

Close this PR.
Because it's expected to be resolved by #3081

It seems this issue is still happens after updated ClrMD version to 4.x.
https://github.com/dotnet/BenchmarkDotNet/actions/runs/26414467889/job/77755813305

Another patterns of flaky tests are occurred on CI (macos(x64))
It's occurred when using InProcessEmitToolchain. So it need to skip related tests.
(This type of error is not observed on previous CI, so it's expected to be occurred by recent ClrMD version update (#3260))

// ! InProcessEmitExecutor, exception: System.IO.InvalidDataException: Mach-O module at 2875fb000 contains a load command with invalid size 0.
   at Microsoft.Diagnostics.Runtime.MacOS.MachOModule..ctor(MachOCoreDump parent, IDataReader reader, MachHeader64& header, UInt64 address, String path, DataTargetLimits limits)
   at Microsoft.Diagnostics.Runtime.MacOS.MachOCoreDump.FindDylinker(Boolean firstPass)
   at Microsoft.Diagnostics.Runtime.MacOS.MachOCoreDump.ReadModules()
   at Microsoft.Diagnostics.Runtime.MacOS.MachOCoreDump.EnumerateModules()
   at Microsoft.Diagnostics.Runtime.MacOS.MachOCoreReader.EnumerateModules()
   at Microsoft.Diagnostics.Runtime.DataTarget.EnumerateModules()
   at Microsoft.Diagnostics.Runtime.DataTarget.GetOrCreateClrVersions()
   at Microsoft.Diagnostics.Runtime.DataTarget.get_ClrVersions()
   at BenchmarkDotNet.Disassemblers.ClrMdDisassembler.AttachAndDisassemble(ClrMdArgs args) in /_/src/BenchmarkDotNet/Disassemblers/ClrMdDisassembler.cs:line 76
   at BenchmarkDotNet.Diagnosers.DisassemblyDiagnoser.HandleAsync(HostSignal signal, DiagnoserActionParameters parameters, CancellationToken cancellationToken) in /_/src/BenchmarkDotNet/Disassemblers/DisassemblyDiagnoser.cs:line 106
   at BenchmarkDotNet.Diagnosers.CompositeDiagnoser.HandleAsync(HostSignal signal, DiagnoserActionParameters parameters, CancellationToken cancellationToken) in /_/src/BenchmarkDotNet/Diagnosers/CompositeDiagnoser.cs:line 42
   at BenchmarkDotNet.Toolchains.InProcess.Emit.InProcessEmitRunner.Run(IHost host, ExecuteParameters parameters) in /_/src/BenchmarkDotNet/Toolchains/InProcess/Emit/InProcessEmitRunner.cs:line 49
   at BenchmarkDotNet.Toolchains.InProcess.Emit.InProcessEmitExecutor.ExecuteCore(IHost host, ExecuteParameters parameters) in /_/src/BenchmarkDotNet/Toolchains/InProcess/Emit/InProcessEmitExecutor.cs:line 83
Executable /Users/runner/work/BenchmarkDotNet/BenchmarkDotNet/tests/BenchmarkDotNet.IntegrationTests/bin/Release/net10.0/BenchmarkDotNet.IntegrationTests-Dry-228Emitted.dll not found
ExitCode != 0 and no results reported

Please report that to https://github.com/microsoft/clrmd/issues

Please report that to https://github.com/microsoft/clrmd/issues

I've tried to reproduce InProcessEmitToolchain issue as commented at #3279.
Though, I can't collect required information to report issue.

Navigation Menu

Repository navigation

DisassemblyDiagnoserTests randomly failed on CI (macos(arm64)) #3086

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

DisassemblyDiagnoserTests randomly failed on CI (macos(arm64)) #3086

Description

Activity

filzrev commented on Apr 25, 2026

filzrev commented on Apr 28, 2026

filzrev commented on May 1, 2026

filzrev commented on May 25, 2026

filzrev commented on Oct 2, 2026

timcassell commented on Oct 2, 2026

filzrev commented on Oct 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions