Exception in test infrastructure - System.InvalidOperationException: Collection was modified after the enumerator was instantiated

[4creators]

For the second time I have encountered the following error during CI OSX test run.

https://ci.dot.net/job/dotnet_coreclr/job/master/job/x64_checked_osx10.12_innerloop_tst_prtest/5550/

02:51:26   Unhandled Exception: System.InvalidOperationException: Collection was modified after the enumerator was instantiated.
02:51:26      at System.Collections.Generic.Stack`1.Enumerator.MoveNext()
02:51:26      at Xunit.Sdk.DisposalTracker.Dispose() in C:\projects\xunit\src\xunit.execution\Sdk\DisposalTracker.cs:line 26
02:51:26      at Xunit.Sdk.ExtensibilityPointFactory.Dispose() in C:\projects\xunit\src\xunit.execution\Sdk\ExtensibilityPointFactory.cs:line 54
02:51:26      at Xunit.Sdk.TestFramework.Dispose() in C:\projects\xunit\src\xunit.execution\Sdk\Frameworks\TestFramework.cs:line 49
02:51:26      at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
02:51:26   --- End of stack trace from previous location where exception was thrown ---
02:51:26      at System.Threading.ThreadPoolWorkQueue.Dispatch()

[danmoseley]

@ViktorHofer does this look familiar?

[ViktorHofer]

that's something that dotnet/arcade#1613 should fix.

[ViktorHofer]

@echesakov any numbers how often this happens? have somebody already root-caused the issue? Seems like this lives in xunit/xunit and not in the runner.

[danmoseley]

xunit/xunit#1855

[safern]

Happened again in: https://dev.azure.com/dnceng/public/_build/results?buildId=646104&view=results

https://helix.dot.net/api/2019-06-17/jobs/6259c638-1c48-444a-9df5-b3d8b89354e4/workitems/PayloadGroup0/console