Desktop integration tests (Debug) hang frequently

[JunielKatarn]

Environment

Pull Request pipeline:
https://dev.azure.com/ms/react-native-windows/_build?definitionId=210

• Build Configuration: Debug (either x64 or x86)

Steps to Reproduce

1. Submit a pull request against this repository. The request's contents are irrelevant.

Expected Behavior

Job Desktop PR X86Debug or Desktop PR X64Debug succeeds within 30 seconds.

Actual Behavior

Job Desktop PR X86Debug or Desktop PR X64Debug times out (threshold was originally 50 minutes, current one is 5 minutes, which is still too long).

Sample build output:

##[section]Starting: Run Desktop Integration Tests
==============================================================================
Task         : Visual Studio Test
Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version      : 2.159.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : (FullyQualifiedName!~WebSocketJSExecutorIntegrationTest)&(FullyQualifiedName!=RNTesterIntegrationTests::WebSocket)&(FullyQualifiedName!~WebSocket)
Search folder : d:\a\1\s\vnext\target\x86\Debug
Run in parallel : false
Run in isolation : true
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Source filter: React.Windows.Desktop.IntegrationTests\React.Windows.Desktop.IntegrationTests.dll
SystemVssConnection exists true
[command]d:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.159.1\Modules\DTAExecutionHost.exe --inputFile d:\a\_temp\input_e7c5b931-f5c8-11e9-b14e-2d463724ae86.json
======================================================
##########################################################################
DtaExecutionHost version 18.159.29407.1.
Starting TestExecution Model...
Updated Run Settings:
<RunSettings>
  <RunConfiguration>
    <BatchSize>1000</BatchSize>
    <ResultsDirectory>d:\a\_temp\TestResults</ResultsDirectory>
  </RunConfiguration>
</RunSettings>
**************** Starting test execution *********************
[command]C:\hostedtoolcache\windows\VsTest\16.4.0-preview-20191007-01\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@C:\Users\VssAdministrator\AppData\Local\Temp\tmp4A98.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.4.0-preview-20191007-01
Copyright (c) Microsoft Corporation.  All rights reserved.
vstest.console.exe "d:\a\1\s\vnext\target\x86\Debug\React.Windows.Desktop.IntegrationTests\React.Windows.Desktop.IntegrationTests.dll"
/TestCaseFilter:"(FullyQualifiedName!~WebSocketJSExecutorIntegrationTest)&(FullyQualifiedName!=RNTesterIntegrationTests::WebSocket)&(FullyQualifiedName!~WebSocket)"
/Settings:"d:\a\_temp\kxnaqrtpgqr.runsettings"
/InIsolation
/Logger:"trx"
/TestAdapterPath:"d:\a\1\s\vnext\target\x86\Debug"
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  √ MakeIsNotNull [< 1ms]
  √ RequestGetSucceeds [237ms]
  √ RequestGetFails [2s 314ms]
  √ Dummy [5s 39ms]
  √ Logging [2s 647ms]
  ! LoggingWithNoCallback
  √ XHRSample [4s 294ms]
##[error]The operation was canceled.
##[section]Finishing: Run Desktop Integration Tests

Reproducible Demo

See Environment section.
This can happen in any given pull request.

Additional context

• This has not been observed when building locally. The hangs appear to only happen in the Microsoft-provided Azure VMs.
• VSTest, the underlying framework for running tests in Azure Pipelines, does not have an out-of-the-box mechanism to log what test is about to run, thus it's not clear which specific test causes this hang.
• This is likely NOT a timing "slowness" issue with the React bundler interaction, for the original timeout value for the job was 50 minutes, and the set of integration tests is reasonably small.

[chrisglein]

@licanhua You submitted a PR against this. Is the issue resolved? Do we need to track the unstable cases to reenable later? (if so add details to this issue so we can track it that way).

[JunielKatarn]

I have not submitted a PR yet.
Someone else disabled the Debug flavor of these tests for now to unblock most PRs.

I'm using this issue for tracking until the root cause is resolved.
I'll add the details (including which are the offending tests) as soon as I find out which are they.