Mouse/pointer events blocked while using arrow keys continuously

[ScepraX]

Describe the bug

Issue:
When using the arrow keys continuously pointer input stops responding.

Background:
I'm creating an application in Winui 3. Since updating from version 1.2 to version 1.3, pointer input stops working completely after a few seconds in the scenario where I simultaneously use the arrow keys continuously. In this app this results in no part of the application responding to any pointer input, except for the buttons in the title bar. When using other keys on the keyboard it resumes to normal behavior, but this is less than optimal.

This occurs even when no keyboard events are handled and a Canvas is used to display a object at the pointer position like in the example below.

Steps to reproduce the bug

1. Create a new Black App, Packaged (WinUI 3 in Desktop)

Note: This should be a project targeting the 1.3 version of the Microsoft.WindowsAppSDK. This issue did not occur in version 1.2 or earlier versions to my knowledge.

3. Replace the MainWindow.xaml content (the StackPanel) with the following code:
   <Grid> <Canvas Background="White" x:Name="PointerTestCanvas" PointerMoved="CanvasOnPointerMoved" HorizontalAlignment="Stretch" VerticalAlignment="Stretch" IsHitTestVisible="True" > <Rectangle x:Name="PointerTestRectangle" Fill="Red" Canvas.Left="0" Canvas.Top="0" Canvas.ZIndex="0" Width="100" Height="100" /> </Canvas> </Grid>
4. Remove the myButton_Click method from the code behind of MainWindow.xaml.cs.
5. Add the following code to the code behind of MainWindow.xaml.cs:
   private void CanvasOnPointerMoved(object sender, PointerRoutedEventArgs e) { var position = e.GetCurrentPoint(sender as UIElement); Canvas.SetLeft(PointerTestRectangle, position.Position.X); Canvas.SetTop(PointerTestRectangle, position.Position.Y); }
6. Run the application, make sure it has focus, and move the mouse/pointer so the rectangle will follow it. Keep one (or more) of the arrow keys pressed a couple of seconds (does not seem to be consistent and might sometimes take up to 10 seconds ) and watch as the pointer stops responding. In this case, as soon as you release the arrow key the pointer resumes normal behaviour.

If the application doesn't have focus this issue will not occur.

Expected behavior

I expected the mouse/pointer to keep responding to events like it did before version 1.3, even while the arrow keys are continuously used (like in for instance a game).

Screenshots

No response

NuGet package version

Windows App SDK 1.3.2: 1.3.230602002

Packaging type

Packaged (MSIX)

Windows version

Windows 11 version 22H2 (22621, 2022 Update)

IDE

Visual Studio 2022

Additional context

Doesn't seem to matter if:

• It's a packaged or non-packaged app.
• It's .NET 6 or .NET 7
• It's a SwapChainPanel instead of a Canvas UI element.
• Key modifiers are added to the input.

[jeffstall]

It's recommended to use SwapChainPanel.CreateCoreIndependentInputSource from a background thread to handle the scenario of getting continuous mouse messages that are not affected by what's in the UI thread's queue.

From your description, the UI thread's input queue is getting saturated from the keyboard messages (and potentially others as well). By design, all Win32 messages are processed in priority order (sent, post, quit, input, paint, timer) in essentially timestamp order. See GetMessage for details. If the app's UI thread can't keep up with processing the input messages quickly enough, new input messages will be added into the queue and coalesced where appropriate. This ensures that input event processing order occurs in the same order as the events were generated, which is important for changing focus with the mouse while typing.

[ScepraX]

@jeffstall: Thank you for your feedback. I will certainly look into the suggested implementation.

However, this does not explain why it only happens with the arrow keys. I'd also like to point out that this is a new "feature" since the 1.3 update and occurs in the WinUI 3 Gallery as well. If you'd open the gallery and keep an arrow key pressed for a couple of seconds the complete user interface will stop responding to pointer events until you release the arrow key. This does not seem related to any code I've implemented.

[codendone]

I'm told this issue still repros on WinAppSDK 1.6. Althrough SwapChainPanel.CreateCoreIndependentInputSource is a recommended approach for SwapChainPanel scenarios, mouse/pointer input should not be blocked by keyboard input.

[sbaeumlisberger]

Can still be reproduced in the WinUI 3 gallery and in my own applications. Releasing the arrow key does not help. I always have to press another key. Any missed mouse input than seems to be processed at once.
Could be related to #10009

[iablaauw-MS]

We found this to be a duplicate of #10009 and has been fixed in WinAppSDK 1.6.7 and 1.7.1.

(Despite the name of the linked issue, this was not affecting only remote desktop scenarios.)

Closing this issue but let us know if you run into this again!