HoloLens native keyboard issues

[MaxWang-MS]

Describe the bug

A few external bugs are affecting the native keyboard for HoloLens 2 (and likely HoloLens Gen 1 and WMR as well). This issue documents known bugs related to the UWP native keyboard in MRTK 2.6/2.7.

MRTK 2.7

Please make sure that you are using Unity 2019.4.31f1 or later / 2020.3.19f1 or later. Earlier Unity versions contain known keyboard issues and/or have different keyboard behaviors. MRTK 2.7 aligns with the keyboard behavior of Unity on or after the specified Unity versions.

If you are on Unity 2018.4, please scroll down and refer to the Unity 2018 LTS (2018.4) section under MRTK 2.6.

Note: If you are upgrading from an earlier MRTK version to 2.7, you MUST make sure you are on the recommended Unity version or there could be regression in keyboard functionalities.

Known issues (Unity-side, all resolved in latest versions):

• (Fixed in TextMeshPro 2.2.0-pre.1 & 3.2.0-pre.1)TextMeshPro Input Field is not working properly
  • Prior to TMP 2.1.5 (for Unity 2019) / 3.0.5 (for Unity 2020) TMP Input Field has a known display issue during character composition when typing in languages that use IME (i.e. Chinese/Japanese/Korean)
  • On or after TMP 2.1.5 / 3.0.5 but before 2.2.0-pre.1 / 3.2.0-pre.1 the caret does not show up when using TMP Input Field
• (Fixed in Unity 2019.4.31 & 2020.3.19)TouchScreenKeyboard.set_selection sometimes throws "ArgumentOutOfRangeException: Selection is out of range"
  • Note: the error should actually be benign The error can lead to unexpected behaviors in certain cases
  • Originally posted in Keyboard on the Hololens 2 not working #10025
  • Special note to developers working with Chinese/Japanese/Korean fonts: please re-import the TMP Essential Resources by using the Window - TextMeshPro - Import TMP Essential Resources menu after updating to the latest version TMP if the font does not display correctly
• (Fixed in Unity 2019.4.28 & 2020.3.10)The word suggestion disappears after pressing the backspace key
• (Fixed in Unity 2019.4.31 & 2020.3.19)HoloLens touchscreenkeyboard closes 4 seconds after focusing an input field
  • Originally posted in Keyboard disappearing all of a sudden #10191 and Inconsistent keyboard triggering from TMP input field #10058

MRTK 2.6

Unity 2020.2/3

We have not conducted extensive testing with 2020.2/3 and MRTK 2.6. We recommend that you use MRTK 2.7 along with the recommended Unity versions (see above) for the best experience, but in case you have to stay on MRTK 2.6 please only use Unity 2020.2.1f1 as there has been a Unity side behavior change along with an new issue impacting 2020.2.2f1 and later. Unity has fixed the issue in subsequent versions but the behavior change requires modification on the MRTK side (implemented in 2.7, see above for details).

Unity 2019 LTS (2019.4)

We recommend that you use MRTK 2.7 along with the recommended Unity versions (see above) for the best experience, but in case you have to stay on MRTK 2.6 please only use Unity 2019.4.9f1 - 2019.4.18f1 (inclusive) as there has been a Unity side behavior change along with an new issue impacting 2019.4.19f1 and later. Unity has fixed the issue in subsequent versions but the behavior change requires modification on the MRTK side (implemented in 2.7, see above for details). Unity versions prior to 2019.4.9f1 includes another UWP keyboard bug that is fixed in 2019.4.9f1.

Please see the table below for the status of HoloLens 2/UWP native keyboard on Unity 2019.4.16f1 with TMP 2.1.3:

Language	TMP Input Field	Unity UI Input Field	Other keyboard invocation method (using SystemKeyboardExample or MixedRealityKeyboard(Base))
Most languages (e.g. English)	⚠️Auto suggestion not available	⚠️Continuous deletion (i.e. holding the backspace key) is not supported & Insertion in the middle of the input field is not supported (pressing arrow keys can move the caret but the new letter is always appended to the end)	✔️Fully functional
East Asian languages that require IME (i.e. Chinese/Japanese/Korean)	⚠️Only the last input character of a word being typed in is shown before finalizing the word	⚠️Duplicate display of last input character, resolved when confirming the whole word	⚠️Arrow keys disabled in languages requiring IME to prevent from running into Unity related issues

Unity 2018 LTS (2018.4)

There are some complex issues in Unity 2018's implementation of UWP TouchScreenKeyboard that they are not planning to fix as it is approaching end of support. Thus the native keyboard experience on Unity 2018.4 is not as reliable as that of Unity 2019.4 and that applies to all invocation methods (e.g. via Unity UI/TMP input field or SystemKeyboardExample). For users that require a more robust experience please switch to Unity 2019.4.

If you have to use Unity 2018.4 for your project, here are some workaround for some known issues:

• For input fields, if pressing keys on the keyboard does not generate new letter the user needs to click on the enter key to hide the keyboard, click on the input field again and then trying typing.
• When directly using SystemKeyboardExample or MixedRealityKeyboard, try to avoid using the arrow keys as that sometimes causes characters to be inserted wrong place. Arrow keys are disabled for East Asian languages that require IME (i.e. Chinese, Japanese, Korean) as they almost certainly don't work for these languages.

The status shown above is based on Unity 2018.4.29f1 with TMP 1.4.1.

[Merichbier]

Any news about the issue affecting Unity 2019.4.19f1 and later ? Do we have more info on it ?

[MaxWang-MS]

Thanks for the follow up! The post has been updated to reflect the latest updates @Merichbier

[stale]

This issue has been marked as stale by an automated process because it has not had any recent activity. It will be automatically closed in 30 days if no further activity occurs. If this is still an issue please add a new comment with more recent details and repro steps.

[IssueSyncBot]

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all-new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visit https://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!