Few issues with New Project Setup (Face, Eye, And Body)

[dilmerv]

Guys great work on this project I am having a lot of fun !

Few things which I believe to be problems with new projects:

Looks like there are a couple of layers not included when you create a new project and import the Movement SDK.

These layers currently need to be created manually:

• HiddenMesh
• Character
• MirroredCharacter

Now I've Body, Face, And Eye tracking enabled in the OVR Camera Rig also Android Manifest has the proper permissions. Oculus app also has all the settings to be able to run this project with Oculus Link as shown below:

Now I think body tracking may be missing from above? or is this error something different?

[OVRPlugin] [XRCMD][failure] [XR_ERROR_RUNTIME_FAILURE]: xrLocateBodyJointsFB(m_xrBodyTracker, &info, &locations), arvr\projects\integrations\ovrplugin\src\util\compositoropenxr.cpp:5217
UnityEngine.Debug:LogWarning (object)
OVRManager:OVRPluginLogCallback (OVRPlugin/LogLevel,intptr,int) (at Assets/Oculus/VR/Scripts/OVRManager.cs:1628)

The result is that I can do eye tracking and face tracking successfully but body tracking is not working at all, I have a feeling this is due to not having body tracking available through Oculus Link ? I can try to run this right after from the device to see if it works and report back with my results.

Thanks again for all you do !
Dilmer

[dilmerv]

Ok I can confirm that deploying the demo scenes to a Meta Quest Pro work with body tracking, looks like Oculus Link may not have that option within the BETA tab in the Oculus App just yet.

[dilmerv]

Ok looks like everything is working now when I deploy it to the device, I also had to use Vulkcan since the OpenGLES3 made all the materials have a magenta type color.

Thanks guys false alarm but hopefully this helps other people.

[BattleAxeVR]

Hey there, I just had this same exact issue in my branch of hello_xr via Link. What do you mean, "false alarm?" This seems very much like body tracking simply doesn't work over Link yet.

My test code works fine on standalone mode (tested on Quest Pro but I believe the body tracking works on Quest 2, maybe even Quest 1, not sure).

I know eye tracking works both through Link and standalone, thankfully. But it needs an explicit permission request (besides the manifest) to use it on standalone

[dilmerv]

@BattleAxeVR you are correct I think the main issue here is that Oculus Link doesn't provide a body tracking option, I see face expressions and eye tracking but not body tracking.

Perhaps this is not really an issue with this repo but an update needed on the Oculus / Meta App.

[sohailshafiiWk]

Hello, thank you for the comments and I'm sorry you are dealing with Link body tracking issues. We will need to investigate on your behalf to see what's going on in regards to that problem.

Also, as you pointed out we need to add some notes regarding layers that are useful for these samples.

[BattleAxeVR]

Thank you so much! I would really appreciate having body tracking working on PC VR, not just for debugging / rapid iteration but to actually ship games that use it.

Above you can see my branch of hello_xr which reproduces this issue in C++ / OpenXR native code, where Quest over Link has a runtime error in xrLocateBodyJointsFB on PC, which returns XR_SUCCESS in standalone builds.

https://github.com/BattleAxeVR/OpenXR-SDK-Source/blob/42b78bdcb96adc27b7516b015e051d67830a6e7b/src/tests/hello_xr/openxr_program.cpp#L1432

[BattleAxeVR]

Btw, this runtime error happens on Quest 1 (in standalone mode) not just Link / AirLink on Quest 2 or Pro. I figured if Quest 1 supports Hand Tracking 2.0 via the generic hand tracking extension it might support the new body tracking. It would be really nice having all the Quest line work with this, when you get time. And on PC too.

[sohailshafiiWk]

Btw, this runtime error happens on Quest 1 (in standalone mode) not just Link / AirLink on Quest 2 or Pro. I figured if Quest 1 supports Hand Tracking 2.0 via the generic hand tracking extension it might support the new body tracking. It would be really nice having all the Quest line work with this, when you get time. And on PC too.

Thank you for the detailed notes. We have some folks looking at this issue right now and I'll be sure to provide some updates when a resolution is found.

[BattleAxeVR]

Thank you so much! I know Meta's going through a rough time but those of us who build VR games and apps know that Meta HMDs are the majority of the market, so we're rooting for you all to succeed. At least I am. Please keep us lowly PC VR folks in mind (I do work in both areas and having the same exact code on PC as on mobile is such a time-saver and what I love about OpenXR in general, but having feature parity would be awesome. I really wish Quest 3 would have eye/face tracking too, so the stuff we're building now on Pro can trickle down to consumer space next year).

[sohailshafiiWk]

Thank you so much! I know Meta's going through a rough time but those of us who build VR games and apps know that Meta HMDs are the majority of the market, so we're rooting for you all to succeed. At least I am. Please keep us lowly PC VR folks in mind (I do work in both areas and having the same exact code on PC as on mobile is such a time-saver and what I love about OpenXR in general, but having feature parity would be awesome. I really wish Quest 3 would have eye/face tracking too, so the stuff we're building now on Pro can trickle down to consumer space next year).

Thank you for the support. We are still trying to sort the issue out and will let you know when a solution has been found.

Also, we've added some notes to our repo's Readme regarding layers, near the end of the "Getting Started" section.

[Soumya323]

Hello Guys,
With the recent 1.2.1 sample and updating the layer as mentioned, body tracking works in Quest builds for my Quest-1. yeaaa

But still doesn't work over Oculus link and I get this warning
"[OVRPlugin] [XRCMD][failure] [XR_ERROR_RUNTIME_FAILURE]: xrLocateBodyJointsFB(m_xrBodyTracker, &info, &locations), arvr\projects\integrations\ovrplugin\src\util\compositoropenxr.cpp:5217".

It will be easier to do testing and dev stuff over link. Not sure if it's the same for other devs with the link.
If the link issue with body tracking something you guys are still fixing, please let us know once fixed.

[BattleAxeVR]

I tested Quest 1 on Android for body tracking, in standalone mode, two days ago and I was still getting that runtime error getting the body joints, for which the exact same code works fine on Quest 2 and Pro. Weird.

Same problem here with body tracking over Link on Quest 2/Pro.

[mattmerry-meta]

Hey all - again, thanks for the feedback here! With the details you provided, we were able to reproduce, root cause and fix the issue. The fix is currently in our public test channel, you can try it here: https://www.meta.com/help/quest/articles/headsets-and-accessories/oculus-rift-s/public-test-channel/