Get depth value (distance) of IR marker

[jovinteng]

I'm tracking a spherical retroreflective ball marker using the Kinect Azure IR camera but am unable to extract its depth information accurately. I discovered that as long as the ball is directly in-line with the depth camera it will be able to detect it up to a certain distance, however when I move the marker around (e.g. towards the edges and corners of the frame) the depth value is invalidated and therefore resorts to zero value.

I also had a look at the depth image to see if the depth camera was able to detect my marker but it showed up as a black circle which according to the SDK documentation indicates that this is due to a saturated/weak IR signal or ambiguous depth.

Furthermore, I've tried using the OpenCV inpaint function to try and fill in the missing depth pixels but it proves to be largely inaccurate.

I'm convinced that there is a solution to this but I'm stuck at the moment so any help would be greatly appreciated! Cheers!

[qm13]

This may be a result of saturation due to an overly reflective retroreflector. Some questions / pointers:
• Are you using NFOV or WFOV? WFOV will be less prone to saturation but has a shorter range.
• Do you have retroreflectors of different brightness's? It would be good to experiment to see at what distance they fail and whether you can duplicate the behavior with another retroreflective object.
• If you can’t get a retroreflector with lower apparent brightness, then one potential (ad-hoc) workaround would be to attach a neutral density filter to the front of the camera system. If the system is indoors and not in a high ambient level the simplest approach is to place over the illuminators. Be careful to not place over the illuminator and circular depth imager window as this will more than likely result in direct coupling between the illuminators and the depth sensor thus degrading accuracy.

[nh236]

I am facing the same problem.
I'm using NFOV mode, but the problem is also present in the other modes. I have tried reflectors of different sizes (down to 9mm), but without much of an improvement.

@qm13 : Could you be more specific on the type of neutral density filter? There seem to be many different options with different properties and I have no idea which one would be suited for use with the Kinect.

Any help would be much appreciated.

[qm13]

@nh236 Try something like this. Contrary to my previous comment you may want to put the filter over the camera as the heat of the illuminator may melt the gel.

[jelling]

Has anyone made progress on this problem? I want to track an object/marker at ~6 feet distance and am trying to understand the best options with Kinect or otherwise. ARKit on iOS works up to about 1 meter / 3.5 feet.

[stytim]

I made a hack to track retroreflective markers using Azure Kinect. To put it simply, I sampled the pixels around the invalid depth regions and used the average of them as the depth value. If anyone is interested, you can take a look at the repo: https://github.com/stytim/AzureKinect-ToolTracker