What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

AldurDisciple
Hello,

I would like to understand what is happening when we turn on the 'depth to RGB' registration mode in OpenNI. I have been thinking of several alternatives, but cannot decide for one or another from what I have found on the internet:
1. The raw depthmap could be converted into a raw point cloud, transformed using factory settings, and projected back into the registered depthmap.
2. The raw depthmap could be directly transformed into a registered depthmap using factory settings.
3. The registered depthmap could be simply computed from data acquired from the RGB camera (some are also sensitive to IR)

Does someone know how this registration is done ? If factory settings are used, which seems the most probable, is there a way to know the exact transformation which is being applied ?

Thanks,
Nicolas

PS: I am using the Asus Xtion Pro camera, but any insight would be appreciated :)
Reply | Threaded
Open this post in threaded view
|

Re: What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

Radu B. Rusu
Administrator
Here's a good explanation on what's going on: http://www.ros.org/wiki/openni_launch/Tutorials/QuickStart#Registration:_matching_depth_to_color

Best,
Radu.

On Aug 20, 2013, at 9:14 AM, AldurDisciple <[hidden email]> wrote:

Hello,

I would like to understand what is happening when we turn on the 'depth to
RGB' registration mode in OpenNI. I have been thinking of several
alternatives, but cannot decide for one or another from what I have found on
the internet:
1. The raw depthmap could be converted into a raw point cloud, transformed
using factory settings, and projected back into the registered depthmap.
2. The raw depthmap could be directly transformed into a registered depthmap
using factory settings.
3. The registered depthmap could be simply computed from data acquired from
the RGB camera (some are also sensitive to IR)

Does someone know how this registration is done ? If factory settings are
used, which seems the most probable, is there a way to know the exact
transformation which is being applied ?

Thanks,
Nicolas

PS: I am using the Asus Xtion Pro camera, but any insight would be
appreciated :)



--
View this message in context: http://www.pcl-users.org/What-is-happening-behind-the-Depth-to-RGB-registration-mode-in-OpenNI-tp4029195.html
Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users


_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

AldurDisciple
Hi Radu,

Thanks for sharing this link, now I know what is going on.

In my application, I need to reverse the built-in "Depth to RGB registration" in oni sequences (which I cannot easily re-acquire in un-registered mode) in order to use a custom and hopefully more accurate calibration of the Asus Xtion.

Hence, I would like to determine exactly the "factory calibration" mentioned in the page you linked. Being mainly a PCL user, I am not sure how to get the 'dynamic_reconfigure gui' working. Alternatively, is there a way to determine the factory calibration directly using OpenNI ?

Cheers,
Nicolas
Reply | Threaded
Open this post in threaded view
|

Re: What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

PercocetPenguin
Hello,
I worked with a chap who was able to toggle depth registration on and off using pcl, so you should be able to do the same and avoid having to actually reverse the transformation.

Sent from my android device.

-----Original Message-----
From: AldurDisciple <[hidden email]>
To: "[hidden email]" <[hidden email]>
Sent: Wed, 21 Aug 2013 3:34 AM
Subject: Re: [PCL-users] What is happening behind the 'Depth to RGB' registration mode in OpenNI ?


Hi Radu,

Thanks for sharing this link, now I know what is going on.

In my application, I need to reverse the built-in "Depth to RGB
registration" in oni sequences (which I cannot easily re-acquire in
un-registered mode) in order to use a custom and hopefully more accurate
calibration of the Asus Xtion.

Hence, I would like to determine exactly the "factory calibration" mentioned
in the page you linked. Being mainly a PCL user, I am not sure how to get
the 'dynamic_reconfigure gui' working. Alternatively, is there a way to
determine the factory calibration directly using OpenNI ?

Cheers,
Nicolas



--
View this message in context: http://www.pcl-users.org/What-is-happening-behind-the-Depth-to-RGB-registration-mode-in-OpenNI-tp4029195p4029205.html
Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

AldurDisciple
I would be interested to have more details about this, because when I tried to use the function openni_device::setDepthRegistration(bool) in order to change the registration mode while reading my .ONI sequence, I received an exception saying "Turning registration off failed. Reason: The node is locked for changes!" (which makes sense) ...

I also found this page which describes closely what I am trying to do : http://www.ros.org/wiki/kinect_calibration/technical. However using the indicated values did not provide the same result as the built-in registration.
Reply | Threaded
Open this post in threaded view
|

Re: What is happening behind the 'Depth to RGB' registration mode in OpenNI ?

AldurDisciple
In case anyone else is looking for more details on the built-in registration mode, I just found this very interesting discussion between several of the people developping libfreenect :
https://groups.google.com/forum/#!searchin/openkinect/registered/openkinect/VaaFVUnvVVk/0GZWa6NGd5kJ

The actual registration processing occurs in the primesense driver, in file XnDeviceSensorV2/Registration.cpp (see for instance function Apply1080):
https://github.com/PrimeSense/Sensor/blob/master/Source/XnDeviceSensorV2/Registration.cpp

Cheers,
Nicolas