RGB corruption with two or more kinects

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

RGB corruption with two or more kinects

Icarus123
This post has NOT been accepted by the mailing list yet.
Hello everyone,
I have searched the mailing list and seen this issue before. That being said there does not seem to be a working solution. The issue is that when 2 or more kinects are active at once they appear to corrupt each other's RGB data (but not depth) at random when viewed via the PCL visualization system. I have compiled PCL from source (via the GIT) as well as trying the prebuilt 1.7 binaries. In both cases I have seen this error. I am running ubuntu 12.04 LTS on a x64 system.

My openni package versions:

dpkg -l | grep openni
rc  libopenni-sensor-primesense0                  5.1.0.41-2+precise1        
rc  libopenni0                                                  1.5.4.0-4+precise1                                
ii  openni-dev                                                  1.3.2.1-4+precise2                            

Please let me know if there is something that I am missing or any solution to this issue.

Thank you,
- Michael


Reply | Threaded
Open this post in threaded view
|

Re: RGB corruption with two or more kinects

kwaegel
Administrator
Icarus123 wrote
Hello everyone,
I have searched the mailing list and seen this issue before. That being said there does not seem to be a working solution. The issue is that when 2 or more kinects are active at once they appear to corrupt each other's RGB data (but not depth) at random when viewed via the PCL visualization system. I have compiled PCL from source (via the GIT) as well as trying the prebuilt 1.7 binaries. In both cases I have seen this error. I am running ubuntu 12.04 LTS on a x64 system.

My openni package versions:

dpkg -l | grep openni
rc  libopenni-sensor-primesense0                  5.1.0.41-2+precise1        
rc  libopenni0                                                  1.5.4.0-4+precise1                                
ii  openni-dev                                                  1.3.2.1-4+precise2                            

Please let me know if there is something that I am missing or any solution to this issue.

Thank you,
- Michael
I encountered (and fixed) this issue in my own code after I blindly copy-pasted the OpenNI grabber implementation.

The problem is that the convertToXYZRGBPointCloud(..) function uses statically allocated resize buffers to copy the RGB and depth data. If more than one sensor is being used at the same time, this means they will happy trample each others data when creating a point cloud. This is a race condition and can happen (or not) randomly.

The RGB resize buffer is used every time an image is copied, even if the output is the same size, hence the frequent data corruption. The depth data only goes through a resize buffer if the depth image actually needs to be resized, so you usually don't see the same problem. I'm not sure why the two cases are handled differently.

The easiest solution is to convert the two resize buffers to be instance members of the grabber, not function-scope static. This is what I fixed in the link above.

(I'd be happy to fix this myself, but the OpenNI 1.x grabber is not forward-compatible with my compiler.)
Reply | Threaded
Open this post in threaded view
|

RE: RGB corruption with two or more kinects

Icarus123
This post has NOT been accepted by the mailing list yet.
Hello kwaegel,
Thank you for this insight! After examining the DIFF that you linked I was able to implement something similar in standard 1.x grabber and BAM, the corruption is gone.

Initially I thought about wrapping those methods in a a shared mutex but quickly realized that would lead to odd(er) code and would introduce a performance degradation. Instead I made all of the static variables in question member variables, much like your approach in the 2.x version.

Thanks again! You've been a great help.

- Michael 


Date: Tue, 8 Apr 2014 20:26:10 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: RGB corruption with two or more kinects

Icarus123 wrote
Hello everyone,
I have searched the mailing list and seen this issue before. That being said there does not seem to be a working solution. The issue is that when 2 or more kinects are active at once they appear to corrupt each other's RGB data (but not depth) at random when viewed via the PCL visualization system. I have compiled PCL from source (via the GIT) as well as trying the prebuilt 1.7 binaries. In both cases I have seen this error. I am running ubuntu 12.04 LTS on a x64 system.

My openni package versions:

dpkg -l | grep openni
rc  libopenni-sensor-primesense0                  5.1.0.41-2+precise1        
rc  libopenni0                                                  1.5.4.0-4+precise1                                
ii  openni-dev                                                  1.3.2.1-4+precise2                            

Please let me know if there is something that I am missing or any solution to this issue.

Thank you,
- Michael
I encountered (and fixed) this issue in my own code after I blindly copy-pasted the OpenNI grabber implementation.

The problem is that the convertToXYZRGBPointCloud(..) function uses statically allocated resize buffers to copy the RGB and depth data. If more than one sensor is being used at the same time, this means they will happy trample each others data when creating a point cloud. This is a race condition and can happen (or not) randomly.

The RGB resize buffer is used every time an image is copied, even if the output is the same size, hence the frequent data corruption. The depth data only goes through a resize buffer if the depth image actually needs to be resized, so you usually don't see the same problem. I'm not sure why the two cases are handled differently.

The easiest solution is to convert the two resize buffers to be instance members of the grabber, not function-scope static. This is what I fixed in the link above.

(I'd be happy to fix this myself, but the OpenNI 1.x grabber is not forward-compatible with my compiler.)


If you reply to this email, your message will be added to the discussion below:
http://www.pcl-users.org/RGB-corruption-with-two-or-more-kinects-tp4033392p4033393.html
To unsubscribe from RGB corruption with two or more kinects, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: RGB corruption with two or more kinects

nizar sallem
In reply to this post by kwaegel
Thank you very much,

This is very helpful indeed, can you please open an issue in github so
that we don't forget to fix it.
Thank you very much.

--
Nizar
On 09/04/2014 05:26, kwaegel wrote:

> Icarus123 wrote
>> Hello everyone,
>> I have searched the mailing list and seen this issue before. That being
>> said there does not seem to be a working solution. The issue is that when
>> 2 or more kinects are active at once they appear to corrupt each other's
>> RGB data (but not depth) at random when viewed via the PCL visualization
>> system. I have compiled PCL from source (via the GIT) as well as trying
>> the prebuilt 1.7 binaries. In both cases I have seen this error. I am
>> running ubuntu 12.04 LTS on a x64 system.
>>
>> My openni package versions:
>>
>> dpkg -l | grep openni
>> rc  libopenni-sensor-primesense0                  5.1.0.41-2+precise1
>> rc  libopenni0
>> 1.5.4.0-4+precise1
>> ii  openni-dev
>> 1.3.2.1-4+precise2
>>
>> Please let me know if there is something that I am missing or any solution
>> to this issue.
>>
>> Thank you,
>> - Michael
> I encountered ( and fixed
> <https://github.com/kwaegel/pcl/commit/ebd82de938627787078c57fe9ca9bd1458b3b365>
> ) this issue in my own code after I blindly copy-pasted the OpenNI grabber
> implementation.
>
> The problem is that the *convertToXYZRGBPointCloud(..)* function uses
> statically allocated resize buffers to copy the RGB and depth data. If more
> than one sensor is being used at the same time, this means they will happy
> trample each others data when creating a point cloud. This is a race
> condition and can happen (or not) randomly.
>
> The RGB resize buffer is used every time an image is copied, even if the
> output is the same size, hence the frequent data corruption. The depth data
> only goes through a resize buffer if the depth image actually needs to be
> resized, so you usually don't see the same problem. I'm not sure why the two
> cases are handled differently.
>
> The easiest solution is to convert the two resize buffers to be instance
> members of the grabber, not  function-scope static
> <https://stackoverflow.com/questions/246564/what-is-the-lifetime-of-a-static-variable-in-a-c-function>
> . This is what I fixed in the link above.
>
> (I'd be happy to fix this myself, but the OpenNI 1.x grabber is not
> forward-compatible with my compiler.)
>
>
>
> --
> View this message in context: http://www.pcl-users.org/RGB-corruption-with-two-or-more-kinects-tp4033392p4033393.html
> Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: RGB corruption with two or more kinects

Icarus123
This post has NOT been accepted by the mailing list yet.
Looks like kwaegel created the issue: https://github.com/PointCloudLibrary/pcl/issues/629
Reply | Threaded
Open this post in threaded view
|

Re: RGB corruption with two or more kinects

Kuta
In reply to this post by Icarus123
Is it also a problem with the OpenNI2 grabber?