How to combine depth with RGB data?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to combine depth with RGB data?

goodgodgd
Hi.
Today I installed openNI and tested it.
Now I can get RGB image and depth image.
However, I don't know how to find pixel correspondences between RGB and depth images.
When I used Kinect SDK, there is a function, which has a very long name, "NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution".

Is there any function like this in openNI?
Please let me know.
I want to use a cloud of PointXYZRGB.
Reply | Threaded
Open this post in threaded view
|

Re: How to combine depth with RGB data?

aichim
Administrator
Hi,

They are already registered, so the pixel at position (u, v) corresponds to the (u, v) dexel.

Cheers,
Alex

On May 10, 2013, at 4:01 PM, goodgodgd <[hidden email]> wrote:

> Hi.
> Today I installed openNI and tested it.
> Now I can get RGB image and depth image.
> However, I don't know how to find pixel correspondences between RGB and
> depth images.
> When I used Kinect SDK, there is a function, which has a very long name,
> "NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution".
>
> Is there any function like this in openNI?
> Please let me know.
> I want to use a cloud of PointXYZRGB.
>
>
>
>
> --
> View this message in context: http://www.pcl-users.org/How-to-combine-depth-with-RGB-data-tp4027771.html
> Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: How to combine depth with RGB data?

goodgodgd
No, that's not true.
Here are the depth and color images I captured.
 

You can see that the images are slightly misaligned.
And here's the main function.

int main()
{
        XnStatus nRetVal = XN_STATUS_OK;
        xn::Context context;
        // Initialize context object
        nRetVal = context.Init();
        // TODO: check error code
        if (nRetVal != XN_STATUS_OK)
        {
                printf("context init failed: %s\n", xnGetStatusString(nRetVal));
                return (nRetVal);
        }

        // Create a DepthGenerator node
        xn::DepthGenerator depthnode;
        nRetVal = depthnode.Create(context);
        // TODO: check error code
        if (nRetVal != XN_STATUS_OK)
        {
                printf("No depth node: %s\n", xnGetStatusString(nRetVal));
                return 1;
        }
        // depth meta data 받기
        xn::DepthMetaData depthMD;
        depthnode.GetMetaData(depthMD);

        // Create a ImageGenerator node
        xn::ImageGenerator imagenode;
        nRetVal = imagenode.Create(context);
        // TODO: check error code
        if (nRetVal != XN_STATUS_OK)
        {
                printf("No image node: %s\n", xnGetStatusString(nRetVal));
                return 1;
        }
        // image meta data 받기
        xn::ImageMetaData imageMD;
        imagenode.GetMetaData(imageMD);

        // Make it start generating data
        nRetVal = context.StartGeneratingAll();
        // TODO: check error code
        if (nRetVal != XN_STATUS_OK)
        {
                printf("Failed to generate all: %s\n", xnGetStatusString(nRetVal));
                return 1;
        }

        // depth 데이터 받을때까지 기다리기
        nRetVal = context.WaitOneUpdateAll(depthnode);
        // image 데이터 받을때까지 기다리기
        nRetVal = context.WaitOneUpdateAll(imagenode);


        // depth resolution 확인
        const XnDepthPixel* pDepthRow = depthMD.Data();
        printf_s("imgsize : total=%d, fullx=%d, fully=%d, xres=%d, yres=%d\n",depthMD.DataSize(), depthMD.FullXRes(), depthMD.FullYRes(), depthMD.XRes(), depthMD.YRes());
        // IplImage로 복사하여 화면에 띄우기
        IplImage* depth = cvCreateImage(cvSize(depthMD.XRes(), depthMD.YRes()), IPL_DEPTH_8U, 1);
        for(int i=0;i<(int)(depthMD.XRes()*depthMD.YRes());i++)
                depth->imageData[i] = (char)(BYTE)((double)pDepthRow[i]/(double)depthnode.GetDeviceMaxDepth()*256.0);
        cvNamedWindow("OpenNI depth");
        cvShowImage("OpenNI depth", depth);
        cvSaveImage("depth.jpg", depth);


        // image resolution 확인
        const XnRGB24Pixel* pImageRow = imageMD.RGB24Data();
        printf_s("imgsize : total=%d, fullx=%d, fully=%d, xres=%d, yres=%d\n",imageMD.DataSize(), imageMD.FullXRes(), imageMD.FullYRes(), imageMD.XRes(), imageMD.YRes());
        // IplImage로 복사하여 화면에 띄우기
        IplImage* image = cvCreateImage(cvSize(imageMD.XRes(), imageMD.YRes()), IPL_DEPTH_8U, 3);
        for(int i=0;i<(int)(depthMD.XRes()*depthMD.YRes());i++)
        {
                image->imageData[i*3] = pImageRow[i].nBlue;
                image->imageData[i*3+1] = pImageRow[i].nGreen;
                image->imageData[i*3+2] = pImageRow[i].nRed;
        }
// memcpy(image->imageData, pImageRow, imageMD.XRes()*imageMD.YRes()*3);
        cvNamedWindow("OpenNI image");
        cvShowImage("OpenNI image", image);
        cvSaveImage("image.jpg", image);

        cvWaitKey();

        return 0;
}


Is there anybody knows a solution to the pixel correspondence problem?


Reply | Threaded
Open this post in threaded view
|

Re: How to combine depth with RGB data?

aichim
Administrator
Hi,

You are not using PCL at all :-). If you were using our OpenNIGrabber, then things would have been like I said in my previous mail.
Please go to the appropriate forums for solutions on this.

Cheers,
Alex
---
http://www.alexichim.com
http://www.openperception.org




On May 11, 2013, at 7:21 AM, goodgodgd <[hidden email]> wrote:

> No, that's not true.
> Here are the depth and color images I captured.
> <http://www.pcl-users.org/file/n4027777/depth.jpg>  
> <http://www.pcl-users.org/file/n4027777/image.jpg>
>
> You can see that the images are slightly misaligned.
> And here's the main function.
>
> int main()
> {
> XnStatus nRetVal = XN_STATUS_OK;
> xn::Context context;
> // Initialize context object
> nRetVal = context.Init();
> // TODO: check error code
> if (nRetVal != XN_STATUS_OK)
> {
> printf("context init failed: %s\n", xnGetStatusString(nRetVal));
> return (nRetVal);
> }
>
> // Create a DepthGenerator node
> xn::DepthGenerator depthnode;
> nRetVal = depthnode.Create(context);
> // TODO: check error code
> if (nRetVal != XN_STATUS_OK)
> {
> printf("No depth node: %s\n", xnGetStatusString(nRetVal));
> return 1;
> }
> // depth meta data 받기
> xn::DepthMetaData depthMD;
> depthnode.GetMetaData(depthMD);
>
> // Create a ImageGenerator node
> xn::ImageGenerator imagenode;
> nRetVal = imagenode.Create(context);
> // TODO: check error code
> if (nRetVal != XN_STATUS_OK)
> {
> printf("No image node: %s\n", xnGetStatusString(nRetVal));
> return 1;
> }
> // image meta data 받기
> xn::ImageMetaData imageMD;
> imagenode.GetMetaData(imageMD);
>
> // Make it start generating data
> nRetVal = context.StartGeneratingAll();
> // TODO: check error code
> if (nRetVal != XN_STATUS_OK)
> {
> printf("Failed to generate all: %s\n", xnGetStatusString(nRetVal));
> return 1;
> }
>
> // depth 데이터 받을때까지 기다리기
> nRetVal = context.WaitOneUpdateAll(depthnode);
> // image 데이터 받을때까지 기다리기
> nRetVal = context.WaitOneUpdateAll(imagenode);
>
>
> // depth resolution 확인
> const XnDepthPixel* pDepthRow = depthMD.Data();
> printf_s("imgsize : total=%d, fullx=%d, fully=%d, xres=%d,
> yres=%d\n",depthMD.DataSize(), depthMD.FullXRes(), depthMD.FullYRes(),
> depthMD.XRes(), depthMD.YRes());
> // IplImage로 복사하여 화면에 띄우기
> IplImage* depth = cvCreateImage(cvSize(depthMD.XRes(), depthMD.YRes()),
> IPL_DEPTH_8U, 1);
> for(int i=0;i<(int)(depthMD.XRes()*depthMD.YRes());i++)
> depth->imageData[i] =
> (char)(BYTE)((double)pDepthRow[i]/(double)depthnode.GetDeviceMaxDepth()*256.0);
> cvNamedWindow("OpenNI depth");
> cvShowImage("OpenNI depth", depth);
> cvSaveImage("depth.jpg", depth);
>
>
> // image resolution 확인
> const XnRGB24Pixel* pImageRow = imageMD.RGB24Data();
> printf_s("imgsize : total=%d, fullx=%d, fully=%d, xres=%d,
> yres=%d\n",imageMD.DataSize(), imageMD.FullXRes(), imageMD.FullYRes(),
> imageMD.XRes(), imageMD.YRes());
> // IplImage로 복사하여 화면에 띄우기
> IplImage* image = cvCreateImage(cvSize(imageMD.XRes(), imageMD.YRes()),
> IPL_DEPTH_8U, 3);
> for(int i=0;i<(int)(depthMD.XRes()*depthMD.YRes());i++)
> {
> image->imageData[i*3] = pImageRow[i].nBlue;
> image->imageData[i*3+1] = pImageRow[i].nGreen;
> image->imageData[i*3+2] = pImageRow[i].nRed;
> }
> // memcpy(image->imageData, pImageRow, imageMD.XRes()*imageMD.YRes()*3);
> cvNamedWindow("OpenNI image");
> cvShowImage("OpenNI image", image);
> cvSaveImage("image.jpg", image);
>
> cvWaitKey();
>
> return 0;
> }
>
>
> *Is there anybody knows a solution to the pixel correspondence problem?*
>
>
>
>
>
>
> --
> View this message in context: http://www.pcl-users.org/How-to-combine-depth-with-RGB-data-tp4027771p4027777.html
> Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: How to combine depth with RGB data?

goodgodgd
Sorry that I misjudged you. You're right.
I saw the opennigrabber in some simple viewer examples, but I couldn't know how to access RGBD data.
So I read the openni samples.
Now I have studied more about opennigrabber and found the access to RGBD.
Thank you.