Transforming Point Cloud to get Top Down image

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view

Transforming Point Cloud to get Top Down image

This post has NOT been accepted by the mailing list yet.

My task :
I have a task where i am asked to track parcels(carton boxes) of different dimensions moving on a conveyor. I am using Asus Xtion pro camera mounted on top of a conveyor in any inclined angle. I am looking for a model free object tracker that will detect boxes in the scene,track them & gives their 6DOF ? My target object is just a box and i want to eliminate all other things in the scene.

My approach :

Step 1) I do Point cloud preprocessing like downsampling,pass through filtering and segmentation. All these should give me a final point cloud containing only the objects on the conveyor.

Step 2) I planned to make the "z" values in each point(depth value) as zero, thereby making the point cloud of the box to be flat on the ground.

step 3) I planned to transfer the view of the camera from any inclined position to a top down view so that i can view any number of carton boxes moving on the conveyor from a top down view . I feel the top down view will prevent perpective viewing problems

The process flow of step 2 and 3 is shown below

step 4) After the top down view of the point cloud is achieved, i need to convert the 3d point cloud to 2d image ,so that i can perform object tracking with so many OpenCV based tracking algorithms available.

A Sample point cloud is shown below in different views

Original View from camera:

Point Cloud View 1:

Point Cloud View 2:

Point Cloud Target/Desired View for converting to 2d :(The box is the target.All the ground plane and unnecessary points would be eliminated)

Is my approach correct? How will i achieve steps 2,3 and 4?
Reply | Threaded
Open this post in threaded view

Re: Transforming Point Cloud to get Top Down image

If you find the floor plane, then you can project the object points to the plane if you wish.
pcl::ProjectInliers does that for you.

If you want to transform your cloud, you can use
There are also a number of pcl::getTransformation... functions you might find helpful.

Or you could mount your camera from above to begin with?

You should be able to fill in a cv::mat from the points quite easily. There is probably some discussion here regarding that if you have a search (haven't had the need to do myself).