3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
Hello everyone,

I am trying to use the correspondence_grouping.cpp from http://pointclouds.org/documentation/tutorials/correspondence_grouping.php to identify a 3d object in a scene as explained in the tutorial.
It is working fine with the given pcd files milk.pcd and milk_cartoon_all_small_clorox.pcd.

I have similar pcd files with "just object" and "object with scene" obtained from kinect that I have. I am trying to find instance of the object in the scene to get the rotation matrix and translation vector of the object from the scene.  But when I try I get an ouput that the model instances found are 0. I have tried to change model and scene sampling  radius and few parameters but result doesn't seem to have changed.
Please help me out in finding the correct parameters to recognize the 3D object from the scene and get its translation and rotation matrices. Or help me out in figuring out if I am doing something wrong.

Here is my log:


sanmarino@ubuntu:~/Desktop/Pcl_experiments/object recognition/build$ ./correspondence_grouping  scene_with_just_object.pcd scene_modified_ascii_removed_nan.pcd -c --model_ss 0.005 --scene_ss 0.01
Model total points: 16437; Selected Keypoints: 2238
Scene total points: 103831; Selected Keypoints: 5401
[pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 746
[pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 749
Correspondences found: 4717
Model instances found: 0

I am attaching the pcd files that I am using.
Any help is greatly appreciated.

Sincerely,
Ramsri Gouthamscene_modified_ascii_removed_nan.pcdscene_with_just_object.pcd
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

fedassa
Hi Ramsri

I tried to run the tute with your data and found this suitable set of params for recognizing your object:

--model_ss 0.01 --scene_ss 0.0125 --rf_rad 0.08 --descr_rad 0.08 --cg_size 0.05 --cg_thresh 10 --algorithm Hough

although I'm sure that by playing around a bit with the params you can get even better results in terms of pose estimation.

Since tweaking these params seem to be a bit tricky for most tute users, me and Tommaso are about to add a new feature to the tutorial where all metric params will be specifiable also by means of the mean cloud resolution, so that the values you use for the parameters should become less data dependent. Stay tuned.

Federico
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
Thank you very much Federico ! It works very well.

 I plan to do this real-time so I would like to have consistent recognition all the time. So can you please let me know if this can be done real-time with reasonable recognition and also what params are crucial and need to be taken care of carefully?  
Thank you for your time and effort.


Sincerely,
Ramsri
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

fedassa
Ramsri,

real-time object recognition is yet not available on pcl_recognition, and efficiency mainly depends on the keypoint detection and descriptions stages, since matching and grouping are often (not always though) reasonably faster. So choosing the right detector and descriptor might turn out to be important for the computational aspect of your application. Also, remember that in order to provide a robust object recognition you should complete the 3D pipeline by plugging in also a final Hypothesis Verification step (you can find appropriate methods in pcl:recognition::HypothesisVerification).

As for the tute (which, remember, is only a tute!), the key params are the support size for the descriptor and the bin size for the grouping algorithm. You might find that choosing different values will turn out in quite different results on different kind of data.

Hope this helps,
Federico
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
Hi Federico,

Thank you for the information provided. That was very helpful to get my doubts cleared.
-Ramsri.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

schubbube
In reply to this post by fedassa
Hey Federico,

that would be great if you could provide us with more data about: how to change the parameters.
I'm facing a similar problem here with a model that I try to fit in a scene. Tried also already the template matching tutorial, but also there I don't come to the final result. It stops rotation on it's way.

I'll attach my files, so maybe you can also help me out with some parameters, I'm trying here since a week but couldn't figure out the right combination.

test-figure.pcd
SRcloud2.pcd
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

fedassa
Hi schubbube

I had a quick test with your data and i notice a few problems: first of all the point density between the two clouds is highly different (denser on the scene). You should first try to downsample/upsample the two clouds or try using a voxelGrid in order to normalize the two densities. then you should make sure the model is at the same scale as its instance in the scene, cause this tutorial does not handle scale variations. Finally, the shape of the model you are using seems not to be particularly descriptive, especially with respect to the background of the scene which locally reports a similar geometry, so you might tend to get a (slightly) inaccurate pose when deploying local descriptors for recognition. In any case, you might try to follow these suggestions to see if your results improve...

best
Federico
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

schubbube
This post was updated on .
Hey Federico,

thanks a lot for your reply. Yes I was thinking something similar that the amount of points is too different in the 2 scans. However, I found a solution by combining the NDT-tutorial with your Correspondence Grouping and in the end I get a really good solution.

Thanks,
Tobias
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
In reply to this post by fedassa
Hi Federico,

I am using two "ply" files , one contains a complete 360 degree scan of an object and the other contains a scene with the same object in it.

I have modified to load ply files and tried to run this tutorial but experienced problems.

If I run I get the following -

If I run with "-r" enabled i get pcl Shot estimation the local reference frame is not valid and it crashes.

I am attaching my source file , and the two "ply" files . Please let me know what modifications are needed to get it working with the given ply files.
correspondence_grouping.cpp

coffee.ply

coffee_scene.ply

Thanks,
Ramsri
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

gioia
Hi,

first of all, the writings that the PLYReader gives you as output are not a problem. Indeed, I think they should be "warning" and not "errors". Furthermore, as you can see, the tutorial correctly loads all the points contained in your files. I've quickly tested the tutorial on the files you've provided and:
- In the tutorial there is not the option "-r" so I don't know what you were doing.
- It could happen that the local reference frame is not valid for few points. As long as this happens only for few points is okay.
- In order to achieve results from the tutorial by using your personal files, you should always play with the algorithm parameters. In this case, I found good the following parameters:

//Algorithm params
bool show_keypoints_ (false);
bool show_correspondences_ (false);
bool use_hough_ (true);
float model_ss_ (0.005f);
float scene_ss_ (0.01f);
float rf_rad_ (0.015f);
float descr_rad_ (0.02f);
float cg_size_ (0.015f);
float cg_thresh_ (5.0f);

If you don't use the Hough algorithm you should try to play again with the parameters.

Cheers,
Gioia
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
Hi Gioia,

Thank you very much for the reply. Sorry, I thought ply files caused  trouble with out much further thought.
I was using "-r" to compute model cloud resolution.
The parameters given by you work very fine.
I would be glad if you can let me know how each of  rf_rad_ , descr_rad_  , cg_size_ , cg_thresh_  affect the final output in general case.

Thanks,
Ramsri.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

fedassa
Ramsri,

here is a brief description of the meaning of those parameters:

descr_rad_  : the radius that defines the support on which the descriptor is computed (i.e. the subset of keypoint neighbors being described). It should be enough large to include at least a few tens of points, but not too large not to include clutter in the description of keypoints close to the object border. Usually a value between 10-20 cloud resolution does the job.
rf_rad_: the radius that defines the support on which to compute the local Reference Frame associated to each keypoint. Should be similar (as per order of magnitude) to the descriptor support

cg_size_ : Grouping parameter, that defines the spatial length of each Hough bin. Should be enough large to encompass oscillations due to noise of the 3D Hough votes but not too large to create spurious peaks in the Hough space. A few cloud resolution might be a reasonable value in most cases.
cv_thresh_ : The other grouping parameter, that thresholds the consensus (number of votes received) of each object hypothesis.

Please also refer to the tutorial and related pcl doc for info on these and other parameters.

Hope this helps
Fede
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

ramsrigouthamg
Hi Federico,

Thank you for the explanation. I shall go through the pcl doc for more info.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

zq07075335
In reply to this post by ramsrigouthamg
hi, I have the same problem, and set the params many times ,but cannot get a  good result.
someone can help me?
thanks in advance
glass_7.pcd
scene_1.pcd
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

kan2k
This post was updated on .
Hi,

I tested your data with my code, and I managed to find a good alignment.

I really really recommend to all you guys that you preprocess the pointcloud before using correspondence grouping.
Remove the planes, and extract the clusters and use correspondence grouping for each detected cluster.

Another point is, regarding to this tutorial http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6299166 the best search radius for the SHOT descriptors is 0.06m! And I made the same observation.

Another interesing point is, that it is not always the case, that the true correspondence of one keypoint is the keypoint with the nearest feature. In this case we have a cup, so nearly every keypoint has the same feature, since they are all on a cylindric surface, so the correspondences can get messed up.
What you should do, select the nearest N features as correspondences.
for (size_t i = 0; i < scene_descriptors->size (); ++i)
  {
    std::vector<int> neigh_indices (1);
    std::vector<float> neigh_sqr_dists (1);
    if (!pcl_isfinite (scene_descriptors->at (i).descriptor[0])) //skipping NaNs
    {
      continue;
    }
    int found_neighs = match_search.nearestKSearch (scene_descriptors->at (i), 1, neigh_indices, neigh_sqr_dists);
    if(found_neighs == 1 && neigh_sqr_dists[0] < 0.25f) //  add match only if the squared descriptor distance is less than 0.25 (SHOT descriptor distances are between 0 and 1 by design)
    {
      pcl::Correspondence corr (neigh_indices[0], static_cast<int> (i), neigh_sqr_dists[0]);
      model_scene_corrs->push_back (corr);
    }
  }

REPLACE WITH
for (size_t i = 0; i < scene_descriptors->size (); ++i)
  {
    std::vector<int> neigh_indices (knn);
    std::vector<float> neigh_sqr_dists (knn);
    if (!pcl_isfinite (scene_descriptors->at (i).descriptor[0])) //skipping NaNs
    {
      continue;
    }
    int found_neighs = match_search.nearestKSearch (scene_descriptors->at (i), knn, neigh_indices, neigh_sqr_dists);
    for(int k = 0; k < found_neighs; k++)
    {
 if(found_neighs == 1 && neigh_sqr_dists[k] < 0.25f) //  add match only if the squared descriptor distance is less than 0.25 (SHOT descriptor distances are between 0 and 1 by design)
    {
            pcl::Correspondence corr (neigh_indices[k], static_cast<int> (i), neigh_sqr_dists[k]);
            corrs->push_back (corr);
        }
    }
  }
 A good value for knn is 10.

At least, if you somehow find good correspondences, and somehow manage to align them, but the alignment is not so perfect, than use ICP to refine the alignment.

Here is the alignment I found for your test data:


Cheers,
Serkan.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

zq07075335
In reply to this post by ramsrigouthamg
hi,Serkan
thank you for your help, I'll have a try and read that paper.
Best wishes.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

dhrupal93
This post has NOT been accepted by the mailing list yet.
In reply to this post by fedassa
cloud_cluster_0.pcdrobot1_downsampled.pcd

May you please help me too find the parameters to fit the kinect box in the robot1_downsampled.pdc with reference box (same box but extracted from some other file in same orientation)

Can u describe clear meaning of each parameters and how to find them?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

teddybouch
This post was updated on .
dhrupal93 - Federico provided a description of the parameters earlier in this thread. I've copied it here for convenience, but you'll find a lot of tips for tuning them if you read the whole thread. I'm in the same boat right now - new to correspondence grouping and trying to use a model produced with the in_hand_scanner app to find an object in a scene provided by a Kinect sensor. It seems to me that the bottom line is you need to work with the parameters to learn how to use them right now.

"descr_rad_  : the radius that defines the support on which the descriptor is computed (i.e. the subset of keypoint neighbors being described). It should be enough large to include at least a few tens of points, but not too large not to include clutter in the description of keypoints close to the object border. Usually a value between 10-20 cloud resolution does the job.
rf_rad_: the radius that defines the support on which to compute the local Reference Frame associated to each keypoint. Should be similar (as per order of magnitude) to the descriptor support

cg_size_ : Grouping parameter, that defines the spatial length of each Hough bin. Should be enough large to encompass oscillations due to noise of the 3D Hough votes but not too large to create spurious peaks in the Hough space. A few cloud resolution might be a reasonable value in most cases.
cv_thresh_ : The other grouping parameter, that thresholds the consensus (number of votes received) of each object hypothesis.

Please also refer to the tutorial and related pcl doc for info on these and other parameters."

That having been said, especially now that this library comes as a standard part of PCL 1.7, I imagine there will be a growing demand for people looking to understand the parameters and how to adjust them intelligently to find their objects and debug when it's not working. Tips such as those found in this thread - pre-processing methods for models that will help, alternative algorithms that are beneficial to different scenarios, and what types of features help the algorithm. If I can ever figure out these things myself, I'll gladly contribute to such a tutorial, but right now I think that I would only be a consumer of such a tool.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

laca89
hello,

I have a problem with this part.

pcl::UniformSampling<PointType> uniform_sampling;

// model part
uniform_sampling.setInputCloud (model_cloud); // here it exits
uniform_sampling.setRadiusSearch (model_ss_);
uniform_sampling.compute (sampled_indices);
pcl::copyPointCloud (*model_cloud, sampled_indices.points, *model_keypoints);

It exists suddenly. I tried with multiple input variables but nothing helps. What can be the problem? i;m running this code in a GUI, inside a "loop", where I do a lot of other things. Everything works, it just exits when I use the corr. grouping code.

Any help is really appreciated.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3d object recognition with Correspondence grouping -- help needed to recognize object with my provided pcd files

teddybouch
laca - If it is exiting when you try to set the input cloud, my hunch would be that you are passing it an invalid cloud, or a type that is different from what it expects. Do you have any error handling on your input cloud? You really need to see the conditions of the error in order to intelligently pick a path to go down. I tend to run my programs from the command line, so if a program exits abnormally it generally reports the error when it exits (e.g. segmentation fault). The information that you've provided really isn't sufficient to offer much help beyond this.
12
Loading...