Global SHOT descriptor

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Global SHOT descriptor

Andreas786
Hi!

I'm experimenting with the global descriptors in PCL for CAD object
recognition with an Intel RealSense depth sensor. As a simple initial
approach I'm trying to use the SHOT descriptor as a global descriptor by
computing a single SHOT descriptor at the center of mass and expanding the
search radius to include the hole object. My code is as follows:

        pcl::PointCloud<pcl::PointXYZ>::Ptr scene_cloud (new
pcl::PointCloud<pcl::PointXYZ>);
        pcl::PointXYZ center_of_mass;

       ...

        // Compute normals for scene
        pcl::NormalEstimationOMP<pcl::PointXYZ, pcl::Normal> normal_estimator;
        normal_estimator.setKSearch (k_search_normals);
        pcl::PointCloud<pcl::Normal>::Ptr normals (new
pcl::PointCloud<pcl::Normal>);
        normal_estimator.setInputCloud (scene_cloud);
        normal_estimator.compute (*normals);
               
        // Compute global SHOT descriptor for scene
        pcl::PointCloud<pcl::PointXYZ>::Ptr input_cloud (new
pcl::PointCloud<pcl::PointXYZ>);
        input_cloud->push_back (center_of_mass);
        pcl::search::KdTree<pcl::PointXYZ>::Ptr tree (new
pcl::search::KdTree<pcl::PointXYZ> ());
        pcl::SHOTEstimationOMP<pcl::PointXYZ, pcl::Normal, pcl::SHOT352>
feature_estimator;
        feature_estimator.setSearchMethod (tree);
        pcl::PointCloud<pcl::SHOT352>::Ptr feature (new
pcl::PointCloud<pcl::SHOT352>);

        feature_estimator.setSearchSurface (scene_cloud);
        feature_estimator.setInputCloud (input_cloud);
        feature_estimator.setInputNormals (normals);
        feature_estimator.setRadiusSearch (0.1);
        feature_estimator.compute (*feature);

where scene_cloud is the segmented object and center_of_mass is the center
of mass for the segmented object. I create a "dummy" point cloud as the
input cloud which just contains the center of mass point. When doing
descriptor matching with my CAD database this gave descent results.

My first question is why this works? The input cloud contains the center of
mass which is a PointXYZ structure that does not contain any normal. So why
is the feature estimator able to compute a SHOT descriptor when the input
cloud (center of mass) doesn't have any normal? Would it be better to add
center_of_mass to scene_cloud and the viewing direction to normals (assuming
that the viewing direction is the normal of the center of mass point)?

My second question is whether the function "computePointSHOT" is more
suitable for my approach?



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Global SHOT descriptor

vs-tec
Hi Andreas786,

as far as I can see, you actually are using normals.
The input point cloud is just the cloud that has to contain the
key points where the descriptors should be estimated at - no need for
normals here.
However, the method "setInputNormals" takes the normals that correspond
to the points passed in to "setSearchSurface".
Taking a look at your normal estimation: you are passing scene_cloud to
estimate normals, then later you are passing the scene_cloud as
searchSurface and the computed normals as normals.

So all good - everything is correct and you are using the computed normals.

Your first question:

It works because you are using normals. However, even if you DO NOT use
normals (e.g. by passing all normals as zero vectors) the descriptor
calculation works.
The SHOT descriptor is computed on a 3D grid with 32 cells. In each of
the cells a 11-bin histogram encodes the normal's angles. If you only
pass zero normals the 11-bin histogram will have 10 empty bins with 1
bin having all the "zero angles" between neighboring normals.

Your second question:

No, you are already doing it the right way (as far as my PCL
understanding goes). The "compute" function wraps the "computePointSHOT"
function and does some additional checks. They do no harm, nor do they
cost you any (significant) amount of runtime.

If you are interested in (global) descriptors, take a look at ESF, e.g.
from the very nice tutorial here:
http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#ESF

Best,
V

On 12.09.2017 14:49, Andreas786 wrote:
Hi!

I'm experimenting with the global descriptors in PCL for CAD object
recognition with an Intel RealSense depth sensor. As a simple initial
approach I'm trying to use the SHOT descriptor as a global descriptor by
computing a single SHOT descriptor at the center of mass and expanding the
search radius to include the hole object. My code is as follows:

	pcl::PointCloud<pcl::PointXYZ>::Ptr scene_cloud (new
pcl::PointCloud<pcl::PointXYZ>);
        pcl::PointXYZ center_of_mass;

       ...

        // Compute normals for scene
	pcl::NormalEstimationOMP<pcl::PointXYZ, pcl::Normal> normal_estimator;
	normal_estimator.setKSearch (k_search_normals);
	pcl::PointCloud<pcl::Normal>::Ptr normals (new
pcl::PointCloud<pcl::Normal>);
	normal_estimator.setInputCloud (scene_cloud);
	normal_estimator.compute (*normals);
		
	// Compute global SHOT descriptor for scene
	pcl::PointCloud<pcl::PointXYZ>::Ptr input_cloud (new
pcl::PointCloud<pcl::PointXYZ>);
	input_cloud->push_back (center_of_mass);
	pcl::search::KdTree<pcl::PointXYZ>::Ptr tree (new
pcl::search::KdTree<pcl::PointXYZ> ());
	pcl::SHOTEstimationOMP<pcl::PointXYZ, pcl::Normal, pcl::SHOT352>
feature_estimator;
	feature_estimator.setSearchMethod (tree);
	pcl::PointCloud<pcl::SHOT352>::Ptr feature (new
pcl::PointCloud<pcl::SHOT352>);

	feature_estimator.setSearchSurface (scene_cloud);
	feature_estimator.setInputCloud (input_cloud);
	feature_estimator.setInputNormals (normals);
	feature_estimator.setRadiusSearch (0.1);
	feature_estimator.compute (*feature); 

where scene_cloud is the segmented object and center_of_mass is the center
of mass for the segmented object. I create a "dummy" point cloud as the
input cloud which just contains the center of mass point. When doing
descriptor matching with my CAD database this gave descent results. 

My first question is why this works? The input cloud contains the center of
mass which is a PointXYZ structure that does not contain any normal. So why
is the feature estimator able to compute a SHOT descriptor when the input
cloud (center of mass) doesn't have any normal? Would it be better to add
center_of_mass to scene_cloud and the viewing direction to normals (assuming
that the viewing direction is the normal of the center of mass point)?

My second question is whether the function "computePointSHOT" is more
suitable for my approach?



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

-- 
___________________________________________________________
Senior Developer

Tau Epsilon C GmbH & Co. KG
Viktor Seib
Universitätsstraße 3
56070 Koblenz

fon: 	+49 261 45093140
fax: 	+49 261 45093149

mail:   [hidden email]
web:    www.tau-epsilon-c.com

Geschäftsführer: Andreas Rieser

Rechtsform GmbH & Co. KG - Sitz: Koblenz
Registergericht: Amtsgericht Koblenz HRA 21384

Persönlich haftende Gesellschafterin: Tau Epsilon C Verwaltungs GmbH
Rechtsform GmbH - Sitz: Koblenz
Registergericht: Amtsgericht Koblenz HRB 24137

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) please notify
the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or
distribution of the material in this e-mail is strictly forbidden.

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Global SHOT descriptor

Andreas786
Hi Viktor!

Thank you for your answer! This was exactly my reasoning as well when I
first wrote the program. However, when I went back to check the code I
noticed that the input point cloud (which is the center of mass point) does
not exist in the cloud in setSearchSurface (scene_cloud). The same reasoning
goes for the normals where the center of mass normal doesn't exist in
normals. My understanding is that the keypoints and their normals are also
supposed to be present in setSearchSurface and setInputNormals. In my
example the "keypoint" does not exist in setSearchSurface nor in
setInputNormals. When the SHOT descriptor is computed it determines the
angle between the normal points in each cell and the angle between the
feature normal. So how does this work if I haven't set the feature angle
(i.e. the angle of center of mass)?

Thank you also for your recommendation. I will definitely have a look at the
ESF global descriptor and its performance.

Best,
Andreas



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Global SHOT descriptor

Viktor Seib
Hi Andreas,

ok, now I understand what you mean. I just took a look into the PCL code
(features/.../impl/shot.hpp)

The query point does not need to have a normal, only its neighbors need
normals.

In the code the angle (actually the dot product) is computed between the
normals of the neighbors and the query point's z-axis of the reference
frame.

Best,

V



On 13.09.2017 11:00, Andreas786 wrote:

> Hi Viktor!
>
> Thank you for your answer! This was exactly my reasoning as well when I
> first wrote the program. However, when I went back to check the code I
> noticed that the input point cloud (which is the center of mass point) does
> not exist in the cloud in setSearchSurface (scene_cloud). The same reasoning
> goes for the normals where the center of mass normal doesn't exist in
> normals. My understanding is that the keypoints and their normals are also
> supposed to be present in setSearchSurface and setInputNormals. In my
> example the "keypoint" does not exist in setSearchSurface nor in
> setInputNormals. When the SHOT descriptor is computed it determines the
> angle between the normal points in each cell and the angle between the
> feature normal. So how does this work if I haven't set the feature angle
> (i.e. the angle of center of mass)?
>
> Thank you also for your recommendation. I will definitely have a look at the
> ESF global descriptor and its performance.
>
> Best,
> Andreas
>
>
>
> --
> Sent from: http://www.pcl-users.org/
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Global SHOT descriptor

Andreas786
Hi Viktor!

Ok, that makes more sense. Thank you again for your reply!

Best,
Andreas



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users