VoxelGridFilter: Leaf size is too small

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

VoxelGridFilter: Leaf size is too small

Robert Huitl
Hi,

I'm having trouble with the voxel grid filter:

"Leaf size is too small for the input dataset. Integer indices would overflow"

 I apply it to the scan of a building, which extends a few hundred meters in x
and y direction. When I set the resolution of the voxel grid filter to ~ 10 cm,
the program aborts with the error mentioned above.

Has anyone run into this problem and found a solution?

I tried making the indices larger (64 bit), but there are a lot of other
variables that have to be changed to wider types, too. I'm not sure if this is
the way to go, because as far as I understood, there's a vector storing an
item for each cell (leaf_layout_). In my case where the 32 bit indices are not
enough, this means that there would be more than 4 billion entries in that
vector...

Any ideas? Maybe split the point cloud into smaller cubes (properly aligned)
and filter them one by one? Or maybe an Octree using the
pcl::octree::OctreePointCloudVoxelCentroidContainer?

Cheers,
  Robert

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Radu B. Rusu
Administrator
Robert,

I confirm that some of the latest changes in trunk have made VoxelGrid unstable. We need to carefully review the code
and make sure that it is correct for all cases. Please add your comments to the existing VoxelGrid issue on the tracker.
Thanks.

Cheers,
Radu.
--
http://openperception.org

On 01/21/2013 08:43 AM, Robert Huitl wrote:

> Hi,
>
> I'm having trouble with the voxel grid filter:
>
> "Leaf size is too small for the input dataset. Integer indices would overflow"
>
>   I apply it to the scan of a building, which extends a few hundred meters in x
> and y direction. When I set the resolution of the voxel grid filter to ~ 10 cm,
> the program aborts with the error mentioned above.
>
> Has anyone run into this problem and found a solution?
>
> I tried making the indices larger (64 bit), but there are a lot of other
> variables that have to be changed to wider types, too. I'm not sure if this is
> the way to go, because as far as I understood, there's a vector storing an
> item for each cell (leaf_layout_). In my case where the 32 bit indices are not
> enough, this means that there would be more than 4 billion entries in that
> vector...
>
> Any ideas? Maybe split the point cloud into smaller cubes (properly aligned)
> and filter them one by one? Or maybe an Octree using the
> pcl::octree::OctreePointCloudVoxelCentroidContainer?
>
> Cheers,
>    Robert
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
>
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Julius Kammerl-2
In reply to this post by Robert Huitl
Hi Robert,

On 01/21/2013 08:43 AM, Robert Huitl wrote:

> Hi,
>
> I'm having trouble with the voxel grid filter:
>
> "Leaf size is too small for the input dataset. Integer indices would overflow"
>
>  I apply it to the scan of a building, which extends a few hundred meters in x
> and y direction. When I set the resolution of the voxel grid filter to ~ 10 cm,
> the program aborts with the error mentioned above.
>
> Has anyone run into this problem and found a solution?
>
> I tried making the indices larger (64 bit), but there are a lot of other
> variables that have to be changed to wider types, too. I'm not sure if this is
> the way to go, because as far as I understood, there's a vector storing an
> item for each cell (leaf_layout_). In my case where the 32 bit indices are not
> enough, this means that there would be more than 4 billion entries in that
> vector...
>
> Any ideas? Maybe split the point cloud into smaller cubes (properly aligned)
> and filter them one by one? Or maybe an Octree using the
> pcl::octree::OctreePointCloudVoxelCentroidContainer?
>
Probably, there is a way to patch the voxel grid filter to work with
larger datasets. However, as long as your point data is not completely
randomly distributed, an octree would organize the point set in a much
more efficiently way leading to reduced memory usage.

Here is an example of an octree container class that can be used for
calculating the centroid of all points within the voxel.

    class OctreeCentroidContainer : public
pcl::octree::OctreeContainerBase<int>
    {
      public:
        /** \brief Class initialization. */
      OctreeCentroidContainer ()
        {
          this->reset();
        }

        /** \brief Empty class deconstructor. */
        virtual ~OctreeCentroidContainer ()
        {
        }

        /** \brief deep copy function */
        virtual OctreeCentroidContainer *
        deepCopy () const
        {
          return (new OctreeCentroidContainer (*this));
        }

        /** \brief Add new point to voxel.
          * \param[in] new_point the new point to add
          */
        void
        addPoint (const pcl::PointXYZRGB& new_point, unsigned int
time_stamp)
        {
          ++point_counter_;

          x_sum_ += new_point.x;
          y_sum_ += new_point.y;
          z_sum_ += new_point.z;

          color_r_ += new_point.r;
          color_g_ += new_point.g;
          color_b_ += new_point.b;

          time_stamp_ = time_stamp;
        }

        /** \brief Calculate centroid of voxel.
          * \param[out] centroid_arg the resultant centroid of the voxel
          */
        void
        getCentroid (pcl::PointXYZRGB& centroid_arg) const
        {
          if (point_counter_)
          {
            float fc = static_cast<float> (point_counter_);
            centroid_arg.x = x_sum_ / fc;
            centroid_arg.y = y_sum_ / fc;
            centroid_arg.z = z_sum_ / fc;

            centroid_arg.r = color_r_ / fc;
            centroid_arg.g = color_g_ / fc;
            centroid_arg.b = color_b_ / fc;
          }
        }

        /** \brief Calculate centroid of voxel.
          * \param[out] centroid_arg the resultant centroid of the voxel
          */
        unsigned int
        getTimeStamp () const
        {
          return time_stamp_;
        }

        /** \brief Reset leaf container. */
        virtual void
        reset ()
        {
          point_counter_ = 0;

          color_r_ = 0.0;
          color_g_ = 0.0;
          color_b_ = 0.0;

          x_sum_ = y_sum_ = z_sum_ = 0.0;

          time_stamp_ = 0;

        }

      private:
        unsigned int point_counter_;

        unsigned int color_r_;
        unsigned int color_g_;
        unsigned int color_b_;

        float x_sum_;
        float y_sum_;
        float z_sum_;

        unsigned int time_stamp_;
    };

Please let me know if you have further questions.


Cheers,
Julius


_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

signature.asc (917 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

y666
In reply to this post by Robert Huitl
Split point cloud into smaller ones and voxel grid them one by one.
Or make a patch, that changes internal integer used by VoxelGrid from 32 bit to 64.
The latter will hurt performance somewhat due to larger memory footprint.

----------------------------------

Białym śniegiem zimą
Górą i doliną
Mrozem i kolędą
Tym dla ciebie będę
Będę ciepłem lata
Lepszą stroną świata
Będę kiedy jesień
Babie lato niesie
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Robert Huitl
In reply to this post by Julius Kammerl-2
Hi Julius,

> Probably, there is a way to patch the voxel grid filter to work with
> larger datasets. However, as long as your point data is not completely
> randomly distributed, an octree would organize the point set in a much
> more efficiently way leading to reduced memory usage.

Thanks for the suggestion! I took your code and made a drop-in replacement for
VoxelGrid (see attached file) for the time being. It looks like it works fine
and it has no problems with the dataset size. Although I didn't do benchmarks,
it didn't seem to be much slower.

Cheers,
  Robert

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

OctreeVoxelGridFilter.h (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Robert Huitl
In reply to this post by y666
Hi Radoslaw,

> Split point cloud into smaller ones and voxel grid them one by one.

I thought about that, an adapter that does this kind of splitting  and result
merging for anything derived from Filter could be useful in other scenarios,
too.

> Or make a patch, that changes internal integer used by VoxelGrid from 32 bit
> to 64. The latter will hurt performance somewhat due to larger memory
> footprint.

I tried that but ended up changing many other variables that are computed from
the original int32 keys, until I realized that there's a vector with size
dx*dy*dz . For huge datasets this would quickly eat up all the memory. On the
other hand it would work if the vector was replaced by a map, as discussed in
http://dev.pointclouds.org/issues/780, but that hurts performance, too.

Cheers,
  Robert

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

IrinaF
In reply to this post by Radu B. Rusu
Dear Radu,

I have exactly the same error when implementing Pairwise incremental registration   with two datasets 200.000 points each. I've installed PCL recently.  Is it possible to find a solution without making a lot of changes in the code?

Best regards,

Irina
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Radu B. Rusu
Administrator
Irina,

Unfortunately VoxelGrid has been modified several times since I first introduced the class, so I'm not sure what's going on. All unit tests seemed to have been passing, so maybe this is a corner case? Someone should try an older version in the repository and see where it stopped working. Alternatively, you can downsample using an octree.

Best,
Radu.

On Sep 16, 2013, at 9:19 AM, IrinaF <[hidden email]> wrote:

> Dear Radu,
>
> I have exactly the same error when implementing Pairwise incremental
> registration   with two datasets 200.000 points each. I've installed PCL
> recently.  Is it possible to find a solution without making a lot of changes
> in the code?
>
> Best regards,
>
> Irina
>
>
>
> --
> View this message in context: http://www.pcl-users.org/VoxelGridFilter-Leaf-size-is-too-small-tp4025570p4029695.html
> Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

IrinaF
This post was updated on .
Radu,

I tried to use OctreeVoxelGridFilter as was explained before. I just put the OctreeVoxelGridFilter.h file in the program directory and used OctreeVoxelGrid instead of VoxelGrid in the 209th line of the pairwise_incremental_registration code  (209 pcl::VoxelGrid<PointT>grid;). I have the following mistake when compiling:
<



[100%] Building CXX object CMakeFiles/pairwise_incremental_registration.dir/pairwise_incremental_registration.cpp.o
In file included from /home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:48:0:
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:72: error: expected template-name before ‘<’ token
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:72: error: expected ‘{’ before ‘<’ token
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:72: error: expected unqualified-id before ‘<’ token
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:67:91: error: invalid use of incomplete type ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:7: error: declaration of ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:81:93: error: invalid use of incomplete type ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:7: error: declaration of ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:96:55: error: invalid use of incomplete type ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:7:7: error: declaration of ‘class OctreeCentroidContainer<pcl::PointNormal>’
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp: In function ‘void pairAlign(pcl::PointCloud<pcl::PointXYZ>::Ptr, pcl::PointCloud<pcl::PointXYZ>::Ptr, pcl::PointCloud<pcl::PointXYZ>::Ptr, Eigen::Matrix4f&, bool)’:
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:209:3: error: ‘OctreeVoxelGrid’ is not a member of ‘pcl’
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:209:3: note: suggested alternative:
In file included from /home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:48:0:
/home/fateeva/pcl_projects/OctreeVoxelGridFilter.h:105:7: note:   ‘OctreeVoxelGrid’
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:209:30: error: expected primary-expression before ‘>’ token
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:209:32: error: ‘grid’ was not declared in this scope
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:260:45: warning: ‘void pcl::Registration<PointSource, PointTarget, Scalar>::setInputCloud(const PointCloudSourceConstPtr&) [with PointSource = pcl::PointNormal; PointTarget = pcl::PointNormal; Scalar = float; pcl::Registration<PointSource, PointTarget, Scalar>::PointCloudSourceConstPtr = boost::shared_ptr<const pcl::PointCloud<pcl::PointNormal> >]’ is deprecated (declared at /usr/local/include/pcl-1.7/pcl/registration/impl/registration.hpp:43): [pcl::registration::Registration::setInputCloud] setInputCloud is deprecated. Please use setInputSource instead. [-Wdeprecated-declarations]
/home/fateeva/pcl_projects/pairwise_incremental_registration.cpp:278:47: warning: ‘void pcl::Registration<PointSource, PointTarget, Scalar>::setInputCloud(const PointCloudSourceConstPtr&) [with PointSource = pcl::PointNormal; PointTarget = pcl::PointNormal; Scalar = float; pcl::Registration<PointSource, PointTarget, Scalar>::PointCloudSourceConstPtr = boost::shared_ptr<const pcl::PointCloud<pcl::PointNormal> >]’ is deprecated (declared at /usr/local/include/pcl-1.7/pcl/registration/impl/registration.hpp:43): [pcl::registration::Registration::setInputCloud] setInputCloud is deprecated. Please use setInputSource instead. [-Wdeprecated-declarations]
make[2]: *** [CMakeFiles/pairwise_incremental_registration.dir/pairwise_incremental_registration.cpp.o] Error 1
make[1]: *** [CMakeFiles/pairwise_incremental_registration.dir/all] Error 2
make: *** [all] Error 2





I tried to use just OctreeVoxelGrid instead of pcl::OctreeVoxelGrid, it does not help...What could be a problem? Or maybe newer code exists?

thank you,

Irina

OctreeVoxelGridFilter.h


Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

IrinaF
In reply to this post by Radu B. Rusu
Maybe there are some other options to use octree for downsampling? It's really important for me to implement this algorithm...

Regards,

Irina
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Rewtrash
In reply to this post by Robert Huitl
Hi Robert,

I am pretty new to PCL and I'm currently trying to downsize a pointcloud using the voxel grid tutorial, but got the stated error.


May I know how to use your 'OctreeVoxelGridFilter.h' file in the tutorial or how is it supposed to work?

Sorry for such a newbie question..
Hope you can help me

Regards,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

VictorLamoine
Administrator
Here is a code example :

#include <pcl/filters/voxel_grid.h>

// start main function etc..

PointCloudT::ConstPtr &cloud_in (new PointCloudT);
PointCloudT::Ptr &cloud_out (new PointCloudT);

// Put some points in cloud_in

float leaf_size = 0.05f;
pcl::VoxelGrid<PointT> vg;
vg.setInputCloud (cloud_in);
vg.setLeafSize (leaf_size, leaf_size, leaf_size);
vg.filter (*cloud_out);

What was your leafsize parameter ?
Reply | Threaded
Open this post in threaded view
|

Re: VoxelGridFilter: Leaf size is too small

Rewtrash
Hi Victor,

That was exactly what I had in my codes. I was trying leafsize 0.01f to 0.1f and it gave me the 'Leaf size is too small for the input dataset' error.

However after changing the leafsize to 0.5f, and 1.0f, both worked and gave the same number of data points as the input. Increasing the leafsize > than 1.0f will result in the desired function.

My Input is a 3D scan of a part of a desk of about 3m using a kinect.
What does the leafsize value actually represent?
The tutorial states that 0.01f is 1cm, so if I'm using 0.5f to obtain the same value as my input, does it mean that my points are 50cm apart? Am I missing something?

Regards,
Shawn