Rectangular Prism RANSAC

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Rectangular Prism RANSAC

conormcmahon
Hi all,

I have an application where I'd like to be able to recognize within a cloud
a rectangular prism of known length/width/height but unknown location and
orientation. I was thinking a simple RANSAC approach would probably be
suitable and computationally manageable for this!

Example applications
- find the walls within a pointcloud captured in a rectangular room
- find a 'brick' or 'box' sitting on a table

The closest existing functionality to this I've been able to find is the
ExtractPolygonalPrismData class - but if I'm understanding this correctly I
think that's just used to find the points within an extrusion of a KNOWN
prism face? So I think it would not be useful for this case. Alternatively,
maybe the Objection Recognition library in PCL would be useful? I don't have
any experience using it yet.

This looked relevant but never got any answers:
http://www.pcl-users.org/RANSAC-to-detect-cubes-td3710525.html

I also read through this:
http://www.pcl-users.org/Defining-new-SACModel-td2645441.html#a2668870

I don't think the actual math of implementing a rectangular prism
segmentation routine would be too bad! But I wanted to check first to see if
there's any existing functionality to support this before I reinvent a
wheel, since this seems to me like such a basic and important use case.

I'm imaging defining the prism based on:
- 3 points to define one face of the prism (-Z for example)
- 1 point each on the OPPOSITE face (+Z) and two of the orthogonal adjoining
faces (+X, +Y)

If the length/width/height of the box were unknown this could still be done
with:
- 3 points to define one face of the prism (-Z for example)
- 1 point each remaining face (+X, -X, +Y, -Y, +Z)

The disadvantage with the latter case being that you'd need to have at least
one point included in the dataset on the every face! Which probably isn't
reasonable except for the special case where you're inside the box (finding
the walls case), or you have multiple sensors aimed at it from multiple
sides.

Thanks for your input!



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Rectangular Prism RANSAC

Sérgio Agostinho


On 21/02/2018 16:57, conormcmahon wrote:
>
> The closest existing functionality to this I've been able to find is the
> ExtractPolygonalPrismData class - but if I'm understanding this correctly I
> think that's just used to find the points within an extrusion of a KNOWN
> prism face? So I think it would not be useful for this case. Alternatively,
> maybe the Objection Recognition library in PCL would be useful? I don't have
> any experience using it yet.

This is more a thing for the SAC module as you've identified.

>
> This looked relevant but never got any answers:
> http://www.pcl-users.org/RANSAC-to-detect-cubes-td3710525.html
>
> I also read through this:
> http://www.pcl-users.org/Defining-new-SACModel-td2645441.html#a2668870
>
> I don't think the actual math of implementing a rectangular prism
> segmentation routine would be too bad! But I wanted to check first to see if
> there's any existing functionality to support this before I reinvent a
> wheel, since this seems to me like such a basic and important use case.
I don't believe there is anything of the sort in the library.

> I'm imaging defining the prism based on:
> - 3 points to define one face of the prism (-Z for example)
> - 1 point each on the OPPOSITE face (+Z) and two of the orthogonal adjoining
> faces (+X, +Y)
>
> If the length/width/height of the box were unknown this could still be done
> with:
> - 3 points to define one face of the prism (-Z for example)
> - 1 point each remaining face (+X, -X, +Y, -Y, +Z)
>
> The disadvantage with the latter case being that you'd need to have at least
> one point included in the dataset on the every face! Which probably isn't
> reasonable except for the special case where you're inside the box (finding
> the walls case), or you have multiple sensors aimed at it from multiple
> sides.
I have the impression you have more restrictions that it is actually
necessary.
- First case: 3x3 + 3 + 3 + 3 = 18D vector
- Second case : 3x3 + 5x3 = 24D vector

Assume an axis oriented cuboid and that one of the points is at the
origin, so no need to store it. To fully define your cuboid you just
need the position of the opposite point.

How to make this generic to any transformation? Encode the
transformation as well.

Any 3D transformation can be encoded in a 6D vector. Adding up that
extra point from the opposite corner, the minimum you need for your
cuboid is a 9D vector.

I believe we should move this a conversation to
[hidden email] or to the issue tracker.

Cheers



_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Rectangular Prism RANSAC

Fabien Rozar
I do not know if there is an issue or a new thread in pcl-developers regarding
this  feature, but I let a message here just to an eye over it and because I'm 
highly interested :)

frozar

2018-02-24 12:26 GMT+01:00 Sérgio Agostinho <[hidden email]>:


On 21/02/2018 16:57, conormcmahon wrote:
>
> The closest existing functionality to this I've been able to find is the
> ExtractPolygonalPrismData class - but if I'm understanding this correctly I
> think that's just used to find the points within an extrusion of a KNOWN
> prism face? So I think it would not be useful for this case. Alternatively,
> maybe the Objection Recognition library in PCL would be useful? I don't have
> any experience using it yet.

This is more a thing for the SAC module as you've identified.

>
> This looked relevant but never got any answers:
> http://www.pcl-users.org/RANSAC-to-detect-cubes-td3710525.html
>
> I also read through this:
> http://www.pcl-users.org/Defining-new-SACModel-td2645441.html#a2668870
>
> I don't think the actual math of implementing a rectangular prism
> segmentation routine would be too bad! But I wanted to check first to see if
> there's any existing functionality to support this before I reinvent a
> wheel, since this seems to me like such a basic and important use case.
I don't believe there is anything of the sort in the library.
> I'm imaging defining the prism based on:
> - 3 points to define one face of the prism (-Z for example)
> - 1 point each on the OPPOSITE face (+Z) and two of the orthogonal adjoining
> faces (+X, +Y)
>
> If the length/width/height of the box were unknown this could still be done
> with:
> - 3 points to define one face of the prism (-Z for example)
> - 1 point each remaining face (+X, -X, +Y, -Y, +Z)
>
> The disadvantage with the latter case being that you'd need to have at least
> one point included in the dataset on the every face! Which probably isn't
> reasonable except for the special case where you're inside the box (finding
> the walls case), or you have multiple sensors aimed at it from multiple
> sides.
I have the impression you have more restrictions that it is actually
necessary.
- First case: 3x3 + 3 + 3 + 3 = 18D vector
- Second case : 3x3 + 5x3 = 24D vector

Assume an axis oriented cuboid and that one of the points is at the
origin, so no need to store it. To fully define your cuboid you just
need the position of the opposite point.

How to make this generic to any transformation? Encode the
transformation as well.

Any 3D transformation can be encoded in a 6D vector. Adding up that
extra point from the opposite corner, the minimum you need for your
cuboid is a 9D vector.

I believe we should move this a conversation to
[hidden email] or to the issue tracker.

Cheers



_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users



_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Rectangular Prism RANSAC

conormcmahon
In reply to this post by Sérgio Agostinho
Thanks for your response Sergio! Sorry for the delay in getting back to you.

I'm completely new to this forum - is the issue tracker something on a PCL
Github somewhere, or something within this interface?

I agree for sure that the system I outlined contains more dimensionality
than is necessary. The cuboid can definitely be STORED in a 9D vector.
However I was trying to talk about the most elegant way to specify a model
during RANSAC from a minimum selection of points in the dataset.

The cuboid could be minimally specified using three points, each at a
different corner. I don't really like this method though because it requires
that the corners of the cuboid all be contained within the cloud - if for
example the bottom part of the cuboid is occluded by something else in the
scene it wouldn't be able to find it. I think a face-based approach should
be robust to that problem, and has the extra advantage of being able to find
even cuboids without sharply defined corners. The particular application I'm
interested in would probably not work well with a corner-based approach.

I haven't actually started doing anything on this yet, but I'm hoping to get
around to actually implementing this in the nearish future.



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users
Reply | Threaded
Open this post in threaded view
|

Re: Rectangular Prism RANSAC

Sérgio Agostinho


On 09-03-2018 17:47, conormcmahon wrote:
> Thanks for your response Sergio! Sorry for the delay in getting back to you.
>
> I'm completely new to this forum - is the issue tracker something on a PCL
> Github somewhere, or something within this interface?

We have a GitHub repo.

> I agree for sure that the system I outlined contains more dimensionality
> than is necessary. The cuboid can definitely be STORED in a 9D vector.
> However I was trying to talk about the most elegant way to specify a model
> during RANSAC from a minimum selection of points in the dataset.
>
> The cuboid could be minimally specified using three points, each at a
> different corner. I don't really like this method though because it requires
> that the corners of the cuboid all be contained within the cloud - if for
> example the bottom part of the cuboid is occluded by something else in the
> scene it wouldn't be able to find it. I think a face-based approach should
> be robust to that problem, and has the extra advantage of being able to find
> even cuboids without sharply defined corners. The particular application I'm
> interested in would probably not work well with a corner-based approach.
You focused too much on the word 'point'  in my previous answer. Let me
rephrase: cuboid is defined by the 6D transformation and size x, y and z
(it's equivalent to what I said before). From this you can extract all
the 6 plane equations.


Cheers



_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Rectangular Prism RANSAC

conormcmahon
Hi Sergio,

Just wanted to update you on this - I cloned from the PCL github repo last
week and think I have mostly finished hacking together a system for the
Cuboid RANSAC we talked about a bit ago on the PCL Listserv. Depending on
the needs of other projects, I'll hopefully be working on testing it this
week and I'll let you know if I get it to a place where I think it could
conceivably be looked at for inclusion with the public repo.

I had a potentially silly question about the implementation of some of the
existing RANSAC methods. Most of them seem to rely on expressing points as
Eigen::Vector4f structures - is this just to allow them to be operated on
with homogeneous transforms, or is this a data storage efficiency thing? I'm
not very knowledgeable about the latter. If I don't need to do any
homogeneous transforms is there any benefit to keeping my internal point
handling as Vector4f instead of just Vector3f?

Thanks,
Conor



--
Sent from: http://www.pcl-users.org/
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users