Point Cloud Library (PCL) Users mailing list

Running PCL on a tablet? To be used on set of a feature film.

Classic

List

Threaded

11 messages Options

48Letters

Running PCL on a tablet? To be used on set of a feature film.

Hello everyone! I can't go into details, but I am trying to put together a handheld 3d scanning unit to be used on set of a major feature film. I am building the unit using a Xtion Pro Live, some lights, and a tablet of some sort. How much power do I need just to be able to record the data streams from the Xtion? Then I will be taking the data and crunching the numbers on a more robust computer. It would be excellent to be able to view a simple point cloud on the tablet when capturing.

Does anyone here have experience running PCL on a tablet? I assume the tablet has to run windows 7 or some form of linux? I was thinking this could be a good option:
http://www.amazon.com/Acer-Iconia-W500-BZ467-10-1-Inch-Tablet/dp/B004SBI2PW/ref=sr_1_1?s=electronics&ie=UTF8&qid=1340659127&sr=1-1&keywords=windows+7+tablet

48Letters

Re: Running PCL on a tablet? To be used on set of a feature film.

https://www.gumstix.com/store/product_info.php?products_id=253

Do you guys think this computer would be fast enough to record the data stream from the Xtion? It would be excellent if we could run the unit off of something so small.

martyvona

Re: Running PCL on a tablet? To be used on set of a feature film.

We spent a while last summer & fall working with Kinect on Pandaboard, which has a very similar armel architecture to the gumstix you are looking at. We did eventually get both openni and the libfreenect drivers to work. Despite our best efforts openni was very slow, only a few fps even just to display live VGA depth. We are getting more like 10-15fps with libfreenect. Maybe things have improved since then.

About 6 months ago we even packaged libfreenect for ubuntu 11.04 armel, you can get the patch and debs here

http://code.google.com/p/ohmm-sw/source/browse/#svn%2Ftrunk%2Fhlp%2Fext%2Ffreenect

The patch is pretty minimal, it fixes a few things particular to our use but not essential, and one compilation bug that was present at that time on armel; I believe that has since been fixed in libfreenect trunk so you might just try that first.

We did not experiment with saving the live data but I expect that is going to be a bit of a project. I would be a little surprised if you manage to sustain more than ~10fps to flash (these things use SD cards; you can attach USB storage but that is going to share the same USB bandwidth used by the sensor itself).

I just did a quick check using the libfreenect "record" utility with Kinect on Pandaboard running Ubuntu 11.04 and a good sandisk high speed SD card, it looks like it was managing to save about 5-10 FPS to flash *except* a good 3-8s hiccup every 5 seconds or so, probably as buffers are getting flushed to flash. This is with it trying to save both RGB and depth images uncompressed; maybe you can do better with compression or for depth only.

We won't have time to work on this further for the next few months but we may revisit it in the early fall and see if updating to ubuntu 12.04 (which newer kernels with more armel support afaik) and newer openni/libfreenect gives any improvements.

Marty Vona
http://ccis.neu.edu/research/gpc
http://ohmmbot.org

48Letters

Re: Running PCL on a tablet? To be used on set of a feature film.

Thanks for the info! It looks like the fastest way to getting a working handheld unit would be to use a tablet.

http://www.amazon.com/ASUS-Slate-B121-A1-12-1-Inch-Tablet/dp/B005HV3IMY

I assume that one would probably be fast enough to save a stream and maybe view a point cloud while capturing? As far as simplicity for installation do you think windows or linux would be a better option? It seems like linux may be a better performer, but I see a lot of posts on here about issues getting it to install correctly in the first place!

-= Robin

On Tuesday, June 26, 2012 at 3:08 PM, martyvona [via Point Cloud Library (PCL) Users] wrote:

We spent a while last summer & fall working with Kinect on Pandaboard, which has a very similar armel architecture to the gumstix you are looking at. We did eventually get both openni and the libfreenect drivers to work. Despite our best efforts openni was very slow, only a few fps even just to display live VGA depth. We are getting more like 10-15fps with libfreenect. Maybe things have improved since then.

About 6 months ago we even packaged libfreenect for ubuntu 11.04 armel, you can get the patch and debs here

http://code.google.com/p/ohmm-sw/source/browse/#svn%2Ftrunk%2Fhlp%2Fext%2Ffreenect

The patch is pretty minimal, it fixes a few things particular to our use but not essential, and one compilation bug that was present at that time on armel; I believe that has since been fixed in libfreenect trunk so you might just try that first.

We did not experiment with saving the live data but I expect that is going to be a bit of a project. I would be a little surprised if you manage to sustain more than ~10fps to flash (these things use SD cards; you can attach USB storage but that is going to share the same USB bandwidth used by the sensor itself).

I just did a quick check using the libfreenect "record" utility with Kinect on Pandaboard running Ubuntu 11.04 and a good sandisk high speed SD card, it looks like it was managing to save about 5-10 FPS to flash *except* a good 3-8s hiccup every 5 seconds or so, probably as buffers are getting flushed to flash. This is with it trying to save both RGB and depth images uncompressed; maybe you can do better with compression or for depth only.

We won't have time to work on this further for the next few months but we may revisit it in the early fall and see if updating to ubuntu 12.04 (which newer kernels with more armel support afaik) and newer openni/libfreenect gives any improvements.

Marty Vona
http://ccis.neu.edu/research/gpc
http://ohmmbot.org

If you reply to this email, your message will be added to the discussion below:
http://www.pcl-users.org/Running-PCL-on-a-tablet-To-be-used-on-set-of-a-feature-film-tp4019672p4019723.html

To unsubscribe from Running PCL on a tablet? To be used on set of a feature film., click here.
NAML

james

Re: Running PCL on a tablet? To be used on set of a feature film.

In reply to this post by 48Letters

Have you seen this? http://pointclouds.org/news/pcl-goes-mobile-with-ves-and-kiwi.html

martyvona

Re: Running PCL on a tablet? To be used on set of a feature film.

In reply to this post by 48Letters

That tablet does look promising, intel processor, SSD. I don't know exactly what performance you'll manage to get but it might be possible to approach full frame rate (30fps). Depends a lot on exactly what you want to do and your coding expertise. Getting high (or even reasonable) performance may require knowledge about how to optimally write to the SSD and display hardware on that particular platform.

I don't know regarding Windows vs Linux either. Normally I use Linux but with a new platform you need to verify device driver availability and stability.

Luck,
Marty

Pat Marion

Re: Running PCL on a tablet? To be used on set of a feature film.

Hi Robin,

You might also want to read Raymond Lo's blog posts here:

http://www.pointclouds.org/blog/nvcs/raymondlo84/index.php

Pat

On Wed, Jun 27, 2012 at 11:00 AM, martyvona <[hidden email]> wrote:

That tablet does look promising, intel processor, SSD. I don't know exactly
what performance you'll manage to get but it might be possible to approach
full frame rate (30fps). Depends a lot on exactly what you want to do and
your coding expertise. Getting high (or even reasonable) performance may
require knowledge about how to optimally write to the SSD and display
hardware on that particular platform.

I don't know regarding Windows vs Linux either. Normally I use Linux but
with a new platform you need to verify device driver availability and
stability.

Luck,
Marty

--
View this message in context: http://www.pcl-users.org/Running-PCL-on-a-tablet-To-be-used-on-set-of-a-feature-film-tp4019672p4019753.html

Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

48Letters

Re: Running PCL on a tablet? To be used on set of a feature film.

In reply to this post by martyvona

Yea considering our timeframe sticking with windows for now will probably be best. Do you think it would be hard to get to 30fps even if we were just saving the data and not processing it?

Patrick Mihelich-2

Re: Running PCL on a tablet? To be used on set of a feature film.

Actually running ROS on a tablet is probably too much trouble, but check out the advice at http://ros.org/wiki/openni_launch/Tutorials/BagRecordingPlayback.

For recording you can do pretty well recording the raw uint16 depth image and raw Bayer RGB image, totalling 3 bytes / pixel. For VGA depth+RGB at 30fps, that's 3*640*480*30 ~= 27 MiB/s, so your storage medium needs a max write speed of at least that (probably a bit more). An SD card won't cut it, but any SSD or hard disk drive should be just fine.

Compression (I've used PNG for depth and JPEG/Theora for RGB) will reduce the bandwidth some, but obviously requires more processing power.

Cheers,
Patrick

On Wed, Jun 27, 2012 at 11:15 AM, 48Letters <[hidden email]> wrote:

Yea considering our timeframe sticking with windows for now will probably be
best. Do you think it would be hard to get to 30fps even if we were just
saving the data and not processing it?

--
View this message in context: http://www.pcl-users.org/Running-PCL-on-a-tablet-To-be-used-on-set-of-a-feature-film-tp4019672p4019764.html

Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

48Letters

Re: Running PCL on a tablet? To be used on set of a feature film.

That is a excellent link! Thank you! Yea right now we are going with a high end tablet that should be able to record everything at 30fps. It is a 1.6ghz dual core so actually it should be able to compress to png and jpg in real time as well.

Thanks again for the link. I will see if I can figure this out and run some tests. I don't suppose there are already created windows binaries for this is there?

-= Robin

On Thursday, June 28, 2012 at 12:08 PM, Patrick Mihelich wrote:

Actually running ROS on a tablet is probably too much trouble, but check out the advice at http://ros.org/wiki/openni_launch/Tutorials/BagRecordingPlayback.

For recording you can do pretty well recording the raw uint16 depth image and raw Bayer RGB image, totalling 3 bytes / pixel. For VGA depth+RGB at 30fps, that's 3*640*480*30 ~= 27 MiB/s, so your storage medium needs a max write speed of at least that (probably a bit more). An SD card won't cut it, but any SSD or hard disk drive should be just fine.

Compression (I've used PNG for depth and JPEG/Theora for RGB) will reduce the bandwidth some, but obviously requires more processing power.

Cheers,
Patrick

On Wed, Jun 27, 2012 at 11:15 AM, 48Letters <[hidden email]> wrote:
Yea considering our timeframe sticking with windows for now will probably be
best. Do you think it would be hard to get to 30fps even if we were just
saving the data and not processing it?

--
View this message in context: http://www.pcl-users.org/Running-PCL-on-a-tablet-To-be-used-on-set-of-a-feature-film-tp4019672p4019764.html

Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users

Patrick Mihelich-2

Re: Running PCL on a tablet? To be used on set of a feature film.

On Thu, Jun 28, 2012 at 12:14 PM, Robin Graham <[hidden email]> wrote:

Thanks again for the link. I will see if I can figure this out and run some tests. I don't suppose there are already created windows binaries for this is there?

ROS on Windows is a work in progress - definitely not to the point of having pre-built binaries yet. But this is pretty straightforward, you just need to hook up OpenNI/libfreenect to something like FreeImage.

Patrick

_______________________________________________
[hidden email] / http://pointclouds.org
http://pointclouds.org/mailman/listinfo/pcl-users