writeBinaryCompressed() too slow

> Hi,
>
> I am trying to dump the XYZRGBA cloud stream from my Kinect on to my SSD
> with PCDWriter::writeBinaryCompressed(). the openni callback with the write
> command then takes around 100ms to finish and the size of one pcd file is
> around 1,66 MB. Is this to be expected or am I missing something? I am using
> the PCL trunk from 2 days ago on Windows 7x64, all libs are 32 bit, CPU Core
> i5 M560 2.6 GHz
>
> thanks
> Christoph Pacher
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p3840585.html
> Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> thanks for your fast response.
>
> since i don't have a desktop to compare, I benchmarked a 2.5" usb 3.0 drive
> and a 3.5" esata drive on my laptop. usb 3 is worse with write times between
> 220 and 140 ms, esata is basically the same like my internal SSD with write
> times between 75 and 120ms. Anything else I can try?
>
> Some more background info: i am using OpenFrameworks for my main OpenGl App
> which runs in its own boost thread. The PCL pointcloud callback is a member
> of this App. I already disabled all the visual stuff, and just write the
> file in the callback with the current App running time in millisecs in the
> filename.
>
> Thanks!
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p3842093.html
> Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> this is the list of the files the openni_grab_frame tool has written:
> 20.03.2012  14:49         1.960.897 frame_20120320T144930.853091.pcd
> 20.03.2012  14:49         1.962.489 frame_20120320T144930.954096.pcd
> 20.03.2012  14:49         1.962.085 frame_20120320T144931.015100.pcd
> 20.03.2012  14:49         1.962.370 frame_20120320T144931.113105.pcd
> 20.03.2012  14:49         1.962.395 frame_20120320T144931.184109.pcd
> 20.03.2012  14:49         1.966.375 frame_20120320T144931.253113.pcd
> 20.03.2012  14:49         1.965.632 frame_20120320T144931.313117.pcd
> 20.03.2012  14:49         1.967.354 frame_20120320T144931.383121.pcd
> 20.03.2012  14:49         1.963.961 frame_20120320T144931.442124.pcd
> 20.03.2012  14:49         1.962.817 frame_20120320T144931.513128.pcd
> 20.03.2012  14:49         1.966.599 frame_20120320T144931.611134.pcd
> 20.03.2012  14:49         1.966.473 frame_20120320T144931.683138.pcd
> 20.03.2012  14:49         1.964.266 frame_20120320T144931.769143.pcd
> 20.03.2012  14:49         1.964.655 frame_20120320T144931.853148.pcd
> 20.03.2012  14:49         1.964.032 frame_20120320T144931.970154.pcd
> 20.03.2012  14:49         1.963.021 frame_20120320T144932.052159.pcd
> 20.03.2012  14:49         1.967.307 frame_20120320T144932.141164.pcd
> 20.03.2012  14:49         1.960.302 frame_20120320T144932.213168.pcd
> 20.03.2012  14:49         1.963.080 frame_20120320T144932.310174.pcd
> 20.03.2012  14:49         1.966.794 frame_20120320T144932.414180.pcd
> 20.03.2012  14:49         1.967.251 frame_20120320T144932.483184.pcd
> 20.03.2012  14:49         1.964.649 frame_20120320T144932.553188.pcd
> 20.03.2012  14:49         1.965.271 frame_20120320T144932.654194.pcd
>
>
>
> I also made a small grab test project with just PCL writing the files
> without the OpenFrameworks app. when writing XYZRGBA files i get a callback
> runtime between 40 and 50 ms, which is still too slow
>
> class testGrabber
> {
> public:
> testGrabber (){}
>
> void cloud_cb_ (const pcl::PointCloud<pcl::PointXYZRGBA>::ConstPtr&cloud)
> {
> pcl::ScopeTime t ("grabber");
>
> w->writeBinaryCompressed(std::to_string(counter) + ".pcd", *cloud);
> counter++;
> }
>
> void run ()
> {
> interface = new pcl::OpenNIGrabber();
> w = new pcl::PCDWriter;
> counter = 0;
>
> boost::function<void (const
> pcl::PointCloud<pcl::PointXYZRGBA>::ConstPtr&)>  f =
> boost::bind (&testGrabber::cloud_cb_, this, _1);
>
> interface->registerCallback (f);
>
> interface->start ();
>
> while (true)
> {
> sleep (1);
> }
>
> interface->stop ();
> }
>
> // vs10 missing to_string(int) workaround
> long long counter;
> pcl::PCDWriter* w;
> pcl::Grabber* interface;
>
>
> };
>
>
>
> //========================================================================
> int main( ){
>
> testGrabber t;
> t.run();
>
> }
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p3842451.html
> Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

>
> class testGrabber
> {
> public:
> testGrabber (){}
>
> void cloud_cb_ (const pcl::PointCloud<pcl::PointXYZRGBA>::ConstPtr &cloud)
> {
> pcl::ScopeTime t ("grabber");
>
> w->writeBinaryCompressed(std::to_string(counter) + ".pcd", *cloud);
> counter++;
> }
>
> void run ()
> {
> interface = new pcl::OpenNIGrabber();
> w = new pcl::PCDWriter;
> counter = 0;
>
> boost::function<void (const
> pcl::PointCloud<pcl::PointXYZRGBA>::ConstPtr&)> f =
> boost::bind (&testGrabber::cloud_cb_, this, _1);
>
> interface->registerCallback (f);
>
> interface->start ();
>
> while (true)
> {
> sleep (1);
> }
>
> interface->stop ();
> }
>
> // vs10 missing to_string(int) workaround
> long long counter;
> pcl::PCDWriter* w;
> pcl::Grabber* interface;
>
>
> };
>
>
>
> //========================================================================
> int main( ){
>
> testGrabber t;
> t.run();
>
> }
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p3842451.html
> Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> Any news on this topic? Should I test something else?
>
>
> Christoph Pacher wrote
>>
>> i tried to profile the grab frame tool. unfortunately the release
>> profiling results are not really helpful, as you cant see the code in
>> question, only the dlls. see the result in the following image:
>>
>> http://imageshack.us/photo/my-images/17/performanced.jpg/
>>
>
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p3860882.html
> Sent from the Point Cloud Library (PCL) Users mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> I'm sorry for posting on such an old thread, but I too am having the same
> issue. I'm running pcl-trunk (1.7) on revision 7754. To rule out disk write
> performance, I ran the following benchmark:
>
> dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output
>
> Which returned me 75MB/s in one platform and 90MB/s in another. Strangely,
> the first one (specs  here
> <http://www.notebookreview.com/default.asp?newsID=6047&review=msi+gx660+gx660r>
> ) would only reach around 10fps, while the second (a core i5 desktop) had an
> average of 15fps. Both are Linux based and the data was being written by
> openni_grab_frame to ext4 partitions.
>
> In another platform (with 109MB/s of disk write performance and PCL 1.7 rev
> 7754), I decided to perform a test that would generate random clouds and
> save them with writeBinaryCompressed. Sice their average size was of 3.7MB,
> I shouldn't expect my system to reach 30fps (that would require a throughput
> of around 112MB/s), but I suppose it could reach around 27fps (100MB/s).
> Instead, I got an average of 11fps. So I decided to run gprof on this
> application, I'm uploading the resulting profile  here
> <http://www.pcl-users.org/file/n4023508/profile>  . (The testGrabber::fill
> function was created for generating random point clouds, and is not directly
> affecting the writer performance).
>
> I hope this will give a hint and help optimize this function further.
>
>
>
> --
> View this message in context: http://www.pcl-users.org/writeBinaryCompressed-too-slow-tp3840585p4023508.html
> Sent from the Point Cloud Library (PCL) Users mailing list mailing list archive at Nabble.com.
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
>