Running KinFu in "offline" mode?

> Hi,
>
> This happens when KinFu processes data at lower FPS than in ONI-file.
> While kinfu is busy, OpenNI Grabber continues playing ONI file and drops
> frames.
> Need to implement triggered (not streaming) mode for ONI files.
>
> Anatoly
>
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Hannes Ovren
> Sent: Wednesday, July 04, 2012 1:56 PM
> To: [hidden email]
> Subject: [PCL-users] Running KinFu in "offline" mode?
>
> Hi!
>
> I am trying to run kinfu_app on data recorded into a OpenNI .oni file by
> running 'kinfu_app -oni myrecording.oni'.
>
> However, it seems like not all frames are used. Feeding the application a
> 5 second clip seem to only process 17 frames of the available 150. And a
> 10 second clip 35 frames of 300 available.
> This was discovered by inserting a debug printout in KinFuApp::execute()
> just before the kinfu_(depth_device) call.
>
> It seems like it is running in the frame rate that my GPU can handle (~3
> fps on a Quadro 600).
>
> This gives me trouble for two reasons:
> 1) Running kinfu_app on the same recording twice generates different
> results. But, if I am correct, the algorithm involves no randomness, so I
> should get the same result every time for the same data? Please correct me
> if I am wrong here.
>
> 2) With a moving Kinect the result is really bad which I am guessing is
> due to there being too long beween frames, which produce too much movement
> for the algorithm to handle.
>
>
> So, my actual question:
>
> How can I get kinfu_app to use all frames in my .ONI recording?
>
> I have tried following what goes on in pcl::io for ONI files but I fail to
> find an explanation.
>
> Steps to reproduce:
> 1) Add debug printout to KinFuApp::execute() before or after
> kinfu_(depth_device) and compile
> 2) Record a .ONI file using the NiBackReg application from OpenNI
> ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds have
> passed)
> 3) Run kinfu_app - oni recording.oni
> 4) Observe that the debug line is printed only ~17 times and not the
> expected ~150 times.
>
> Regards,
>
> Hannes Ovrén
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> Hi!
>
> I am trying to run kinfu_app on data recorded into a OpenNI .oni file by
> running 'kinfu_app -oni myrecording.oni'.
>
> However, it seems like not all frames are used. Feeding the application
> a 5 second clip seem to only process 17 frames of the available 150. And
> a 10 second clip 35 frames of 300 available.
> This was discovered by inserting a debug printout in KinFuApp::execute()
> just before the kinfu_(depth_device) call.
>
> It seems like it is running in the frame rate that my GPU can handle (~3
> fps on a Quadro 600).
>
> This gives me trouble for two reasons:
> 1) Running kinfu_app on the same recording twice generates different
> results. But, if I am correct, the algorithm involves no randomness, so
> I should get the same result every time for the same data? Please
> correct me if I am wrong here.
>
> 2) With a moving Kinect the result is really bad which I am guessing is
> due to there being too long beween frames, which produce too much
> movement for the algorithm to handle.
>
>
> So, my actual question:
>
> How can I get kinfu_app to use all frames in my .ONI recording?
>
> I have tried following what goes on in pcl::io for ONI files but I fail
> to find an explanation.
>
> Steps to reproduce:
> 1) Add debug printout to KinFuApp::execute() before or after
> kinfu_(depth_device) and compile
> 2) Record a .ONI file using the NiBackReg application from OpenNI
> ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds
> have passed)
> 3) Run kinfu_app - oni recording.oni
> 4) Observe that the debug line is printed only ~17 times and not the
> expected ~150 times.
>
> Regards,
>
> Hannes Ovrén
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
>

> Hi,
>
> This happens when KinFu processes data at lower FPS than in ONI-file.
> While kinfu is busy, OpenNI Grabber continues playing ONI file and
> drops frames.
> Need to implement triggered (not streaming) mode for ONI files.
>
> Anatoly
>
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Hannes Ovren
> Sent: Wednesday, July 04, 2012 1:56 PM
> To: [hidden email]
> Subject: [PCL-users] Running KinFu in "offline" mode?
>
> Hi!
>
> I am trying to run kinfu_app on data recorded into a OpenNI .oni file
> by running 'kinfu_app -oni myrecording.oni'.
>
> However, it seems like not all frames are used. Feeding the
> application a
> 5 second clip seem to only process 17 frames of the available 150. And
> a
> 10 second clip 35 frames of 300 available.
> This was discovered by inserting a debug printout in
> KinFuApp::execute() just before the kinfu_(depth_device) call.
>
> It seems like it is running in the frame rate that my GPU can handle
> (~3 fps on a Quadro 600).
>
> This gives me trouble for two reasons:
> 1) Running kinfu_app on the same recording twice generates different
> results. But, if I am correct, the algorithm involves no randomness,
> so I should get the same result every time for the same data? Please
> correct me if I am wrong here.
>
> 2) With a moving Kinect the result is really bad which I am guessing
> is due to there being too long beween frames, which produce too much
> movement for the algorithm to handle.
>
>
> So, my actual question:
>
> How can I get kinfu_app to use all frames in my .ONI recording?
>
> I have tried following what goes on in pcl::io for ONI files but I
> fail to find an explanation.
>
> Steps to reproduce:
> 1) Add debug printout to KinFuApp::execute() before or after
> kinfu_(depth_device) and compile
> 2) Record a .ONI file using the NiBackReg application from OpenNI
> ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds
> have
> passed)
> 3) Run kinfu_app - oni recording.oni
> 4) Observe that the debug line is printed only ~17 times and not the
> expected ~150 times.
>
> Regards,
>
> Hannes Ovrén
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> KinfuApp class should understand that device is passed in triggered mode,
> so it will call 'capture_.start()' in main loop for each frame.
>
> Also you need to init ONIGrabber in triggered mode (disable stream mode
> for this).
> capture.reset (new pcl::ONIGrabber (oni_file, true, bool stream  =
> false));
>
> If you send us patch that would be helpful.
>
> Cheers,
> Anatoly
>
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Hannes Ovren
> Sent: Wednesday, July 04, 2012 3:33 PM
> To: [hidden email]
> Subject: Re: [PCL-users] Running KinFu in "offline" mode?
>
> Hi!
>
> Thank you for the swift reply.
>
> Do you mean that the OpenNI ONIGrabber (pcl::ONIGrabber I suppose?) need
> to be adapted in some way?
>
> Or is "triggered mode" something that I should be applying to my ONI
> recording application?
>
> /Hannes
>
> On Wed, 2012-07-04 at 15:26 +0300, Anatoly Baksheev wrote:
> > Hi,
> >
> > This happens when KinFu processes data at lower FPS than in ONI-file.
> > While kinfu is busy, OpenNI Grabber continues playing ONI file and
> > drops frames.
> > Need to implement triggered (not streaming) mode for ONI files.
> >
> > Anatoly
> >
> >
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On Behalf Of Hannes Ovren
> > Sent: Wednesday, July 04, 2012 1:56 PM
> > To: [hidden email]
> > Subject: [PCL-users] Running KinFu in "offline" mode?
> >
> > Hi!
> >
> > I am trying to run kinfu_app on data recorded into a OpenNI .oni file
> > by running 'kinfu_app -oni myrecording.oni'.
> >
> > However, it seems like not all frames are used. Feeding the
> > application a
> > 5 second clip seem to only process 17 frames of the available 150. And
> > a
> > 10 second clip 35 frames of 300 available.
> > This was discovered by inserting a debug printout in
> > KinFuApp::execute() just before the kinfu_(depth_device) call.
> >
> > It seems like it is running in the frame rate that my GPU can handle
> > (~3 fps on a Quadro 600).
> >
> > This gives me trouble for two reasons:
> > 1) Running kinfu_app on the same recording twice generates different
> > results. But, if I am correct, the algorithm involves no randomness,
> > so I should get the same result every time for the same data? Please
> > correct me if I am wrong here.
> >
> > 2) With a moving Kinect the result is really bad which I am guessing
> > is due to there being too long beween frames, which produce too much
> > movement for the algorithm to handle.
> >
> >
> > So, my actual question:
> >
> > How can I get kinfu_app to use all frames in my .ONI recording?
> >
> > I have tried following what goes on in pcl::io for ONI files but I
> > fail to find an explanation.
> >
> > Steps to reproduce:
> > 1) Add debug printout to KinFuApp::execute() before or after
> > kinfu_(depth_device) and compile
> > 2) Record a .ONI file using the NiBackReg application from OpenNI
> > ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds
> > have
> > passed)
> > 3) Run kinfu_app - oni recording.oni
> > 4) Observe that the debug line is printed only ~17 times and not the
> > expected ~150 times.
> >
> > Regards,
> >
> > Hannes Ovrén
> >
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
>
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> I tried setting stream = false, like this:
>
>     else if (pc::parse_argument (argc, argv, "-oni", oni_file) > 0)
>     {
>       bool repeat = false;
>       bool stream = false; // false = trigger, true=stream [default]
>       capture.reset (new pcl::ONIGrabber (oni_file, repeat, stream));
>     }
>
> Unfortunately this just made kinfu process the first image, and then
> stop. has_data in execute is set to true once, and is false after that.
>
> So, is there something that need fixing in ONIGrabber? Or any other
> ideas?
>
> Also, you are sure that there is nothing that must be done to the ONI
> files?
>
> I will be happy to supply patches for an offline mode when I get it
> running. :)
>
> /Hannes
>
>
> On Wed, 2012-07-04 at 15:45 +0300, Anatoly Baksheev wrote:
> > KinfuApp class should understand that device is passed in triggered mode,
> > so it will call 'capture_.start()' in main loop for each frame.
> >
> > Also you need to init ONIGrabber in triggered mode (disable stream mode
> > for this).
> > capture.reset (new pcl::ONIGrabber (oni_file, true, bool stream  =
> > false));
> >
> > If you send us patch that would be helpful.
> >
> > Cheers,
> > Anatoly
> >
> >
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On Behalf Of Hannes Ovren
> > Sent: Wednesday, July 04, 2012 3:33 PM
> > To: [hidden email]
> > Subject: Re: [PCL-users] Running KinFu in "offline" mode?
> >
> > Hi!
> >
> > Thank you for the swift reply.
> >
> > Do you mean that the OpenNI ONIGrabber (pcl::ONIGrabber I suppose?) need
> > to be adapted in some way?
> >
> > Or is "triggered mode" something that I should be applying to my ONI
> > recording application?
> >
> > /Hannes
> >
> > On Wed, 2012-07-04 at 15:26 +0300, Anatoly Baksheev wrote:
> > > Hi,
> > >
> > > This happens when KinFu processes data at lower FPS than in ONI-file.
> > > While kinfu is busy, OpenNI Grabber continues playing ONI file and
> > > drops frames.
> > > Need to implement triggered (not streaming) mode for ONI files.
> > >
> > > Anatoly
> > >
> > >
> > > -----Original Message-----
> > > From: [hidden email]
> > > [mailto:[hidden email]] On Behalf Of Hannes Ovren
> > > Sent: Wednesday, July 04, 2012 1:56 PM
> > > To: [hidden email]
> > > Subject: [PCL-users] Running KinFu in "offline" mode?
> > >
> > > Hi!
> > >
> > > I am trying to run kinfu_app on data recorded into a OpenNI .oni file
> > > by running 'kinfu_app -oni myrecording.oni'.
> > >
> > > However, it seems like not all frames are used. Feeding the
> > > application a
> > > 5 second clip seem to only process 17 frames of the available 150. And
> > > a
> > > 10 second clip 35 frames of 300 available.
> > > This was discovered by inserting a debug printout in
> > > KinFuApp::execute() just before the kinfu_(depth_device) call.
> > >
> > > It seems like it is running in the frame rate that my GPU can handle
> > > (~3 fps on a Quadro 600).
> > >
> > > This gives me trouble for two reasons:
> > > 1) Running kinfu_app on the same recording twice generates different
> > > results. But, if I am correct, the algorithm involves no randomness,
> > > so I should get the same result every time for the same data? Please
> > > correct me if I am wrong here.
> > >
> > > 2) With a moving Kinect the result is really bad which I am guessing
> > > is due to there being too long beween frames, which produce too much
> > > movement for the algorithm to handle.
> > >
> > >
> > > So, my actual question:
> > >
> > > How can I get kinfu_app to use all frames in my .ONI recording?
> > >
> > > I have tried following what goes on in pcl::io for ONI files but I
> > > fail to find an explanation.
> > >
> > > Steps to reproduce:
> > > 1) Add debug printout to KinFuApp::execute() before or after
> > > kinfu_(depth_device) and compile
> > > 2) Record a .ONI file using the NiBackReg application from OpenNI
> > > ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds
> > > have
> > > passed)
> > > 3) Run kinfu_app - oni recording.oni
> > > 4) Observe that the debug line is printed only ~17 times and not the
> > > expected ~150 times.
> > >
> > > Regards,
> > >
> > > Hannes Ovrén
> > >
> > > _______________________________________________
> > > [hidden email] / http://pointclouds.org
> > > http://pointclouds.org/mailman/listinfo/pcl-users
> > > _______________________________________________
> > > [hidden email] / http://pointclouds.org
> > > http://pointclouds.org/mailman/listinfo/pcl-users
> >
> >
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
>
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> I tried setting stream = false, like this:
>
>     else if (pc::parse_argument (argc, argv, "-oni", oni_file) > 0)
>     {
>       bool repeat = false;
>       bool stream = false; // false = trigger, true=stream [default]
>       capture.reset (new pcl::ONIGrabber (oni_file, repeat, stream));
>     }
>
> Unfortunately this just made kinfu process the first image, and then
> stop. has_data in execute is set to true once, and is false after that.
>
> So, is there something that need fixing in ONIGrabber? Or any other
> ideas?
>
> Also, you are sure that there is nothing that must be done to the ONI
> files?
>
> I will be happy to supply patches for an offline mode when I get it
> running. :)
>
> /Hannes
>
>
> On Wed, 2012-07-04 at 15:45 +0300, Anatoly Baksheev wrote:
> > KinfuApp class should understand that device is passed in triggered
> > mode, so it will call 'capture_.start()' in main loop for each frame.
> >
> > Also you need to init ONIGrabber in triggered mode (disable stream
> > mode for this).
> > capture.reset (new pcl::ONIGrabber (oni_file, true, bool stream  =
> > false));
> >
> > If you send us patch that would be helpful.
> >
> > Cheers,
> > Anatoly
> >
> >
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On Behalf Of Hannes Ovren
> > Sent: Wednesday, July 04, 2012 3:33 PM
> > To: [hidden email]
> > Subject: Re: [PCL-users] Running KinFu in "offline" mode?
> >
> > Hi!
> >
> > Thank you for the swift reply.
> >
> > Do you mean that the OpenNI ONIGrabber (pcl::ONIGrabber I suppose?)
> > need to be adapted in some way?
> >
> > Or is "triggered mode" something that I should be applying to my ONI
> > recording application?
> >
> > /Hannes
> >
> > On Wed, 2012-07-04 at 15:26 +0300, Anatoly Baksheev wrote:
> > > Hi,
> > >
> > > This happens when KinFu processes data at lower FPS than in

> > > While kinfu is busy, OpenNI Grabber continues playing ONI file and
> > > drops frames.
> > > Need to implement triggered (not streaming) mode for ONI files.
> > >
> > > Anatoly
> > >
> > >
> > > -----Original Message-----
> > > From: [hidden email]
> > > [mailto:[hidden email]] On Behalf Of Hannes
> > > Ovren
> > > Sent: Wednesday, July 04, 2012 1:56 PM
> > > To: [hidden email]
> > > Subject: [PCL-users] Running KinFu in "offline" mode?
> > >
> > > Hi!
> > >
> > > I am trying to run kinfu_app on data recorded into a OpenNI .oni
> > > file by running 'kinfu_app -oni myrecording.oni'.
> > >
> > > However, it seems like not all frames are used. Feeding the
> > > application a
> > > 5 second clip seem to only process 17 frames of the available 150.
> > > And a
> > > 10 second clip 35 frames of 300 available.
> > > This was discovered by inserting a debug printout in
> > > KinFuApp::execute() just before the kinfu_(depth_device) call.
> > >
> > > It seems like it is running in the frame rate that my GPU can
> > > handle
> > > (~3 fps on a Quadro 600).
> > >
> > > This gives me trouble for two reasons:
> > > 1) Running kinfu_app on the same recording twice generates
> > > different results. But, if I am correct, the algorithm involves no
> > > randomness, so I should get the same result every time for the
> > > same data? Please correct me if I am wrong here.
> > >
> > > 2) With a moving Kinect the result is really bad which I am
> > > guessing is due to there being too long beween frames, which
> > > produce too much movement for the algorithm to handle.
> > >
> > >
> > > So, my actual question:
> > >
> > > How can I get kinfu_app to use all frames in my .ONI recording?
> > >
> > > I have tried following what goes on in pcl::io for ONI files but I
> > > fail to find an explanation.
> > >
> > > Steps to reproduce:
> > > 1) Add debug printout to KinFuApp::execute() before or after
> > > kinfu_(depth_device) and compile
> > > 2) Record a .ONI file using the NiBackReg application from OpenNI
> > > ('NiBackReg time 5 depth vga' and press 'd' when more than 5
> > > seconds have
> > > passed)
> > > 3) Run kinfu_app - oni recording.oni
> > > 4) Observe that the debug line is printed only ~17 times and not
> > > the expected ~150 times.
> > >
> > > Regards,
> > >
> > > Hannes Ovrén
> > >
> > > _______________________________________________
> > > [hidden email] / http://pointclouds.org
> > > http://pointclouds.org/mailman/listinfo/pcl-users
> > > _______________________________________________
> > > [hidden email] / http://pointclouds.org
> > > http://pointclouds.org/mailman/listinfo/pcl-users
> >
> >
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
>
>
> _______________________________________________
> [hidden email] / http://pointclouds.org
> http://pointclouds.org/mailman/listinfo/pcl-users

> Hi Hannes,
>
> I think the ability to do Kinfu offline will be great. It will allow
> data collection in the field using cheap hardware with no GPU.
>
> Data processing can happen later on good desktop hardware, or even
> on cheap hardware with an old GPU.
>
> Please keep us updated on your progress with this. I will help with
> this as soon as I have KinFu running myself (hardware is in the
> mail).
>
>    -k.
>
>
> On Wed, 4 Jul 2012, Hannes Ovren wrote:
>
> > Hi!
> >
> > I am trying to run kinfu_app on data recorded into a OpenNI .oni file by
> > running 'kinfu_app -oni myrecording.oni'.
> >
> > However, it seems like not all frames are used. Feeding the application
> > a 5 second clip seem to only process 17 frames of the available 150. And
> > a 10 second clip 35 frames of 300 available.
> > This was discovered by inserting a debug printout in KinFuApp::execute()
> > just before the kinfu_(depth_device) call.
> >
> > It seems like it is running in the frame rate that my GPU can handle (~3
> > fps on a Quadro 600).
> >
> > This gives me trouble for two reasons:
> > 1) Running kinfu_app on the same recording twice generates different
> > results. But, if I am correct, the algorithm involves no randomness, so
> > I should get the same result every time for the same data? Please
> > correct me if I am wrong here.
> >
> > 2) With a moving Kinect the result is really bad which I am guessing is
> > due to there being too long beween frames, which produce too much
> > movement for the algorithm to handle.
> >
> >
> > So, my actual question:
> >
> > How can I get kinfu_app to use all frames in my .ONI recording?
> >
> > I have tried following what goes on in pcl::io for ONI files but I fail
> > to find an explanation.
> >
> > Steps to reproduce:
> > 1) Add debug printout to KinFuApp::execute() before or after
> > kinfu_(depth_device) and compile
> > 2) Record a .ONI file using the NiBackReg application from OpenNI
> > ('NiBackReg time 5 depth vga' and press 'd' when more than 5 seconds
> > have passed)
> > 3) Run kinfu_app - oni recording.oni
> > 4) Observe that the debug line is printed only ~17 times and not the
> > expected ~150 times.
> >
> > Regards,
> >
> > Hannes Ovrén
> >
> > _______________________________________________
> > [hidden email] / http://pointclouds.org
> > http://pointclouds.org/mailman/listinfo/pcl-users
> >
>