Problem in implementation

[dashesy]

High frame rate capture problem:

When I change line in GrayscaleSample code (COM-based code written for OptiTrack) to full speed
i.e. I change
camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 25);

to

camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 75);

most of the frames are corrupted and empty frames are returned.
My question is, is this because of COM interface and could be solved by using TT2.0?
While I capture each frame using the command:
camera->GetFrameImage(frame, windowWidth, windowHeight, windowWidth, 8, (byte *) frameBuffer);
what does the NP_OPTION_FRAME_RATE do? I mean shouldn't framme-per-second solely depend on how fast we capture the frames? Or maybe NP_OPTION_FRAME_RATE increases some hardware timer speed in order to capture with higher rate and otherwise calling GetFrameImage would return the previous frame?

The reason I suspect it is COM fault (not USB) is that inside NaturalPoint TT2.0 I can get the 100FPS.

I decided to ask my software question here instead of the other thread that I started, because I did not want to mix different issues.
I may keep all my questions in these two threads if you suggest so

[Birch]

Its likely you are seeing dropped or corrupt frames due to USB bandwidth limitations combined with not servicing the API fast enough to keep up.

The full grayscale video mode consumes a large amount USB bandwidth.
http://www.naturalpoint.com/optitrack/s ... .html#q205

If you call GetFrame faster than the camera produces frames then it will return NULL if new frames are not yet available. If it is called at a rate slower than the camera produces frames, then camera frames may begin to get backlogged in the frame queue. If the frame queue is full then incoming frames may get dropped until it has been serviced to make room available. We recommend making sure the frame queue gets drained down regularly :
http://forum.naturalpoint.com/forum/ubb ... #Post27662

The OptiTrack SDK doc is a good reasource :
http://www.naturalpoint.com/optitrack/s ... .1.035.doc

NP_OPTION_FRAME_RATE :
This option changes the internal speed at which the camera captures frames. At 100FPS the camera captures a frame every 10ms regardless of whether it is transferred to the PC or used by an application. Likewise, at 25FPS it captures a frame every 40ms. Since the only frame rates available for V100/V100 R2are 25, 50 and 100, when you set it to 75 it rounds that up (to 100). Lower frame rates use less bandwidth

The following option is also useful for reducing bandwidth consumption.

NP_OPTION_FRAME_DECIMATION :
This option Controls the cameras auto-frame discarding (decimation) feature, this allows the camera to expose at fast shutter speeds while delivering a reduced number of frames (every Nth) to the PC. Range is from 0 to 5 (larger values result in more auto-discarded frames). Default value is 0 (no frames discarded). 0=drop no frames, 1=send every other, 2=send every forth, 3=send every eighth, 4=send every sixteenth, 4=send every thirty-secondth

[dashesy]

Thank you for your informative post.

Is your TT2.0 software using TT SDK? The reason I am asking this is that I can get full speed (100%) with best-quality grayscale image without decimation using TT software. So USB bandwidth shouldn't be an issue specially since I have only one camera connected directly to PC (no hubs).
TT also says my camera is V100 so it should not be V100R2 that supports compressed JPeg.

I am going to change my code from OptiTracK COM to TT, although I will use 2D features for now. I think TT is better supported and also has features not available in OptiTrack (e.g. AGC, AEC and compress JPeg).
Besides looking at TT software, I have milestone how far I can go if I go in the right direction.

BTW, you have done a great job with TT

Ehsan

[dashesy]

More on this:
I tried the following code with GrayScale example, I do not think I can service camera any faster but still I can not get full speed. I set camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 100); then removed the sleep command from inside the while loop. Then to service all queued frames I did this,

bool bEmpty = false;
while (!bEmpty){
camera->GetFrame(0, &frame);
bEmpty = true;
if(frame!=0)
{
bEmpty = false;
//== New Frame Has Arrived ==========================------
frameCounter++;
camera->GetFrameImage(frame, windowWidth, windowHeight, windowWidth, 8, (byte *) frameBuffer);
for (int index=0; indexFree();
frame.Release();
}
}

Compiled in release configuration. Further I changed the priority of test.exe process to realtime! Now consuming 50% of a 2GHz CPU time but frames are corrupted.

More info:
camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 50); works however:

camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 100);
camera->SetOption(NP_OPTION_FRAME_DECIMATION,(CComVariant) 1);
and even
camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 100);
camera->SetOption(NP_OPTION_FRAME_DECIMATION,(CComVariant) 2);
do not work!

[Birch]

The following inner loop produces reliable 100FPS grayscale on an older P4 2.8ghz (V100 connected directly to the PC). The callback mechanism may be an easier way to get full frame rate. The "C++ Win32 VC8/2005 Sample" displayed 100FPS grayscale with no modification.

Code: Select all

while(!TimeToClose)
{
    MSG msg;
    if( PeekMessage( &msg, NULL, 0, 0, PM_NOREMOVE ) )
    {
        if(GetMessage( &msg, NULL, 0, 0 ) )
        {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
        }
    }

    Sleep(5);

    // drain down frame queue and use last valid frame extracted
    camera->GetFrame(0, &tempframe);
    frame = tempframe;

    while (tempframe!= NULL)
    {
        frame = tempframe;
        tempframe.Release();
        camera->GetFrame(0, &tempframe);
    }

    if(frame!=0)
    {
        //== New Frame Has Arrived ==========================------
        frameCounter++;

        camera->GetFrameImage(frame, windowWidth, windowHeight, windowWidth, 8, (byte *) bufFrameCurrent);

        // free up the frame
	frame->Free();
						frame.Release();

        // display the frame data
        for (int index=0; index<windowWidth*windowHeight; index++)
        {
            noise = bufFrameCurrent[index];
            pixel[index] = (noise<<16) | (noise<<8) | noise;
        }
        ptc_update(pixel);
    }
}

[dashesy]

Hi,

Code at the end is the full code I am using after applying your suggestion, but still I get corrupt frames.
Even adding the line

Code: Select all

camera->SetOption(NP_OPTION_FRAME_DECIMATION,(CComVariant) 2);

does not help. I reduced the call to ptc_update and that also did not do the trick.
I am not sure if that is because of a defect in camera either, because as I said TT2.0 software works fine with effective full frame rate.

Code: Select all

//========================================================================-----
//== OptiTrack Sample Application
//== Copyright (c) NaturalPoint
//== 
//== This sample is intended to show how to properly access and control
//== OptiTrack cameras.  The application does the following:
//==
//== 1. Initializes the OptiTrack COM Component.
//== 2. Displays information about all connected cameras.
//== 3. Starts the first camera in the system and displays a grayscale image
//==    (grayscale is only supported on OptiTrack cameras.  TrackIR and 
//==     SmartNAV cameras will display filtered tracking image)
//== 4. Unintializes the OptiTrack cameras, COM, and terminates.

//== This sample uses TinyPTC to display grayscale data ==================-----
//== >> TinyPTC by Gaffer
//== >> www.gaffer.org/tinyptc

//== INCLUDES ============================================================-----

#include "tinyptc.h"

//== NECESSARY OPTITRACK INCLUDES AND DEFINITIONS ========================-----

#include <objbase.h>
#include <atlbase.h>
#include "./optitrack.h"
#import  "./optitrack.tlb"

   
static int noise;
static int carry;
static int index;
static int seed = 0x12345;
static int pixel[640*480];
static unsigned char frameBuffer[640*480];
bool       TimeToClose = false;

//== SAMPLE APPLICATION ENTRY POINT ======================================-----

int main()
{
	//== Initialize Microsoft COM Interop ================----
	CoInitialize(NULL);

	//== Initialize OptiTrack COM Component ==============----
	CComPtr<INPCameraCollection> cameraCollection;
    CComPtr<INPCamera>           camera;
	CComPtr<INPCameraFrame>		 frame, tempframe;

    cameraCollection.CoCreateInstance(CLSID_NPCameraCollection);

	//== Enumerate (Identify) Available Cameras ==========----
	cameraCollection->Enum();

	long cameraCount  = 0;
	unsigned int  frameCounter = 0;

	//== Determine Available Cameras =====================----
    cameraCollection->get_Count(&cameraCount);
	
    int windowWidth = 0;
    int windowHeight = 0;

	//== Display Camera Information for All Cameras ======----
	for(int index=0; index<cameraCount; index++)
	{
		cameraCollection->Item(index, &camera);

		long serial,width,height,model,revision,rate;

		camera->get_SerialNumber(&serial);
		camera->get_Width       (&width);
		camera->get_Height      (&height);
		camera->get_Model       (&model);
		camera->get_Revision    (&revision);
		camera->get_FrameRate   (&rate);

        if(index==0)
        {
            windowWidth = width;
            windowHeight = height;
        }

		//== Set Some Camera Options ====================----

		//== Set Grayscale Mode =========================----
		camera->SetOption(NP_OPTION_VIDEO_TYPE        , (CComVariant) 1 );

        //== Don't drop frames ==--
        camera->SetOption(NP_OPTION_FRAME_DECIMATION  , (CComVariant) 0 );

		//== Display 99 on the Camera ===================----
		//== Note: This only works for cameras with a display
        camera->SetOption(NP_OPTION_NUMERIC_DISPLAY_ON, (CComVariant) 99);

        //== Add frame information to the top left of the frame ==--
        camera->SetOption(NP_OPTION_TEXT_OVERLAY_OPTION,(CComVariant) 255);

        //== Send corrupt frames ==
        camera->SetOption(NP_OPTION_SEND_FRAME_MASK,(CComVariant) 255);

        //== Slow camera frame frame to 25% ==--
        camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 100);
//Even with decimation (following line) full frame rate is not achieveable
// camera->SetOption(NP_OPTION_FRAME_DECIMATION,(CComVariant) 2);
		//== Always Clean-up COM Objects ================----
		camera.Release();
	}

	//== Open the first camera ==========================----
	
	if(cameraCount>0)
	{
        if (!ptc_open("OptiTrack Grayscale Sample", windowWidth, windowHeight)) return 1;

		cameraCollection->Item(0, &camera);
		{
			camera->Open();
			camera->Start();
            //camera->SetOption(NP_OPTION_EXPOSURE, (CComVariant) 300);
			//camera->SetOption(NP_OPTION_FRAME_RATE,(CComVariant) 100);
			//camera->SetOption(NP_OPTION_FRAME_DECIMATION,(CComVariant) 4);
            {
				while(!TimeToClose)
				{
					MSG msg;
					if( PeekMessage( &msg, NULL, 0, 0, PM_NOREMOVE ) )
					{
						if(GetMessage( &msg, NULL, 0, 0 ) )
						{
							TranslateMessage(&msg);
							DispatchMessage(&msg);
						}
					}

					Sleep(5);

					// drain down frame queue and use last valid frame extracted
					camera->GetFrame(0, &tempframe);
					frame = tempframe;

					while (tempframe!= NULL)
					{
						frame = tempframe;
						tempframe.Release();
						camera->GetFrame(0, &tempframe);
					}

					if(frame!=0)
					{
						//== New Frame Has Arrived ==========================------
						frameCounter++;

						camera->GetFrameImage(frame, windowWidth, windowHeight, windowWidth, 8, (byte *) frameBuffer);

						// free up the frame
					frame->Free();
										frame.Release();

						// display the frame data
						for (int index=0; index<windowWidth*windowHeight; index++)
						{
							noise = frameBuffer[index];
							pixel[index] = (noise<<16) | (noise<<8) | noise;
						}
						ptc_update(pixel);
					}
				}

            }
			camera->Stop();
			camera->Close();
		}

		camera.Release();
	}


	//== Always Clean-up COM Objects ================----
	cameraCollection.Release();

	//== Uninitialize Microsoft COM Interop =============----
	CoUninitialize();

    ExitProcess(1);
}

[dashesy]

Please let me know what you think is the problem of the posted code.
BTW, apart from not being able to run at 100fps I also noticed that running at lower fps like 50 I can see a grid in the picture that does not move or change size. Is there a way to explain this artifact? I thought it might be related to fps issue.

[Birch]

We tested the code you sent and it is working reliably (no corrupt frames) at 100FPS with a FLEX:V100. We did have to manually ensure that only one copy of the application was running at a time, since it doesn't have a built in facility for protecting against that scenario (which would degrade camera performance).

Please contact support@naturalpoint.com and send the following information :

* a screenshot of the device manager set to display devices by connection, showing all of the EHCI nodes on your system expanded

* the serial number of your camera(s)

* a screenshot of the grid you are seeing

[dashesy]

I sent the screen shots to the support, thank you