Camera->GetFrame(), GetLatestFrame(), and frame queues

[LinusA]

Hi, I'm using Camera SDK 1.0.1 beta 1 and V100R2 cameras. I noticed there are 2 similar functions:

Code: Select all

Frame *     GetFrame();                       //== Fetch next available frame =======----
Frame *     GetLatestFrame();                 //== Fetch latest frame (empties queue) ---

What is the exact difference between them?

Is there a way to see how many frames the cam has waiting for me (or just to see if GetFrame() would return something != NULL)?

Can I use GetLatestFrame() to skip/drop individual frames (when my CPU load is too high)?

Is there a way to detect a frame queue overflow? When does it happen?

Thanks, hope you get the idea what I'm on to. Right now I'm just using GetFrame() and it seems to work.

[beckdo]

Hi. For 99% of the applications using the Camera SDK, users will always want to use Camera->GetFrame(). I would always recommend that when your application goes to process a frame it processes all available frames until Camera->GetFrame() returns NULL. The only reason for GetLatestFrame() is in the event where the application is inefficient or starved for CPU and is unlikely to be able to service the camera(s) at regular intervals.

There is no way to see the number of frames in the que. But ideally, if you're in a place where you want to know that you should just empty it by processing them all.

The only time you'll get a frame queue overflow is when you're not processing frames fast enough. I would simply recommend looking at the FrameID in each frame you get. If you start getting gaps in the FrameIDs for the frames you're receiving you are overflowing.

[LinusA]

Thanks, the frameID is a good tip .

So if I get those overflows, I could use GetLatestFrame() in this one case to flush the frame queue, and keep on using GetFrame() again as long as I can keep up?

[LinusA]

I just noticed: When using cModuleSync with FrameGroups, there is neither a function called GetLatestFrameGroup, or GetLatestFrame within the FrameGroup. The thing is: I started to like GetLatestFrame, as it would get me the newestup2date frame, even in situations were I can't keep up (due too starved CPU). It would then automatically flush the buffer, if I understood you correctly.

Latency is most important in my application, so it would be nice to automatically get the newest FrameGroup (and discard all others, if there are some). Is this happening, or is the "GetLatestFrame" feature indeed missing when working with synchronized groups?

[LinusA]

The problem is more serious than I thought

Even if I call FrameGroup->GetFrame() as quickly as I can, a "backlog" inside the frame-queue builds up. This causes a significant lag with my application, the markers are visible "behind in time". I was very glad that I had solved this problem by using GetLatestFrame(). This caused small variations in my processing frame-rate to have no bad affect.

Ok, what do I do now?

• Can I somehow get the equivalent of Camera->GetLatestFrame() with a FrameGroup?
• What if I keep using Camera->GetLatestFrame(), without cModuleSync and FrameGroups. Are the frames synchronized? Can I check this (same FrameID means in sync)?
• Could I use a FrameGroup for my real frame data, but somehow just throw away old frames (by using GetLatestFrame and ignoring the result?). I tried this, but as your docs say: As soon as I use a FrameGroup, GetFrame() and GetLatestFrame() don't work anymore.
• Or is there not other way, but to create a separate thread that keeps on requesting synchronized frames, and that provides those frames for the rest of my application?

Thanks so much for looking at this!

[beckdo]

Hey LinusA,

I think you're getting caught up in the details a little too much here. The Camera->GetLatestFrame() is really just a helper function that empties the que for you giving you the latest frame, you could write this function yourself. Actually here, let me just post the code for the actual function so there are no misconceptions about what is exactly happening when you call GetLatestFrame()

Code: Select all

Frame *CameraLibrary::Camera::GetLatestFrame()
{
    //== Flush que, yielding the latest frame ==--

    Frame* frame = GetFrame();

    if(frame==0)
        return 0;

    Frame* temp = GetFrame();

    while(temp)
    {
        frame->Release();
        frame=temp;
        temp=GetFrame();
    }

    return frame;
}

And as a result, I could potentially write an equivalent function for the cModuleSync so you have the same thing for frame groups. Here is a function that provides exactly the GetLatestFrameGroup() functionality you're requesting, since it's not part of the cModuleSync class, I'll just pass in a pointer to the cModuleSync object:

Code: Select all

FrameGroup* GetLatestFrameGroup(cModuleSync *SyncGroup)
{
    //== Flush que, yielding the latest frame group ==--

    FrameGroup* frameGroup = SyncGroup->GetFrameGroup();

    if(frameGroup==0)
        return 0;

    FrameGroup* temp = SyncGroup->GetFrameGroup();

    while(temp)
    {
        frameGroup->Release();
        frameGroup=temp;
        temp=SyncGroup->GetFrameGroup();
    }

    return frameGroup;
}

Does that help?

My inclination is to drop the Camera->GetLatestFrame() function from the SDK because it seems to become a point of confusion for users. It really is just there to prevent users from having to do the work of emptying the que themselves when all they really want is the latest frame data available at the time of the call.

D

[LinusA]

[quote=NaturalPoint - Doug]Hey LinusA,
Does that help?
[/quote]
It did indeed, thank you so much! You saved me the hustle to implement my own frame-getter-thread.

It's all very obvious what you posted -- it makes perfect sense. I don't really get why I didn't come up with this myself in the first place. The reason might be that the SDK functions are not well documented, so I couldn't imagine what those functions "really do". Also there is no explicit documentation of a frame queue and how big it is (i.e. how the whole concept works).

Anyway, I've got a much better understanding of everything now. I've been working with your cameras and SDK on a daily basis for three months now, and I must say I enjoy it very much. Thanks!

Linus

[kb_kat]

Hello, NP!
I use the v120:Slim camera for my application. Namely, i need to synchronize the camera shot with a IR-LED impulse, lets say every 1 second. The impulse lasts some 20 ms and the camera is set to 100 fps. So here is the code of what i do:

Code: Select all

// read serial until the signal comes (it comes every 1 second)
// .... 

Frame * frame = NULL;
camera->GetLatestFrame(); // clear the queue
while(!frame)
  {frame = camera->GetFrame(); }
frame->Rasterize(framebuffer);
frame->Release();

This works all right for the first 10 seconds (9 frames). After that the algorithm never leaves the "while" loop, i.e. "frame" always remains NULL.
If, say, the period is 0.5 sec then the problem starts after 5 seconds correspondingly (making again 9 frames altogether).


Here is another approach i tested:

Code: Select all

// read serial until the signal comes (it comes every 1 second)
// .... 

Frame * frame = NULL;
frame = camera->GetFrame(); // "while" loop is not needed here since most probably there will be frames in the queue
frame->Rasterize(framebuffer);
frame->Release();
camera->GetLatestFrame(); // clear the queue

This works all right. However i get the synchronized frame from the previous synchronization, which is logical, since before the camera-GetFrame() is called
1 second has elapsed, i.e. 100 frames are in the queue at this point. So i would like to make the first version work. Could you comment what is wrong with it?

[beckdo]

Ok, I'm trying to figure out exactly how you're detecting the IR LED. It seems reasonable that you're just waiting for a centroid to show up. If you have 'SetSendEmptyFrames' to false, then GetFrame would always return NULL until the frame with the IRLED on.

The easiest way might be to send your code/project over and I'll just take a look at it and fix it.

[kb_kat]

Hello, NP.
Thank you for your response.
I partially implemented your idear, namely:
1. The controller (Arduino) lights the LED
2. The camera makes the shots until it finds the object (spot from LED)
3. The camera sends the reply to the controller that the short is done.
4. The controller waits 1 second and lights the LED...(the loop repeats)

So in this way the LED makes impulses.

The code looks as follows:

Code: Select all

int objCount = 0;
while (objCount == 0) // repeat until an object is found
{
	frame = NULL;
	while(!frame) { frame = camera_01->GetLatestFrame(); }
	objCount = frame->ObjectCount();
}
if(frame)
{
	frame->Rasterize(framebuffer);
	frame->Release();
	obj1 = frame->Object(0);
}
WriteFile(hSerial, chBuff, 1, &dwBytesRead, NULL); // send signal to controller to turn off the LED

So the problem is that after just one successful cycle the algorithm stays in the
while(!frame) { frame = camera_01->GetLatestFrame(); }
loop. The frame remains always NULL. This, however does not happen in following cases:

1. When the breakpoints are set and the code is run slowly with debugger.
2. LED are always on, that is there is no permanent turning on/off of the LED.

Could you please help me understand this? Thank you.