Block visible markers using SDK

Post Reply
JeDi
Posts: 41
Joined: Thu Mar 10, 2011 3:59 am

Block visible markers using SDK

Post by JeDi » Fri May 29, 2015 2:12 am

Hi,

I want to implement the same functionality as the "block visible" button in Motive, using the camera SDK. I can see the methods available (SetEnableBlockingMask and the ones underneath), but I cannot find any documentation or examples.

Is the functionality I want available using the SDK? Or do I have to implement it myself? If so, I guess I have to detect the visible markers in precision mode, and then iterate over the markers' masks to block them?

Greetings,
JeDi

steven.andrews
NaturalPoint Employee
NaturalPoint Employee
Posts: 555
Joined: Mon Jan 19, 2015 11:52 am

Re: Block visible markers using SDK

Post by steven.andrews » Fri May 29, 2015 11:41 am

Hello JeDi,

Our Camera SDK provides all of the low level functionality to create masks and apply them to the cameras.

We don't have a sample application that demonstrates how to create images, but we can describe some details here:

Based on the location & size of centroids on the camera image, you can call
camera->AddBlockingRectangle for each rectangular region you would like to mask. Then call
camera->UpdateBlockingMask() to apply the updated mask to the camera.

In order to draw something besides rectangles, the approach is to create a 'mask image' which is 1 byte per mask pixel. The mask image is a lower image than the camera's resolution. You can determine how much lower by using
camera->BlockingMaskWidth and camera->BlockingMaskHeight.

If you want to leverage the Camera SDK's Bitmap class to prepare the image and also give easy access to drawing things like lines, rectangles, and circles then you can instantiate one with something like:
blockBitmap = new CameraLibrary::Bitmap(camera->BlockingMaskWidth(),camera->BlockingMaskHeight(),0,CameraLibrary::Bitmap::EightBit, ,0));

Then you can draw on this bitmap using the CameraLibrary::Bitmap pixel, line, rectangle, and circle functions. Then apply the mask with:
camera->SetBlockingMask(blockBitmap->GetBits(),blockBitmap->PixelWidth()*blockBitmap->PixelHeight());
camera->UpdateBlockingMask();

You would likely want to have the camera in SegmentMode or PrecisionMode to observe the shape of the centroids. Camera centroids are reported by the Camera SDK as cObjects. When the camera is in SegmentMode or PrecisionMode, the objects also expose a linked list of line segments that describes the shape of the centroid. Calling object->Segments() will give you the head of the linked list. You can then walk through that list to get a list of line segments that accurately describes the shape of the centroid. Presumably that could be used to draw a similar shape on the mask.


I hope this information helps with your attempts to create camera masks.

Best regards,
Steven
--
Steven Andrews
OptiTrack | Customer Support Engineer

JeDi
Posts: 41
Joined: Thu Mar 10, 2011 3:59 am

Re: Block visible markers using SDK

Post by JeDi » Fri May 29, 2015 11:50 am

Thanks a lot for the detailed info, that will get me going.

When I finish the code, I will share it here, because I think a simple "block visible" method would be helpful for a lot of people.

Greetings, JeDi

steven.andrews
NaturalPoint Employee
NaturalPoint Employee
Posts: 555
Joined: Mon Jan 19, 2015 11:52 am

Re: Block visible markers using SDK

Post by steven.andrews » Fri May 29, 2015 12:05 pm

Thanks JeDi, that would be incredible.

If we could have your permission, I would also like to suggest we include it with the samples we provide.

Steven

JeDi
Posts: 41
Joined: Thu Mar 10, 2011 3:59 am

Re: Block visible markers using SDK

Post by JeDi » Fri May 29, 2015 12:14 pm

Of course, that will be no problem.

JeDi
Posts: 41
Joined: Thu Mar 10, 2011 3:59 am

Re: Block visible markers using SDK

Post by JeDi » Fri May 29, 2015 3:40 pm

OK, this method seems to work (I only did a quick test with one camera though):

Code: Select all

void CameraManager::blockVisibleMarkers(CameraLibrary::Camera *pCamera, CameraLibrary::Frame *pFrame)
{
    assert(pCamera && pFrame);

    // Make sure the hardware mask is enabled
    pCamera->SetEnableBlockingMask(true);

    // Determine hardware mask size (can be different than image resolution)
    int maskWidth = pCamera->BlockingMaskWidth();
    int maskHeight = pCamera->BlockingMaskHeight();

    float scaleWidth = maskWidth / (float) pCamera->Width();
    float scaleHeight = maskHeight / (float) pCamera->Height();

    int nObjects = pFrame->ObjectCount();

#if DEBUG_BLOCK_VISIBLE_MARKERS
    cerr << "[CameraManager::blockVisibleMarkers] Camera #" << pCamera->CameraID() << " blocking mask size: ";
    cerr << maskWidth << " x " << maskHeight << " - mask scale: " << scaleWidth << " x " << scaleHeight << endl;

    cerr << "  " << nObjects << " object(s) found" << endl;
#endif

    for(int i = 0; i < nObjects; ++i)
    {
        CameraLibrary::cObject *pObject = pFrame->Object(i);
        assert(pObject);

#if DEBUG_BLOCK_VISIBLE_MARKERS
        cerr << "  object #" << i << ":" << endl;
#endif
        // Traverse the segments (scanlines) of the object
        CameraLibrary::Segment *pSegment = pObject->Segments();
        while(pSegment)
        {
            int startX = pSegment->StartX();
            int startY = pSegment->StartY();
            int length = pSegment->Length();

#if DEBUG_BLOCK_VISIBLE_MARKERS
            cerr << "    segment: (" << startX << "; " << startY << "; " << length << ")" << endl;
#endif

            // Add the segment to the (scaled) blocking mask
            for(int x = startX; x <= (startX + length); ++x)
                pCamera->SetBitMaskPixel(x * scaleWidth, startY * scaleHeight, true);

            // Next segment
            pSegment = pSegment->Next();
        }
    }

    // Update the blocking mask
    pCamera->UpdateBlockingMask();
}
I use a synchronized frame group, and this is how I get the frames (this is temporary code, but you'll figure it out):

Code: Select all

void CameraManager::blockVisibleMarkers()
{
    // Drain the current frames
    drainCameraFrames();

    // Wait for synced frame group
    //! @todo Timeout
    CameraLibrary::FrameGroup *pFrameGroup = nullptr;
    while(!pFrameGroup)
        pFrameGroup = _pSyncModule->GetFrameGroup();

    if(pFrameGroup)
    {
        int numCameras = pFrameGroup->Count();

        for(int i = 0; i < numCameras; ++i)
        {
            CameraLibrary::Frame *pFrame = pFrameGroup->GetFrame(i);
            blockVisibleMarkers(pFrame->GetCamera(), pFrame);
            pFrame->Release();
        }
    }
}
drainCameraFrames() is just a while loop to get rid of all frames still hanging around, if any. I had to write that because a frame group lacks a GetLastFrames method.

When you don't use a synchronized frame group, you have to iterate over the cameras and query the frames directly.

I hope this is useful to someone. Please let me know if anything looks weird.

Greetings,
JeDi

JeDi
Posts: 41
Joined: Thu Mar 10, 2011 3:59 am

Re: Block visible markers using SDK

Post by JeDi » Tue Jun 02, 2015 12:21 pm

Although the code above works, it only blocks lights that are actually part of a detected marker. When smears of light are visible for example, they won't always be blocked.

I guess this is OK, because the other light is not detected as a marker anyway. But is this also how the block visible button in Motive:Tracker works? Or is the grayscale image used there, in combination with a treshold for the light intensity?

Did anyone else try the code yet, or had a look at it? Thanks!

Greetings,
JeDi

steven.andrews
NaturalPoint Employee
NaturalPoint Employee
Posts: 555
Joined: Mon Jan 19, 2015 11:52 am

Re: Block visible markers using SDK

Post by steven.andrews » Tue Jun 02, 2015 1:03 pm

Hi JeDi,

Motive uses its own high-level methods for masking, but it is essentially equivalent to masking in segment or precision mode. This allows masks to be created over light that is not circular enough to circle fit as objects.

Steven

Post Reply