Streaming Individual Markers

NatNet, VRPN, TrackD, and Plugins
heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Streaming Individual Markers

Post by heath789 » Thu Feb 02, 2012 10:06 am

Hello,

I would like to stream marker locations to Matlab but I am having trouble determining if this is possible after looking through the forums.

I am not sure about the tracked/untracked and labeled/unlabeled nomenclature but I would need the markers to at least be numbered the same from frame to frame. I am using 10-12 markers and no cluster (though I keep one stationary in the frame so Arena can trajectorize).

Is it possible to stream real-time 3D marker locations to Matlab using Tracking Tools or Arena and keep the markers the same numbers from one frame to the next?

I have looked here but again the nomenclature is confusing as to what it actually means.

Thank you!

morgan
NaturalPoint Employee
NaturalPoint Employee
Posts: 200
Joined: Tue Jun 24, 2008 2:01 pm
Location: Corvallis, OR, USA
Contact:

Re: Streaming Individual Markers

Post by morgan » Thu Feb 02, 2012 12:19 pm

Yes - both Arena and TrackingTools stream labeled markers.

RE markers keeping the same number frame to frame, we refer to these as "labeled" markers. For labeled markers to stream you will need to define a tracking model. For TT this would be a RigidBody, For Arena either a RigidBody or a Skeleton.

You can also stream unlabeled markers, however they will not necesarily have the same marker id every frame.

RE MatLab, you can use our NatNet SDK to integrate the real-time labeled marker stream into MatLab using either:

- the exported NatNet DLL and a c wrapper
- using the direct depacketizing example as a basis for establishing a UDP connection to Arena/TT from within MatLab directly (does not require the NatNet DLL).
- with newer versions of MatLab, consume the NatNetML.DLL, which is a .NET component wrapper for the NatNet library.

Refer to the NatNetSDK docs for more info.

hope this helps,

Morgan

heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Re: Streaming Individual Markers

Post by heath789 » Sat Feb 04, 2012 10:34 am

Thanks for the reply, Morgan.

Can you go into more detail about using NatNetML to stream data?

I can load NatNetML.dll into Matlab and I can see the classes in the assembly, but I'm not sure how to make Matlab into a client to receive the streamed data. Maybe I'm not thinking of this in the right way. I am able to get the NatNet samples to receive data so I assume my settings in Arena are okay.

Most of the posts on this type of work seem to be a few years old and I'm not sure what all has changed with updates.

Thank you!

heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Re: Streaming Individual Markers

Post by heath789 » Mon Feb 06, 2012 9:12 am

Just to give a few more details:

Using Arena 1.7.2

Streaming pane settings:
IP Address = current full address for the computer (not 127.0.0.1)
Command port: 1510
Data port: 1511
Set to Unicast, stream Right Hand Coord, 3D Marker Positions, and Rigid Body Data.

I'm trying to read this data on the same computer with Matlab. I try to set up Matlab to 'listen' to port 1511 with the same IP address and it times out every time. I'm not sure how to handle this data in Matlab but I am just trying to receive something for now.

I can see the packets when I run the Packet Client in the Nat Net SDK 2.2 samples.

Maybe I am missing a step? Where does NatNetML come in to play?

Thank you.

morgan
NaturalPoint Employee
NaturalPoint Employee
Posts: 200
Joined: Tue Jun 24, 2008 2:01 pm
Location: Corvallis, OR, USA
Contact:

Re: Streaming Individual Markers

Post by morgan » Mon Feb 06, 2012 11:55 am

From Matlab, you have 3 separate options:

1. Connect via a .NET assembly. This is our NatNetML.dll. Discussion of how to consume .NET assemblies in Matlab is beyond the scope here. I would reference this:

http://www.mathworks.com/help/techdoc/m ... pb5k6.html

Also reference the NatNet Winforms sample for how to use the NatNetML assembly in a generic .NET environment.

2. Connect via direct UDP. For this example, no NatNet library is required. The NatNet direct depacketization sample (PacketClient.cpp) shows how to do this in C code. You would need to port this to the MatLab language. Since MatLab's native language is very C/C++ like, this should be *somewhat* straightforward. USe the Depacketization sample as a guide. I would:

- First try in multicast with the deault port assigments
- Connect to the command port (1510). The command port is used for sending/receving commands with Arena, confirming conenction, host info, etc. See the PacketClient.cpp for an example.
- Once connected, you can then use the data port to listen for packets. Again the PacketClient.cpp shows this.

Note in Unicast a direct connection must be established first, whereas in multicast this is not required. Regardless, it is suggested your client code establish and confirm a command connection first. Again, the PacketClient.cpp sample in the NatSDK illustrates the correct order of operations.

3. Write a C callabale DLL that Matlab can connect to. This would be if you are comfortable writing C code and calling DLLS from other environments.

I would pick the apporach that fits the programming environment you are most comfortable with. If none, I would pick the .NET assembly route myself.

Does anyone here have a MatLab sample they're willing to share here?

thx!


Morgan

heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Re: Streaming Individual Markers

Post by heath789 » Mon Feb 06, 2012 8:51 pm

Thanks for the info.

Regarding the method of using the .NET assembly:

I can make Matlab recognize the NatNetML.dll file and can get the properties and methods for each class within NatNetML except for NatNetClientML. Every other class gives what seem to be valid properties and methods.

Can you think of a reason for this? I see that the sample code for the WinForm example uses the NatNetClientML and FrameOfMocapData classes and then uses those to make the connection to the server (Arena in my case) and process the marker data in each frame received.

I am unsure why the NatNetClientML class does not work with Matlab since the other classes do and the WinForm executable works with my machine.

Thanks for your help.

morgan
NaturalPoint Employee
NaturalPoint Employee
Posts: 200
Joined: Tue Jun 24, 2008 2:01 pm
Location: Corvallis, OR, USA
Contact:

Re: Streaming Individual Markers

Post by morgan » Tue Feb 07, 2012 1:45 pm

OK - there may be an issue here with MatLab's .NET support. We'll need to try and get MatLab in here for testing, or work with them directly.

In the interim - can you use the direct depacketization / UDP approach?

heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Re: Streaming Individual Markers

Post by heath789 » Tue Feb 07, 2012 2:30 pm

I have had no luck transforming the C++ code from the Packet Client sample to Matlab using the udp() function or the simulink block 'udp receive'.

I am attempting to use java within Matlab to receive the multicast sent from Arena. I have the data port set to 1511 which is what I am 'listening' for with the java commands in Matlab.

For reference, this is what I am attempting: java.net.MulticastSocket

I can receive but only zeros are getting returned to matlab.

heath789
Posts: 41
Joined: Wed Aug 10, 2011 7:42 am

Re: Streaming Individual Markers

Post by heath789 » Wed Feb 08, 2012 10:41 am

I am able to read in the packet sent by Arena, which is a series of 8-bit integers.

I cannot tell from the C++ code example how to get the values for marker position, etc. Small values like the number of rigid bodies can be read directly but I don't understand how to get the floating point values for marker position.

Thanks for any help!

morgan
NaturalPoint Employee
NaturalPoint Employee
Posts: 200
Joined: Tue Jun 24, 2008 2:01 pm
Location: Corvallis, OR, USA
Contact:

Re: Streaming Individual Markers

Post by morgan » Wed Feb 08, 2012 11:00 am

Most values in the FrameOfMocapData are 4 bytes. The rigid bodies ids likely work because your just taking the low 8 bits.

I'm really not too fluent in Java, so this may not be the best approach, but you might try using the Java bytebuffer helper. Something like:

Code: Select all

int currentIndex = 0;   // current index into data buffer
byte[] data = ...      // char buffer from multicast socket read
byte[4] curr;

// for each value...
curr = data[currentIndex];
ByteBuffer b = ByteBuffer.wrap(curr);
float fValue = b.getFloat();  // or whatever that data type is - see packet client.cpp
currentIndex +=4;      // increment by size of value

again - dont quote me on the code. you may find it easier just to manipulate the byte array directly - just pack the values into the corresponding data type using memcpy equiv or shift operators (as in the C++ example).

The byte length of each value in the FrameOfMocapData data can be gleaned from the depacketization sample.

Morgan

Post Reply