Muscle driven face mocap

[evengi]

Hi!

I'm currently wrapping up an intensive animation project.
I've done 13 episodes, duration 1-2 min each, with both face and body capture.
Translating our mocap to cartoony characters. Each episode done in one to two weeks.
I've build a custom muscle rig for the face, so my mocap drives this rig and deforms their faces, even though proportions are quite different from actor to model.

12 cam body and 6 cam face.

Here's one of the episodes (I handle all visuals, not script) :
http://www.evengi.no/Evengi/Gallery.html#1

Roger

[DonBurroni]

Roger that was great,

How are you animating the face, I would love to see the custom muscle rig for the face.

All the best
Hogan

[morgan]

lmao! bloggerne - thats a great piece - priceless.

also love the tegneby stuff!

[evengi]

-Don Burroni.
The data from my mocap is the 41 marker setup, including the four head rotate markers.
I export all markers with no head movement, then delete all unnecessary movement. ( I think I have a bad camera, getting a lot of shaking, even with excellent all over)
Then I import my face data to Maya, Scaling and positioning the markers to best fit the model.
I've made muscles for the cartoony face. Some are Mayas standard muscles, and some are custom build, like the circle muscles around the eyes and mouth. These muscles are simply driven by the markers position. I can blend the motion in and out, If say, a mouth pose is to extreme and deforms my model in a unnatural way, I fade it out.
In the end I animate on top of the markers with Maya animation layers, giving them extreme poses, like a smile un-impossible for a human being to achieve.
The muscle contraction triggers a series of displacement maps that blend into each other and create wrinkles.

I'm attaching a picture for you. Also I'm finishing a showreel tomorrow, and I'm explaining about the rig there, I'll post a link then.

http://www.evengi.no/delt/Poster/MuscleTransp.jpg
http://www.evengi.no/delt/Poster/MuscleSide.jpg
http://www.evengi.no/delt/Poster/MuscleFront.jpg

-Morgan, you've been on my homepage? He he, thanks for checking it out!


Here are some more episodes, but I must warn you, this may be offensive to some!
I'm NOT writing this stuff, only animating it!
(the first few episodes was done without the rig, used arena standard face rig)

http://www.tv2.no/play/underholdning/kjendisbloggerne

[evengi]

Hi Guys, here's a clip from my showreel, showing the rig in action.

http://www.evengi.no/delt/Poster/FaceRigg.m4v

Roger

[morgan]

Roger -

Thats a great video. What's the lag on the Maya muscle solver? Is Maya solving in real-time, or is there a significant delay?

Reason is, we're looking at features for our next version of Expression and wondering about adding live streaming markers to our Maya VCS plugin.

Would you use a live data pathway or are you happy with the current Expression -> File -> Maya route?

Also, how clean is you're data now, and what are you using for cleanup? Arena?

great work!

Morgan

[evengi]

I'm sorry for my late answer, I'm starting a crunch period now.

The lag on the muscle solver is quite significant. I'm on a fast mac a real workhorse!
Depending on the mesh driven by the muscle, you can have everything from 1-7 frame second with/25fps.
(I'm using arena in bootcamp, and I't plays beautifully, In case any wonder about this)

My face data is not clean at all. As I said, I think I have a bad camera. I get excellent all over and still at lot of flickering, even with dots placed on rigid objects standing on the ground. My light conditions and space conditions is terrible, doing your system unjust. Or maybe showing how great it is, making me able to capture face and body in no more than 24 square meters, with low ceiling and down lights! I think I could get great data, by moving to a better location, and changing my cameras.

I'm using arena for clean up, great tools! So fast!

I't would be great to stream data into maya, since Maya is my backbone in production.
Testing limits of deformation live and so on, on the face, would be a great asset!
The great thing about streaming data, is that text writers and directors easily can see the character in action, and don't have to imagine.

The file > maya workflow has worked ok for me, but streaming would be even better.

Send me you mail address, and I'll upload link with rig for you guys to test. I might upload it for the community later also.

Roger

[morgan]

OK great.

Maya doesn't have the kind of marker cleanup tools that Arena or MotionBuilder do so it seems like the file pathway for post would still be required, or we just track/stream perfect data .

Depending on what kinds of framerate can be achieved and what is needed, perhaps we could work out some kind of simple cleaner script in mel that at least processes/cleans some of the data (like simple swaps?) as it comes in to keep a pre-vis face holding together.

Also, the new S250 cameras are much higher resolution, and combined with the right lenses could significantly reduce the need for cleanup.

Are you using V100 R2 cameras now? What lenses are you running?

[DonBurroni]

Roger thanks for the insight,

I would love to have a look at your rig too,
Great stuff
Burroni

[evengi]

Sure!
I'll try to upload the rig without face models in a couple of days. That mean you'll have to bind and paint weights yourselves. This because I can't give away the models I've made for my company. But it's quite straight forward, and the rig comes with notes.
In december I will, if time allows it, try to make a tutorial.

Roger