Page 1 of 1
Streaming Delay for Virtual Production (Unreal)
Posted: Wed Sep 16, 2020 2:04 am
by Fabio
Hello,
this is my first post here, hello to everybody!
When working in Virtual Production with mixed reality you need to match video input with tracking streaming. Usually tracking data input is very fast, while the camera signal is delayed by some milliseconds, and when you move the virtual camera is very noticeable. So we use to delay tracking data, by setup an array of values and read it when you need.
I'm wondering is there is already an option in Motive to "delay" this streaming data sent to Unreal, that woukld be very useful, especially if is per rigid body.
Thank you.
Re: Streaming Delay for Virtual Production (Unreal)
Posted: Wed Sep 16, 2020 10:01 pm
by Pixerati
Fabio,
There may well be a simple way to do this in Motive, but I would think you'd have more control over adjustable latency using a simple TQueue container in Unreal.
Re: Streaming Delay for Virtual Production (Unreal)
Posted: Thu Sep 17, 2020 9:57 am
by Fabio
Hello, i am not familiar with TQueue, I don't know what is it for. Mi problem is that I am stucked to Unreal tick/framerate in blueprints, but there is a difference between tracking data and frame from the camera which is less then a frame, can this TQueue solve the problem?
In alternative... what is the way to do this in Motive?
Thank you.
Re: Streaming Delay for Virtual Production (Unreal)
Posted: Thu Sep 17, 2020 2:27 pm
by Pixerati
It looks like TQueue is only available in the C++ API, not Blueprints. (Although I did see a tutorial on how to wrap one for Blueprints:
https://www.parallelcube.com/2017/10/23 ... lueprints/)
Unfortunately I'm not sure you're going to a simple "one dial" solution to this. You will most likely need some kind of queue container in your Blueprint to hold and delay the incoming data until you're ready to consume it. You could maybe use a TArray (which is natively supported in Blueprints) but it won't be as fast to pull the delayed values out the other end as a TQueue.
Re: Streaming Delay for Virtual Production (Unreal)
Posted: Fri Sep 18, 2020 1:29 pm
by Fabio
I found a solution that seems to work very well. I have an array with flow of transform values for every unreal tick. So I look at a "macro" delay and then I lerp between that value and the previous. The alpha value makes the trick. Adjusting the alpha towards one value or the other allow me to sync CG with video in a very precise way.
I'm sure it is not perfect, but it seems to work.