Ideate, Prototype, Test
So for the last two weeks of the project I set out to record with the help of my team members Nils and Daniel, and the talent of Artemis who is a peer in our studio.
First problem we ran into, the Xsens Motion Capture Suit was malfunctioning and couldn't be connected to the the MVN Animate Software, meaning we couldn't record any footage with it. So we had to use the lesser version. The Xsens Awinda Suit. The Awinda suit does the exact same type of motion capture but is less accurate. The setup is also quite different. as you need to stick sensors in different areas of the body and velcro them to safety.
Which had another problem, the battery life. Because each sensor is wireless, each sensor has it's own battery life, sadly the suit wasn't fully charged so we had less time with the suit.
Once we got the suit running we made as many recording as possible with the idea of using the animations and stick them together with different camera cuts. We had to scrap this idea and do a whole run through due to batter life problems. We had run out of battery life on the gloves meaning the hands had no data in he recording, and since we couldn't review the takes, after seeing the recording we noticed the calibration wasn't very good. This meant that the hands clipped into the legs when in a relaxed position, and some more clipping problems were present. The only footage that did seem to work quite well was the facial tracking data.
For the trailer we used the MetaHuman Animator method of facial tracking, this makes recordings that you can then place into unreal and use as an animation.
This made me discover that the MetaHuman Animator pipeline is more accurate as it records the movements really well, so expressions are more expressive and the movements feel more natural. However, with the metahuman animator method nothing can be done in real time, meaning it wouldn't work for the actual play. But for something like a scripted video, its the way to go. It works quite similarly to making a metahuman identity, but instead from footage and not from a mesh. Here's an example of the trackers from footage to a time I worked on the tech for myself. Now, I don't have footage or screenshots of all the errors I encountered. Because it was Incredibly frustrating, it took 2 weeks away of production and the trailer had to be scrapped in favor for a less entertaining one. While I wanted to work with the the MVN Animate Software I needed the license, but those weeks the XRLab needed the license and so did some other teams. I had an issue with the head and face animation being disconnected from the body, and when I would attach it the engine became laggy or the animation would stop playing. After finding multiple tutorials to try and fix this issue, I managed to find the correct menu and settings to tell Unreal to only use the face animations and not the neck and head in order to let the motion capture data to be the ones responsible to move the neck and head. Other issues I encountered where that the body data wouldn't play correctly when importing it to Unreal. Luckily I fixed this by essentially streaming the recorded data as if it were live, but instead of tracking a person, it just tracked the data of the recorded take. Then I recorded the MetaHuman through unreal with the Take Recorder tool, this way i was able to create an animation of the recording within Unreal. Sadly this fix came in about 1 day too late, as I had already scrapped the trailer idea since I needed this fixed to use it for the trailer. All in all, these 2 weeks helped me learn a lot about how finnicky MetaHuman is still. And not having a very powerful computer will be a problem. As I was working on this project and letting stuff process and waiting for them to complete which would take 45 minutes to an hour per process due to the long takes, unreal would often crash, meaning I'd lose all progress as I couldn't save any of the processing parts. I feel confident that I managed to fix all the issues fir the recording but it was just too late to use it for the trailer.
댓글