Anyone out there doing "deepfake" VFX?

Hi All,

A place that I was freelancing at recently is looking for someone who has experience using Deepfake-style ML for VFX to help them out on an upcoming job. If you know of anyone who fits the bill send me a message here or at andy@logik.tv

Thanks!

Andy

Brendan O’Neil was into it. He never joined the forum, so I don’t really know what he is up to now. On FB he is Bren Dan.

Hi Andy,
Maybe ‘Revel AI’ can help them out.

I have done some face swapping Deepfakes for an actor through the pandemic. It was super interesting!

Any of the major studios would offer deepfakes. Careful though. I’ve heard of a few rather high profile projects starting a la deepfake then after some poor flame artists couldn’t rescue it they’ve been redone via traditional effects.

How did you do it?

I cut together 2 minutes worth of footage of the actor , some green screen, some footage from movies and other commercials hes been in. It was Tommy Lee Jones so he had lots of footage to deal with. Tried to get as many directions of his face as possible. The biggest challenge was when we needed to see his side of his face and when he tilted his head. At one point we commissioned a shoot of stills to get even more detail but we didnt end up using that due to the job being pushed. Then i used Deepfake software Deep Face Lab and learnt how to use it and trained on the largest GPU i could get my hands on. So i would bid a week to get footage together and train for 1 or 2 weeks. then i would comp it in flame for 3 or so days. If i had the opportunity again i would use a combo if that and keen tools to get a better base to work from. I really enjoyed reading about how ShangChi was made from getting the 3d head matchmoved, then unwrapping it, doing the deep learning and then re wrapping the head… thats prob how id start to research it if i had the chance to get it better. I hope thats helpful

5 Likes

Great info. How did you like Deep Face Lab? Is it a one time purchase or subs?

its free, from github. its a bit to research it so def not for faint of heart. def no support to this software, but theres websites that go into to detail in how to use it and with some practice you learn what works and what doesnt. I wish wwe had access to what Disney is using, they are doing amazing work with deepfakes and blast everything ive seen from larger vfx companies away.

@theoman i asked a guy who did some star wars stuff and this is what he said

Hi Ben! It’s not something I can discuss thoroughly, but there was some proprietary deepfake style software and traditional CG digidouble work in addition to lots of the usual warping and tucking in comp. The kitchen sink approach, really. If you’re interested you can download Deepfacelab from GitHub, it’s an open source deepfake program. The best guides all live in nefarious parts of the internet which I won’t link to here but are only a google search away. Good luck!

Pretty sure Disney hired a youtuber who was doing some incredible deepfakes.

Not sure if I can post this here, else you can delete it. But recently we did also used deepfake in combination with vfx nuke/flame for A G-Star ad.(I did not personally worked on this), There is a behind the scenes video. Making-of | G-Star Raw - Wear Your Denim Till the End from Ambassadors on Vimeo

4 Likes