Better ML face extraction for fast turnaround?

ML Face is okish, but sometimes noisy. Let’s say I have over 200 shots that I need to do face/skin softening in variable lighting, etc… keying won’t always work because of heavy colored lights (music video) And it’s all due in a week.

We have Rotomaker on speeddial. Can we run UHD through Tiktok?!?!? “Soft glow” “light makeup” :smile:

There is a relatively new product, not a lot of people have heard about it, but it’s gaining some traction.

3 Likes

One thing if you just need the whole face:

Don’t use face detection, use the Salient semantic keyer and set the bounding box loosely around the face. It will find the face.

May not always work, but may save the day on some shots without having to go to Nuke or some other tools.

6 Likes

One of our freelancers using Salient! He prefers it. I had 100% forgotten about it

I find the ML face extractor is much more reliable if I stabilize the face first, then make the ML matte and re-animate.

4 Likes

Today is one of those “I didn’t know that but wish I did” moments for me. I had always assumed they were based on the same trained model so results would be identical. Will revisit the salient keyer now.

The Face/Human keyers are searching for specific features. The Salient keyer searches for the prominent object in the ROI (hence the name). As such it works beyond humans and faces and is more tolerant of turned/partial faces or other things that throw off the face/human keyers.

The BiRefNet ML model we’ve been experimenting with, is a much more advanced version of Salient. Unfortunately even though we have ONNX versions now (ones we converted and the authors have published one as well), it doesn’t work well with the current version of the Inference node.

There are some issues with pre and post processing the image, channel order, etc. We’ve tried to work around them with custom matchboxes to pre and post process but it’s not satisfactory yet. Work in progress.

3 Likes

I was just toying with inference models in the Portal. There’s some interesting stuff!

1 Like