Planar Tracking - External Mask (Alpha)

Hello my fellow Flamers! First post here. (So hope im doing everything correctly :sweat_smile:)

I’ve stumbled across a problem and i would love to share it with you guys and see if there is an actual way of doing this.

In the plate that im working on, i have to retouch a panel behind some actors. My idea was to planar track the panel by occluding the actors through a mask i generated through AI. But i cannot atach this external mask for the “Exclusion” parameter on the planar tracker, the only option it seems its through a proper Gmask done manually.

Is there any way to actually atach into the planar tracker an external alpha?

Of course theres always a milion ways of doing this shot, but it just came to my mind that this should be somehow possible?

Thanks a lot!

Following, never quite figured it out either.

as far as I know it’s not possible to feed it to the planar tracker as an explicit exclusion like you can with gmasks. I’ve had some luck in the past with just cutting a hole in the plate with the external matte or using pixel spread or something similar to infill the occlusion area with the external matte before tracking, but neither works perfectly.

As far as I recall you must make it within the gmask tracer node. However, it only needs to be rough.

Could you cheat this - temporarily insert an infill matchbox, drive it with your AI mask to make the actor area a solid color. Track, then disable the infill.

Haven’t tried it, but might work.

Incidentally I haven’t figured it out with Mocha either. I just rough in an exclusion mask for tracking.

How odd that both Mocha and tracer are lacking that basic functionality of excluding (or including) based on an input alpha…

I guess when you say planar tracking you mean the perspective grid tracking. Exclusion masks for the perspective grid only work if they are gmasks created or copied into your action/gmask tracer/image node.

I’ll take a look on it next week! I would say this should be at least a new feature implemented. Mostly with new AI era being so present in production workflows.

Thank you so much for the tip. I’ll try to keep you updated on the results

Yeah that’s my exact point. This is something that should be considered to be honest.

ML mask in Syntheyes for the win

I think Mocha can, but last time I watched a tutorial it involved exporting a file sequence of your alpha and sideloading it into Mocha. I just want to pipe a matte into the OFX input. I should probably look into it again.

This should be waaaay easier and intuitive (and possible), for both tools.

Flame Learning Channel
Tracking using Exclusive Constraints - Part 3 of 3 - Flame 2016 EXT 2

This might not help on your shot, but some workarounds could be to try and select the best reference frame without occlusions, turn off the Auto Update Reference button, snap ref and turn on the Occlusions button.

Mocha doesn’t support loading a mask into it as a plug in, I’ve been requesting this on Blackmagic forums and they say they will include that feature eventually, but you can load a rendered JPG or PNG black and white matte from external source once you are in the plug in. More steps than it should be, so I hope Blackmagic adds that feature in the next release. Once can hope. It has been requested multiple times.

Did some experiments.

The ‘exclusive’ button is interesting, but doesn’t work as is. It excludes any gMask shape that is assigned to the same media input. So you need your occlusions as a gMask. An externally rendered matte can’t be considered.

I tried other tricks, like using the matte channel and premult on the surface, using the mediafx and the modular keyer to try to hold out the matte input. But none of the combinations I tried had the desired effect.

A bit of research - tools seem to be split whether hold-out masks are just treated as textures and handled on the pixel level, or if there’s a reason to keep them in spline form for the processing in the tracker. Seems to be a range of situations based on the nature of the tool. That would answer why they wouldn’t just allow a hold-out matte channel.

I mentioned earlier just filling the hold-out with infill. It might work in some scenarios. But it’s different than a true hold-out mask. In a true hold-out mask each tracking point will interpolate it’s next position based on last good data and interpolation settings, until it exits the hold-out. Thus there are no signals interpreted. If you infill the area, the tracker doesn’t ignore those pixels, but just doesn’t have anything to hold on to. And particularly at the boundary it could get stuck.

So some testing would be needed to see if this works. It’s a definite hack.

A more solid solution short of a redesign by ADSK, would be an external python script that can take the rendered matte and regenerate a spline and save it as a gMask. Then you can import that gMask into Action and then the ‘exclusive’ button would work.

It’s something that can be done. It’s not entirely trivial, but possible. Easier for simple shapes, harder when you envision dealing with all kinds of corner cases.

The question is if the extra steps of having to run this outside Flame and then to import a gMask is too much friction and not worth it. You might as well do a quick and dirty garbage matte yourself.

That would be up to you guys to comment on.