It seems like catch 22. Splines derive from masks which are not very accurate and they are unstable so the resulting splines inherent that problem and become more trouble then they are worth to fix. Especially over time.
If we had masks that are accurate as splines need to be, we would not need the splines and if we don’t have masks that are as accurate as splines we have more problems than to do it manually.
Is that regarding TokGan?
Yes. I downloaded their sample from their website and loading into silhouette to see what its doing. They also provide depth maps and few other things.
Yes, that is not solving the problem then. It just provides an extra broom to clean up a mess we didn’t ask for in the first place…
Exactly. Catch 22. I would say current masking solutions I’ve seen like Magic Mask, Mask ML and probably one in Flame are better and more stable than theirs, so I wonder if they can create splines from those or only their masking system. Either way, it would require more stable and accurate masks. Or go with the way smart roto went. Manual, hooman made splines and than automated tracking by ML.
Agree. This is why I liked what I saw in SmartRoto. It wasn’t yet another spin at a solution that is known to be difficult, but takes a different approach so we have an actual alternative, not just a different flavor of the same.
And I understand that the problem is hard and there may not be easy answers. If you look at the first frame spline Mocha creates for Matte ML, it’s madness. That’s not a usable spline any human would want to work with. You would need to run another algorithm over that, that rationalizes the spline, and possibly has same extra tools to make it easier to adapt. Like ranking each CP by confidence, something like the Nuke filters in the curve editor to even things out, break the spline based on regions. We can generate pose graphs, so we can detect the elements of hands to give each finger a separate roto spline. SynthEyes has great tools for filtering poor tracking points, like high error, distant, etc. If you apply some of this logic to CPs from auto-generated splines, we could see some progress.
If someone finds a way of creating the types of Splines humans would create as a starting point, then we’d be going somewhere.
Yes, Mask ML in Mocha is really good for holdout mattes, garbage mattes, occlusion masks, things like that, but not for final roto. But being all in one place saves the time when you need to apply remove module or occlusion mask or thing like that. Same as how I demoed it with magic mask in fusion and doing roto there. So its part of a larger set of tools and workflows, but its not a replacement for final roto , for sure, We a pretty far from that.
Two more videos came out on Nuke Smart Roto.
The first one is from Compositing Academy guy, who is good compositor, but he doesn’t strike me as someone who rotoscopes very much, so video is a bit disappointing.
My commentary would be: Smart roto seems to do about similar as traditional methods, because you always, always stabilize the shot and since there is no big change in shape and you eliminate the problem of motion from the process, it should perform pretty much the same as “smart” roto. It would be differnt if you worked the way you do which is to not stabilize but you always , always stabilize first in roto and than adjust for shape change with motion not being an issue. you match move at the end. What you should have done in the test is stabilize the section you are doing roto on, and than compare to normal roto vs smart roto. Otherwise its just not worth to go with the extra license for smart roto in my opinion. Speaking of which, even if it was better, what Foundry does with license is an insult to already paying customers who are paying for the product.
Second video is from a VFX VIBE Indian guy so he knows a lot more about roto as one would expect. And the video provides much better insight although I am waiting on more info from more people.
well.
for me personally, most allready existing free to use AI roto tools are like 80% of things 20% go to outsource.
is that 20% going to make me spend the money? no? ok
Good point. Weather or not the studios you outsource to will adopt this particular tool and Nuke rates, remains to be seen. But its most sensible to do 80% of the work for garbage mattes, holdouts and all sorts of VFX using ML masking tools and than if you need sub pixel precision to let a specialists studio do it that have the expriance, tools and patience to roto all day, every day. As long as they can be budgeted into a job, its the most sensible solution.