AI tool for transparent and reflective objects over green screen

Flame Flame Questions
1

Hello. We have a possible job coming up, talent saying lines over green screen, and occasionally holding up a reflective Emmys statue or a transparent/translucent champagne glass, with some motion blur sometimes. We can of course do all the Flame tricks to make the objects look good – but I’m curious of any AI tools that might do it faster/better?

It’s UHD footage, but the affected areas would be a small fraction of that resolution.

SammiRoto? Anything else suggested?

Thanks in advance for any insight.

2

CorridorKey?

3

I’ve actually pulled usable hair from a horrible blue screen using automatte. It’s always a first go-to now. I plan on testing it out to a greater degree the next time I get a blue/green screen. I personally have no experience with copy-cat, but I’ve seen some pretty convincing stuff come out of it.

4

I would think that the old school method would work here for such things. The additive, subtractive key, multiply background or however you want to call it. Here are some examples. It will preserve even the tiniest hair detail and all the motion blur, but it is very sensitive to even lighting of the blue or green screen. if you don’t have an even lighting you have to add some manual work.but its still the best way to get all the details from the original plate, In my opinion. Perfect for semi transparent, motion blur, out of focus type things.

If its just a small segment of the overall screen you can probably extract just that area, track it and keep it stable and in same small frame, and than revert transformation and motion once you are done keying or whatever.

THE ADDITIVE KEYER HACK

Logik LIve #124: Flame Keying Tips with Inti Martinez

…

Regarding using AI tracking, I suspect most tools on the market that we use have same underlaying models or similar ones with each company adding their own things on top. So one thing that seems to apply to all or most of them of them is that if you are applying ML segmenation tools for tracking and masking like SammiRoto, Magic Mask (Blackmagic) or Mask ML from Boris FX they will benefit from having your subject fill out about 2/3 of the screen.

I think in Flame there is always a bounding box to sort of restrict that. Not sure about SammiRoto, but I know Boris FX with their Mask ML tools in recent updates have added Auto ROI or Region of Interest which adds auto bounding box to keep the subject about 2/3 of the area being searched by the algorithm, and this gives more precise results.

I can illustrate this with Magic Mask in fusion. If I track something small on screen and link pivot point of transform tool I can make it fill in the screen, and keep it in center of the screen. I apply magic mask than to what is now about 2/3 of the screen and I get much sharper and consistent mask and than I just invert transform and merge it back over original.

Here are some examples to what kind of difference it can make. Comparison between just a small object in the shot tracked and when that same objected is made to fill out the screen and tracked. In your case like Emmy statue in hand, would probably benifit from that kind of appraoch, whichever of the ML tools you use, since I think they all stem from same source.

…https://ibb.co/yBhHmBLXhttps://ibb.co/zWs4BY8V

…

For green or blue spill, I’m sure you can use flame tools, personally I use in fusion what I think is probably the best tool on the market, despiller plus.

It is awesome third party free fuse for fusion that I’m sure can be done in flame or something similar exists, but what it does is will not only despill color, but mix the original background input with the foreground and can restore luminance values that are lost after despill. This makes it very easy to apply and yet its very powerful for any kind of despill process.

Here is one example, that might be similar to your situation with glasses and Emmy statues.

Anyway, that would be my recommendation. I would try additive / subtractive method in flame first and than if that doesn’t work, or needs refinement, I would use any of the ML masking tools on a object you need to track, but fill in the screen as much as possible for better more accurate results of masking and than if it needs despilling I would use something like despiller plus or Flame equivalent. If you set it up as a template I think you can work relatively fast and get great results.

5

@GPM looking at the shots what’s your biggest concern? Keying/roto issues or spill and integration?

6

Thank you, Kuno. This is all great info, some of which I knew and some not.

It seems the best solution found by @Mihran (who is doing the actual work for me) is Flame + Silhouette. But all of the above are solid approaches.

7

I’ll also chime in that the other day I needed to add a crowd behind a glass backboard on a basketball court. I used comfy. I was stunned at how well the crowd tracked and how well it recreated the streaks and glare on the backboard. I emphasize the word “recreate” however. Not “retain.” It changed the camera move a tiny bit and the clapping arms were barely convincing, but the comp through the glass was amazing.

8

Thanks everyone for all the information. It’s all very useful, and there are definitely some great approaches here.

For this particular situation, though, I think the winner is a combination of Flame and Boris Silhouette, using their ML tools for the initial keying, and of course the additive keyer, which is still hard to beat for motion blur and semi-transparent areas.

The other AI tools are definitely good as well, but for a high-end production workflow, I don’t find them quite as powerful or flexible. To get a really clean result, they can also require quite a bit of manual work and time.

With Flame + Silhouette, having all those tools available together gives me a very strong result while keeping the workflow efficient. For this particular job, that would definitely be my preferred approach.

9

I still miss keylight….