Excuse my ignorance but is this a thing that can be made into a matchbox or python thing that runs in the BG like the ML timewarp? Looking for something that can generate a decent normals of a person for relighting.
2 Likes
that is super impressive. I wonder how temporally stable it would be. That seems to be one of the main issues with this type of AI stuff.
And it is a standard PyTorch model too.
1 Like
Yeh, absolutely. Temporal consistency is key, I play and use a lot of AI stuff these days on win/linux and i’ve realised the most stable ones takes a lot of gpu vram, 40gb minimun and 85gb on a uhd res shot…I think it’s a matter of time for them to optimize these tools.
Even though these tools are just standalone python commands its actually quite easy to use if you know basic cmd/terminal command.
1 Like
pretty amazing
2 Likes
Anyone used this yet?
hoping to test it out in Nuke over the next couple of weeks
They have native plugins for Nuke? I thought it was its own separate app.
I have used it. Its a standalone app with an upcoming public plugin for nuke (theres private beta)
Its pretty awesome, pretty fast, stable and temporal consistent. The only downside I can see is it needs the input to be masked out, so lets say you cannot relight a whole plate environment, but I guess its part of the idea of integrating stuff into another environment
1 Like
The normals it generates are great! can use the web version to have a little test. Source image below with embedded alpha
The strength of this tool is how good it is working with sequence/video files for the consistency across all the frames you need to work with. For single images we have a lot of other free tools such as DSINE, GeoWizard, ComfyUI, etc…but when it comes to temporal consistency SwitchLight outperforms any other tool I’ve seen out there.
SwitchLight is not free but its not that expensive so in real production job we cant complain. Also, for the free tools like DSINE or GeoWizard, would worth a try to get some copycat/FlameSimpleML training to see if it achives temporal consistency.
DSINE: DSINE
GeoWizard: GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image
1 Like
So you basically have to roto your people first? Can’t just throw it some footage?
No you don’t have to. There is a keying option on the web version but haven’t tested it
The input must have an alpha channel either keyed or rotod but needs an alpha channel and can’t be a white solid whole frame. Also, as Myke said, there’s a automatic roto/key human feature on the software to help you with this and save time…of course, it’s an AI automatic solution so dont expect ABSOLUTE PERFECT RESULTS but it’s quite good
I saw a demo of something similar to this in 1998ish at NAB from a company called MTI. It cost $250K and they said it would replace Flame. I asked them how to do a dissolve. It couldn’t.
3 Likes
OK…Beeble is amazing. I just tried their Studio app, and while the UI is a tad buggy, the results are incredible.
4 Likes
This is a mediocre interview with the creator, but the interesting thing he talks about is shooting green screen with flat lighting, and changing it in post to match the BG. As opposed to the DP trying to create lighting for a BG which is usually un-known at time of filming. The guy says that footage originating with flat lighting is best for the AI model.
3 Likes