Hey all. Just released a ComfyUI that use model GitHub - tpc2233/ComfyUI-TP-OmnimatteZero: Nodes to run native weights of LTX's OmnimatteZero in ComfyUI to do cleanups.
Heres a Tuturial, is a very interesting model i have been using.
Comfy can be installed from Manager or manually.
Amazing @tpo!!! Thanks so much for sharing this. Let’s get you on Logik Live ASAP!
Thanks for the great share. So, do you know a good workflow for upscale that works well? I’ve done a few projects with AI but I have issues like texture loss and blurry images with upscaling. - I’ve seen working workflows that can produce 16-bit EXR images. I think everything is getting deeper.
Very cool indeed.
Did chuckle at Flame, Nuke and After Effects all in there. I was waiting for Resolve to show up 
This is awesome! But i can’t get it to work on my 5080 (16gb vram). Is there a way to make it work (with a smaller model, for example)?
Hi @tpo . Question for you. The workflow is working, but all of my renders come out very noisy. Any suggestions?
Meh. Same. A5000ada and OOM.
@andymilkis noisy you mean random noise bad or in the patched area model created new elements that looks artifacts? I went over to that in the video, this model can reason about and create elements to do seamless transition to link outside the mask region. For example:
if you mask a person to cleanup, and the shadows of that person are outside mask, the chances of the model just create a new person is huge, that would be the most logical to generate a seamless blend to interact with the shadows. That is where the seeds values + mask control come into play.
Try to grab the sample footage and replicate results i showed, will be the fastest way to find out if is installation issue or mask. Attached.
samples.zip (15.0 MB)
@cnoellert @Sean1985 sadly this is a 80Gb++ VRAM model since i port with native weights. Some reports working on 48Gb but with hacks. ComfyUI team has been working on a Quantize version that would allow smaller cards, but, so far, the results are heavy degradation in loss due gguf or smaller precision weights:(((
@CoryJohnson hahahaa bit of diversification lol:) believe or not i dont know how to use Resolve, such a non intuitive app for me.
@dnzyc there is a few, i have my own with my own trained LoRas and fine-tune, i found the best results with custom trained using V2V.
Hi Thiago. Sorry, I should have been more specific. I meant that the area I was matting, which was a macro shot of sand on a beach, was returning results that had a very visible noise pattern that changed slightly over time. My guess that the footage I was using was something outside of the model’s training. I’m going to give it more tests with different kinds of shots.
Thanks again for sharing this!
Hi brothers
it’s great, this is how I would imagine the work to be easier thanks to AI in comfyui. But unfortunately I can’t do that on my graphics. Only if I reduce the sample and shorten the length of the shot. But I don’t want to.
P.S. nice greetings from Prague
This is so cool ! Too bad it require so much VRAM
It won’t work on my gpu so I am not even try to get the workflow
one thing , does it haves steps , if so maybe you could try higher step number ?
Until there is something more gpu friendly
video 01 is creating clean frame , video 02 is using that clean frame and models to clean the video
For VRAM optimal version, there is this version:
that comfy team are getting, im not following or using but is there also.
The way they are building to be open to quantize and/or lower precision versions of the models, while mine, is a direct port to stay true as most as possible to the quality results of the original research
This model works best with .mp4 files. Does a sequence of 16-bit access files work well? Or do I need to convert the access files to 709 recordings?Using LoadEXR in ComfyUI, the model works well with this Acess color space.
I tried testing this on Apple Silicon, can’t seem to get it to work. Does this only work with CUDA capable GFX cards?
Cuda (Nvidia) only yes, maybe community might do a port in somepoint me personally dont have plans for port for MPS (Apple)
I’ve got a somewhat stable MPS port going. Opening a pull request now….
Output is ok for the test shot I worked.
Request is here. It’s not the fastest on MPS… but it works.
[OmnimatteZero] Aligning: 24f×1072h×1280w → 17f×1056h×1280w
[OmnimatteZero] Object removal: 17 frames @ 1280x1056, 30 steps
[OmnimatteZero] condition_latents shape: torch.Size([1, 128, 3, 33, 40])
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [11:00<00:00, 22.03s/it]
Damn @cnoellert you’re on a roll man!
Thank you @cnoellert ! This community is amazing.
I’m noob at this and not all that technicial. How would the pull request work? Do I need to redownload the repo from Github? Will google to figure it out.