Sammie Roto Panda

Hey everyone
I updated my Sammie Roto fork to use LTX.

You can find it here (for now it is Linux only but I’m working on a win/mac versions too) GitHub - lukaszomasta/Sammie-Roto-Panda: Sammie-Roto 2 fork: SAM3, ViTMatte, MEMatte and LTX-2.5 Alpha-Gen video matting · GitHub

To optimize I fused the transformers.

https://huggingface.co/lomasta/LTX-2.5-22b-Alpha-Gen-fp8-fused

This makes it work better on smaller cards. It delivers results on my 4060 8gb test box.

I mimicked VideoMama UX so you can choose resize, batch size and blending to make it work on any metal. If anyone has questions fire away. Also if you want I can send you a 50gb tarball that is ready to use as is even on airgapped machines. It does not try to pull anything, everythign is predownloaded and ready.

:clap:t2: :clap:t2: :clap:t2:

Another update :slight_smile:

I added CorridorKey 2 to the list and made some other small QOL improvements

CorridorKey matting. The tracked SAM masks are its per-frame hint, so use SAM to pick the subject and CorridorKey keys it. Besides the matte it predicts the clean foreground colour: the new CorridorKey-FG, CorridorKey-Despill and CorridorKey-Despill-Alpha views and export types appear after a CorridorKey run.

  • Internal Resolution: Same thing, resize to process clip. I was able to process 2.3K on my 8gb card so I think we are safe with full 🙂
  • Hint Trust: (0.75 is default) Corridor expects a plate or first frame garbage matte or garbage matte on each frame. As we are using SAM generated matte we can go 0.9 for no motion blur and 0.75 for motion blur elements
  • Window Frames: Leave it on auto unless it crashes because of memory. Larger windows need more VRAM but are more temporally stable. This is pretty much frame averaging.
  • Despill: Despill amount 😉 But honestly… despill in Flame. How God intended.
  • Carry matte between windows: should you use SAM hint matte or CorridorKey generated matte from shot before, use for tweaking

Matte one object at a time. A dropdown next to “Combine All Objects” picks the object to matte; the other objects’ mattes are kept, so objects can be processed and exported one by one.

LTX Alpha: 25-frame batches for short shots (batches must be 8n+1 frames) and 0 crossfade overlap.

One code base for online and offline installs. SAM3, LTX Alpha (NVIDIA) and CorridorKey now download their missing files on first use, like the upstream models, including the LTX-2 and CorridorKey code at the tested commits. Every file is checked on its own and verified by MD5, and files already in place are never downloaded again. Gated repositories use your Hugging Face login and explain what to do when access is missing.

IMPORANT!
CorridoKey needs CUDA. So no mac support sorry. Use Linux or a Win box to handle roto.

Hey!

Just tried it on windows (rtx pro box) .. pretty sweet. Thanks! … One question, would it be doable to add some basic colour management in there? …eg aces in aces out but processing in whatever works best for the model in use?

Easy :slight_smile:

I’ll add exr import and export and colour management dropdown in segmentation tab. It might help especially with corridorkey. Give me a bit.