A set of new Matte Refinement tools.
These grew out an impromptu need. I was using LTFX Alpha Gen on a production job, and some of the mattes had quite a few holes.
The native Matte Edge and other nodes didn’t really help. I loved the edge, I didn’t love the holes. I needed something to fix the matte that would respect the existing matte edge.
These days, if the tools don’t work, you just take a 1hr detour and build it rather than futz around with tools that don’t want to be helpful.
And so a node that could fill holes while respecting the existing edge was born. After the job was done I asked Claude, well, what other matte refinement tools could be helpful, and I got list of 20 odd ideas.
These are three nodes that implement the ones that were bearing fruits:
Matte Refine: ways to improve and existing matte while keeping it intact
Matte Refine ML: a separate node that uses ML nodes for refinement, both matte edges and temporal stability
Matte Merge: If you have a good core matte, and a good edge matte, combine them. But more intelligently than just at gMask. Do it with actual matte operations to keep the best of both.
List of operations:
- Clean & Crush: remove islands of specs outside of the existing matte
- Fill Holes: Fill holes inside the matte while keeping the edge intact
- Solid Core: Make anything inside the edge solid white
- Contour Smooth: Takes the wiggle and chatter out of the edge.
- Shrink & Grow: The trusted functions everyone needs
- Edge Width: Make the softness along the edge consistent
- Snap to Image: Use the RGB channels to fix a drifting matte edge
- Edge Color Unmix: Despill adjacent, it unmixes the old background color (must run in linear)
- Analyze Frame: Look at the image & matte and pick appropriate defaults for Clean & Crush, Fill Holes, and Solid Core to make it easier to use
- Smart Fill (ML): Like Fill Holes, but decides by hole if it’s a defect or real gap by using segmentation.
- Edge Refine (ML) Uses the VitMatte edge refiner. Automatically builds trimap to drive refine operation.
- Temporal: Calms flicker between frames. Uses optical flow to line up neighboring frames and apply weighted median.
- Matte Merge: Takes picture and two mattes, one which is a great core matte, and one which is a great edge matte (e.g. LTX) and merges them. Not a straight copy but a careful addition of the second matte to the first, after first registering (aligning) the two mattes and only copying fine detail and bridge zones.
Each node can display the input RGB, the matte, a trimap, and highlight the changes it has made.
Where necessary colorspace transforms for models are automatic.
There is a full User Guide with examples and all the settings.
All three nodes run inline (no process & cache) and are performance optimized. Matte Refine and Matte Merge have installers for both Linux & Mac, Matte Refine ML is Linux only at the moment until I can fix performance issues on Mac.
Some sample screenshots (there are many more detailed examples in the user guide). You can download them from the user guide. These were the synthetic test scenes Claude generated to test the code with. But the also help illustrate how it works.
From left to right:
- Matte merge
- Fill Holes
- Contour Smooth
- What changed highlight mode
- Background color unmix
Let me know if you find it helpful, have other ideas, or find bugs.
Over two days Claude Code wrote 38,500 lines of code to make this all happen.