AI Tool to create Mattes/Alpha (RVM)

Flame Tools and Tech
, ,
1

Hi everyone, my first contribution here

I just want to show you a tool that i’ve been using these days which I think you all should try…another AI Tool, in this case called Robust Video Matting GitHub - PeterL1n/RobustVideoMatting: Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

I just made a video for all of you to understand this, how to install and how to use it.

This works on linux, mac and windows. I’ve done it on Linux with an RTX 3090 and mac with cpu

Hopefully you find it useful

Here are the steps to install and use it.

1 - Install Python 3.9
2 - Install Miniconda
3- cd RobustVideoMatting
4 - conda create -n rvm python=3.9.0
5- conda activate rvm
6 - download the models and paste them into /RobustVideomatting/checkpoint/
7 - Run these commands on a shell

pip install “av==8.0.3” “tqdm==4.61.1” “pims==0.5”

pip install torch torchvision.

If gpu is 30/40x series=

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

8 - run command =

python inference.py --variant mobilenetv3 --checkpoint checkpoint/rvm_mobilenetv3.pth --device cuda --input-source input/ --output-type png_sequence --output-composition output/com.mp4 --output-alpha output/alp.mp4 --output-foreground output/for.mp4 --seq-chunk 10

This is a free open source tool, maybe someone with good enough programming knowlegde can implement it to run directly from flamy

17 Likes
2

I’ve gotten to the point of running the inference but I get an error “unrecognized arguments”. I’ve tried:
–device cuda
–device cpu

But both give that same error.

3

@bryanb Could you share a screenshot of your terminal command?

4

Hi - Yes.

I’m using a png sequence in a folder called ‘test’

and because we’re using png_sequence I modified all the outputs to be a folder instead of a folder/file.mp4

Screenshot 2024-02-19 at 12.17.08 PM
Screenshot 2024-02-19 at 12.17.08 PM1207×542 55.8 KB

5

@bryanb Yes, if you take a look, at the end of your string you have “- – seq-chunk 10”
so you should remove the first “-” after “outputFOR/”… so it would be “outputFOR/ --seq-chunk 10”

6

Thanks - that works. Can you check your run command above? That’s where I got that extra dash from.

7

Thanks. Fixed!

8

cheers! This is going to be fun testing with my current project… roto with crazy curly hair.

1 Like
9

How’d it go?

10

the new 2025.1 release brought this post backup in my head. I noticed there is an ONNX package on the GitHub for RVM. Would this work with the new inference node?

11

@dave - try it

12

I tried it after posting, but it gave me an error loading the model that said “Model must have single-channel or RGB inputs”

I was going to try converting the PyTorch one later today and see if that worked

1 Like
13

@dave - post your results and happy Thursday brother!

1 Like