Anybody have a ComfyUI character swap workflow they like? Beefy VRAM setups welcome. This weekend I wanna play around with driving animation of still frame from a human via reference videos.
I have focused on 97% still frame building but could use some updated ComfyUI character swap workflows.
Thanks!
No special workflow needed.
This Qwen face swap lora Alissonerdx/BFS-Best-Face-Swap · Hugging Face
In combination with this lora eigen-ai-labs/eigen-banana-qwen-image-edit · Hugging Face
With this custom node GitHub - pizurny/ComfyUI-AV-Handles
All of this combined with WanAnimate…
Whole body? I thought this was face only for some reason.
It all depends how you use the tools, even if they are not intended for that exactly propuse you looking…2 weeks ago I used it for a really complex clean up and it was successful! I spent more time dealing with the zoom/distort/warp issue from Qwen Edit than the actual video sampling process…(thanks to the lightx 5steps lora)
Here’s one I use pretty much all the time. I haven’t updated it to Wan2.2 yet… but it’ been bulletproof for me. Change whatever aspects you want to change from the first frame of the sequence. Scale and crop the ref and sequence to what frames your shit correctly and then process. Generates a control net based on Depth and uses that to do its magic. Should run like a bat out of hell on your beefy ass setup.
If your focus has been creating good first frames this should work nicely with a first frame spatially matching sequence and its resulting control net. So you might, per se, restyle the first frame of your control sequence to be what you want, or use the pose info in from the first frame in Qwen to pose your character the same as your control and then use this setup to do move it. I’ve had better luck with depth than pose in Wan, but feel free to try for your self.
Workflow should be embedded in the png, but I don’t know if it get’s stripped off or not so included a zip too.
wan2-1.png.zip (827.1 KB)
So here’s one I started adapting tonight, also seems to just work and should do very much what you’re after. I started throwing in Coco Sequence loaders and inline crops to increase pixel denity but figured I’d pass this earlier (working) version along before I break too much.
Start by bypassing the samplers, set the desired width (ie 1024 for us mortals) and the height is computed based on the AR of the incoming image, set the points for masking the object you want to replace and then use the pose selection to go for DWPose or DepthAnything. I prefer DA–but whatever. Load your reference for the switch and you should be good to go.
Unlike the other one, this one just goes full out on a reference image that can have nothing to do with the first frame of your sequence whatsoever.
My main issue at this point is resolution with my GPU, which shouldn’t be an issue for you @randy
wanimate_v1.png.zip (2.0 MB)
jeebus. Next time I’ll just hire you.
Also, for anyone who wants to try these, the comfy setup is embedded in the screen cap. If you drop the png in Comfy it will load the workflow automatically. At least it should–the zips are definitely clean and tested.
