How to implement ComfyUI as an Plugin for Autodesk Flame?

Here is an example what it can do absolutely easy

1 Like

not sure i understand what i’m looking at. this was a DMP that you put a camera move on and Comfy is generating all the set extension automatically?

1 Like

Is there a Comfy workflow for VideoPainter?

2 Likes

Not yet. In this case it would be a comfyui custom node but yeh, not yet. Sure in less than a week it’ll be out.

1 Like

Now that’s one of the coolest things I’ve seen in a while :grin: ComfyUI is quite the workhorse, and is only getting better and better. Custom nodes plus just about any new model, checkpoints and LoRA will run with it. InvokeAI is nicer for control, but they are hostile to things outside their own ecosystem. I hope this gets developed further.
How do you deal with malloc when running pytorch and flame at the same time?
Wouldn’t your LAN based system kind of need to be the default?

3 Likes

Kind of what I was thinking as well… basically use another node on the network for processing and push the results back. God knows we’ve got memory issues without adding this to the mix.

2 Likes

Yeah, absolutely. That was my first thought months ago and I ended up using it on an isolated environment or running comfy on the same env but flame closed. It would be really helpful to be able to use comfyui nodes inside flame to perform certains tasks like Clean VRAM Usage node from easyuse, this thing is crucial when the gpuvram is filled after any process has been performed or model has been load-processed-not offloaded. @Xteve Is there a way to use some comfyui custom nodes inside flame?

1 Like

Hi Tim, the footage clip is real Camera footage. I only send the 1st frame to comfy, painted a raw mask, let the magic happen… tracked the wrecked spaceship into the whole clip.

1 Like

The Artist can use Flame AND ComfyUI on the same machine (if enough VRAM). But the real feel comes with a network node of ComfyUI. and this could be

  • a nice equipped AI PC
  • or better a stack of 5x mac minis M4 connected via tb5
    or even your own full blown AI renderfarm.
1 Like

@cristhiancordoba ComfyUI nodes work within ComfyUI only. That’s the main reason I wanted address with that pipeline Aux ComfyUI logic; using tools that are established and available without letting flame deal with.

Available now at my patreon

1 Like

Does it work on Linux or mac only?

Any Flame that has Python 3.11.x in the toolbox. Linux, Apple Silicon and Intel as well

I am updating now weekly very useful fine tuned ComfyUI workflows to use with Flame on my patreon.

1 Like

cool stuff @Xteve ! what hardware you running on? is that Mac?

1 Like

yep I use apple silicon as comfy server and an old mac as flame 2023.3 host

But it work on ANY flame with ANY ComfyUI server. I am working on to refine and evaluate … More news next week

1 Like

I Added a Workflow selection dropdown menu. And if the workflow contains a text prompt input, then the Text Inputfields appear.


6 Likes

Do you think you could show us in more detail how this all works and a few worlflows? Would be great to see it working and how easy it is to download and use various models or workflows.

1 Like