How to implement ComfyUI as an Plugin for Autodesk Flame?

Flame Python
128

Thanks Xteve,

Such a tool sounds like it requires constant on going development. With the system you’ve implemented I would have expected a larger user base. Do you have a clear video demonstration of what you created? How it works with features, benefits and limitations? thanks

129

Of course there is a video explaining how to and what if from last year in april:

130

it needs only logical process flow understanding. No magic, no crazy things… just a little bit of processing logic and you can do whatever you want. ComfyUI as Flame’s AI processor can be really awesome in many ways.

131

I can demonstrate it in real time in a real online demo if someone ask me to do it. By the way, it works well any time online as well. 2k, 4k , 6k , 8k footage is fine. 12 and 16k i am not yet prepared for… but could be arranged if a real need comes up.

132

Is there something rolling up sleeves? I can see it. Do we want that? Does it matter?

137

Working on a flame related “self improving” - Knowledge Database for PostgreSQL in all kind of code snippets in all languages C++, Python and whatever script else comes up. This WILL be useful for anyone starting coding scripts or plugins. I am wondering if that could be of interest at all to the community. I only have the older 2025.2.2 ready for tests. Ok, here my Idea:

For my Autodesk Flame examples, the important distinction is not just “store code” versus “store docs.” It is the retrieval contract.

You want the system to answer two very different intents:

  • “Just answer my question quickly”
  • “Give me the exact real material, not a paraphrase”

Those should not use the same retrieval mode, even if they use the same underlying Wisdom database.

I would classify stored code information by evidence level and by retrieval intent.

The key idea

Not all stored code knowledge is equally trustworthy.

For something like Flame 2025.2.2 Python API, I would separate:

  • canonical source material
  • extracted structured knowledge
  • reusable examples
  • derived summaries
  • conversational recollections

Then your AI node can decide how “strict” to be.

A practical classification model

  1. Canonical references
    This is the highest-trust layer:
  • official API docs
  • vendor examples
  • version-specific release docs
  • local mirrored manuals
  • exact code snippets from trusted source files

This is what you want for “real information.”
For Flame, this would be the official Flame 2025.2.2 API examples, signatures, object names, callback names, and supported patterns.

This layer should be versioned and source-attributed.

  1. Verified internal examples
    These are examples you used successfully:
  • working snippets from your own projects
  • tested scripts
  • code adapted from official docs and confirmed in production
  • internal wrappers around the API

This is often even more useful than docs because it reflects what actually worked in your environment.

  1. Structured symbol knowledge
    This is not raw code, but extracted facts:
  • class names
  • method signatures
  • argument patterns
  • return types
  • lifecycle requirements
  • object relationships
  • version applicability

This layer makes quick answers possible without dragging full documents every time.

  1. Pattern recipes
    These are higher-level tasks:
  • “create batch export”
  • “hook into selection changes”
  • “inspect current segment”
  • “iterate clips”
  • “open a dialog”
  • “trigger render/export flow”

These are intent-oriented and often span several functions or classes.

  1. Derived summaries and conversation memory
    This is the lowest-trust but fastest layer:
  • prior explanations
  • chat-derived summaries
  • notes like “Flame APIs are tricky here”
  • human-written memory blurbs

Useful for routing, but not enough for exact answers alone.

That gives you a trust ladder:

  • canonical
  • verified internal
  • structured
  • recipe
  • summary/chat memory

Then the AI can answer differently depending on what you asked.

For “just a question” versus “exact info playout”

I would define response modes explicitly.

  1. Quick Answer mode
    Use:
  • structured symbol knowledge
  • recipe layer
  • summary layer
  • maybe top canonical excerpt if confidence is low

Goal:

  • answer fast
  • keep context small
  • avoid overloading the model

For example:
“Flame 2025.2.2 clip iteration is usually done through X object family; here is the typical entry point and a minimal example.”

  1. Grounded Answer mode
    Use:
  • structured knowledge
  • verified examples
  • short canonical excerpts

Goal:

  • still concise
  • but backed by source-grade material

This is good when the API is tricky and hallucination risk is higher.

  1. Exact Info Playout mode
    Use:
  • canonical source excerpt
  • verified example snippet
  • exact version tag
  • source path / source title / internal archive ref
  • minimal interpretation

Goal:

  • reproduce the real source material
  • avoid AI paraphrase unless requested

For example:
“Here are the stored Flame 2025.2.2 examples and exact method signatures we have for this API area.”

That is the mode you want for vendor APIs where subtle naming differences matter.

I would not rely on one generic semantic search for all of this.
I would add classification fields so retrieval can be filtered before ranking.

Useful classification fields

For each stored record, I would want things like:

  • content_type: doc, symbol, snippet, recipe, transcript, note
  • trust_level: canonical, verified, derived
  • language: python, js, ts, c++, mixed
  • product: Autodesk Flame
  • product_version: 2025.2.2
  • api_area: media panel, clip, timeline, batch, export, hooks, UI
  • source_origin: official-doc, internal-project, client-project, conversation, generated-summary
  • exactness_mode: exact, normalized, summarized
  • tested_status: untested, tested, production-proven
  • retrieval_cost: low, medium, high
  • supersedes or applies_from / applies_to

Then a query like:
“get me the Autodesk Flame Python API examples for Flame 2025.2.2”
could automatically prefer:

  • product = Flame
  • version = 2025.2.2
  • language = python
  • trust = canonical or verified
  • content_type = snippet or doc
  • exclude chat summaries unless nothing else exists

That is how you stop “search overwhelm” while still getting real answers.

I think the most important distinction is between identity and interpretation.

For tricky APIs, store both:

  • identity records: exact snippet, exact signature, exact excerpt
  • interpretation records: what it means, when to use it, caveats

If the AI only retrieves interpretation, it may be helpful but unsafe.
If it only retrieves identity, it may be precise but not usable.
You want both, but separately typed.

For example:

  • Identity:
    flame.PyClip.get_something(...) exact snippet from source/manual
  • Interpretation:
    “This only works in Flame 2025.2.2 when called from X context, not during Y callback.”

That separation is extremely valuable.

How I would route user intent

The node could infer or accept a retrieval mode from phrasing:

  • “What is…” or “How do I…”:
    default to quick or grounded answer

  • “Show me the exact example”

  • “Give me the real API snippet”

  • “Do not summarize”

  • “Version-specific”

  • “Exact 2025.2.2 behavior”
    switch to exact playout mode

You could even let nodes expose a small retrieval policy:

  • fast
  • grounded
  • exact
  • forensic

That would be much cleaner than one-size-fits-all Wisdom lookup.

For your Autodesk Flame example specifically

I would store Flame knowledge in at least four parallel shapes:

  1. Official doc excerpts
    Short exact excerpts from Flame docs, version-tagged.

  2. Exact code examples
    Complete Python snippets from docs or your verified archive.

  3. Structured API entries
    Symbol-level facts:

  • class
  • method
  • params
  • version
  • caveats
  • related examples
  1. Working internal scripts
    Your own tested scripts, tagged by task and version.

Then “just a question” can search layer 3 first and maybe 4.
“Exact info playout” can pull 1 and 2 directly, and optionally 4 if you want practical examples.

My recommendation

If you want this to be reliable, I would define retrieval classes like this:

  1. exact_reference
    Official/vendor or exact archived text

  2. verified_example
    Known working code

  3. symbol_fact
    Structured API facts

  4. task_recipe
    Intent-based workflow examples

  5. advisory_note
    Derived explanation, caveats, conversation memory

Then define answer modes:

  1. quick
    Prefer symbol_fact + task_recipe

  2. grounded
    Prefer verified_example + symbol_fact + short exact_reference

  3. exact
    Prefer exact_reference + verified_example, minimal paraphrase

That would give any AI harness a sane way to answer both casual questions and strict “show me the real thing” requests without blasting the whole archive into every prompt.

The biggest mistake to avoid is storing all of this as undifferentiated “memory text.” Once that happens, your Flame query competes with random client conversations and casual summaries, and the exact API truth gets diluted.

If you want, the next step we can brainstorm is the actual taxonomy for code-memory record types for any AI harness, such as symbol, snippet, doc_excerpt, task_recipe, project_pattern, conversation_fact, and archive_blob.

Just let me know that this is part of what you looking for.
Cheers,
Xteve