Flame Machine Learning Timewarp, now on Linux and Mac

masaki · June 4, 2021, 5:05pm

I compared a mac flame that works correctly and a mac flame that does not work.
The name, clip and path are the same.
Shot name is A. The path is Desktop(Users).

Here are the logs for a correctly working mac ↓

initializing Timewarp ML...
Trained model loaded: /Users/work01/Documents/flameTimewarpML/bundle/trained_models/default/v2.4.model
---
Free RAM: 92.5 Gb available
Image size: 1920 x 1080
Peak memory usage estimation: 4.1 Gb per CPU thread 
Using 18 CPU worker threads (of 20 available)
rendering 9 frames to /Users/work01/Desktop/ML/A_TWML4_2021JUN05_0148_B97/
 56%|███████████████████████▎                  | 5/9 [00:13<00:10,  2.73s/frame]

Here are the logs for a non working mac ↓

initializing Timewarp ML...
usage: inference_sequence.py [-h] [--input INPUT] [--output OUTPUT]
                             [--model MODEL] [--UHD] [--exp EXP] [--cpu]
inference_sequence.py: error: unrecognized arguments: 605_0156_CED/source 605_0156_CED

I didn’t use space for either of these things.

claussteinmassl · June 4, 2021, 5:33pm

So I wonder: where do these come from? „ 605_0156_CED/source 605_0156_CED“

talosh · June 4, 2021, 9:06pm

Hi Masaki, sorry for letting this things with spaces to stay for a bit too long, it is a technical debt I need to fix and I really hope it will be fixed in one of the next of upcoming releases. It looks like there’s somehow a space before 605_0156_CED, so it looks like ‘space + 605_0156_CED’. Could you please double check if it is not in the name of the clip you’re trying to export form Flame. If you stuck just let me know and I can try to teamviewer to your machine and have a look

masaki · June 5, 2021, 12:42am

Thanks for the reply！

The name of the folder exported from flame will always have a space in it.

folder name : A_TWML2_2021 605_0053_FB6

shotname_TWML2_Year +sapce Day_ HourTime

Do you know how to prevent this space command from being issued?

Thank you !

talosh · June 7, 2021, 11:04pm

Hi Masaki, could you please check if this version solves that problem?

masaki · June 8, 2021, 10:12am

Hi talosh.
Perfectly done with the process!

This tool is the most powerful of all time warp tools.
Thank you so much for your help.

MikeyBrown · June 24, 2021, 12:09am

Hi @talosh, wondering if you can shed any light on this error. only happens on 3 out of 5 clips with varying timewarps. Using the beta flame TW effect. v2.4

Received 1 clip to process, press Ctrl+C to cancel
Initializing TimewarpML from Flame setup…
“invalid literal for int() with base 10: ‘10.999999’ in <class ‘ValueError’>”
[‘Traceback (most recent call last):\n’,
’ File ’
'"/opt/Autodesk/flame_2021.0.2/flameTimewarpML/bundle/inference_flame_tw.py", ’
‘line 520, in \n’
’ frame_value_map = bake_flame_tw_setup(args.setup, args.record_in, ’
‘args.record_out)\n’,
’ File ’
'"/opt/Autodesk/flame_2021.0.2/flameTimewarpML/bundle/inference_flame_tw.py", ’
‘line 327, in bake_flame_tw_setup\n’
" tw_timing[int(index)] = {‘frame’: int(frame), ‘value’: float(value)}\n",
“ValueError: invalid literal for int() with base 10: ‘10.999999’\n”]
Traceback (most recent call last):
File “/opt/Autodesk/flame_2021.0.2/flameTimewarpML/bundle/inference_flame_tw.py”, line 520, in
frame_value_map = bake_flame_tw_setup(args.setup, args.record_in, args.record_out)
File “/opt/Autodesk/flame_2021.0.2/flameTimewarpML/bundle/inference_flame_tw.py”, line 327, in bake_flame_tw_setup
tw_timing[int(index)] = {‘frame’: int(frame), ‘value’: float(value)}
ValueError: invalid literal for int() with base 10: ‘10.999999’
Press Enter to continue…

claussteinmassl · June 24, 2021, 4:24am

Not sure if this is caused by index or frame (you could find out by printing both variables), but I guess they are strings containing float values. I‘ve seen this in timewarps where the keyframe is between the actual frames.

If you convert them to float before converting to integer, it should fix the issue:


tw_timing[int(float(index))] = {‘frame’: int(float(frame)), ‘value’: float(value)}

MikeyBrown · June 24, 2021, 5:59am

@claussteinmassl thankyou. in between frames was the issue. not sure how its possible given you have to parked on a frame before you can set one

claussteinmassl · June 24, 2021, 6:03am

Glad it worked!
I had this issue when importing XMLs or AAFs (don’t remember). I would guess it can happen inside Flame, too, if you stretch the track in the animation editor.

talosh · June 24, 2021, 9:33am

Hi Claus, the part that reads and understands timewarp curves is actually using quite ancient ruby code made by Juliik I believe more than ten years ago and amazing that with a couple of patches it really works well. Though that limitation is still there that keyframes has to be integer frame values. I’ve ported some of that to python already and it is a bit of a technical debt to get it done fully. In the meantime the workaround is to rebuild keyframes in flame.

If you know anyone around who would fancy to help with coding it lel him or her pm me!

claussteinmassl · June 24, 2021, 10:38am

Hey Andriy, I only did a quick read of the code (there is really a lot of stuff going on for the calculation! :D), but if I’m not mistaken, the error happens right after reading the flame timewarp setup and before the actual parsing of that data is done. As mentioned above I think fixing the str to int conversion should get rid of the problem:

tw_timing[int(float(index))] = {‘frame’: int(float(frame)), ‘value’: float(value)}

This happens in several places:

github.com

talosh/flameTimewarpML/blob/32e1782231e5b82ad0ab42a79952c65e9c694013/bundle/inference_flame_tw.py#L313

    
      
              TW_Speed = xml.getElementsByTagName('TW_Speed')
              keys = TW_Speed[0].getElementsByTagName('Key')
              for key in keys:
                  index = key.getAttribute('Index') 
                  frame = key.getElementsByTagName('Frame')
                  if frame:
                      frame = (frame[0].firstChild.nodeValue)
                  value = key.getElementsByTagName('Value')
                  if value:
                      value = (value[0].firstChild.nodeValue)
                  tw_speed[int(index)] = {'frame': int(frame), 'value': float(value)}
                  tw_speed_frames.append(int(frame))
          
          
        intp_start = min(start, min(tw_speed_frames))
                  intp_end = max(end, max(tw_speed_frames))
          else:
              tw_timing = {}
              tw_timing_frames = []
              TW_Timing = xml.getElementsByTagName('TW_Timing')
              keys = TW_Timing[0].getElementsByTagName('Key')
              for key in keys:

github.com

talosh/flameTimewarpML/blob/32e1782231e5b82ad0ab42a79952c65e9c694013/bundle/inference_flame_tw.py#L331

    
      
              TW_Timing = xml.getElementsByTagName('TW_Timing')
              keys = TW_Timing[0].getElementsByTagName('Key')
              for key in keys:
                  index = key.getAttribute('Index') 
                  frame = key.getElementsByTagName('Frame')
                  if frame:
                      frame = (frame[0].firstChild.nodeValue)
                  value = key.getElementsByTagName('Value')
                  if value:
                      value = (value[0].firstChild.nodeValue)
                  tw_timing[int(index)] = {'frame': int(frame), 'value': float(value)}
                  tw_timing_frames.append(int(frame))
          
          
        intp_start = min(start, min(tw_timing_frames))
                  intp_end = max(end, max(tw_timing_frames))
          
          
tw_channel_name = 'Speed' if TW_RetimerMode == 0 else 'Timing'
          
          
cmd = parser_and_baker + ' -c ' + tw_channel_name
          cmd += ' -s ' + str(intp_start) + ' -e ' + str(intp_end)
          cmd += ' --to-file ' + parsed_and_baked_path + ' ' + xml_path

github.com

talosh/flameTimewarpML/blob/32e1782231e5b82ad0ab42a79952c65e9c694013/bundle/inference_flame_tw.py#L397

    
      
          TW_SpeedTiming = xml.getElementsByTagName('TW_SpeedTiming')
          keys = TW_SpeedTiming[0].getElementsByTagName('Key')
          for key in keys:
              index = key.getAttribute('Index') 
              frame = key.getElementsByTagName('Frame')
              if frame:
                  frame = (frame[0].firstChild.nodeValue)
              value = key.getElementsByTagName('Value')
              if value:
                  value = (value[0].firstChild.nodeValue)
              tw_speed_timing[int(index)] = {'frame': int(frame), 'value': float(value)}
          
          
if tw_speed_timing[0]['frame'] > start:
              # we need to extrapolate backwards from the first 
              # keyframe in SpeedTiming channel
          
          
    anchor_frame_value = tw_speed_timing[0]['value']
              for frame_number in range(tw_speed_timing[0]['frame'] - 1, start - 1, -1):
                  if frame_number + 1 not in tw_channel.keys() or frame_number not in tw_channel.keys():
                      step_back = tw_channel[min(list(tw_channel.keys()))] / 100
                  else:

What do you think? Unfortunately I have zero knowledge of ruby…

randy · June 24, 2021, 10:46am

I saw this error recently me thinks. Avid aaf. Every frame was a key frame. My solve was to reset them and rebuild them with only a couple key frames and let flame interpolate.

west · July 7, 2021, 7:43pm

Hey @talosh ! I’m digging into this lovely looking tool on the recommendation of @randy as he said it’s amazing. I pulled the latest from GitHub (I’m on a Mac), went through the install without error, but when I try and run it I get this error:

Traceback (most recent call last):
File “/Users/westley/Documents/flameTimewarpML/bundle/inference_sequence.py”, line 3, in
import cv2
ModuleNotFoundError: No module named ‘cv2’
logout
Saving session…
…copying shared history…
…saving history…truncating history files…
…completed.
[Process completed]

It seems it’s not seeing the OpenCV library (my best guess). Any idea as to how I can remedy this issue? I was going to install the OpenCV library using brew but thought I’d ask you before going that route.

Thanks and all the best

West

claussteinmassl · July 7, 2021, 9:04pm

Did you use the installer or did you install manually?

I had the same problem when I installed manually:
The conda env was active, but the pip command that I used to install the requirements.txt still referred to the system pip and not the one from conda (run which pip to see what is called). Therefore the libs weren’t installed correctly. After I used the conda pip opencv and the others were installed as expected.

I still don’t know why it referred to the system pip…

west · July 7, 2021, 9:40pm

Hey @claussteinmass I used the installer (easy method).

Copied flameTimewarpML.py into /opt/Autodesk/shared/python/

And booted up flame. An installer dialogue popped up and it was all pretty much hitting OK from there.

When I try to Timewarp a clip a terminal pops up throwing the error I posted above.

I see there’s a cv2 directory installed: /Users/westley/Documents/flameTimewarpML/miniconda3/lib/python3.8/site-packages/cv2/

But I’m guessing it’s not pointed to properly somewhere along the line?

talosh · July 7, 2021, 11:06pm

Hi Westley,
The easiest way is to get the installer from releases page on github and let it do the job. It should build an isolated anvironment with dependencies needed (you should probably remove the github pull you’ve copied before).
Let me know if it works for you.

It’s been a notion recently on github that having a separate environment for just pne plugin is a bit of an overkill. I atill think the simplicity of getting it up and running should win, but as we’re all moving towards 2022 with python3 being default it might be good to have an option for easier approach.
Currently all deps are embedded into the installer so no internet access or root pw needed, all happens in userspace.

P.S. Iv3 models has been released by the upstream algorythm devs but unfortunately whilst better by pure numbers they suffer visually vs v2 branch. This development lead to the split in the upstream branch. Now its “scientific” vs “practical” and we’re much followiing the latter one though it’s in embryo stage.

P.P.S
I’ve been using interpolation between two stills 90% of the time recently. If its the case for you as well - we’d need a better interface for the thing, I really suffer myself doing complex shots with current approach.
Looks like it should be done in batch and probably old-school furnace repair style should come first, any thoughts?

TonyRichards · July 14, 2021, 8:20pm

This is great and loving it. Is there any chance it will or could be a Batch node at some point?

Cheers
Tony

ALan · July 14, 2021, 8:31pm

https://feedback.autodesk.com/project/feedback/view.html?cap=5afe6c845cb3447ab36ccbd7f0688f84&uf=d2431db839794c3b9c5d5e987e172595&slsid=97243c41218e467d91ff8227dcc095dd

TonyRichards · July 14, 2021, 8:57pm

Yes, thanks for the friendly nudge on that

Topic		Replies	Views
It's a good day - Flame 2025.1 is here! 🔥🚀 Autodesk News update	46	1145	August 8, 2024
ML Timewarp - Tech & Business Flame Questions ml-timewarp	4	401	December 8, 2023
Timewarp ML seems to have stopped working Flame Questions ml-timewarp	8	748	March 9, 2023
Flame Timewarp ML Flame Questions timeline	13	199	May 13, 2025
Flame 2025.2 is here! 🚀🎉 Logik Announcements update	34	1593	January 30, 2025

Flame Machine Learning Timewarp, now on Linux and Mac

Related topics