It’s really weird… I’ve tried to nail it down. Just did a project, smaller one. No freezes. Also working on bigger project (same 2023.1 version) … too many freezes. I’ve been suspecting gmasks for a while as it -feels- like as soon as I start adding many things start to happen…
Sometimes, running strace on flame prevents the freezing…
strace? Can you tell me more please?
I also have been on a new and smaller project, and no freezes. Switch back to an older and bigger project, constant freezes to the point that it’s not a very viable tool to work with.
2022.3.1 Centos
Hi,
Strace is a Linux troubleshooting command.
Although running the command may prevent some issues from occurring, running Flame with strace running is not recommended as it will affect the overall performance.
In a nutshell, running strace slows down everything as each command made by the application to the operating system is recorded and printed in the shell. This slowdown may prevent some issues from happening.
Best,
Yann
Thanks Yann. Yeah I don’t think I’ll be adding anything to the mix to even further slow it down. Sure hope Rocky saves the day after install next week.
Seeing quite some of these in dmesg: [83781.157540] RT Watchdog Timeout: flame[93198] not sure if that has remotely anything to do with it… haha
Update:
I very much wish I could file a different report here.
Rocky Linux, Flame 2023.1, new Lenovo P620, no ML installed, not even any Boris plugins. No third party apps other than Anydesk.
I started a new project, just two batches. The batches are not tiny but they’re not huge. Two 4k source clips total. CACHED to my local storage — not accessing the server.
I still got micro freezes. Possibly less frequent than before, but they seemed to last longer. The difference is that they don’t seem to be prompted by predictable things as before, like trying to access the Batch or Action bins. Just happened seemingly randomly.
I was on a heavy deadline so no I did not check the output logs and did not have time to try to chase down someone in support.
And now, I also had several fatal freezes, hard shutdown required, over the course of 2 days of work. I don’t recall fatal freezes before, amongst all the micro freezes.
This is NO BUENO.
ADSK, is there any progress at all on this? Other than “it’s been fixed” or “it will be fixed?” Is there any new clarity or insight about this?
Also, the autosaves in the version are extremely long compared to before. Worse than the micro-freezes themselves. For a tiny project with relatively small batches. What is up with that?
Hi Greg Paul,
Thanks for reporting again on the subject. We are very much invested in understanding this issue, but to do so we need more than this post. Please contact our technical support team and if possible provide a unlinked version of these setups for us to expedite the investigation process.
Are the hard freezes happening during/after using motion vectors? Because there is a GPU memory leak with motion vector tracking that will crash the GPU requiring reboot. Only fix is to have a6000 or rtx8000.
Nope - no vectors we’re harmed in these setups. Just normal comp work stuff.
I opened up a support ticket and have sent ADSK the requested files. Hope I can help.
Got it, thank you !
This makes for really depressing reading.
I’m 100% @YannLaforest et al are on the case. RE: strace, this is indeed a pain – sometimes when an application locks up due to a race condition, the use of strace can prevent the issue from occurring. Given that the freezes seem in line with project complexity, for a less-invasive diagnostic rule-out you could run something like iostat to monitor disk device reads/writes – if it shoots up during your freeze then that’s highly relevant. The sheer number of tiny files that get written out these days is hench and I wouldn’t rule out saves yet.
Trying to find any comments about micro-freezes on Mac. Anyone have this experience? Is this a linux only issue?
No idea about Mac, but, I don’t think I experience micro freezes at home on my Linux boxes. So I have no idea if its a universal thing or not.
Still on 23.0.1 but there we have them on both Rocky Linux machines, but not on the mac studio.
I’ve only heard of it on Linux. I have some hopefully promising new results, will update here soon.
So here’s an update from my end. Opened a support ticket about this and Beau & team have been on it.
I’m on Rocky, 2023.1 and 2022.3.1. New Lenovo, A6000.
Started a new project, small with a few batches, 4k. Been on it for a few days and I had one fatal freeze, but zero micro freezes that I’m aware of. I don’t think the fatal crash was related to the micro freezes I experienced before.
*This new project is in 2022.3.1.
From Beau re: what he’s done in the background:
I believe we did basic tweaks to the network config, adjusted the memory management tokens to favor Flame @ 5 for BGR and 75/80 for Flame. We also added a script which detects if Flame is running and if strace is not already launched/crashes, launches it and attached to the Flame process. Strace exits after Flame the flame pid released. However just an aside, from the time of your crash till I was on the system, it looks like strace was off.
Once we get the system cleaned out like discussed, I suggest we do a day on and a day off w/ strace running and see if you have any issues. If it is smooth sailing, kill strace completely and see if you run into issues.