Hi Alklier, sure we can do a separate thread for tech and ML details.
Version 0.4.x it is been based purely on RIFE and has been trained on Vimeo90 dataset.
With version 0.5 I’m experimenting with adding RAFT with optimised weights from Autoflow (https://github.com/google-research/opticalflow-autoflow - you can check the datasets they’ve been using to train - in order to get the first forward and backward pass feed into modified RIFE model. I’m also trying to improve warping artefacts by using MultiResUnet inserted inbetween RIFE multi-res passes.
The base data for training it is still Vimeo90 and I have two more sources on my own - one is a lot of 35mm film scans that I’ve filmed myself mostly in India and Nepal but some different places as well back in 2003 - 2008.
Another is a parts from Ukrainian period biopic I was editing and partly finishing - shot on Alexa and for that I have producers permission to use for that task (though it has never been put into a proper contract). This data currenlty is mostly used for validation.
The code is opensource and everyone is invited to participate