Slowed video, it's sloumo, enjoys incredible popularity with viewers, but it's extremely difficult to create. The necessary equipment is very expensive, and the need to store material, filmed at a speed of 300 thousand frames per second, quickly becomes a problem. However, the new technology from Nvidia allows to solve this problem in the best way.
The method is called “multi-frame interpolation of variable length” and is based on machine learning when analyzing the source material so that the neural network can “guess” the missing frames. It does not matter if you want to achieve a virtual slowdown of 8 or 15 times, this technology does not have an upper limit, and the system can generate any number of images that perfectly fit into the frame. More precisely – the viewer will not notice the dirty trick.
In fact, there are two neural networks working here. The first analyzes the video itself on a specified range of frames, creates a video stream map in the forward and backward directions, forms a plan for inserting virtual frames. The second system interpolates the data, comparing the generation capabilities with the plan to exclude the pixel curves, parasitic overlays and other “artifacts”. Now it remains to create from these data an arbitrary number of distorted versions of the first and second frames in tandem to insert them between them and “stretch” the video to the desired length.
For the implementation of the technology, Nvidia Tesla V100 graphics cards and the PyTorch deep analysis system from cuDNN were used. According to the creators, this means that the commercial version will not appear very soon, and in it most of the calculations will have to be transferred to the cloud. But the result is amazing – the video is very smooth, and there is an opportunity to “slow down” the already super-slow video.js.src = “&version=v2.8”; 'script', 'facebook-jssdk'));