Suramya's Blog : Welcome to my crazy life…

May 2, 2021

Infinite Nature: Creating Perpetual Views of Natural Scenes from a Single Image

Filed under: Emerging Tech,Interesting Sites,My Thoughts — Suramya @ 11:28 PM

Found this over at Hacker News , where researchers have created technologies that use existing video’s and images and extrapolate them into an infinite scrolling natural view that is very relaxing to watch and at times looks very tripy. The changes are slow so you don’t see how the image is changing but if you wait for a 20 seconds and compare that image with the first one you will see how it differs.

We introduce the problem of perpetual view generation—long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image. This is a challenging problem that goes far beyond the capabilities of current view synthesis methods, which work for a limited range of viewpoints and quickly degenerate when presented with a large camera motion. Methods designed for video generation also have limited ability to produce long video sequences and are often agnostic to scene geometry. We take a hybrid approach that integrates both geometry and image synthesis in an iterative render, refine, and repeat framework, allowing for long-range generation that cover large distances after hundreds of frames. Our approach can be trained from a set of monocular video sequences without any manual annotation. We propose a dataset of aerial footage of natural coastal scenes, and compare our method with recent view synthesis and conditional video generation baselines, showing that it can generate plausible scenes for much longer time horizons over large camera trajectories compared to existing methods.

The full paper is available here Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image with a few sample generated videos. One of the examples is below:

This is a very impressive technology. I can see a lot of uses for it in video games to generate real estate for flight simulators to fly over or fight over. It can be used for VR world developments or just to relax people. It might also be possible to take footage from TV shows and extrapolate them to allow folks to explore it in VR. (After a lot more research is done on this as the tech is still experimental). We could also simulate alien worlds using pics taken by our probes to train astronauts and settlers realistically instead of relying on fake windows and isolated area’s.

Check the site out for more such videos. Looking forward to future technologies built up over this.

– Suramya

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress