NVIDIA greatly improves the NeRF technique to generate 3D objects from 2D photos almost instantly

A new technology called Neural Radiance Field (NeRF) uses artificial intelligence learning algorithms to enable the creation of 3D objects from 2D photos. NeRF has the ability to fill in the blanks, so to speak, by interpolating what 2D photos couldn’t capture.

This is a very interesting advance that could lead to major advances in a wide variety of fields, such as video games or autonomous driving. Today, NVIDIA has developed a new NeRF technique, the fastest to date according to the company, which only takes a few seconds to learn to generate a 3D scene.

NVIDIA greatly improves the NeRF technique

It only takes seconds to learn the model, Instant NeRF, using dozens of frozen photos and camera angles to take them. After that, the system is able to generate a 3D scene in “a few tens of milliseconds”.

Like any other NeRF technique, you need to have images taken at different positions. And for photos with several subjects, it is better to have images without too much movement, otherwise the result remains blurry.

to generate 3D objects from 2D photos almost instantly

NVIDIA explains that the first NeRF models produce results quite quickly. It only takes them a few minutes to generate a 3D scene, even if the subject is hidden in some of the images, by a pillar or furniture. That being said, learning takes several hours.

With the NVIDIA version, this phase lasts only a few seconds, because it uses a technique developed by the company, a technique called “multi-resolution hash grid encoding”, which is optimized to work optimally on GPUs from the brand.

The system can even run on a single GPU, although the fastest results are achieved on boards with tensor cores offering a performance boost for all things AI.

The company believes Instant NeRF could be used to train robots and help self-driving systems understand the sizes and shapes of real-world objects. NVIDIA also sees a possibility for techniques in entertainment and architecture.

This could be used to generate 3D models of real environments that creators could modify during their planning processes.

Categorized in:

Tagged in: