Nvidia AI turns 2D to 3D in seconds

Nvidia AI turns 2D to 3D in seconds

Nvidia announced their AI can create virtual worlds, avatars, reconstruct scenes for 3D maps or train self-driving cars.

Nvidia’s AI researchers said they are developing new technology that can turn 2D images into a 3D scene in mere seconds.

NVIDIA’s Instant NeRF AI renders 3D scenes from 2D photos in just seconds

The conversion is achieved through inverse rendering. In this process, AI approximates how light behaves in the real world. Allowing researchers to create a 3D scene from a collection of 2D photos taken at different angles.

The Nvidia AI research team said their approach can complete this task almost instantly, using a combination of neural network training and rapid rendering. This approach has been applied to a new technology called neural radiance fields, or NeRF.

This new process, identified as “Instant NeRF”, is 1,000 times faster. Nvidia states this model can be trained in seconds using a collection of still photos, before rendering the 3D scene within tens of milliseconds.

“If traditional 3D representations like polygonal meshes are akin to vector images, NeRFs are like bitmap images: they densely capture the way light radiates from an object or within a scene,” Nvidia VP for graphics research David Luebke said.

“In that sense, Instant NeRF could be as important to 3D as digital cameras and JPEG compression have been to 2D photography – vastly increasing the speed, ease and reach of 3D capture and sharing,” Luebke added.


Nvidia AI researchers state that there are a variety of applications for this technology. It could be used to train robots and self-driving cars to understand the size and shape of real-world objects. This technology could also create avatars for virtual worlds, capture video conference participants and their environments, or reconstruct scenes for 3D digital maps.

Nvidia recently joining Zuckerberg’s metaverse with Omniverse, a platform for real-time 3D simulation and design collaboration. Last week, the tech giant shared plans make this platform accessible in the cloud, to accelerate the development of virtual worlds.

Currently researchers are exploring if their Instant NeRF input encoding technique can help with AI challenges such as reinforcement learning, language translation and general-purpose deep learning algorithms.

Nvidia teamed up with Microsoft last October to create an AI-powered language model. The tech giants claimed their new model was “the largest and the most powerful monolithic transformer language model trained to date”.

Nvidia AI research team presented a demo of the Instant NeRF results at Nvidia GTC
Share it :

Comments are closed