The "sculpting" of a yellow cab image from 2D to 3D.
Clay, glass, wood, and marble are among the materials that can be sculpted, turning ordinary, three-dimensional objects into works of art. Researchers at NYU’s Courant Institute of Mathematical Sciences are working to bring this approach to images—through a pioneering form of sculpture that transforms 2-D pictures into 3-D.
Saining Xie and Daniele Panozzo, professors in Courant’s Department of Computer Science, have created a prototype technology that manipulates geometry and deploys generative AI to allow users to see not only an image’s front or back, but also its sides, top, and bottom—and to move it around based on one’s preferences. For instance, let’s say you wish to add a picture of cherries to the top of a cake image, by turning a 2-D image of cherries (and cake) into 3-D, you can more accurately place the cherries because the extra dimension affords a better match to reality.
The researchers describe their progress, which was reported earlier this year, below: