3D photo manipulation

[dragon]

Common photo-editing software allows users to move objects—a flower from the left side of a photo to the right, for example—along a plane, but that’s about all they can do. Computer scientists at Carnegie Mellon and the University of California at Berkeley have developed the first application that lets users scale, rotate, and move onscreen objects in the 3-D space.

link

[Covenant]

Without seeing this I can only speculate. However, it says in specific that it is the first one designed for the average joe, not for graphics professionals like me who can already do this very thing with more advanced software.

I am, however, eager to see if they turned model-and-texture modification into a seamless thing and saved me precious hours of time if I ever want to rotate objects in 3D space.

[tezunegari]

They have a video. http://www.cs.cmu.edu/~om3d/video/video_hr.mp4

[salm]

I wonder how it handles reflective and refractive surfaces. They said it can not handle translucency but that wouldn´t be as important as good handling of the former two. You very rarely need translucency but need refraction and reflection permanently.

<edit>I just saw that it doesn´t handle reflections at all. The reflective highlight on the rotated taxis in the clip doesn´t change when they transform the car. This reduces the worth of such a programm quite a bit. It is still interesting technology, though. The fact that it fakes shadows is nice.</edit>

[Covenant]

I'm really interested in their light modelling--that looks like an incredibly robust automatic mapping of light sources, which is amazing in itself. I am guessing this is a lot more complex than they are letting on. It is already using 3D models to create hypothetical shapes, I am not sure that will actually work predictably well... though if you have enough good TurboSquid models (not cheap!) then I suppose it works fine.

I suppose the question is, who would want this, as is? Maybe nobody. But it is certainly something great to add into a new revolution of image suites, ideally a CreativeCloud style software package that links to a database of models.

[biostem]

I am also assuming that it can't account for details that aren't present in the original - for instance, it couldn't render the license plate number of a car that had said license plate facing away and not visible. Similarly, if only half yoru face was visible, the best it could probably do would be to mirror the known half onto the other, (so things like birthmarks/moles would be omitted).

Still, it's impressive.

[salm]

It uses textures of other models to fake the backside of objects. You can see that in the taxi example. The car is only seen from above in the original. Then the user flips the car and you can suddently see the car from underneath. This information wasn´t in the image but was taken from a 3D model of a different car. The progam "knows" that it´s a car and "knows" what the bottom side of a car is supposed to look like. So when you flip the car it maps this information to the cars bottom side.

I´m not sure if it could do faces at all because this programm uses stock models as references and unless you have the face in your stock model data base it probably won´t work or look weird. The thing is that the human eye can detect flaws in human faces very easily, so even minor offsets might irritate the viewer. In the example they use bananas and chairs. Humans have a much harder time detecing minor flaws in fruit and furniture so it is easier for the software to fool people.