Fully detecting a scene's lighting is a complex problem. This paper from last year was quite interesting: "Rendering Synthetic Objects into Legacy Photographs"

Fully detecting a scene's lighting is a complex problem. This paper from last year was quite interesting: "Rendering Synthetic Objects into Legacy Photographs" - http://kevinkarsch.com/publications/sa11.html http://kevinkarsch.com/publications/sa11.html This combined with existing techniques of constructing 3D geometry out of videos could work really well if say you wish to construct a game level out of a real scene. Raytracing complex scenes on a hand-held device at interactive frame rates is still quite a bit off. But techniques where a lot of the calculations can be done offline are possible to use today (radiosity, etc).