Science meets art once again. Abe Davis, a computer science PhD student at MIT, recently published a research about interaction with the objects in videos, by measuring and mapping the vibrations of their movements. His project, which he developed with Justin Chen and Fredo Durand, and patented by MIT, aims to be used both in engineering and videography.
By creating interactive dynamic videos, users can interact with objects and move them depending on real-world force effects, to learn more about their dynamics. Currently, Abe Davis, shows this technology on videography, and states that it would be a low cost alternative for VFX in movie industry, where 3D scanning may be expensive. He claims, every video captures enough information to create compelling and realistic results as an alternative to VFX.
On the other side, this technology can also be used in observing and testing the dynamics of structures, and the whole process can be used in different fields of engineering applications.
To sum up, Abe Davis records and maps the different frequencies on moving objects, therefore he creates interactive 2D simulation of the moving objects, and he just uses a simple camera to do this. This unique algorithm, that he and his team created, once again shows us the power of computational photography and its capabilities.
To learn more about this project and Abe Davis, you can visit his website.
That is really clever!