Real-Time In-Camera VFX: Next Generation of Filmmaking

Visual effects (VFX) are a great way of achieving spectacular results when the budget doesn't allow filming for real or it's too risky for the actors. Up till now, these were achieved mostly by filming on green or blue screens. This new way of filmmaking presents a way for shooting scenes with VFX directly in camera.

For the last several years I've seen real-time VFX compositing, but the actors are  still in a green or blue screen environment. In this presentation the guys from Unreal Engine have managed to utilize LED screens for both lighting and projecting visuals. Together with the help of some real-time camera tracking, the filmmakers can create impressive scenes that are naturally blending with the actors and the set. The realism now depends not on the way the characters are keyed-out from a green screen and composited into the computer-generated scene, but from the realism of the projected imagery on the screen and additional on-set lighting. If you have been working with 3D, you will know about using HDRI backgrounds to light a scene which is basically what is being done with these LED walls. The colors of the projected screen reflect perfectly over the actors and the set to create realistic lighting conditions.

While this set up is very handy for directors and DPs, it will be interesting to hear how actors feel like being filmed for VFX in comparison to the green or blue screens.