But how do we make those holograms that show up as a 3D overlay of the real world? Microsoft has been making them for a while, so it’s got the process down and it’s released a video showing us just how it does it.
What’s immediately impressive about the video capture, is how smooth it is. This is no Kinect-like system with heavily aliased or wobbly edges, this is a true 3D video recording.
Of course that’s because the camera set up would make the Matrix’ bullet-time rigs look pedestrian. There’s a total of 106 cameras in use to capture the footage, while the subject (in this case, performers) stands in front of green screens to allow the background to be digitally removed.
What follows is a series of software steps that judge the distance between the performers and the cameras and the topology of their bodies and clothing, with successive steps making for more detailed and noise-free reproductions. Optimizations, that decide where the most detail is required, allow all of this to happen without the need for any major graphical overhead.
Key frames are also used to further compress the mesh motion and textures, without sacrificing quality.
The end result is fully 3D scanned and rendered video, that could be used with Hololens, or potentially VR as well, opening up many exciting opportunities.
What kinds of ways can you see this sort of technology being used in the future?
