Filmmaker Danfung Dennis is no stranger to battlegrounds. He spent four weeks in 2009 shooting with U.S. Marines in Afghanistan for the Oscar-nominated 2009 documentary, Hell and Back Again. Now Dennis is moving on to a very different flavor of battleground with Zero Point, a movie that both reflects on and threatens to disrupt traditional filmmaking by releasing exclusively for the Oculus Rift. It’s an inventive concept, one that immediately brought to mind filmmaker Alexander Sokurov’s beautiful Russian Ark.
Shot on a single day in December 2001, Russian Ark is an avant-garde piece that consists of a single, continuous tracking shot. It weaves a path through the Winter Palace in Saint Petersburg, Russia while a narrator relates a loose narrative. It’s 96 uninterrupted minutes that amounts to a monumental feat of carefully staged choreography, with actors coming in and out of the frame on specific cues, at one point by the hundreds.
“We no longer have control of what frame the audience is going to see.”
The film’s extended use of Steadicam – a stabilizing rig that eliminates shake from a handheld camera – is a key source of inspiration for Dennis. Through Zero Point, he intends to explore the evolved entertainment medium that is rapidly developing around virtual reality by making use of one of its most successful iterations thus far, the as-yet-unreleased headset from Oculus VR.
“Russian Ark is the masterpiece of Steadicam work. I think that will work really well in the Rift, where it’s one unbroken, cohesive story and experience that’s sort of passing by you,” the Zero Point director tells Digital Trends.
By design, the movie will unfold “on rails,” pulling the audience along from one chamber to another, much like Russian Ark gradual passage different rooms and time periods. The idea is to have each space deliver some meditation on emerging VR applications, with audio from various expert interviews playing over an immersive series of presentations. These Holodeck-like chambers are part of a space station on which the whole journey unfolds.
“You’re tying spatial locations to the story, and I think that’s really important for how our memory works,” Dennis explains. “Our brain is constantly mapping environments around us. We think what’s going to be really different from traditional filmmaking is that sense of space and physical location, and what’s happening in that physical location.”
This speaks directly to the principal challenge of VR filmmaking. Conventional concepts like framing and editing don’t apply when you’re plunging your audience into a manufactured reality. The “camera” is, for all intents and purposes, the eyes of the viewer, and it offers an unrestricted viewpoint on the world that’s been created. You may not be able to move around the space like it’s a video game, but you can turn your head to look in any direction. The question is, how do you build a cohesive plot when the focus of the camera’s attention is completely unrestricted and impossible to predict?
“This is a new medium, a new language, and we have to invent the grammar and the syntax of how you tell an effective story in this medium,” Dennis says. “We no longer have control of what frame the audience is going to see. We’re delivering all of this raw information … and then letting them decide what portion of it they want to watch.”
“It’s one unbroken, cohesive story and experience that’s sort of passing by you”
“A lot of [our filmmaking approach] is going to be this merger, I think, between gaming and film,” he says. “There’s a lot we can learn from how first-person games have told those stories where you are a character. It’s completely new territory. We know it’s going to be extremely challenging to figure out how we tell a story or convey an experience, so we’re researching and developing that at the same time the technology is coming around.”
Alongside the more esoteric questions of how you shoot a largely passive VR experience is the very practical dilemma of developing technology to do so. Conventional filmmaking techniques are impractical for this kind of project because traditional camera rigs aren’t built to capture a 3D space. Sokurov’s Steadicam setup worked in Russian Ark, but Zero Point will plunge viewers into a space that they’ll be able to fully explore with their eyes.
“We built a camera system … based on these existing digital cameras that can shoot high frame rate on high resolution,” Dennis tells us. “We have an array of these high-resolution cameras and each one is shooting quite a wide angle [and] we’re stitching those together to create that 360 panoramic video, but we’re also shooting 3D. So you’re getting both depth and the full 360.”
“Doing 360 video, that’s been done before. Doing 3D movies, that’s been done of course. But doing those together is an entirely new type of capture system that we had to build specifically for the Rift. We’re still prototyping that, we’re still adding more cameras [and] there’s a lot of technical issues to still be solved, but it’s getting there.”
The most pressing challenge is getting those images to stitch together correctly. Sokurov created an incredibly convincing illusion in Russian Ark, but he did so from a single camera’s perspective. In Zero Point, Dennis will perform a similar feat, but with an elaborate array of cameras capturing 360-degree spaces. That’s a challenge when the camera isn’t moving; it becomes considerably tougher when you’re working with something more than a static shot.
“It’s a challenge to get [the footage from each camera in our array] to stitch correctly,” Dennis says. “Once the entire camera starts moving and everything is passing through the seams, it gets really hard to create this seamless experience. Motion is really hard for this medium, and keeping that really steady, balanced, smooth shot is so important to reduce motion sickness.”
That’s why Steadicam, one of Hollywood’s most favored filmmaking standbys, is also a potentially handy tool to use in the nascent practice of VR filmmaking. “So there’s something that keeps coming up in these stabilized shots on continuous scenes that I think Russian Ark, that type of storytelling, [provides an ideal framework] for virtual reality.”
(Images and video © Danfung Dennis and Condition One)
