Future stories will blur line between physical and virtual

By:
On:
With: 0 Comments
(Image courtesy NYC Media Lab.)

Story telling is increasingly blurring the line between the physical and virtual worlds, according to presenters at the just-concluded this year’s  Technology + Storytelling: Engaging The Immersive Future event organized by NYC Media Lab.

We’re already seeing this happen on the production sites in series like Disney’s The Mandalorian, as you can see in the video below.

Producers use the Unreal Engine from Epic Games to combine physical people and objects with virtual components and movie character, Connie Kennedy, head of Epic Games Los Angeles Lab, told attendees at the event.  

Connie Kennedy

“One of the tricks is to blend that physical set with the virtual environment so that you can see it is a seamless transition,” she said. “Any of the terrain, the dirt, the set pieces with the rock, all of that needs to be constructed during the early stages of the preparation for the film so that everything is seamless.”

The process also brings special effects, normally done in post-production, right into the heart of the filming process.

“The people who have been working traditionally in the post-production were separated from the collaborative process in the sense that their work was after the movie was shot and the people that were doing the pre-phase, those people were also separated and distinctive,” she said. “Now, everyone is working together in real-time and what that means is that the prep that we were doing in advance of the production is crucial.”

Evolution of the avatar will influence immersion

Additionally, the evolution of the use of avatars in virtual reality, augmented reality, and gaming platforms — from static avatars to lifelike virtual humans — will also be a game-changer for storytelling by making it feel more immersive, said Steve Rosenbaum, executive director at NYC Media Lab.

Steve Rosenbaum

“This is truly crossing the uncanny valley — making digital experiences indistinguishable from real-world experiences,” he told Hypergrid Business. “For storytellers, this is a game-changer. No longer are they making entertainment or diversion — they’re going to be able to have a profound impact on how and where people live in the digital world.”

One example of this is the new Epic Games’ MetaHuman platform which uses scanning to create photo-life-like avatars for game characters or for users participating in virtual training, events, or games.

“We will probably, in the next year or so, see phenomenal new changes with this facial capture and body capture technologies,” he said.

The technology is getting good enough to get us to the other side of the uncanny valley, he added. The uncanny valley is that creepy area of animation where it’s so close to being realistic that it comes off as creepy.

“I think those things are being resolved now with this precise and very exciting new way of scanning,” he said.

Storytelling breaks the fourth wall

Immersive and virtual technologies, by bringing the audience into the heart of the story, and allowing them ot interact with it, are changing the basic rules of storytelling.

“So a creator can frame a world, or a topic, or an environment, but the ideas, contributions, and participation of a group formerly known as the audience is going to be the difference between worlds that fall flat, and worlds and immersive stories that break through,” said Rosenbaum. “It’s really exciting to be at the forefront of something truly new — and know that the potential isn’t limited by tools or budget as much as it is by the imagination of the artist.”

(Image courtesy Wisdom VR.)
Latest posts by David Kariuki (see all)