WeAnimate 2024-04-12 | wam#0037

Storytelling in Virtual Reality stands at a crossroads of many creative disciplines and media. Incorporating elements from theater, cinema, and videogames, VR as a medium remains elusive to define.

Dooley Murphy is a VR researcher at the University of Copenhagen in Denmark, and he is currently finishing his PhD thesis, focusing on storytelling and presence in Virtual Reality. Murphy’s work attempts to create a general theory of VR Experiences, by combining theories and concepts from narratology, ludology, cognitive science, emotion science, and presence research. I sat down with Murphy to discuss his research findings, as well as his thoughts about the current state of VR and its future.

WeAnimate Magazine: Where do you stand on VR experiences being either games or movies?

Dooley Murphy: It’s unusual not to think of consumer VR as a gaming technology and not a film technology, but I don’t think it pays to limit ourselves to those two sources of inspiration. The essence of the technology has yet to be decided. The label “VR experiences” has affinities to both sides but isn’t making commitments. But there are clearly also imports from theater—immersive and participatory theater. I guess I’m just trying to keep an open mind as to what the label “VR experiences” can or might encompass.

Do you have any particular stance on the common term “VR experience”?

An experience is something that belongs to a person. It’s not an instance of an artwork. Yet that’s how we’re using the word. The real catch-all; the most honest thing to call VR experiences would be “expressive VR”. This would cover both popular and fine art, but it just doesn’t roll off the tongue! There are people like Brenda Laurel and Char Davies who have been making fine art in VR since 1992 and 1995, respectively. Now there are people like Laurie Anderson and Hsien-Chien Huang, a Taiwanese artist, who are producing fine artworks that win prizes at Venice Film Festival and the like. Anyway, I think the label “VR experience” works just fine even though it is technically inaccurate. It captures what the works do. They are playful, but they are not games. They are narrative, but they are not silver screen stories. They express something, and that’s supposed to make you feel something: They are intended experiences.

What are the main takeaways from your research on presence and storytelling in VR?

One of the core tenets of presence theory is this unspoken assumption, this unexamined belief that presence means the participant is not aware that the experience is artificial. “An illusion of non-mediation”. I think that’s misleading because participants are always aware to some degree that the experience is artificial. If we are talking about interactive storytelling, you do not want the participant to concentrate on the fact that there is a developer behind it, that there is an author of this experience, and that they — the designer or developer — want the participant to do something. Because if the participant is trying to guess what the developer wants them to do, they are not going to have a naturalistic experience. If you can make them do what you want them to do without making them think, “Is that what they want me to do?”, then it is going to be a more organic thing. The way I see it, what Janet Murray calls “scripting the interactor” means appealing to concerns that participants have. The most universal self-referential concerns are about protecting oneself, one’s interests. So, as aggressive as it sounds, I think one of the most powerful ways to “script” or direct participants’ attention and behavior in VR is to… in some sense, threaten them. I don’t mean point a gun at them!

If you can get them invested in something — “here’s a virtual doggy. Wouldn’t it be sad if something happened to it?” — then you are going to get them acting in a fairly non-reflective way. They won’t go into a self-reflective frame of mind thinking, “Is that what the developer wants me to do?” They will be acting on impulse, and I think this kind of mind-state is more indicative of presence. Presence isn’t being unaware that the experience is synthetic. Rather, it is making sure that the participant isn’t thinking, “Is this what the developer wants me to do?”

Offer the participant something — something too tempting to pass up, too ridiculous or inviting not to do. Say, a big red button that says, “DO NOT PRESS”. In videogames it’s a treasure chest, but then that treasure chest is actually a flesh-eating mimic. That’s fairly specific though since in videogames you have stereotyped goals like collecting treasure. I think there are things you can make VR participants want to do without prototypical game structures like “collect loot”. Valve’s The Lab, which was released in 2016 and is comprised of a series of mini-VR Experiences, has a segment with balloons where you have spikes on your hands. Spikes on my hands… Balloons in the air… I’m going to pop them. It’s impossible that I don’t pop them!

And then you start to doubt the experience. There are plenty of things that are not fun or pleasurable in the moment, one example being horror films but we watch those because we want to feel scared. But if you’re struggling with a puzzle in a VR storytelling experience, and difficulty isn’t why you signed up for the damn thing, then that can be off-putting and, by extension, presence-breaking.

Being aware of the medium isn’t a problem?

Not in and of itself, no. An analogy: You might notice a sublime edit in, say, Stanley Kubrick’s 2001: A Space Odyssey. The ape throws the bone into the air. Cut. Now it’s a space station. You appreciate that moment and you get aesthetic pleasure from it because it underscores that the film is an artificial thing—not despite that fact. A minute ago, you felt immersed in the ape bit. It doesn’t damage your immersion to be reminded that you’re watching artfully assembled pieces of footage. And we can do something with that fact in VR, and it is impressive, it is enjoyable. I don’t think it hurts that VR can feel synthetic or artificial. What’s important is that when you are reminded that that’s the case, you’re in a positive mindset, still appreciating things. If you’re reminded that your experience is artificial because you tripped over a cable and crashed into a wall, obviously that’s shit. But it’s completely fine to be reminded that experience is “fake” and it’s good. I think the 2018 comedic VR Experience Accounting+ from the minds of William Pugh and Justin Roiland does that a lot: It constantly reminds you that you are in VR and yet ends up feeling more immersive than stuff that tries to hide its own artifice.

Are there any ways that VR is currently being underutilized?

There are still way too many VR works—storytelling experiences—that ignore you, sideline you; treat you like an invisible witness. That is clearly—and I don’t mean to be prescriptive—in defiance of the medium’s most obvious affordances. VR has to involve you, implicate you. You do something; you commit a wrong that you then have to right. Get your hands dirty so that you’re bound up in the narrative. So much VR still tries to tell a story in front of you that you are not a part of. Sometimes they’ll have a character make eye contact with you and that is supposed to make you feel acknowledged, but no. You have to be… not guilty of something but drawn into it somehow: Embroiled. Either you, in the capacity of your real-world self, or some character you roleplay as, has to be drawn into the action, whether you like it or not. That provides a superior experience that you cannot turn your attention away from.

Are you optimistic about the future of VR?

Yes, the distant future. Not as long as Facebook reigns, and not given its current form factor. I think VR will continue to evolve as long as humanity stays around. It will continue to evolve to the point that we don’t think of it as VR, in the same way we don’t think of film as photography. After a certain time, there will stop being headsets. I’ve heard people say at academic conferences that within 15 years we’ll have VR contact lenses. I think that’s optimistic. We always think the next big thing is 15 years around the corner. At some arbitrary cut-off point, though, we will stop thinking of it as VR and say, “Do you remember VR?” But there will be some sort of continuation of the medium’s basic principles; spatial computing and embodied, participant-centric representation.

PC-based VR is not going to die and be replaced by standalone systems any time soon. There will be artists and people submitting to film festivals, at animation festivals like Annecy. They often have to work on PCs because Facebook’s closed ecosystem isn’t accommodating those types of indie creation, and the hardware isn’t powerful enough anyway. Smaller creators often do not have the time or resources to get their stuff running smoothly on standalone headsets.

I would assume that in the same way I could still go out and buy a Super 8 camera and shoot a Super 8 film, anybody will continue to be able to create rough-and-ready VR works and have them exhibited somewhere. Mainstream or consumer VR entertainment may even fall by the wayside, but there should always be a thriving scene, no matter how small or obscure. I think whatever VR changes into or becomes or ends up as there will still be people with the creative impulse to engage with a kind of ”exotic” medium and make interesting things with it. I think that’s where the most innovative ideas come from. Even if it transpires that VR is mostly used for gaming, there is hope.

Dooley Murphy’s recommended VR Experiences:

WOLVES IN THE WALLS PC VR, Oculus Quest

Popular Neil Gaiman story transformed into a charming participatory fairy tale with stunning set design by entrepreneur Ed Saatchi’s “virtual human” company.

[2018. Pete Bellington. Fantasy, Drama]

CROW: THE LEGEND PC VR, PlayStation VR, Oculus Quest

Hollywood VR mega-studio reimagines Lenape native American legend as Pixar-esque hero’s journey directed by DreamWorks alumnus Eric Darnell (Antz, Madagascar.).

[2018. Eric Darnell. Baobab Studios, Inc.  Adventure, Folktale, Comedy, Fantasy]

MADRID NOIR PC VR, Oculus Quest

Hyper-stylized noir crime caper in which you alternate between being told a story and becoming a part of it. Excellent onboarding, delighting interactions—not a dull moment!

[2021. James Castillo. No Ghost, Atlas V. Comedy, Drama, Crime Caper, Noir

PIGGY PC VR, Google Daydream

Short but sneaky interactive VR animation looks like a film but behaves like a game. Ever had a character watch you watch it? One for Schubert fans.

[2018. Jan Pinkava and Mark Oftedal. Google Spotlight Stories, Google, Inc. Comedy, Animation]

RAIN OR SHINE PC VR, Google Daydream

A responsive family-friendly narrative that has all the trappings of traditional animation. You’ll be focused on the characters, but keep an eye on the background—especially on subsequent viewings!

[2016. Felix Massie. Nexus Studios, Google Spotlight Stories, Google, Inc. Animation, Comedy]

AMENDS PC VR

Touching interactive story of a troubled maternal relationship that keys into some of VR’s stunning strong points: Verticality, impossible spaces, and scenographic spectacle.

[2020. Mikkel Battefeld. Amends Team. Drama, Autobiography]

Credits