Those familiar with Stephen King’s novella The Breathing Method (1982) will remember this inscription on the keystone of the main fireplace of a mysterious London club: “It is the tale, not he who tells it.” Well, step aside, Stephen—these days it is not the tale, but s/he who tells it, especially those with innovative technologies to facilitate storytelling.
One of the many mind-boggling yet immensely popular technologies of our modern world is Virtual Reality, or VR, as demonstrated by Storytelling for VR, a hands-on concept workshop recently held in Melbourne, Australia. Together with Writing for TV and Online and Writing for Games, it is part of the Plot Twist: Stories of the 21st Century program supported by Film Victoria, a government agency that provides strategic leadership and assistance to the film, television, and digital media sectors in the State of Victoria.
In the words of Plot Twist, the Storytelling for VR Workshop is designed for “anyone who has not yet worked professionally in the field of Virtual Reality and who has an idea that would be well suited to be told in this medium. You could be already working in theatre, visual arts or other narrative mediums. You just need to be new to VR”.
As a writer daring to explore new horizons, I was selected as one of 12 participants in the one-day Storytelling for VR Workshop, which took place on May 26 at ACMI-X, a 2,000-square-metre state-of-the-art office space in the heart of Melbourne’s arts precinct. Part of the Australian Center for the Moving Image, ACMI-X is an eye-opening place where filmmakers, digital and visual artists, digital producers, web developers, screenwriters, and designers gather to develop film, TV, video games and digital art that feature collaboration, innovation, and sustainability. As a mature-age emerging writer with limited knowledge of and experience in digital technologies, I felt overwhelmed upon arrival.
But Katy Morrison, a recent VR Artist in Residence at the Royal National Theatre in London, put us at ease with a brief introduction on Real-Time VR. Katy explained that Real-Time VR is its own genre, involving not only the use of film, but also theatre, games, and physical performance. While VR projects are produced using the game engine Unity, the story is the most important part, as the focus of any narrative form is to tell a good story.
To paraphrase Katy, VR narrative is not a screen that you wrap around your head. It is not an extended cinema. Instead, VR is an interactive, user-initiated experience that is incredibly empowering and has a physical presence around the viewer or visitor, so they have to be able to affect that “reality”. In short, VR is an artistic medium.
Next, Oscar Raby, award-winning multimedia artist and fellow VR Artist in Residence at London’s Royal National Theatre, gave us an awesome talk on the concept and theoretical/philosophical background of VR. According to Oscar, the territory of experience is the body. “The body is not just space but concerns time as well. VR deals with the extension of the body and asks how one can convey this experience to other bodies.”
We all know that artists convey what they see via visual, audio and/or textual mediums such as drawings/paintings, music/sounds/noise/silence and/or written/printed words. Other mediums include but are not limited to physical performance, e.g. dance, theatre, and installation art. But Oscar focused on practical and artistic efforts to overlap one body’s experience on another body, using “gadgets” as extensions of the body.
For example, accountants of the ancient Inca Empire traveled and recorded the happenings across the land. Dangling from their belts were multiple strings, known as Quipu or “talking knots,” which they used to tie knots as a way to recorded data. As snapshots of the land, the data recorded by this device contained more than numbers and values, as these were intertwined with people’s lives in order to sustain the empire.
In his presentation, Oscar paid special attention to efforts to “objectify” experiences via visual devices. Since as early as Albrecht Dürer’s The Four Books of Measurement with Compass and Ruler (1525) and The Four Books of Human Proportion (1528), artists have learned to see the world with lines and numbers. But these lines and numbers are significant only because we use them to describe our experiences.
As René Descartes famously put it: “I think, therefore I am.” We are the center of the experiences. We are the starting point. Any attempt to record and refine our universe has to begin by mentally placing ourselves at the intersecting point of the x, y, and z axes.
Oscar took this notion one step further. He stressed that this whole process—this mental positioning of one as the origin—can be done by anyone and then transferred to anyone else. In other words, at the central point to capture the surroundings can be any Self, any Body, or even an object. Equally important is the notion that to be captured can be real, unreal, and even imagined, fabricated and/or constructed, as seen in Scott Roth’s “Ray Casting” algorithm (1982), which renders three-dimensional scenes to two-dimensional images in computer graphics.
Utilizing this algorithm, id Software’s Wolfenstein 3D (1992) was a critical and commercial success, a prominent example of creating a 3D illusion in a game. The player assumes the role of a spy trying to escape WWII Nazi German prison Castle Wolfenstein while carrying out a series of important missions.
According to Oscar, we have what the camera can give us. This is dominating in storytelling, but it is not the only way. If we can share a reality—if we can pass on a reality—then we can tell the story differently. Things can be described in different ways—not just through language, not just what we can say or do, but the whole describing process as well. You get to experience it—you as the viewer or visitor get to perform it.
As if this conceptualization is not complex enough, Oscar went further, inviting us to imagine ourselves as the camera. That is, in the VR environment, we no longer depend on a machine to tell us what it is seeing.
We are the eye. Stop thinking: “How does a machine tell me what it is seeing?” Instead, start considering: “How do I pass on to another body what I am seeing/hearing/experiencing?”
Oscar proceeded to explain the capacities of VR technologies and how they operate from the viewer’s or visitor’s perspective at intellectual, emotional and psychological levels. These are crucial in designing and producing user-friendly and empowering VR experiences, shifting our focus from technologies to physical performances while still keeping us aware of what technologies can do.
Oscar’s conclusion: “What is reality? Any reality can be shared.” We were left with these fascinating words from Soviet filmmaker and cinema theorist Dziga Vector: “I am an eye. A mechanical eye. I, the machine, show you a world the way only I can see it. I free myself for today and forever from human immobility… My way leads towards the creation of a fresh perception of the world. Thus, I explain in a new way the world unknown to you.”
Afterward, Oscar asked all the participants at the Writing for VR Workshop to share our story ideas and how we think VR can help with the storytelling. This is what I like about workshops, learning from others and being amazed at how unique each and every one of us as artists can be.
Many of us are already familiar with theater, visual arts, game design, and/or other narrative mediums. Most of us have some sort of idea about using VR to help build empathy and exploring previously unknown, underestimated or unconditioned emotions. Now, recognizing where we are and how we can be so much more with VR, all of us want to use stories and storytelling to capture our audiences.
Having received priceless one-on-one feedback from Oscar on our story ideas, we proceeded to learn the VR production structures. Similar to film production, a VR project typically features the creative vision of its director and a wide range of creative technologies supported by his/her development team, which consists of 3D artists and developers, sound specialists, and visual capture technicians.
After that, the 12 of us were divided into four teams to brainstorm our own VR project. Each team then had two hours to work with a student from RMIT University who is already familiar with the game engine Unity, to turn our rough idea into an actual VR experience. This is where the fun began, as none of us realized how complex the process is to turn a simple and mostly verbally-composed-and-expressed idea into a real functional application on the computer screen.
For example, we wanted an image here, a background there, a segment of audio/visual material inserted at an odd spot, a color flashing or a shape morphing, or an object moving from one corner of the screen to another. We had to specify the relevant requirements in order for the student programmer to understand what we needed and to locate and adjust the corresponding functions of the game engine to deliver a rough result.
The team members took turns wearing the “gadget”—the VR headset—to experience it. After that, the whole process started all over again: “Can you make this a bit bigger? Can you cut that a bit shorter? Oh, while we are here, can you also make that other thing a bit smoother/rounder/larger/louder/whatever…???”
When I put on the VR headset, I saw a gridded dome-shaped world in black and white, much like Homer Simpson’s 3D world. I was told to move to the center of a circle, but when I looked down to adjust my position, I could not see my feet! It was a really curious sensation, knowing I was there but unable to see myself. I suppose this is something that even Oscar could not easily explain in words, something that we had to experience by ourselves.
I immediately began to appreciate how tough it is for the character Kitai Raige to comprehend and master the technique “ghosting” in After Earth (2013). I also gained a better understanding of the confusion and fear felt by one of the characters in Hong Kong science fiction author Ni Kuang’s The Obscuring Point (1982). With his brain waves altered by a mysterious power, the poor guy looks into a mirror and sees everything but himself.
When I was wearing the VR headset, I kept craning my neck and standing on my tiptoes in an attempt to look beyond the figure’s shoulders. At one stage I even tried to push the figure away, and other team members had to restrain me from moving forward and bumping into the computer table. On another occasion I jumped up and down, trying to peek out a window that was placed high above me. Sounds silly, but all of it felt really real.
Our template had a featureless grey-colored 3D figure standing in front of us as the viewer or visitor. Behind the figure were a sphere and a cube suspending in the air. Focusing our gaze at the cube beyond the figure’s right shoulder would lead to something happening while looking at the sphere beyond the figure’s left shoulder would trigger something else. I have to say, this was perfect for my story idea “Chance Encounter,” initially a “choose your own adventure” digital novel where readers have two options at the end of each chapter regarding how the story should develop.
Needless to say, all four teams managed to come up with their own brief, rough but highly amusing VR experience, and we all giggled like three-year-olds, delighted by our small yet amazing achievements. Perhaps we also felt that mysterious pull of something greater than us—that almighty power of cutting-edge technologies: how it can impact us and how we can use it to affect others in numerous (positive) ways.
I imagine this is how Harry Potter felt when he finally accepted his magical powers, before commencing the journey to actively study the art and craft of being a wizard. All that learning in front of you—all those possibilities, all those opportunities! You just need to go for them.
Interestingly, in writing for VR, there is presently no established format for writers to depend on and be trained for. Neither is there any particular “gateway” through which emerging writers can be properly initiated into the industry. As Oscar put it: “People’s expectations and assumptions affect the powers of production. Sometimes people think too much. Sometimes something goes wrong and triggers new ideas.”
Highlight those parts of the story where an action is used, or at least indicate you need action in specific scenes. Keep in mind that the viewer or visitor is experiencing the VR world in real time. There is no fixed point in either storyline or space-time continuum.
At the conclusion of the Writing for VR Workshop, Oscar encouraged emerging writers to pair up with people that have technological skills and know-how, and to then promote themselves to the VR industry. Content matters, as does storytelling that finds the common language among existing and emerging industries catering to the human race.
If you have a good story and know how it can impact users in a Virtual Reality landscape—through setting, characters and/or events—then the industry needs it. How can you engage the viewer or visitor and help them have fun? That is the ultimate challenge for VR writers.
**
Image Credits:
1. Courtesy of Film Victoria
2. Courtesy of AZ Quotes
3. Courtesy After Earth
4. Courtesy VRTOV, organizer of the Writing for VR Workshop
Born in Taiwan and now an Australian Citizen, Christine Yunn-Yu Sun (@ChristineSun1) is a Melbourne-based writer, translator, reader, reviewer, occasional journalist and independent scholar in English and Chinese languages. Her essays, reviews and creative writings have appeared in Sydney Review of Books, Mascara Literary Review, Australian Poetry Journal, Overland Literary Journal, Westerly Magazine, Limina Journal, The Victorian Writer Magazine, The Good Weekend Magazine, International Journal for People-Oriented Programming, and American Journal for Chinese Studies. You can find Christine at Voices under the Sun.