Ole Christoffer Haga – Film Style in Cinematic VR

In recent years, filmmakers have started to look at Virtual Reality (VR) as a platform for cinematic storytelling (Bucher 2018; Opam 2016; Melnick 2019). When I initially encountered short films for VR and the 360-degrees view, some of my first thoughts were regarding the limitations of the medium. A 360-degree field of view upends many of the stylistic features that cinema has historically relied upon. Cinema’s predefined framing and editing positions the spectator as the ideal observer, or, how the film seeks to arrange optimal camera positions in relation to the preceding narrative (Bordwell 1985, pp. 9-10, 110-113; Bordwell et al. 1985). With VR, predefined framing is gone, and editing is usually kept to a minimum. These are part of a film’s technical and aesthetic devices, which is often referred to as film style (Bordwell, 1985; Bordwell & Thompson, 2013), and are obviously experienced differently when watched through a VR headset, or head-mounted display (HMD). My research question is therefore:
How can film style be used to tell stories in VR movies, and do these uses differ from cinema?
A distinctive feature of VR is that the spectator can get the impression of being encircled by virtual existents. This is often referred to as immersion, a term that, according to Murray (2017, p. 124) has its origin in the feelings we experience when our body descends into water – the sense of being totally enveloped by alternative surroundings. This again can lead to a sense of being transported into the virtual environment; that is, a sense of presence, a term originally derived from Minsky’s term telepresence (Calleja 2011, p. 18; Minsky, 1980). To achieve these sensations, the virtual environment needs to be what Ryan (2015, p. 48) describes as “fully spatial.” In short, this requires ubiquitous visual information, the ability to alter the field of view, and a visual illusion of depth, for example, stereoscopy (LaValle 2017, pp.145-156; Ryan 2015, pp. 47-49; Sherman &Craig 2003, pp. 119-120). VR offers a spatial distribution of digital existents in relation to real-time tracking of the spectator’s movements and position. Often this is referred to as six degrees of freedom (6DoF), wherein the spectator can alter both the rotation and position of view in three dimensions (Tricart 2018, p. 55). This enables the spectator to study, for example, a character from different angles, by altering her own real-world position. This can increase the sense of immersion and presence, but can simultaneously exclude some of the narrative tools provided by cinema, such as predefined framing. I have therefore also looked at how these aspects of VR influence the narrative in my study objects:

Does the illusive sense of spatial presence contribute to, or conflict with the narrative in these movies?

For example, a sense of presence requires the ability for the spectator to shift her field of view, which automatically excludes predefined framing. With the absence of several cinematic features, is the VR movie still able to perform intricate narration? Or is the concept of narrative VR just a gimmick? Are filmmakers stretching and cutting the limbs of cinema to shape it into a new platform of hardware, just as Procrustes did in attempting to fit his victims to the iron bed?

Theory

Of course, assumptions like these can seem valid. The narrative form of many VR movies faithfully follows established canonical formats. On the other hand, however, story structure has often been characterised by similarities regardless of the form of expression, whether it be folk tales, literature, film, or videogames (Bordwell, 1985; Chatman, 1978; Genette, 1980; Juul, 2005; Murray, 2017; Propp, 1968; Ryan, 2015).

A narrative can be understood as a recounting of events and their interrelations (Genette, 1980: 25). According to Chatman (Story and Discourse, 1978: 19) a narrative consists of story and discourse. Story is often referred to as fabula, a “chronological, cause-and-effect chain of events occurring within a given duration and a spatial field” (Bordwell1985, p. 49).

 

 

 

 

Figure 1: Illustration by author, based on Chatman (1978, p. 19, 26).

A story’s events and existents can be expressed and conveyed in unlimited variations, and this will change how the receiver perceives and interprets the story. The act of representation is called discourse, what Propp, and occasionally Bordwell, has referred to as syuzhet. Simply put, Story is what is being told, and Discourse is how it is told.

Figure 2 shows how a story’s existents and events are discursively selected and distributed to an audience in non-interactive media such as cinema. If we take the fairytale of Cinderella as an example, it is crucial for the audience to be informed that Cinderella loses her glass slipper on her way out of the castle – the lost slipper eventually enables the prince to find her.

 

 

 

Figure 2: Illustration by author combined with photo from Colourbox.com.

Figure 3 illustrates how interactivity can influence, and potentially interfere with, the intended representation as a result of an extended access to story space and/or a freer disposal of the discourse time. If the audience looks somewhere else when Cinderella leaves the ball, they may miss information required for comprehension of the events that follow. Simply put, watching a film through an HMD allows the spectator to see different things than the filmmakers have foreseen.

 

 

 

Figure 3: Illustration by author combined with photo from Colourbox.com.

As an underlying framework I have used Bordwell’s model for narration as presented in Narration in the Fiction Film (1985). As Bordwell’s model suggests, narration in film is largely influenced by the film’s style, that is, its “systematic use of cinematic devices” (p. 50).

 

 

 

 

Figure 4: Illustration by author, based on Bordwell (1985, p. 50).

The figure above illustrates the relationships between narration’s underlying systems. To avoid confusion, I will use Chatman’s terms discourse and story, which largely correspond to, respectively, syuzhet or plot, and fabula, as used by other theorists.

Bordwell (1985, p. 50, 53) highlights the close linkage between style and discourse (syuzhet). Both are strongly involved in the phenomenological process, that is, how the spectator receives the story, but where discourse takes care of the dramaturgical elements, style handles the technical side. Both elements are of course strongly intertwined; for example, discourse space is largely determined by the framing (cinematography) and the mise en scène.

The figure includes another element, excess. This involves a film’s “non-narrative” aspects, that is, any intrinsic element that enhances the spectator’s experience but for which a narrative function cannot be traced directly (Bordwell,1985, p. 53). Though “excess” can be highly relevant within VR, I will ignore this aspect, for reasons of scope. However, what I have looked at is film style, which according to Bordwell and Thompson (2013), can be divided into four main categories; Mise en scène, Cinematography, Editing, and Sound.

Method

To find out how film style can be used in VR movies, I decided to study and analyse three different films, all intended for screening through an HMD. I picked films that I found particularly interesting both theoretically and analytically, what Grønmo (2016, p. 103) calls a strategic selection. My study objects are not necessarily interesting because they tell unique or important stories, but they are technically innovative in the way they deal with formalistic and stylistic features. All of them have at least one trait that distinguishes them from the others, and they are produced by professionals with considerable experience within major animation studios in the USA. My selections are Invasion! (2016), Pearl (2016), and Piggy (2018).

My primary concern was to cover a certain diversity in the way narrative is conveyed. I chose Invasion! mainly because it is one of the earliest attempts to make animated movies for HMDs. The director, Eric Darnell, has a considerable track record from his time at Dreamworks – he directed Antz and several productions in the Madagascar franchise (Bucher 2018, pp. 214-215; Internet Movie Database, n.d.; Nafarrete 2016). Invasion! shares numerous attributes with other popular animation from the 1990s and beyond. At the same time, it offers something new with its ubiquitous imagery and with the spectator being granted an avatar.

My second selection, Pearl, is considered a milestone within the medium as the first 360-degree animated short to receive a nomination for an Academy Award (Hall 2017). This short is directed by Patrick Osborne, who has extensive experience as an animator, and who directed Disney’s 2014 Feast (Internet Movie Database, n.d.). Dramaturgical Pearl stands out from other works by its spatial constraints: the complete story is told through one single camera position. The story also stretches over a long period of time, which is not usual in this medium.

My final selection is another short, Piggy, which was released in Cannes in 2018 (Failes 2018). Unlike the others, this film includes narrative elements that proceed in real time, and the pacing and order of events can thus partly be influenced by where the spectator turns her gaze. Even though the film is simple and experimental, Piggy is a step closer to the fusion of cinema and video games and might anticipate how cinematic stories will be told in the future.

Researching how film style is used to convey narrative in VR movies relies heavily on data that cannot easily be quantified. This is because, among other things, most stylistic features in cinema do not have absolute functions or meanings. Rather, everything depends on the “film’s overall form and the immediate context” (Bordwell & Thompson 2013, p. 191). To find valid data, I needed to perform a close reading and research-based interpretation, which is why I chose a qualitative approach. This paper can thus be included in the tradition of text analysis, which is a common research design within the humanities (Østbye et al. 2007, p. 55-77).

The close reading was performed by viewing the short films on different platforms. An optimal experience requires more high-end equipment, and for this I used an HTC Vive with Siberia 200 Gaming Headset. I also watched the films through a Samsung Gear VR, which offers a midlevel experience but is quicker to boot up. In addition to this, I watched the movies multiple times on YouTube (360 degrees). The latter makes it easier to pause, rewind, and take notes while watching.

Invasion!

The six-minute animated short Invasion! tells the story of a little bunny named Chloe who witnesses two aliens landing in her forest with the intention of invading Earth. Intimidated at first, she eventually gathers courage and scares them away. We first encounter Chloe from a distance as she snaps out of her den. But around two minutes into the film, she jumps closer to the spectator’s position (Figure 5, right).

 

 

 

Figure 5: Illustration by author inspired by Invasion! (2016).

In the way that Chloe approaches the spectator’s vantage point, her actions mimic interaction with another character. Since the field of view is controlled by the spectator’s head movement, this can give the impression that the bunny is sniffing and looking at the spectator. The spectator’s optical viewpoint does not go unaffected but has a certain significance inside the fictional world. This is known in drama as breaking the fourth wall. In theatre and cinema, this dramatic mechanism has mainly been a way to abrupt a coherent illusion (Bucher 2018, p. 64; Silberman et al. 2015, pp. 23-24). In Invasion! on the other hand, Chloe’s approaching the spectator is designed not to break, but to be part of the diegesis. Chloe is not aware of being a fictional character; rather, she is simply communicating with a fellow diegetic native. This is proven to the spectator if she bows her head to look down. In Invasion! the spectator is equipped with a graphical representation of herself inside the diegesis, an avatar, in the form of the white body of a cartoony rabbit, with an appearance similar to that of the protagonist.

As the spectator both can have an impression of being placed – of being “fully spatial” – in the middle of the virtual environment, and is additionally granted an avatar, breaking the fourth wall does not necessarily have the same function in Invasion! as in traditional cinema. Chloe does not address the spectator to step out of the diegesis and into the real world, but to pull the spectator with her into the fictional world. It is thus a way of acknowledging the spectator as an existent than Chloe’s self-awareness of being a fictional character. Thus, in the case of Invasion! the intention of breaking the fourth wall is not to dissolve the illusion, but rather to strengthen it by making the spectator part of the diegesis.
Banakou, Groten and Slater (2013) argue for believable changes in perception and attitude that can be derived from the sense of inhabiting a different body. The study was conducted through two experiments and in both, the adult participants experienced virtual environments from a first-person perspective, represented by a downsized virtual body (avatar). The experiments had two conditions for how participants were represented: a child body avatar (condition C) and an adult body avatar (condition A), both equally scaled down in relation to the virtual environment. In the first experiment, the avatar’s movement corresponded in real-time with the participants’ real-world movements. Subsequent to their engagement in each condition, the participants completed a questionnaire. The questions revolved around how they perceived the virtual environment and the illusion of embodiment, based on scoring scales. Data from the questionnaire showed that participants perceived body ownership in relation to their avatars, and that there was no significant difference between condition A and C.

To examine size estimation, the participants’ hand positions were tracked as they positioned their arms (and hand controllers) to represent the size of selected virtual objects. The researchers found that participants overestimated object sizes when they were represented with a child body avatar, condition C, compared with condition A.

While still within the virtual environment the participants completed an implicit association test, or IAT. This test is performed on a computer where participants are supposed to rapidly pair different categories. In these tests the reaction time is also measured and is used to assess the input. In general, it is assumed that the stronger the association, the shorter the reaction time. For example, if the concepts presented are “Tiger – Lamb” and “Harmless – Dangerous,” the associated combination “Tiger – Dangerous” is most likely completed faster by the participant than “Tiger – Harmless.” The IAT in this study paired “child” and “adult” with the categories “me” and “others.” The presentation also included images and personal characteristics of each participant.

The conclusion from experiment 1 is that a child body led to overestimation of object sizes, and greater IAT scores. To determine whether these effects derived from the illusion of body ownership, the research group conducted the second experiment where the avatar’s movements did not correspond with the participant’s real-world movements – so-called visuomotor asynchrony. The information provided in a questionnaire showed a reduced sense of body ownership for both conditions, A and C. This sense influenced the results from size estimation and IAT in experiment 2, where the differences discovered between the conditions A and C from experiment 1 were no longer present without the illusion of body ownership. In summary, the study indicated changes in how subjects experienced and interpreted their surroundings when they perceive embodying another figure.

This outcome resembles my own experiences with Invasion! When the movie opens, I am watching Chloe at a distance, and she appears to be an ordinary small bunny. But when she approaches my vantage point, she turns out to be much bigger than I expected. With the HMD visually replacing the real world, the sense of scale is different than it would be with a close-up shot in cinema. In the physical world, I am standing six feet above the ground, but my perception is that of being the size of a little bunny. This is amplified when I look down and discover my altered bodily proportions. According to Banakou, Groten and Slater, this perception might also change the way I perceive the experience of Invasion!

As with many other VR movies, Invasion! is sparsely edited. The action proceeds mainly within one take, and it is up to the spectator herself to turn her head to keep track of what takes place within the surrounding virtual environment. In cinema, predefined framing enables more constraints on how the events and existents will appear to the spectator and what the spectator can focus on. Within a 360-degree view, this constraint is absent. To compensate for the lack of predefined framing, Invasion! broadly uses staging of action.

Staging is a broad term that can be regarded as the placement of existents as well as the choreography of movement and behaviour within the movie frame (Bordwell and Thompson, 2013). Staging is also one of Thomas and Johnston’s (1995; 1981) twelve principles of animation, which outline a model for composing a clear and comprehensible presentation of events. Proper staging of action ensures that ideas are effectively communicated, something that might be even more important for films in 360 degrees.
An example from Invasion! is the moment right before the alien arrival, around 02:10 into the movie. Here, Chloe suddenly shifts her attention away from the spectator, turning around and looking up at the sky. She holds this pose for approximately twenty seconds, with only slight turns to adjust her field of vision to correspond to the position of the flying item she is following. Her action here, anticipating extraterrestrial visitors, is sustained for a remarkably long time, which might be the film’s way of ensuring that the spectator does not miss this event. Chloe’s persistent glancing toward the sky cues the spectator to look in the same direction as well, ensuring the crucial moment will not go unnoticed. In cinema, staging is largely influenced by shot composition, a stylistic device that collapses with the 360-degree view. To compensate for this, Invasion! use the characters’ actions to cue our attention. So, even if we have the agency to look around, the film is discreetly trying to constrain this freedom by cuing our attention with cleverly laid-out guides.

Pearl

Pearl is a coming-of-age film that follows a girl from childhood to adulthood. The protagonist, Sarah, spends most of her adolescence on the road together with her father, and their car acts as an anchor for all the events that unfold. The story is compressed into a total runtime of just below six minutes and is represented through a third-person point of view.

A remarkable characteristic of Pearl is the continual presence and dominance of the car; no matter where the spectator turns her head, the interior is always within the field of view. (An exception is if the spectator lifts her head out of the car’s windows or sunroof). But though the interior takes up much of the space, it does not draw attention to itself. The sculptor Richard Serra used a similar camera-subject relationship in his 1976 movie Railroad Turnbridge (Bordwell & Thompson 2001, p. 130). Unlike Pearl, in Serra’s movie the camera is anchored to a bridge, obtaining both its locomotion and orientation. In this way, the structure of the bridge is fixed within the frame while the surroundings remain moving. The static and symmetrically balanced appearance of the bridge’s structure make it the centrepiece, drawing attention to its architectural qualities on behalf of the scenery. In Pearl, however, only the initial positional relationship between the vehicle and camera is fixed, while the spectator herself can orient the virtual camera freely with her head and body movements. Her attention to the vehicle is curbed, making the car more akin to inventory she is present within. The ever-present car interior constitutes a visual origin, justifying the spectator’s sense of presence.

Compared to Invasion!, Pearl offers far more visual information as a result of frequent cutting. But it is not cutting in the usual sense, with abrupt changes of the camera’s position or lens; the cinematography of Pearl is left to the spectator. Pearl’s cuts can rather be regarded as jump cuts, where stretches of footage are removed, enabling sudden jumps in time within the same continuous shot.

It is perhaps because the cutting is solely temporal that the makers of Pearl can allow themselves increased pacing. As the entire movie takes place within the old car, seen from the same vantage point, large amounts of visual information remain unchanged from cut to cut; consequently, there is less new information to be absorbed by the spectator. This solely temporal cutting therefore enables more travelling back and forth in time – without wearing out the audience.

It seems that Pearl’s combination of being assembled as a narrative summary and its large volume of static visual information allows for more extensive cutting than in other VR movies.

Piggy

Piggy is a single-take movie, completely without editing, and the setting is simple. The narrative revolves around a pig trying to resist a tempting cake, which spawns around 45 seconds into the movie. In several ways, the film tries to include the spectator in this struggle. If the spectator fixes her eyes on the cake for too long, the pig tries to lure her away. He does this by repeatedly appearing at the edges of the field of view, waving and making faces to entice the spectator to follow him with her gaze. And if the spectator tries to constantly follow the pig with her eyes, he might suddenly take a quick detour, hoping to shake her off.

The protagonist pig, his rival – a female pig, and the side table with the cake are the only existents in the movie, which otherwise features an empty, white background. This concentrates all attention towards character and action. Yet at the same time, the blank background makes it hard for the spectator, deprived of any reference points, to orientate herself. When the protagonist suddenly runs out of the field of view, all that remains is blank white space. In these moments, when the spectator turns around and sees neither the pig nor the cake within her field of view, she receives no visual feedback to indicate how many degrees she has rotated. While staging in Invasion! is firmly anticipated and slowly paced to make sure the spectator does not miss crucial action, the character in Piggy intentionally tries to distract and draw the spectator away. The result is a game of hide-and-seek in which the spectator is encouraged to rotate her focus and look for the character that is trying to escape her gaze.

According to Google (Google Spotlight Stories, n.d.), Piggy offers a “new level of interaction between character and audience.” Piggy does actually deviate slightly from the linear succession of events occurring within a fixed duration of time that we are used to in film. If we consider Chatman’s (1978) conceptions of discourse time, Piggy’s order, duration, and frequency will presumably never be identical for two different screenings. In the middle of the film, the staging of actions is influenced by the spectator. Nonetheless, the outcome – that is, the ending – remains the same. This is possible because each event can have either more or less importance in relation to the overall narrative (Chatman 1978, p. 53). Chatman uses the terms kernel and satellite to distinguish narrative events based on their hierarchical status. Events crucial for the narrative progression are called kernels. These are the key events influencing a story’s course of direction. If, in Piggy, the spectator misses the protagonist’s greedy glance at the cake in the beginning, she might not understand his effort to direct her attention. It would, in Chatman’s words, disturb “the logic of the plot” (Chatman 1978, p. 54). But a narrative consisting of only kernels, that is, events that efficiently drive the story causally forward, is likely to appear less rich and vivid. That is why most films are, additionally, filled with events that deviate slightly from the causal advancement. At one moment in Piggy, the protagonist removes the glass dome covering the cake to have a bite. When he senses being watched, he pretends he is cleaning the dome by rubbing it gently. This little gag can be recognised as a satellite; the event is not decisive for understanding the story yet it still orbits around the kernel, because it gives more depth to the pig’s personality and intensifies the comic situation. The figure below illustrates the relationship between kernels and satellites.

 

 

Figure 6: Illustration by author, inspired by Chatman (Story and Discourse 1978, p. 54).

In Piggy, all the events are pre-authored but the emergence of some of the satellites is based on the spectator’s field of view. This does not alter the narrative in any notable way since the kernels, especially in the beginning and end, are unchanged. But the sense of agency deriving from the pig’s awareness of the spectator’s gaze and the resulting behaviour (of both the pig and the spectator) might lead to the spectator’s engaging more deeply with the film. Of course, cinema depends on involvement as well, at least cognitively. But so-called interactive media requires additional physical involvement via motor interfaces, which again are returned as feedback through sensorial interfaces. The feedback confirms the spectator’s influence over the representation – for example, causing the pig to put down the cake when caught red-handed by the spectator. Still, as previously noted, the protagonist’s antics combined with the lack of spatial reference points make it hard for the spectator to orientate herself. The acquired agency in Piggy comes with a cost, as the protagonist does whatever he can to mislead the viewer and slip away. I previously compared Piggy with a game of hide-and-seek, and this may increase the spectator’s self-centring of her emotions when it comes to the outcome. For example, joy if she catches up with the pig, or frustration if he disappears. In this way, it is likely that a VR movie with elements of interactivity can be a stronger source of self-centred emotions compared to traditional cinema.
In general, without the discursive tools that derive from predefined framing, a VR movie needs to come up with workarounds to be able to cue and constrain the spectator. At first glance, this might appear a Procrustean means of accommodating the technical features of a ubiquitous field of view. But my analyses have also revealed traces of medium-specific uses. The following sections will present a discussion of my main findings.

Inclusion

What largely characterises the three study objects of this paper is inclusion of the spectator. Both Invasion! and Piggy acknowledge the diegetic presence of the spectator, including her as a character in the narrative. Invasion! additionally grants the spectator an avatar to confirm this presence. Pearl, on the other hand, is experienced through a third-perspective invisible observer, as is common within cinema. The characters are not aware of the spectator, but Pearl nevertheless offers a sense of perceptual presence because the spectator is anchored to the old car and perceives the discourse space as volumetric.

All three films share the ability to shift the vantage point within the diegesis, letting the spectator look around and observe the world from all angles. With this, parts of the film style (cinematography) are left to the spectator, which again will influence how she perceives and constructs the story. Film style and discourse work together to create a symbiosis of mutual influence. When the spectator is handling aspects of film style, she is consequentially included in the systems that facilitate story creation.

 

 

 

 

Figure 7: Illustration by author, based on Bordwell (1985, p. 50).

Exclusion

There is something compulsory in the way VR is experienced. Strapping on a headset breaks the ties to reality, at least the visual and auditory sensation of it. This implies that the spectator loses sight of herself as well. While VR includes the spectator, it simultaneously excludes her own real-world appearance. Sometimes she is provided with a substitute, like the avatar-body in Invasion!, but other times she is reduced to a disembodied vantage point, as in Pearl. The paradox is that even though the spectator’s body is perceptually erased or replaced, it seems the awareness of it is not weakened – rather the opposite. When perceived visual information alters with every tiny bodily movement, awareness of oneself is likely to increase (Grau, 2004, p. 278). The encapsulation the headset introduces also conceals every other real-world reference, enabling a VR-experience to change how the spectator perceives herself in relation to the world. I discussed an example of this in Invasion!, where the spectator can get the sense of being scaled down to the size of a little rabbit. The encasing of the spectator will naturally draw her total focus of vision and hearing toward the mediated, unlike other media, where surroundings can more easily distract. But what is just as important as unobstructed attention is what this does with the subjective awareness of the spectator’s own role, and how this affects her interpretation.

Accustomisation

The exclusion of real-world references brought about by the encapsulating HMD can accustom the spectator’s senses to other types of “realities.” As described in the previous section, the sensation of a different world scale can lead to perceiving oneself as shrunken or enlarged. Based on a study of Banakou, Groten and Slater (2013), I have suggested that this can cause a change in perception and attitude. Additionally, it is reasonable to assume that this again can affect how a spectator makes inferences and hypothesises in relation to story construction, but this needs to be confirmed by further research. Nevertheless, accustomising oneself to an alternative reality can increase the emotional immersion, since the spectator is practically experiencing the narrative from the inside. The exclusion of the spectator’s real-world reference points accustoms her to another reality which makes it harder to distance herself from the diegesis and narrative (Grau 2004, p. 152).

Volumisation

To a greater extent than with other narrative media, watching events unfolding through an HMD adds space and volume to the representation. As explained in the introduction, 6DoF leads to an increased depth perception, where the existents appear to have volume. Basically, this conceals the act of mediation. An oil painting can make use of monocular depth cues to give a sense of depth and volume, but this is soon revealed; anytime the viewer changes her position, the angles, distances, and foreshortening in the motif will remain unchanged. In VR however, all of this will change with every tiny body movement. The desire to depict subjective perspectives that underlie artistic movements such as Cubism can, in this way, be considered fulfilled (Ryan 2015, p. 14). The VR experience might expedite the acceptance of the existents as real, strengthening the spectator’s willing suspension of disbelief.

The volumisation also applies to the overall setting, which no longer is limited to a defined area such as the cinematic screen or the theatre stage but can embrace the spectator ubiquitously. Events unfolding in a perceived space rather than on a flat screen give the spectator an increased sense of being included, not least because she must make physical effort herself to follow the action encircling her. This simulates the way one witnesses events in the physical world – as an embedded attendant, instead of a distant, third-party spectator.

Narrative time

In a 360 degree-view, time is a central obstacle to linear narration. The ability to alter the field of view combined with actions that unfold at predefined moments of time increases the likelihood the spectator will miss out on events crucial for narrative comprehension. Invasion! bypasses this with its combination of a single take and a few, carefully anticipated key events. In Pearl, there is more cutting, and the story time is much more comprehensive. But since the cutting in Pearl involves temporal jump cuts, large amounts of visual information remain unchanged from cut to cut. This makes the constraints introduced by the spectator’s activity less decisive for the overall comprehension. In Piggy, the order, duration, and frequency of satellites are altered relative to the spectator’s gaze. In this way, parts of the discourse time are influenced by the spectator, resulting in a more unconstrained temporal linearity, with less risk of missing out on events crucial for narrative comprehension.

Continuity

Digital media has the ability to composite, rather than juxtapose, separate elements (Bolter & Grusin 1999, pp. 25, 27-28; Manovich 2001, p. 142). This condition is largely true of my study objects; a common trait is that they all implicitly claim their own continuity. First, the spherical field of view totally occupies the spectator’s vision. The images are not cropped by frame edges but continue without interruption in 360 degrees. Second, the films attempt to merge the spectator and work into one entity. The spectator does not watch a representation at a distance but perceives herself to be part of it because she is able to move her field of view around the encircling imagery. In this way, she also constructs elements of film style, a system that contributes to the narration. The line between the film and spectator is removed, which is why the notion of “breaking the fourth wall” does not necessarily have the effect of breaking the willing suspension of disbelief in these instances, but rather augments it. Being addressed by a virtual environment’s existents can confirm a spectator’s already established sense of spatial presence. This makes the reality of the spectator and the reality of the existents appear continuous.

This also applies to the issue of editing. Like many other VR movies, both Invasion! and Piggy (as well as Pearl to some extent) use a single take to avoid jump cuts. At first glance this might be perceived as a limitation of the medium as if, in VR, the powerful cinematic tool of editing must be used sparingly to avoid discomfiting or confusing the spectator.

However, if we turn this around, we might rather look at many of the tools we regard as film style as pragmatic ways of defying the limitations of cinema as a medium. The fundamental premise of cinema is juxtaposition and montage (Bolter & Grusin 1999, p. 75); fragmentation is in fact one of its main characteristics. Over time, we have learned to perceive the medium as natural even though the assembly of separate, independent images in succession, many of which often have little connection to real life, is undeniably arbitrary. Still, movies seek to be perceived as a whole, as one coherent diegesis.

In cinema, the 180° system was invented to maintain the audience’s spatial comprehension. Filmmakers cut on action to bind shots together and make the shift easy on the eyes. An actor’s voice can be heard off-screen to assure us she is still there, even though we cannot turn our heads and see for ourselves. All these and many more are examples of how the act of mediation can be camouflaged and affirm a film’s wholeness. In Invasion! the filmmakers did not need to pay attention to the 180° rule or shot/reverse-shot editing, as the spectator is located amidst Chloe and the aliens and thus is provided full spatial orientation by turning her head back and forth between them. When seen from this perspective, a narrative VR does not mean merely the loss of storytelling devices, but rather the disappearance of some of the obstacles its predecessor, cinema, bypasses to tell stories. Editing and framing have developed into powerful instruments but might also have originated as circumventions to pass beyond the limitations of representing a fictional reality through a rectangular image. The disappearance of frame boundaries makes way for new experiences. Indeed, the desire for immersion and presence is nothing new; it has been the objective of many filmmakers through the history of cinema. But until now, without the ability to actually place the audience onto the silver screen.

Conclusion: The road ahead

A film will always contain non-narrative elements, whether one calls this excess (Thompson 1977) or the third meaning (Barthes 1977). If the spectator turns away from Chloe to enjoy the beautiful landscape, the framing is not motivated by the narrative. The sense of spatial presence derives partly from the spectator’s agency to look around freely, but the same ability can simultaneously interfere with the story’s unfolding. Equally, stylistic features widely used to conduct the temporal and spatial discourse within cinema can in some instances break the illusion of being inside the diegesis. Cutting between two shots can violate the sense of presence, since the spectator is abruptly pulled out of the time and space she has accustomed to.

Still, even if there are signs of contention between narration and spatial presence, my study has not revealed any clear, universal patterns. Whether they pull in opposite directions or trace the same path relies heavily on the actual context. Narration and spatial presence are not two parts of a dichotomy in which only one can be active at a time. I have also found examples where they work together, driven by the same motivation. For example, Piggy takes advantage of how the spectator can – potentially – lose sight of the protagonist and uses this as a premise for the plot.

VR movies up until this date have to a great extent been early experiments. Maybe in the future my selected study objects will have the same status as Lumière’s The Arrival of a Train at the La Ciotat Station (Bolter & Grusin 1999, p.155) or Catmull’s A Computer Animated Hand (Sito 2013/2015, pp. 64-66) – dawning proof of concept and reference points for future productions. Future works within narrative VR will probably be even more multifaceted, which demands further investigation of the tension between narrative comprehension and the sense of spatial presence.

This article is based on the author’s Master’s thesis, Animated storytelling in 360 degrees (Haga, 2019).

Prior to entering academia, Ole Christoffer Haga was an animator working on features and TV series. He also directed and produced his own short films, which have been screened at numerous international festivals.

 

References

Banakou, D., Groten, R. and Slater, M. (2013) ‘Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes.’ Proceedings of the National Academy of Sciences 110(31): pp. 12846-51.

Barthes, R. (1977) Image Music Text. London: Fontana Press.

Bolter, J.D. and Grusin, R. (1999) Remediation: Understanding New Media. 1st edn. Massachusetts: MIT Press.

Bordwell, D. (1985) Narration in the Fiction Film. 1st edn. London: Routledge.

Bordwell, D. and Thompson, K. (2001) Film Art: An Introduction. 6th edn. New York: McGraw-Hill.

—. (2013) Film Art: An Introduction. 10th edn. New York: McGraw-Hill.

Bordwell, D., Thompson, K. and Staiger, J. (1985) The Classical Hollywood Cinema: Film Style & Mode of Production to 1960. London: Routledge.

Bucher, J. (2018) Storytelling for Virtual Reality: Methods and Principles for Crafting Immersive Narratives. 1st edn. New York: Routledge.

Calleja, G. (2011) In-Game: Immersion to Incorporation. Cambridge, Massachusetts: MIT Press.

Chatman, S.B. (1978) Story and Discourse. Ithaca, New York: Cornell University Press.

Failes, I. (2018) Annecy: Google Spotlight Stories Releases ‘Piggy,’ A New Approach To VR Interaction [Online] Available at: https://www.cartoonbrew.com/shorts/annecy-google-spotlight-stories-releases-piggy-a-new-approach-to-vr-interaction-159969.html [Accessed: 25 Jan 2021].

Genette, G. (1980) Narrative discourse. Ithaca, New York: Cornell University Press.
Google Google Spotlight Stories [Online] Available at: https://atap.google.com/spotlight-stories/ [Accessed: 07 May 2020].

Grau, O. (2004) Virtual Art: From Illusion to Immersion. Cambridge, Massachusetts: The MIT Press.

Grønmo, S. (2016) Samfunnsvitenskapelige metoder. 2nd edn. Bergen: Fagbokforlaget.

Haga, O.C. (2019) Animated storytelling in 360 degrees. Hamar: Inland University of Applied Sciences. Available at: http://hdl.handle.net/11250/2628757.

Hall, C. (2017) Watch VR’s first Oscar-nominated short film. Polygon. [Online] Available at: https://www.polygon.com/2017/1/24/14370892/virtual-reality-first-oscar-nominated-short-film-pearl [Accessed: 25 Jan 2021].

Internet Movie Database Eric Darnell [Online] Available at: https://www.imdb.com/name/nm0201509/?ref_=fn_al_nm_1 [Accessed: 25 Jan 2021].

Internet Movie Database Patrick Osborne [Online] Available at: https://www.imdb.com/name/nm2444148/ [Accessed: 25 Jan 2021].

Invasion!. 2016. [Animated short] Redwood Shores, CA: Eric Darnell.

Juul, J. (2005) Half-Real: Video Games between Real Rules and Fictional Worlds. London: The MIT Press.

LaValle, S.M. (2017) Virtual Reality. Illinois: Cambridge University Press. Available at: http://vr.cs.uiuc.edu/.

Manovich, L. (2001) The Language of New Media. Cambridge, Massachusetts: The MIT Press.

Melnick, K. (2019) vrscout.com [Online] Available at: https://vrscout.com/news/disneys-cycles-emotional-vr-film/ [Accessed: 6 May 2020].

Minsky, M. (1980) ‘Telepresence’. Omni Magazine. Available at: https://web.media.mit.edu/~minsky/papers/Telepresence.html.

Murray, J.H. (2017) Hamlet on the Holodeck: The Future of Narrative in Cyberspace. 2nd edn. New York: The Free Press.

Nafarrete, J. (2016) VR Short ‘Invasion’ Will Become A Hollywood Movie [Online] Available at: https://vrscout.com/news/vr-short-invasion-hollywood-movie/ [Accessed: 25 Jan 2021].

Opam, K. (2016) The Verge [Online] Available at: https://www.theverge.com/2016/4/21/11479752/pearl-google-spotlight-story-patrick-osborne-scot-stafford-tribeca-2016 [Accessed: 6 May 2020].

Østbye, H., Helland, K., Knapskog, K., Larsen, L.O. and Moe, H. (2007) Metodebok for mediefag. 3rd edn. Bergen: Fagbokforlaget.

Pearl. 2016. [Animated short] San Francisco, CA: Patrick Osborne.

Piggy. 2018. [Animated short] USA: Jan Pinkava.

Propp, (1968) Morphology of the Folktale. 2nd edn. Austin, Texas: University of Texas Press.

Ryan, M.-L. (2015) Narrative as Virtual Reality 2: Revisiting Immersion and Interactivity in Literature and Electronic Media. Baltimore: John Hopkins University Press.

Sherman, W. R. and Craig, A., B. (2003) Understanding Virtual Reality: Interface, Application, and Design. San Francisco: Morgan Kaufmann Publishers.

Silberman, M., Giles, S. and Kuhn, T. (2015) Brecht on theatre. 3rd edn. London: Bloomsbury Publishing.

Sito, T. (2013/2015) Moving Innovation: A History of Computer Animation. Cambridge: The MIT Press.

Thomas, F. and Johnston, O. (1995;1981) The Illusion of Life: Disney Animation. New York: Hyperion.

Thompson, K. (1977) ‘The Concept of Cinematic Excess.’ Ciné-Tracts. 1(2). pp. 54-63.

Tricart, C. (2018) Virtual Reality Filmmaking: Techniques & Best Practices for VR Filmmakers. New York: Routledge.

Uren, A. The secret behavior of New York movie-goers [Online] Available at: https://mashable.com/2014/12/13/movie-theater-behavior/ [Accessed: 7 May 2020].

© Ole Christoffer Haga
Edited by Amy Ratelle