towards the phenomenology of Angry Birds

draft:

The phenomenology of Angry Birds, or, how to live in virtual gravity

[PDF download here]

Abstract

Taking the popular mobile game Angry Birds (Rovio 2009) and its simulation of gravity as a starting point, this essay explores the nature of sensation, perception and proprioception in contemporary digital and mobile culture, as exemplified in digital games. It argues that the application of theories of the phenomenology of perception to digital media and games needs to be extended and adapted to acknowledge and describe the sensing, haptic and proprioceptive abilities of technological as well as human bodies. It is argued that the perception of virtual physics such as touch and gravity – and their effects in gameplay experience – must be understood as distributed across and through human and nonhuman sensoria.

 

Keywords: mobile games, sensing, virtual physics, phenomenology of perception

 

What kind of an animal are Angry Birds? What kind of world do they live in, what niche in the emerging digital mobile media ecosystem do these rancorous creatures occupy? Their popularity springs in part from familiar popular screen media tropes of animated violence and the aesthetics of branding, perhaps also from the cute appeal of their hardwired irascibility, the cartoon abstraction of their gameworld, or their players’ satisfaction of destroying their enemies through luck, skill or tactics. Their world echoes the dyadic simplicity of earlier computer simulated ecosystems with predators and prey, Foxes and Rabbits. This world too is a simple, eternal, war between two species: virtual nature, red in tooth and claw. Its microtemporality is repetitive and minute, its landscape and structures nothing but architectonic puzzles. If this digital cartoon is the immediate environment of the Angry Birds, then their hypermediate domain is the new communication networks and ludic experiences of the smart phone, the app store, the touch screen. Its virtual time-space has evolved to nest within the crannies and hollows of players’ actual time and space: in empty moments at the bus-stop, the waiting room, the lecture hall. These interludes themselves are shaped by these new techno-ecologies and economies: the game’s reliance on the touch screen’s haptic interface has trained a new generation of consumers how to use smart phones (Cheshire 2011).

This essay is less concerned with the game’s simulated zoology than with its physical laws however. Or – to be more precise – it will explore the distribution of simulated and actual physical forces across the gameworld and the player’s body and environment to rethink both the relationships between the virtual and the actual worlds of games and the creatures and bodies that populate this expanded field. There are numerous popular games for smartphones and videogame consoles that are built around a model of virtual physics in which simulated gravity, collision, friction, and acceleration is key to the gameplay. Taking the nature and operations of virtual gravity in particular, I am concerned here with the relationship between – or co-constitution of – virtual and actual worlds in digital game culture at large. Not in terms of space, but in terms of forces, affects, sensation, and bodies – material and actual, human and nonhuman: this is a new natural-artificial ecosystem, grounding both software creatures and players’ experience. As mobile phone hardware and software draw on the development of kinaesthetic and sensing technologies in games culture more generally (notably the Nintendo Wii and Microsoft Kinect systems), this game, or more precisely the playing of this game – is indicative of significant yet under-acknowledged developments in everyday digital technoculture. It does not just model a world of virtual physical forces – acceleration, gravity, momentum, collision, etc. – it brings into being new relationships, new circuits, between the human sensorimotor system and computer simulation.

Drawing on recent theoretical and empirical work in game studies on gesture, technology and play (e.g. Ash 2010, Crogan 2010, Giddings & Kennedy 2010, Simon 2009), this essay will argue that we cannot limit our understanding of sensing, and kinaesthesia to the (human) players. Building on, but critiquing, the influence of phenomenology on screen studies and game studies, it will explore the nature of human and nonhuman proprioception in everyday gameplay and ask how the sensing of gravity is distributed and achieved across the virtual and actual, software, hardware, nerves and perception, across human and nonhuman players. It suggests that we think seriously about digital media technologies not only as extensions of the human body, but as sensing bodies themselves, alongside – in collusion with – human bodies, a collusion that we might think of as distributed proprioception.

 

Angry Birds

The birds themselves have a backstory, a motive for their fury: a herd of green pigs have stolen their eggs. To defeat these swinish adversaries, the birds hop into old-fashioned toy catapults or slingshots to be propelled into the pigs’ ramshackle fortifications. The aim is to take out with each volley an optimum number of pigs, or strategic sections of their defences. The pigs’ infuriatingly smug grunts only render their destruction more satisfying, or near-misses all the more frustrating. With its cartoon graphics, innocuous violence, ramshackle structures and technologies, and near mythological dramatic framing, perhaps the closest analogue for Angry Birds in pre-digital popular screen media is the enduring Warner Brothers’ cartoon Road Runner. These Looney Tunes characteristics have herded the game’s characters down a familiar path through commercial culture into licensed imagery and merchandising.

The Birds have a kamikaze mission: on impact, or soon after, each explodes gently in a puff of feathers. The game mechanic is simple and accessible, a two-dimensional, side-scrolling, world that shows its genealogy in the non-realist graphics and physics of Flash-based web browser games as well as limited animation for television. The player’s only significant input into this gameworld, in the early levels at least, is the triggering of the catapult by placing a fingertip on the waiting bird (on the phone’s touchscreen), drawing the catapult elastic taut through a single slide of the finger, and releasing by removing the fingertip from the screen. The momentum of the bird on release, and its initial trajectory, are determined by the length of the stretched elastic and its angle, all established by the same single finger-swipe. Angle and tension are the only variables accessible to the player and the relationship between the two must be carefully judged for maximum or strategic effect.

The pigs construct a different arrangement of materials and objects for each level, each offering a puzzle to the player that is matched in the number and characteristics of the birds available for the level. In this regard, this is less a war than a collusion between pigs, birds and architecture in the staging of a particular challenge or puzzle to the human player. This is not, then, an avian-porcine conflict at all; the battle is between player and software.

 

What is virtual gravity?

The interplay between elastic tension and angle of launch is only part of the game’s virtual physics. Propelled high into the air, the bird traces a graceful arc as its momentum is countered by the pull of a simulated gravity. Most videogame worlds are consolidated with a simulation of gravity, though more often that not this is as simple as keeping their characters on the ground, rendering predictable movement when they jump, or plunging them plausibly to their death when they stray from the edge of a cliff or platform. In games such as Angry Birds however, virtual gravity is fundamental to the play mechanic and appeal, and its pull has been felt throughout the history of computer games. The development of one of the very first, SpaceWar! (1962), was transformed by the introduction of virtual gravity. This two-player game featured battling spaceships, animated on the oscilloscope display of a PDP-1 mainframe computer was transformed from a eye-catching example of real-time interaction into a compelling game by the addition a ‘sun’ with a simulated gravitational pull to the centre of the game screen. With this a key strategic element was introduced. Rather than a simple shoot ‘em up relying on motor skills, now players had to work with the sun’s gravitational pull. This would  ‘give you speed as you circled it, but if you weren’t careful and got too close, you’d be drawn into the sun’ (Levy 1984: 63). This virtual gravity opened up new possibilities for players to develop their own strategies and play styles. These included ‘the “lie in wait” strategy, in which the player ‘stayed silent while the gravity whipped you around the sun, then straightened out and began blasting torps at your opponent’ (Levy 1984: 64)[1].

So, what is virtual gravity? How does its simulated pull act on actual bodies? On one level, virtual gravity is nothing but the operation of computer code on objects within the game as a program. In a playful but incisive article in Wired, Rhett Allain (2010) has decoded the simulated physics of Angry Birds, reverse-engineering and analysing the game’s relationships of scale and force. Allain’s analysis highlights the relatively simple mathematics that generate virtual gravity, and, by extension, the fact that virtual gravity has no direct physical relationship with actual gravity. The interaction of mathematical variables in the algorithms of a dynamic software simulation, the ‘gravity’ here is merely graphical output (see figs. 1 and 2). There are no bodies with mass, and hence attraction, in this gameworld. The simulation of gravity is effected by the plotting of a bird’s movement through the relationship between the x and y axes, not by the action or pull of the ground. However the graceful parabolic trajectory produced by this interplay, despite its algorithmic generation and the stylised and abstracted cartoon world that it crosses, ‘feels’ convincing and somehow familiar from our actual experience of, say, firing a catapult or throwing a ball.

fig.1: Horizontal motion of an Angry Bird in flight (Allain 2010).

fig 2: Vertical motion of the Angry Bird in flight (Allain 2010).

The player then is not shooting the bird-projectile in a straight line, as the first-person shooter player might with his or her more familiar armoury and ballistics[2]. Rather the player must predict the possible curved lines of flight up and over the landscape. Whilst he or she doesn’t sense the drag of virtual gravity in their own viscera, they must think it, anticipate it as possibility. Similar code could generate onscreen movement that simulated quite different physical forces, for instance the forward motion of a boat or swimmer crossing a river as they interact with a strong current. They would be pushed sideways rather than pulled downwards, but the arc of movement could be the same.  Importantly though, the player’s sensori-motor investment in the bird’s trajectory, his or her visual tracking of its flight, and his or her embodied memories of other missile-based games, together form an experience that cannot be separated into its constituent stimuli, nor straightforwardly translated into other simulated movement. So when we play Angry Birds the pull of virtual gravity is a synaesthetic assemblage of screen imagery and movement, player investment through interactive agency (and moments of loss of control), and memories of other, actual, interactions of play and gravity such as ball games or swings. This feeling takes the form of either intense satisfaction and kinaesthetic pleasure – as the bird hurtles towards precisely the spot that will collapse the tottering pieces of scaffolding that will dispatch the level’s last pig, or – as the player realises that the bird will overshoot and bounce harmlessly beyond the smugly grunting green snout of this same last pig – in a palpable yet impotent willing of the arc to tighten itself, for gravity to exert an extra pull, for the bird to plummet faster. This feeling of virtual physics is a familiar one to players of videogames of many genres, and all new players are embarrassed by their redundant movement of controllers in vain attempts to keep a racing car on the track on a tight bend, or to propel the avatar to achieve a dangerous jump over a chasm.

A number of game scholars have discussed these corporeal reactions in the videogame player. The most relevant to this article draw directly on Vivian Sobchack’s influential work on another screen-based medium – cinema – to explore the player’s embodied relationship with videogame images and action (Swalwell 2008, Crick 2011, Behrenshausen 2007). These writers apply Sobchack’s phenomenological framework to game players, whilst acknowledging and opening up the key differences from cinema in the kinaesthetic experience that videogames as interactive media generate. On one level then, videogames are screen media and operate a similar play of animated imagery on the vision of their players. On this sensual level, the videogame player’s physical experience shares some of its aspects with intense engagements with other screen representations in which the forces of gravity play an integral role, particularly in action cinema and television where the audience is gripped, willing the protagonist to make the near-impossible leap over a ravine, or to recover his or her balance as they teeter on the edge of a tall building, grappling the enemy. As Tim Crick puts it,

Similar to filmgoing, videogaming is a holistic experience and it is precisely our capacity as sensual embodied beings in the world that allows us to engage with a game’s artificial world in a way that would engage those senses in real life. Our imagined perceptions are, as Merleau-Ponty claims, just as much a part of experience as nonimagined ones: ‘‘My field of perception is constantly filled with a play of colors, noises, and fleeting tactile sensations, which I cannot relate precisely to the context of my clearly perceived world, yet which I nevertheless immediately ‘place’ in the world’’ (2002, p. x)” (Crick 2011: 266).

Later I will describe in more detail the synthesis and generation of virtual gravity itself as computer code, but it might be helpful at this point to suggest what the specific implications of paying attention to simulated gravity might be. I will briefly explore the nature and significance of actual gravity for biological embodiment and proprioception, and then, having established its importance to theories of embodiment and media experience, some of the anxieties that the ostensibly gravity-free virtual realities of digital media have generated.

 

Gravity, ecology, virtuality

In a remarkable book on the ecology of children’s imaginative play Edith Cobb (1977) offers some rich ways of thinking about the body, mind, play and proprioception in a holistic, ecological framework. Cobb was a contemporary and friend of Margaret Mead and Gregory Bateson and as such was part of an intellectual milieu that synthesised anthropology and cybernetics up to half a century before the cybercultural studies of the 1990s. Cobb posits perception as a ‘first order drive’ shared across the animal kingdom and by even quite simple organisms. Even the most primitive animal, Cobb argues, may be defined as ‘something that perceives’ (Cobb 1977: 40). The first evidence of mammalian proprioception is the ability of the developing foetus to adjust its orientation within the womb in relation to gravity. Citing Gesell, she asserts that the foetus is a “growing action system… Its first and foremost function is to adjust to the ceaseless pull of gravity” (in Cobb 1977: 41). Through this function this animal system fundamentally organises itself in relation to the earth, to up and down. Moreover, this orientation provides a foundation for all experience and behaviour:

This experience permeates all later behaviour and is the primary adaptation to the logic of nature’s aesthetic […]. Eventually “postural attitude issues into postural action” as these early layers of information extend into behavioural forms and patterns” (Cobb 1977: 41).

 Or, more succinctly, ‘… the counterpoint with gravity is fundamental to the effort to know and to be’ (Cobb 1977: 43).

This antenatal sensing is the first engagement with and reaction to the world, and, Cobb asserts, is developed and explored as the child grows, primarily through physical play and its kinaesthetic joy in body and environment.

Cobb began her research in the 1940s and her account of childhood is of a natural realm largely separate from the adult world and without reference to entertainment media. The general transformations of childhood and children’s culture in the developed world wrought by media culture from comics and television to videogames are not of immediate concern to this essay, but it might be useful to consider the terms and assumptions bound up in anxieties that directly touch on this essay’s interest in embodiment, ecology and gravity in digital media culture. In the early to mid 1990s tremendous excitement and anxiety was generated across popular fiction, journalism and the academy about the apparent, imminent dissolution of a fixed boundary between the physical and biological world and the virtual worlds of computer graphics, simulations and networks. The human mind, as at once informational and intangible (and newly understood as informational and intangible in the light of new computer technologies), was seen as particularly susceptible to transit from the biological to the digital. Whilst the wildest claims for the imminent absolute separation of mind and body (with the possible ‘uploading’ of consciousness out of the body and into virtual networks) were questioned within critical work on emergent electronic / digital media technoculture, a sense of threat to the body and its sense of location and physical presence was evident in much of this serious work. Vivian Sobchack, in an influential and much-anthologised essay, addressed the fate of the body in new electronic screen media and spaces in specifically gravitational terms:

… at this historical moment in our particular society and culture, the lived-body is in crisis. Its struggle to assert its gravity, its differential existence and situation, its vulnerability and mortality, its vital and social investment in a concrete life-world inhabited by others is now marked in hysterical and hyperbolic responses to the disembodying effects of electronic representation […]… constant action and “busyness” replace the gravity that grounds and orients the movement of the lived-body with a purely spectacular, kinetically exciting, often dizzying sense of bodily freedom (and freedom from the body). In an important sense, electronic space dis-embodies (Sobchack 1994).

 Given the primacy of gravity to embodied orientation and agency in the world, any such loss of proprioception in the emergent digital media environment would clearly have significant if not catastrophic consequences. Thus whilst – as will be explained (and with the benefit of 20 years of digital cultural hindsight) – I do not share Sobchack’s diagnosis of the proprioceptive gulf between electronic worlds and human corporeality, the issues she raised – and the theoretical tools she deployed – are of direct relevance to this essay. I will argue that the virtual worlds of videogames do effect a gravitational pull on the body, but that this pull is of a different order to actual gravity alone. To explain this I will draw on work on the phenemonology of perception and media technology, as well as some key concerns and notions from cybernetics, to open out the relationships between human and nonhuman embodiment and perception, and their inter-relationships[3].

 

Beyond the phenomenology of perception

What then are the implications of this generation of virtual gravity across computational and biological domains for the phenemonology of perception as applied to the study of digital media technology?  Much of the most interesting recent work on technoculture and everyday lived realities has closely engaged with phenomenological and post-phenomenological thought; from Paul Dourish and Lucy Suchman’s technomethodological studies of interactive systems to Vivian Sobchack and Laura Marks’s work in film studies (Dourish 2004, Marks 2002, Sobchack 2004, Suchman 2006). All are centrally concerned with the inseparable relationships between mind, body, senses and technologies.

This attention to embodiment and perception, in game studies in particular, is a welcome correction to formalist, semiotic and cognitivist approaches to media experience in which the materiality of bodies and technologies is downplayed or elided. It opens up the field study of gameplay and culture to the physical and social environment as well as the player’s embodied experience. Merleau-Ponty’s famous example of the embodied perception of an everyday technologically-augmented human is illustrative:

The blind man’s stick has ceased to be an object for him, and is no longer perceived for itself; its point has become an area of sensitivity, extending the scope and active radius of touch, and providing a parallel to sight. In the exploration of things, the length of a stick does not enter expressly as a middle term (Merleau-Ponty, 1962: 165-6).

Here, in the act of walking, the blind man and his white cane are one, the simple technology of the latter an extension of the former. Crick applies this insight to digital game play:

the experience of one’s body is not fixed or rigid but adaptable to the numerous tools or technologies that may be embodied. This furthers our understanding of how players form an embodied relationship with the avatar in the game world through their habitual mastery of the control device in the actual world (Crick 2011: 266).

Thus the phenomenology of perception opens up cultural and media research to the integral place of technologies in everyday life, embodied experience, action and cognition. A game controller then ‘acts an extension of the player’s body’, that, once mastered, ‘rarely requires any conscious thought to navigate the avatar’s body’ (Crick 2011: 266).

Yet the usual start and end point for work in this tradition is the human body and human experience. In the spirit of Marshall McLuhan’s dictum, media are ‘extensions of man’, of the human motor capabilities and sensory apparatus (McLuhan 1964). Here, then, media augment the human senses, but are not considered sensate in and of themselves. Though the phenomenology of perception understands the human body as extended out to its environment through prostheses, media, and other technologies, this body remains central, the agential, cognitive and sensorial core from which its augmentations take their direction, meaning and purpose. The phenomenology of perception then persists as primarily a philosophy of the human body and sensorium. As I will argue through the rest of this article, sensory interaction in the digital era is not adequately accounted for in this anthropocentric worldview. There are heterogeneous bodies in play in media culture in general, and in videogame play in particular. These bodies are nonhuman and human, synthetic and biological, virtual and actual.

So the persistent centring of the human body in a model predicated on simple human-technical couplings leaves little conceptual space for the consideration of the obduracy and agency of technologies as nonhuman bodies. This conceptual gap is more evident and more significant when the more complicated technologies and machines form more complicated relationships of action and affect. So, the white stick does not in and of itself sense its environment – or rather it does not register and react to the effects of the environment on its own body itself. Rather its physical properties and shape transmit its engagement with the environment as vibrations to the sensorium of the human component of this assemblage.  Videogame controllers operate differently. Through rumble features they transduct virtual collisions in the gameworld into physical stimuli felt in the player’s hands – and they are controlled by the game code as much as they are by the player and so the player can be seen as much as an extension of the gameworld (hardware and software) as vice versa. The environment is actively and intentionally triggering a sensory response in the player, who in turn responds and acts within the environment. Here then is a complicated and non-linear set of informational and perceptual circuits.

Sensing and cybernetics

Computer science, by practical necessity, has entertained no such qualms about the reality and agency of nonhuman sensing. Cybernetics and robotics in particular are predicated on it. Norbert Wiener described ‘instruments which act as sense organs’ (Wiener 1950: 157), and a fundamental characteristic of robots is their ability to sense and respond to their environment (Winfield 2012). In their influential book on digital game design, Katie Salen and Eric Zimmerman identify sensing as one of the three fundamental actions of a cybernetic system. In such a system, the sensor “senses something about the environment or internal state of the system”, the comparator determines is a change to the system is needed in the light of the sensor’s reading, then the activator puts that change into operation (Salen & Zimmerman 2003: 214). Salen and Zimmerman are here explaining the workings of videogame software, but it is crucial to note that cybernetics in general makes no assumptions about, nor establishes any hierarchy between, the elements of a system, be they animal, human or artificial. It follows that there is no a priori division of sensory labour in a cybernetic system. The governor on a steam engine or thermostat in a heating system sense physical pressure and temperature respectively and respond to change the state of the system. For Wiener, the cybernetic system ‘corresponds’ to the animal as an organism in terms precisely of the sensate:

the all-over system will correspond to the complete animal with sense organs, effectors, and proprioceptors, and not, as in the ultra-rapid computing machine, to an isolated brain, dependent for its experiences and for its effectiveness on our intervention (Wiener 1950: 157).

It is important to establish here that Wiener is not using terms such as ‘sense organs’, proprioception and ‘experience’ metaphorically or anthropomorphically. They are analogous, they ‘correspond’ to animal perception. These machines and devices are not biological but like animals they sense and act in response to certain aspects of their environment. Ian Bogost has applied recent philosophical critiques of the anthropocentrism of phenomenology to the technology of videogames, questioning why human experience and perception should be at the centre of the world. The videogame phenomenologist should not ‘seek to understand how a human player perceives the sounds and images and tactile sensations that comprise the videogame playing experience’ but rather should attend to ‘the way the machine perceives its own internal and external states independently of whether and how the human player views or manipulates the artefact’ (Bogost 2008: 36).  Here then is his description of a videogame platform in which only machinic perception and action is in operation:

Atari VCS players see the same sorts of images that they would have come to expect from television broadcasts – the sense of a moving image like film. But the Atari VCS itself does not ever perceive an entire screen’s work of graphical data in one fell swoop. It only apprehends the syncopations of changes in registers. Its components see things still differently: The 6502 processor encounters an instruction read sequentially from program flow, performing a lookup to execute a mathematical operation. The TIA graphics chip modulates sends [sic] electrical signal when it witnesses a change on one of its input registers. The RF conversion box coupled to console and television transmutes an endless stream of data into radio frequency. Time moves forward in syncopated bursts of inbound bits and bursts of signal, then of color from joystick to motherboard to television. Despite the fact that the machine must manually synchronize itself to the television display at 60Hz, it has no concept of a screen’s worth of image or a note’s worth of sound (Bogost 2008: 36).

As a challenge to anthropocentric thinking, this vivid ethology of the autonomous life of an artificial system is compelling. It a sustained model of technocultural enquiry however it runs the risk of inadvertently mirroring the humanist worldview it seeks to overthrow. A videogame system can be studied independently of whether a human is manipulating it[4], but by and large videogame systems are operational only when a human is playing them. To look ‘inside’ the black box of the videogame hardware, as Bogost does here, reveals the sophisticated perceptual circuits of this nonhuman system as in a peepshow. However, stepping back to scope the videogame system in its operation, in play, we are reminded that the game is designed to initiate, sustain, and be constituted by, aesthetic, kinaesthetic, social and proprioceptive behaviour with and through human minds and bodies. It is possible to acknowledge the human links in the circuits without necessarily granting them agential or sensory sovereignty. In particular, virtual gravity as I discuss it here cannot exist solely in either the hardware / software of the videogame system or the body and mind of the human player, it is inseparable from either its biological or technical substrates. The move I want to make then, is to factor in human embodiment and perception but to resist re-centring these, to resist seeing technologies (hardware, software, techniques) as extensions of the human body and human action.

Accelerometers and nonhuman proprioception

Technological systems that sense their own environments or internal states can be dated back to at least the beginnings of the steam age, everyday digital entertainment and communication applications are increasingly predicated on nonhuman sensing in new ways and in actual space (satellite positioning that facilitates satnav in cars, GPS mobile phone maps, and innovations in location-aware apps and games), and military/corporate research is casting a net of sensing systems over the globe. To understand the specific proprioceptive operations of recent videogames however, we will now need to focus on in a key technical component: the accelerometer.

Accelerometers in smart phones and the Nintendo Wii controller (wiimote), allow these devices to recognise their own acceleration and orientation, and respond both physically to the player’s movement, and virtually to positioning and action within the devices’ software. Linking solid state circuits and simple mechanics, accelerometers measure movement in three axes, and one of their simplest functions is to trigger the rotation of a smartphone display as the device itself is rotated, keeping the screen image oriented to the earth’s gravitational pull. Another is the detection of and compensation for ‘shake’, small movements of a digital camera user’s hand that might result in a blurred image. Numerous smartphone games use this proprioceptive sensing as a core gameplay mechanic (though not incidentally Angry Birds). In one of the earliest, Crayon Physics for the iPhone (Petri Purho 2008), the player drew simple geometric shapes to help navigate a ball through each puzzle level. The shapes took on simulated mass and kinetic energy and thus the gameplay was generated through the interaction between accelerometer, player, physics simulation, and screen display.

The movement-detection of the Nintendo Wii console is effected in part by accelerometers in wiimotes. However, as Bart Simon explains, the Wii system relies on a multisensory apparatus (proprioception and vision):

The Wiimote has an internal accelerometer (an ADXL 330) that allows for the measurement of motion in a three dimensional space and this information is communicated to the console wirelessly via a Bluetooth radio chip. This alone would be enough to measure horizontal, vertical and forward motion, as well as rotation, but the measurements produced by the accelerometer alone are not accurate enough to correctly correlate the motion with what is happening on the screen. For this, Nintendo developed the infrared LED “sensor bar” which is actually not a sensor at all. The bar contains infrared LEDs at either end that emit light which is detected by a simple IR camera in the tip of the Wiimote. Position calibration is made possible then by triangulating the position of the controller relative to the light emitting diodes (Simon 2009:10-11).

Thus on the one had we are presented with domestic entertainment and communication technologies that – whilst not biological – must be understood as proprioceptive, as self-sensing bodies, and on the other, these bodies must in turn be understood not simply as two discrete interacting entities, but as constituting and constituted by an assemblage of technical and biological subsystems.

In arguing for a correspondence or collusion between human and nonhuman sensing bodies we should not assume identity between these bodies and their sensoria. The bodies in these playful assemblages sense the world and their movement through it in very different ways. An accelerometer perceives its motion in space and in relation to other bodies, including the earth, very differently from mammals. In the Wii system, this difference is evident in two key ways: first, as every Wii player’s body quickly learns, the wiimote does not fully capture the body’s movements and gestures and nor then does it faithfully transcribe these movements and gestures into those of the onscreen characters. Learning the game includes figuring out which gestures will have an instrumental effect in the game, and which are redundant. Indeed this difference can provide the very basis for gameplay practices and pleasures. Simon has noted how Nintendo’s marketing reorients players to the screen and shifts the perceptual relationship from one dominated by vision to one of kinaesthesia. He argues that the Wii apparatus, its ‘gestural interface system’, whilst perhaps not as revolutionary as the marketing suggests, is concretely implicated in a significant phenomenological shift, and demands of its players new bodily techniques and experimentation. Counter to the insistence of Nintendo’s marketing that the system breaks down the virtual / actual boundary as the player’s existing bodily expertise is directly registered and replicated in the gameworld, there is a clear disjuncture between the movement or gesture required of the player by the system to – say – swing a tennis racquet, and the movement or gesture – of the tennis swing – enacted by the screen avatar. In Wii Sports for instance, regular players

discover quickly that not only is there a very weak correspondence between their arm motion and the screen effect but also that, understandably, the many nuances of bodily motion that go into performing better or worse swings cannot be captured by the Wiimote. The implication then is that the Wiimote captures some of the player’s bodily motion and not all. At the other end of the spectrum it also becomes clear as one plays that the Wii console and the game software does not use all the positional information it captures. The game software only used just what it needs, as it were, to produce a correlative screen effect’  (Simon 2009: 18-19).

One does not have to play a Wii game for long to establish something of the bandwidth of this gestural transduction. Through play, the player builds up ‘the minimal gestural map to facilitate the most efficient gameplay’ (Simon 2009: 21). This may be achieved through instrumental trial and error, or incrementally – and more or less subconsciously – as the player is taught by the system over hours of play. The swiping of a sword in The Legend of Zelda: Twilight Princess (Nintendo 2006) can be just as effective through a flick of the wrist as by a swing of the whole arm, as satisfying as the latter may be: Simon points out that ‘gestural excess’ of the sword swipe may be sustained for kinaesthetic pleasure even once its redundancy for instrumental effect in the gameworld is recognised.

Secondly, whilst we can understand the wiimote as a body that registers and responds to its own position and movement through space, it becomes clear that this space has a peculiar topology. The device registers movement in a space of gesture, not the position of the player’s body in some notional grid nor in relation to the domestic environment of furniture and walls. The player can wander around the room without affecting the position of the tennis-playing Mii, whilst his or her arm movements are captured in some detail. For example the familiar momentum-building walk to the lane of the actual ten-pin bowling player, ball in hand, cannot be captured by Wii Bowling. It does not register the whole body’s movement through space. The movement of the player’s arm can be captured, and so the walk is condensed into and effected by the arm swing. Thus the arm swing to release the ball also moves the onscreen avatar through virtual space.  Interestingly, whilst the virtual gameworld simulates gravity (the bowling ball falls to the floor when released), the wiimote is less concerned with obeying natural laws. It is possible to play Wii Bowling by standing with one’s back to the screen and bowling with an overhead (in gestural terms ‘upside down’) motion. The system registers this as a normal, if generally not very effective, movement. The movement-sensing process takes no account of the player’s or wiimote’s bodies’ orientation to the Earth. The system keeps a crude check on the player’s position in space as it notes, with an onscreen message, if the player moves out of the zone within which the wiimote’s infrared sensor can see the ‘sensor’ bar, but this is registered by a loss of signal – by ‘vision’, not a proprioceptive sense of place or position. The well-documented accidents as new Wii players collide with furniture is evidence of the disjunction between biological and machinic sensory regimes. The system issues its screen-based warnings as the console is switched on, but it pays no attention thereafter as it takes hold of the player’s attention and demands it responds only to its own non-Euclidean world.

The point I am making here is not to account for performative gestures, as important as these may be to the kinaesthetic pleasures of Wii play. Rather it is to draw attention to the fact that the gesture’s excess or precision is always in relation not merely to an input signal to the game, but also in relation to the Wiimote system’s proprioception in any particular gameworld.

 

Games and gravity

 

The discussion above opens up ways of thinking about the more or less interwoven workings of human and nonhuman sensing in technological systems, providing a broader context for the consideration of natural and artificial proprioception in general and virtual gravity in videogame play in particular. I will now address the primary assertion made in this essay, that virtual gravity cannot be fully understood as located within either the code of the gameworld software or the player’s body and sensorimotor system. It is a function or product of their collusion. To pursue this I will need to argue – contra Sobchack – that simulated gravity has actual world effects, that it acts – through graphic display, interaction, and feedback – on the player’s sensorium and proprioception. It fuses with the current and remembered gravitational pull of the actual world on the player’s body. Rather than dis-embodying it re-embodies.

Fig. 3: Lunar Lander (drawn from memory).

With its white wireframe graphics on a vacuum black background, the early home computer game Lunar Lander looked something like Spacewar!, its mainframe ancestor (fig.3). And, like Spacewar!, virtual gravity was fundamental to its gameplay. Versions of Lunar Lander were written and rewritten for home microcomputers in the early 1980s; I played it on my father’s BBC Micro. The game is simple: the lander – something like the Eagle landing module from the Apollo space missions – descends towards a treacherous moonscape, its speed increasing exponentially as though accelerating under the influence of a gravitational pull. The player must press a key to fire retrorockets to slow the lander’s descent. Press too long and the lander will begin to ascend again, with the risk of running out of time or fuel. Not enough retrothrust and the craft will land too heavily and be destroyed.

Ace space pilot, Captain Flash, is sitting next to you as you take the final part of your Advanced Spacecraft Handling Test (Part III). Your lightweight, two-man landing craft is rapidly approaching the Moon’s surface. Your velocity must be almost zero as you touch down. Deftly you control the thrust, pressing A to increase it and D to decrease it, watching your progress on the screen all the time. If you use too much thrust you will begin to go back up again. Too little and you will make a new crater on the Moon. Can you impress Captain Flash with your skill? (Isaaman and Tyler 1982: 30).

Captain Flash’s approval notwithstanding, this proved a compelling game, the responsiveness of the virtual physics belied by the simplicity of the graphical display of environment and vehicle. Isaaman and Tyler’s book listed versions of the game’s BASIC code for players to type into their microcomputer:

(For TRS-80 and VIC 20).

power = 0.3;

yspeed = 0;

gravity = 0.1;

thrust = 0.75;

 

if (Key.isDown(Key.UP))

yspeed -= power*thrust;

if (Key.isDown(Key.DOWN))

yspeed += power*thrust;

yspeed += gravity;

_y += yspeed;

The gameplay and controls of the game are very simple, as are its algorithms. The code above sets up only four key factors – including one controlled directly by the player (thrust), a value for gravity (against which the player exercises thrust through a key press), and the resulting (one-dimensional) movement of the lander – yspeed. Yet I would ask those of you who have played this game, or one like it, to try to recall how it feels to attempt to negotiate the interplay between these simulated forces. Pressing the key to fire the retrorockets does not immediately propel the lander upwards. Rather it slows its descent. Similarly, releasing the key does not immediately halt this counter-force: if the lander has slowed almost to a hovering halt, the upward thrust initiated by the player may be enough to send it back upwards again. The resistance built into the algorithmic interplay between gravity and thrust is experienced in the player’s body as the key press either pulls against gravity. The sense of willing the lander to slow as it heads towards the ground too quickly, or the agonising realisation that one has overcooked the retrorockets and over-run the time limit is palpable. Moreover, the thematic, temporal and ludic milieu of the gameworld itself intensifies this feeling. As the lander sails back up away from a near fatal impact and the clock ticks towards its deadline, the virtual physics set a spatio-temporal boundary to the possibility of success. The artificial gravity drags more powerfully on the player’s body as we will it to relax or ease just a little. Thus virtual gravity cannot be separated from either the human body’s proprioceptive experience or the simulated physics. In fact ‘inseparable’ is not the right term: this sense is generated by and across all these bodies – human, software, and hardware – and through both embodied memory and immediate cybernetic feedback. It is distributed, and an artefact.

 

Conclusion

The Angry Birds are worthy of study in their own right as a hugely popular game, emblematic of the arrival of mobile gaming as a media cultural phenomenon. But they are also a popular and paradigmatic example of new sensual relationships with media technology. These relationships are reducible to neither the human body and senses nor digital hardware and software. Rather they emerge through their distribution across these platforms: artefacts of code and physical input, haptics and proprioception, both human and nonhuman.

This essay has argued that in the study of contemporary digital technoculture, firstly – following the insights of the philosophical traditions of the phenomenology of perception – that questions of embodiment and perception are key, but secondly, that we should not limit our understanding of sensing, kinaesthesia, proprioception and embodiment to the human players alone. The software of Angry Birds and Lunar Lander might be considered virtually proprioceptive – i.e. it operates through the tracking of its various bodies or components in virtual space, and the various simulated forces acting upon them. With current haptic, gestural, sensing and location aware media in games (Wii, Kinect, mobile device screens and accelerometers) and beyond (satnav, infra-red motion detection, automatic CCTV cameras, military sensing systems), media technologies should not be thought of merely as extensions of the human sensorium, but on the one hand fully environmental – both virtual and actual, surrounding the human, enfolding and co-opting it, and on the other they are sensing bodies themselves. This collusion between human and nonhuman bodies, between actual and virtual environments offers a way into addressing contemporary digital technoculture in all its sensate and experiential dimensions.

 

 

References

Allain, Rhett (2010), ‘The physics of angry birds’, Wired.com, Oct 8. http://www.wired.com/2010/10/physics-of-angry-birds/

http://www.wired.com/wiredscience/2010/10/physics-of-angry-birds/

Ash, James (2010). ‘Teleplastic technologies: charting practices of orientation and navigation in videogaming’. Transactions of the Institute of British Geographers 35(3), 414-430. http://www.badnewthings.co.uk/playfulsubjects/TAreadingroom.html

Behrenshausen, Bryan G. (2007). ‘Toward a (kin)aesthetic of video gaming: the case of Dance Dance Revolution’. Games and Culture 2(4). 335-354.

Bogost, Ian (2008). ‘The phenomenology of videogames’, in Stephan Gunzel, Michael Liebe and Dieter Mersch (eds) Conference Proceedings of the Philosophy of Computer Games 2008. Potsdam: Potsdam University Press. https://publishup.uni-potsdam.de/opus4-ubp/frontdoor/index/index/docId/2553

Cheshire, Tom (2011). ‘In depth: how Rovio made Angry Birds a winner (and what’s next)’. Wired (UK). 7^th March. http://www.wired.co.uk/magazine/archive/2011/04/features/how-rovio-made-angry-birds-a-winner

Cobb, Edith (1977), The Ecology of Imagination in Childhood,

Crick, Timothy (2011), ‘The game body: towards a phenomenology of contemporary videogame play’, Games and Culture 6(3) May

Crogan, Patrick (2010). ‘The Nintendo Wii, virtualisation and gestural analogics’. Culture Machine 11(1). http://www.culturemachine.net/index.php/cm/article/view/374/397

Dourish, Paul (2004). Where the Action Is: The Foundations of Embodied Interaction. Cambridge MA: MIT Press.

Farman, Jason (2012). Mobile Interface Theory: embodied space and locative media. New York: Routledge.

Giddings, Seth & Kennedy, Helen W., (2010). ‘Incremental speed increases excitement’,Television and New Media. 11(3),.

Giddings, Seth (2014). Gameworlds: virtual media and children’s everyday play. New York: Bloomsbury.

Isaaman, Daniel & Tyler, Jenny (1982). Computer Spacegames, London: Usborne

Keogh, Brendan (2014). ‘Paying attention to Angry Birds: rearticulating hybrid worlds and embodied play through casual iPhone games’, in Gerard Goggin & Larissa Hjorth (eds) The Routledge Companion to Mobile Media. New York: Routledge, 267-275.

Levy, Stephen (1984). Hackers: heroes of the computer revolution, Harmondsworth: Penguin.

Marks, Laura U. (2002). Touch: sensuous theory and multisensory media. Minneapolis: University of Minnesota Press.

McLuhan, Marshall (1964). Understanding Media: the extensions of man. London: McGraw Hill.

Merleau-Ponty, Maurice (2002 [1945]). Phenomenology of Perception. London: Routledge.

Salen, Katie & Zimmerman, Eric (2004). Rules of Play: game design fundamentals. Cambridge MA: MIT Press.

Simon, Bart 2009. ‘Wii are out of control: bodies, game screens and the production of gestural excess’. Social Science Research Network. Online at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1354043

Sobchack, Vivian (1994). ‘The Scene of the Screen: envisioning cinematic and electronic ‘presence’’

Sobchack, Vivian (2004). Carnal Thoughts: embodiment and moving image culture, University of California Press.

Suchman, Lucy (2006). Human-Machine Reconfigurations: Plans and Situated Actions. Cambridge: Cambridge University Press.

Swalwell, Melanie (2008). ‘Movement and kinaesthetic responsiveness’, in Melanie Swalwell & Jason Wilson (eds) The Pleasures of Computer Gaming: essays on cultural history, theory and aesthetics, Jefferson NC: McFarland, 72-93.

Wiener, Norbert (1988 [1950]). The Human Use of Human Beings: cybernetics and society. Da Capo Press.

Winfield, Alan (2012). Robotics: a very short introduction. Oxford: Oxford University Press.