Virtual Dance and Motion-CaptureSkip other details (including permanent urls, DOI, citation information)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. Please contact firstname.lastname@example.org to use this work in a way not covered by the license. :
For more information, read Michigan Publishing's access and usage policy.
A general view of various ways in which virtual dance can be understood is presented in the first part of this article. It then appraises the uses of the term “virtual” in previous studies of digital dance. A more in-depth view of virtual dance as it relates to motion-capture is offered, and key issues are discussed regarding computer animation, digital imaging, motion signature, virtual reality and interactivity. The paper proposes that some forms of virtual dance be defined in relation to both digital technologies and contemporary theories of virtuality.
computer animation, dance, interactivity, motion-capture, motion signature, rotoscopy, telematics, virtual dance, virtuality
Ever since the motion picture recording of Loïe Fuller’s Serpentine Dance in 1895, dance and media technology developed an intimate relationship. Electric light not only made film projections possible but also revolutionized scenography. Modern theatre pioneer Adolphe Appia considered body and light as essential elements of the stage, but the body seemed to disappear in Loïe Fuller’s novel art form, contributing through its twists and turns to a dynamic light spectacle that entranced poets as well as popular audiences. As Fuller animated yards of silk on stage, the audience perceived luminous kinetic forms, dancing light shapes, such as butterflies, flowers or flames. The interplay between electric light and fabric generated a virtual display insofar as the perceived shapes were ephemeral and immaterial in spite of the presence of a body more or less hidden by the fantasmagoria it created. In the case of the Serpentine Dance film, the projected image of the dancer, a realistic illusion created by perceptual factors and specialized technical apparatus, is arguably a virtual body. There are of course various meanings and uses of the term “virtual” besides those implied in these two instances, in fields as diverse as optics, aesthetics, philosophy, and computing. All bear on what we mean by “virtual dance,” a term greatly informed by the technologies that make it possible.
Virtual dance can be understood in ways not necessarily related to technology. Suzanne Langer expanded the use of the term “virtual” beyond its conventional associations with illusory images. She wrote about the virtual character of images, virtual space, virtual powers, and virtual memory. For her, an image is a purely virtual “object,” while a virtual image is the illusion of a space that is new; the creation of something that never existed before.
Following Gilles Deleuze who revived the concept of the virtual in philosophy, Brian Massumi, Erin Manning, and José Gil have addressed topics such as the virtual body, virtual gesture, virtual power, and virtual space. José Gil, for example, writes, “It follows that there is no single body, like the “proper body” of phenomenology, but rather multiple bodies. The body of the dancer, Cunningham’s body, but in fact the body of all dancers, is composed of a multiplicity of virtual bodies.” Burt Ramsay has written on virtual dance, drawing on Langer and on Deleuze’s development of Bergson’s ideas about memory and imagination. In the dance pieces he discusses, “a dancer or dancers take on the role of narrator, so that what develops is in effect a virtual dance.” In a similar vein, Steve Dixon writes:
But in an important sense, theatre itself has always been a “virtual reality” where actors imaginatively conspire with audiences to conjure a belief (otherwise known, after Coleridge, as a ‘suspension of disbelief") that a bare stage is in fact the courtyard of an ancient Theban palace, or the 1692 witch trial courtroom in Salem.
For Pierre Lévy, the virtual is one of four mode of being, along with the actual, the possible, and the real; furthermore, these four modes are described as being almost always at play in a given analyzable concrete phenomenon. Lévy summarises their relation thus: “the real resembles the possible, whereas the actual answers to the virtual.” In new media, virtual reality usually refers to the users" navigation in a three-dimensional digitally imaged world, but can be understood much more broadly in relation to interactive multimedia and digital networks.
2. Virtual Dance as Image
At the very beginning of the electric age, stage and cinema can be seen as implying two different ideas of virtual dance and two different types of images. In order to understand what a virtual dance is in an information technology context, it is important to grasp what has become of images, and how computer generated ones differ radically from previous types.
We can classify images as either unmediated, “natural” images or mediated, “material” images, granting that this is provisional and simplistic. An unmediated, natural image of a mailbox is the mailbox we see on the street. In the nineteenth century, a mediated, material image of a mailbox could be a drawing, a painting, a photograph, or a film of it.
In Fuller’s staged production, we are dealing with a natural image, regardless of the fact that it is illusory. There are no butterflies, flowers, or flames actually present, but a dancer who creates virtual shapes somewhat as a puppeteer. In the case of the film, one sees a material image, a represented image, an image whose support is tied to certain technological developments characteristic of cinema.
From photography to virtual reality, mediated images have acquired new and seemingly magical properties: they can be transmitted instantaneously from viewer to viewer, be interactive, immersive, and so on. Recorded electronic images are unlike previous images because the electronic image is not immediately visible on its material support but only by means of specialized equipment. If you look at a film strip, you can find the corresponding image of what is projected on the screen, but not with videotape. Today, as we moved out of the analogical electronic era and into the digital era, images consist of binary data. Analogical coding has been replaced by digital coding, which can be processed by multiple supports. Computer generated digital images allow the creation of characters onto which motion data captured from performers can be mapped. In this sense, virtual dance refers to the dancing of virtual characters, which are themselves computer generated images, not truly computer animated, given that the motion data are originally captured from a live performer and that these data are processed in ways that can essentially retain its authenticity. In this sense we are not dealing with computer animation but with computer aided animation. As we will see, the relationship of motion-capture to animation is contentious.
3. Virtual Dance, Virtual Dancer, Virtual Body
Dance, dances, dancers, body: all are now often referred to as “virtual” though one is not always sure if the term has an equivalent meaning in each case. Virtual dancer and virtual body can however be understood in terms of the relationship between iconic image and physical body.
It likely all started with CGI (computer generated images) of animated characters. “When the eight-minute short film Tony de Peltrie was presented to the world in 1985, the eponymous character was widely considered the first computer-animated character to truly express emotion through his face and body language.” In those years, “virtual humans” or “digital clones” were designed by Nadia and Daniel Thalmann for Rendez-vous à Montréal (1987), another milestone in computer animation history. Motion-capture was not involved, but characters in the likeness of Marilyn Monroe and Humphrey Bogart were realistically animated in 3D. Synthetic actors or synthespians animated through movement capture were then also in the works at Jeff Kleiser and Diana Walczak’s studio. “Intrigued by the potential of motion-capture to link natural human motion to our synthetic characters, we created Don’t Touch Me, a music video piece…in which singer/songwriter Perla Batalla was optically motion-captured…to drive a singing synthespian called Dozo.”
With motion-capture, we can sometimes recognize the identity of a motion-captured individual through the kinetic likeness of the virtual dancer. An example for this would be Bill T. Jones’ dancing in Ghostcatching (1999) and much of what comes under the heading “motion signature,” where realism of image, image resolution, or even morphological likeness matters less. As E. de Aguiar writes: “Motion is a fundamental visual cue in human perception and slight inaccuracies are directly noticed. Hence, even if the rendering is perfect, the motion will still tell you that it is an artificial scene.”
Jones and his Ghostcatching collaborators pose an important question regarding motion-capture: “What is human movement in the absence of the body?” I will later on reframe that question to whether or not the relationship between motion-capture stored as binary data and its later computer use for animating screen characters can be likened to that between virtualization and actualization.
4. Virtual Reality
"Virtual" became a buzzword associated with new digital technologies in the late 1980’s, most notably with Jaron Lanier’s use of the term “virtual reality.” In the following quotation, “virtual reality” is broadly defined, in line with its multiple present applications: “Virtual reality thus seems like a catch-all field within the engineering sciences. It manipulates images that are interactive, multimodal (3D, sound, tactile, kinesthetic, proprioceptive), realistic, animated in real-time, and shared on computer networks.” The European Center for Virtual Reality defines it even more narrowly:
"Virtual reality is a scientific and technical field using information technology and behavioural interfaces to simulate the behaviour of 3D entities in a virtual world. They interact with each other and with one or more users in real time, through pseudo-natural immersion via sensory motor channels".
This definition corresponds to virtual reality as a “place” where users can collaborate in various endeavors through their respective avatars. As an art form, VR (virtual reality) was designed in order to provide immersive and interactive experiences in 3D simulated environments, which are not necessarily replicating an existing physical environment but can be novel worlds to discover through navigation. The corporeal quality of immersion and the importance of the embodied presence of the user is often emphasized in the artists’ statements, though the cumbersomeness of devices and slowness and roughness of image rendering could impede the quality of the experience provided. Since only one user at a time can be immersed in it, these works were also presented as installations, designed to be appreciated for their exterior appearance.
Legible City (1989) by Jeffrey Shaw and D. Groeneveld.
Sitting on a stationary bicycle, the user can explore a virtual city displayed on a large screen in front of her, according to the directions she gives the bar handles and the speed at which she pedals.
The Virtual Museum (1991) by Jeffrey Shaw.
In this installation, the user navigates through the museum displayed on a large screen in front of him depending on how he moves his center of gravity, that is to say by impelling slight inclinations and twists to the armchair he is sitting on.
Home of the Brain (1992) by Monika Fleischmann and Wolfgang Strauss.
In this immersive installation, the user dons a HMD (head-mounted display) helmet and a dataglove and physically as well as virtually moves through an installation that comprises screens that allow external observers to see what she sees.
Cluny (1993) by Medialab.
This work combines immersive VR and telepresence. Stereoscopic HMD wearing users can meet in a simulation of the abbey of Cluny through their respective avatars, small icons that each can move with their 3D mouse. The physically distant users can also hear each other through telephone lines used in parallel to the digital network connection.
Dancing with the Virtual Dervish (1991-1994) by Diana Gromola, Yakov Sharir, and Markos Novak.
The immersion in VR is provided by stereoscopic HMD and is described as one of five cumulative spaces along with the physical space, the cyberspace, the telematic space and body space.
Osmose (1994–95) by Char Davies.
This installation consists of two spaces, one where a user (the “immersant”) wears a HMD and a motion-capture vest equipped with breathing and balance sensors, and the other where spectators watch screens from which to see both the immersant’s VR point-of-view and shadow.
Other works, even more uncommon, were created for CAVE (cave automatic virtual environment) or even necessitated full body datasuit for the users to wear.
When virtual dance is understood as being related to VR, sensors allow navigation through body motions, providing a more physically and kinaesthetically engaging experience than clicking on a mouse while looking at a computer screen. The type of interface used determines to a great degree the sensorial quality of the experience. The preceding examples show that dancing in virtual reality is to be understood as one of the possible meanings of virtual dance.
5. Overview of the Term ‘Virtual Dance’
The meaning of the term ‘virtual’ when joined to “dance” in a technological context can take many forms and exploit various properties of computer images, such as can be inferred from artistic statements, research reports, technical papers, historical accounts, press releases, and various publications relating to staged works, installations, software programs, electronic devices, networks (including internet), “interactive” DVDs, and the rest. Johannes Birringer points to two different concepts of virtual in relationship to dance as it appears on screens:
On the one hand, it is true that video dance, as the precursor of digital dancing and web-based dance, is a hybrid form, existing in a virtual space contextualized by the medium and method of recording.…On the other hand, the impact of digital technology on the moving image (video, cinema) is quite paradoxical, if we recall that the history of fictional films as live action films is grounded in lens-based photographic recordings of reality--actions that took place in real physical space.
It is indeed important to point out that digital technologies allow not only non-linear editing but, through the use of motion-capture, to dispense with an optical camera and therefore with point of view; the original “real physical space” is erased. As we move from traditional cinema to CGI cinema and motion-captured feature films, the term “virtual” takes on a precise technological meaning wholly different from that in Langer, Deleuze and others.
Birringer offers no definition for the term “virtual” in his 1999 article, nor for the numerous associated terms he writes about: “virtual space,” “virtual dance installation,” “virtual performance,” “virtual implications of interactivity,” “virtual environments,” “virtual studio,” “virtual geography of potentially infinite computational possibilities,” “virtual consciousness,” “virtual movement,” “virtual touch with the eyes,” “virtual site,” “virtual concert,” “virtual reality,” “virtual performance space,” “virtual body,” “virtual images,” “virtual-body environment,” “virtual stage” and “virtual stage space.” In a subsequent article (2002), he refers to the “contentious internet debates about emerging definitions of “virtual” or “digital” dance,” which goes to show that these two terms were already considered equivalent. Throughout the literature, and even within a given article, the term “virtual” is often used in place of “telematic,” “cyber,” “web,” “intangible,” “quasi,” “unreal,” or even for all sorts of screen images, projected or displayed, synthetic or natural. Within the context of art digital technologies, the term “virtual” somehow works as a wild card, given the conceptual and technological complexities of the field.
In discussions on virtual dance, the notion of interactivity usually crops up, even though the term “interactive” is often used in lieu of the more suitable “reactive,” that is to say when reciprocity, mutuality, or feedback are not involved. Whatever the case, virtual dance can be deemed interactive, as in Cécile Babiole’s DO (not) DISTURB (2004), an installation in which spectators are invited to “disturb the choreography” by moving themselves in front of the screen.
Virtual dance can be immersive, such as in Steve Paxton’s video installation Phantom Exhibition (2009), which consists of five large screens showing video footage of Paxton and other performers, as well as computer-generated images analyzing their dance moves. Surrounded by images projected on the installation’s four walls and ceiling, visitors are presumed to be perceiving “with all their senses” the relationship between the human body and gravity, between the interior and exterior of the body, and the structure of its bones. Virtual dance can also be collaborative dancing in 3D virtual space, as in Collaborative Dancing in Tele-immersive Environment, studied at University of Illinois at Urbana-Champaign’s Department of Computer Science in 2006.
Though not expressly presented as a work in virtual reality but as a performance / installation, Rebecca Allen’s The Brain Stripped Bare (2002) is nonetheless described in reference to virtual reality. This work considers a future where we live simultaneously in multiple realities, where the boundaries between physical and virtual reality are blurred and thoughts are expressed telepathically. ...Surrounded by a circle of screens the audience is free to shift their point of view. Live performers merge with shadows, projected images and sounds, revealing stark human forms that move in startling and perplexing ways.
As for dancing in virtual reality per se, here is how Yakov Sharir, (dancer and co-creator In Dancing with the Virtual Dervish project, see) described it:
So the zero gravity has changed the notion of how dancers move in cyberspace. Not physical and human dancers, but cyber-humans and cyber-dancers move differently. But still I have to compose that, so I have to relate to that. And in another way, as in virtual reality, as opposed to animation, then you, as a performer and as a human, you are then immersed in the cyberspace with a helmet and a goggle. Right? And you navigate in cyberspace with an electronic glove. Okay. So even though you are grounded in a physical space, you are immersed in cyberspace, and you live now two lives; one in the physical space and one that you are immersed in, which is cyberspace via your goggles. Disconnected from the physical world. Entering the cyber world that is designed on the computer.
A 2003 Dance Magazine article revealed some of many ways virtual dance can be understood as it dealt with telematics, “intelligent stage” (where detection of dancers’ motions trigger various effects), motion-capture and animated characters; “data can be used to manipulate “cyber-humans” in virtual spaces in order to apply authentic movement quality to animated characters,” and the use of Life Forms  animation software.
More recent articles on virtual dance reflect what newer technologies permit, such as the interplay of live dance and 3D scenographic real-time animated virtual-environments (RAVEs). Interest in virtual scenography has been growing and is apparently greater in theater than in dance. Virtual dance can even be understood outside the visual realm from a kinaesthetic and haptic perspective, such as with the Immersence project at Munich’s Technical University:
Dancing is one of the two IMMERSENCE scenarios for direct Person-to-Person interaction in virtual environments. TUM aims at creating a virtual dancing partner based on the concept of Record – Replay – Recreate: First the interaction between dancing partners is recorded. Then one of the partners will be substituted by a robot providing the recorded haptic information to the partner. Finally, the main challenge will be to construct a haptic and visual agent which is realistically interacting with the human partner.
Haptics, a research domain pioneered by Claude Cadoz and collaborators, presents great challenges but corresponds to a need to expand new media beyond the ocular and aural modes, and the development of more complex and engaging interfaces than the usual keyboard, computer mouse or touch screen. That being said, what is presented elsewhere as virtual haptics in a stage performance context is in fact a visual illusion astutely fashioned in order to give the impression that the dancer actually touches the projected visuals.
A mainstream conception of virtual dance has mostly to do with motion-capture, physical modelling and animation techniques since those technologies are increasingly used in cinema. Motion-capture can be traced back to the work of Eadweard Muybridge and Etienne Jules Marey. Cinema, as the term “motion picture” literally expresses it, implies the capture of motion. Rotoscopy is a technique invented in 1915 by Max Fleischer and used by Rebecca Allen for Twyla Tharp’s Catherine Wheel (1982).
At the New York Institute of Technology Computer Graphics Lab, Rebecca Allen used a half-silvered mirror to superimpose videotapes of real dancers onto the computer screen to pose a computer generated dancer for Twyla Tharp’s “The Catherine Wheel.” The computer used these poses as keys for generating a smooth animation. Rotoscoping is by no means an automatic process, and the complexity of human motion required for “The Catherine Wheel” necessitated the setting of keys every few frames. As such, rotoscoping can be thought of as a primitive form or precursor to motion-capture, where the motion is “captured” painstakingly by hand.
Rotoscopy can be seen as a form of motion-capture, while others would argue that it is an animation technique. It consists of extracting the movement of a figure from a filmed sequence by means of sketching it on a transparent support placed over each photogram. Computerized motion-capture can be regarded as an automated way of doing what was previously done manually through rotoscopy. “Motion-capture is basically 3-D rotoscoping. If you accept rotoscoping as a form of animation then you have to accept motion-capture.” If motion-capture is a form of animation, then motion-captured dance is computer animation, though one can argue that the motion is the performer’s and not the computer’s. But since the computer is absolutely necessary, motion-capture should arguably be understood as a form of computer animation, which is why I would refer to it as computer-aided animation. Going back to the debate over the status of rotoscopy, Maureen Furniss writes:
Certainly, mocap shares with roto animation the close relation to a model’s form (human motion). The extent to which the two are related can vary dramatically, some feeling of the “presence” of a human being still exists in most animation of these types. When watching a film like FIeischer’s Gulliver’s Travels (1939), it is of course easy to sense that Gulliver is quite different in essence from the little people around him; you don’t have to know that rotoscoped footage was used to create him in order to sense that difference.
However, for reasons of its own, The Academy of Motion Picture Arts and Sciences decreed in 2010 that “Motion-capture by itself is not an animation technique.” As with rotoscopy so with dance capture, so to say, some believe that motion-captured dance is not computer animation, but “dance freed from the body.”
The main kinds of motion-capture systems are mechanical, optical and electromagnetic, each with a given set of advantages and disadvantages. Moviegoers are familiar with motion-capture in blockbuster animation features, such as Robert Zemeckis" Beowulf (2007) and James Cameron’s Avatar (2009). Since such “films” are more closely related to acting than dancing and involve capturing fine details, like facial expressions and movements of fingers, the term “performance capture” has gained currency. Such high degree of image definition is perhaps not necessary in dance, insofar as full body motion is sufficient to convey it kinaesthetically, especially since the dancing character’s form can be abstracted, as in Ghostcatcher or in Cunningham’s Biped (1999). In virtual dance provided by motion-capture, it is not the lifelikeness of the form that matters, but the convincingness of its movements.
With motion-capture, one of the ways dance is virtual is in that it can be seen from practically any point of view (that which a “virtual camera” provides); from any angle or distance chosen by the viewer / user. A selected playback of it is but one actualization among limitless others since several motion sensors strategically placed in the capture area are used all at once to record the movement of markers on the dancer’s body. Interestingly, motion-capture technologies evolved from a system developed at MIT in 1983 called the Graphical Marionette, and a low resolution but real time “computer controlled puppet” named Waldo was introduced in 1988 by Jim Henson Productions (of The Muppets fame).
At variance with what a camera in video and cinema records, motion sensors (or trackers) register only the location and motion of markers, which are placed on the performer’s suit. Neither figure nor background are recorded, those are to be applied later on. For the capture session, several motion sensors are laid out in the dance space, the more the better in terms of accuracy, so that all the markers on the body can be tracked through time. The data can then be mapped onto a physically modelled figure (i.e. virtual dancer) in order to animate it, making it possible to view the dance sequence from any perspective. Given technical and physical constraints, a limited number of sensors and markers can be used; the system may allow the computer to determine some values through interpolation. Nonetheless, the data are considered objective or absolute in that they do not correspond to a point of view, as an optical camera necessarily must in order for it to provide coherent images. A “virtual camera” can go through or around the scene in ways that would be impossible on a physical stage; in other words it is the user’s perspective (i.e. navigating) that constitutes the virtual camera. In film and video, camera placement and movements have of course to be established beforehand unless they are improvised. With motion-capture, one can decide afterwards where the virtual camera is placed and how it moves. In other words the camera movements can be choreographed over and over, while the performance quality remains the same insofar as it corresponds to a single take (capture session).
Motion-capture is the most objective form of dance notation insofar as it does not rely on subjective appreciation and verbal descriptions of individuals but rather on predetermined mathematical means of specifying spatial coordinates along x, y and z axes at given moments for each marker. These data can be interpreted (inscribed, “read,” and “performed”) cybernetically (human-machine communication) while previous dance notation methods are based on symbolic representations, written and read by humans alone.
Motion-capture data can be used to create new art forms, and data visualization is one of many new hybrid art forms that digitization makes possible. One recent and outstanding instance of this is William Forsythe and Ohio State University’s Synchronous Object For One Flat Thing Reproduced (2009) project, first phase of the so-called Motion Bank, a “repository of ideas developed through choreographic investigation.”
7. Motion Signatures
Motion-capture allows idiosyncratic qualities to be recorded and played back on characters that are morphologically different from their source; one could get a Laurel to move like a Hardy and vice versa. M. A. O. Vasilescu, a researcher and developer in the field of “Biometric Computing using Perceptual Signatures,” published in 2002 an algorithm that extracts motion signatures on the basis of a person’s gait (walking, dancing, or running) and that can be used to animate characters.
Given motion-capture samples of Charlie Chaplin’s walk, is it possible to synthesize other motionsin his distinctive style? More generally, in analogy with handwritten signatures, do people have characteristic motion signatures that individualize their movements? If so, can these signatures be extracted from example motions? Furthermore, can extracted signatures be used to recognize, say, a particular individual’s walk subsequent to observing examples of other movements produced by this individual?
Insofar as the motion-captured corresponds to selected points on the body’s surface, mapping those points on an avatar that is morphologically dissimilar creates some kind of a kinetic aberration. The captured data is superficial, a body’s movement does not originate from the body’s surface, and the data relative to the muscular, skeletal, tissue components are not taken into account, as they are in kinanthropometry. Body volume data, body mass properties data, center of mass data and moment of inertia data, of both whole-body and body segments are not accounted for.
To illustrate this problem, let us turn to the animal world. A giraffe does not move like an anteater, mapping motion data from one onto the other results in something that is physically impossible because of the extreme anatomical and morphological variance. Of course, the resulting computer animation could be displayed on a screen, recognized as biological motion per se, and perhaps the anteater will be perceived as moving somehow like a giraffe and vice versa. But that amounts to kinetic teratology, such creatures can exist in a virtual world, but are unlikely to be found in the physical one. In her paper about both Ghostcatching and Biped, Ann Dils expresses the following critique of motion-capture:
But motion-capture only records movement in space and time, omitting any direct indication of flow (the relative tension or relaxation of muscles) or changes of weight (our relationship to gravity). In a live mover, flow would show up on the surface of the body, as the dancer tensed or eased muscles. Weight changes show up in the thousands of accommodations movers make in their muscles and skeletons as they drop into or overcome the force of gravity or interact with other people or objects. Motion-capture sensors record the motion of a finite number of points, not really the whole body, so some of the pliancy and articulateness of the body is lost.
Contemporary research in Japan shows that quantification of dance movement can be more comprehensive than with the usual motion-capture protocols when it combines optical motion data and biophysical information.
According to Dick Tomasovic, Loïe Fuller did not wish to be filmed dancing La Danse Serpentine because she sensed that her dance could not bear it. Her work on light and movement was somehow essentially too choreographic to be cinematographed. As a new type of show resting on the advent of electric lighting, and full participant in the dream of a revolutionary scenography formulated by modern stage and staging theorists, the Serpentine Dance presented danced movement as a disembodied energy going beyond the defining boundaries of the human body, even annihilating the idea of corporeality, transforming stage presence into a pure animated image.
Indeed, when one looks at the 30 second film featuring Annabelle More, one cannot fully understand what the fuss was all about. Of course the Parisian luminaries were not raving about the Edison Company film but about Fuller’s stage performance, which technically speaking could not have been filmed anyhow since shoots had to be done outside, in the blazing sun, given that films were not light sensitive enough for indoor scenes with artificial lighting.
Sally Banes did a lot to rescue Fuller from the oblivion of forgetfulness in which she fell for decades after her death. Not only did Fuller begin to lay the groundwork for modern dance before Isadora Duncan and others, but also “made radical changes in art dance [that] would remain latent until the 1960s.” Many dance historians had not considered her as a “real” dancer nor her work to be “dance.” Perhaps history is repeating itself insofar as appreciation of virtual dance is concerned.
9. Virtual Dance as Motion-Captured Dance
If motion-capture through rotoscopy is not animation, then neither is motion-capture with digital technology. Motion-capture should be granted an autonomous ontological status; after all, it does make virtual performing possible. Kevin De Spain asks three seemingly important questions regarding the ontology of dance as dance is transformed by digital technology:
[S]hould a motion-captured documentation of an existing live, human dance (later viewed on computer as performed by digital dancers) also be considered a dance?.…Why not simply reach out welcoming arms to encompass all movement under the aegis of dance? Why should the technology affect the ontology if movement is the essence of the work? I have personally argued for what might be called a “medium- based” (as opposed to a movement-based) criteria for parsing the dance from the dance-related, because film and video and graphical computers have as much claim to being movement-based media as does dance.
Surely virtual dance must be conceptualized in a way that distinguishes it from animation film, kinetic art, robotics and so on. During the 1990s, the term ‘virtual’ became fashionable and began to be applied retrospectively to earlier media, as I did earlier in my introduction. While it is arguably legitimate to claim that a telephone conversation takes place in a virtual world of some sort or that the interlocutors are virtually present to each other, it remains that what one means by virtual in a context that could be contemporary with Alexander Graham Bell is wholly different from what it can mean in the digital age. Equating telematic with virtual reveals in some cases a confusion of the spatiotemporal configurations that technologies foster, those where distance is abolished with those where time is abolished. In the actualization of the virtual according to navigational preferences, as is the case with virtual reality and motion-captured performances, the stored data are accessed and deployed according to a timeline, an ordered succession of configurations or events, in other words, an instance or an effectuation. The real time of a telephone conversation or of a telematic dance performance is not the real time of computer processing.
We can only conjecture about what Deleuze would have had to say about virtual dance and motion-capture, but as Mark Poster states, “…Deleuze did not theorise technology. Even worse from the standpoint of investigating new media, Deleuze not only does not theorise media, he rarely mentions the term.” If virtual dance is neither dance nor animation, maybe we should consider it simply as “motion-capture” which would also includes films like Beowulf and Avatar, along with dance based works like Ghostcatching, Biped and Synchronous Object For One Flat Thing Reproduced.
Motion-capture could be a criterion for parsing the virtual dance from the virtual dance-related, given the universality of the principles involved in it as a digital technology, despite the singularity of custom-designed systems, devices and methods for specific artistic projects.
Yet if we consider that dance is a performing art, requiring embodied presence, virtual dance (in the technological sense of the word) does not qualify as dance. Motion-captured virtual dance has no weight, no breath, no spontaneity. No interpretative skills come into play, since there are no performers interpreting a choreography or improvising a dance but data being processed according to the hardware and software used. What does drawing from an archive have to do with breathing across a space where the laws of gravity apply, or even from dancing in micro-gravity as Kitsou Dubois does?
It is not too early to try and answer De Spain’s previous three questions, given the widespread use of the term ‘virtual dance,’ the variety of phenomena it refers to, and the rapid succession of newer technologies and devices used in dance and related arts: “Should a motion-captured documentation of an existing live, human dance (later viewed on computer as performed by digital dancers) also be considered a dance?” As long as the term “dance” itself is not defined, it depends on one’s underlying assumptions about what dance is, so one could consider such documentation a dance if we made no distinction between presentation and representation, choreography and motion-image editing, unidirectional time and manipulated time.
The expression “performed by digital dancers” here is, of course, a metaphor and its use is open to critique. It is unclear if it is to be interpreted as irony or as part of an ambient technological rhetoric that plays on the difference in meanings of the word ‘perform.’ On another sobering note, Brian Massumi reminds us that, from his perspective, “What’s on the screen is an icon. What’s behind it is a set of permutations and algorithms and logical possibilities. None of these things are [sic] the virtual.” The digital dancers on the screen result from computer processing (motion and character modeling data) according to selected items listed on the menu. The computer performs in the sense that it carries out instructions: it processes information. A far cry from the dancer who does not merely “execute” a choreography but embodies it, interprets it, infuses it with his or her own artistry.
Yet “dance” is an adequate enough word to describe some of the activities displayed by virtual characters when compounded with “animation,” “computer,” “motion-captured,” or “virtual.” “Why not simply reach out welcoming arms to encompass all movement under the aegis of dance?” It would be preposterous to lump kinetic arts, robotics, gymnastics, cinema, multimedia and what not under the dance umbrella. A more sensible proposition is to encompass all dance under the aegis of movement. “Why should the technology affect the ontology if movement is the essence of the work?”
Though choreography and cinema can be placed under the aegis of movement, both are clearly distinct, regardless of the interesting rapprochements that can be made which would place cinema, at least some films, under the aegis of choreography. As seen, De Spain provides an answer to his own questions when he writes: “I have personally argued for what might be called a ‘medium-based’ (as opposed to a movement-based) criteria [sic] for parsing the dance from the dance-related, because film and video and graphical computers have as much claim to being movement-based media as does dance.” It would be tempting to say that the media are the essence of the work (pun intended). However, we must often redefine what ‘media’ means in an era not only increasingly defined by “social media” but also where telephones are smart, able to do so much more than most home computers could not do so long ago.
We need not portray dance as a prisoner of the body in order to commend ventures into digital dance, nor for any other reasons. There is no dance without a body, though we can wax poetic about the light dance of the aurora borealis or anything else that shimmers, flickers or quavers. For that matter, coining expressions like ‘virtual haptics’ when referring to a process that does not involve the performer’s sensory experience of touch but creates an illusion of causal relationship between ostensible gesture and visual display has little to do with virtuality and haptics per se but with the audience’s imagination.
We should reserve the term ‘virtual’ for dances that conform to a stricter definition of it than that often implied when referring to web dances, telematic dances, or projected pre-recorded dancing figures. The term ‘virtual’ can be used casually to refer to anything that appears on a screen or, with greater insight, such as when understood in its relation to the term ‘actual.’ Motion-capture virtualizes dance; physical modeling and character animation actualize it. We can only perceive actualized instances of virtual dance. The term “virtual dance” could also be used in relation to robotics (and teleoperation) with free standing autonomous robots.
When dealing with dance, we can refer to very different types of experiences, whether that of the spectator, the dancer, the accompanist, or the choreographer. Likewise with virtual dance insofar as the technology conditions it; the experience of a “user” that interacts with a computerized system via a gesture interface differs from that of a participant ("immersant") in virtual reality. But, of course we are never truly immersed in virtual reality; our sense of weight is grounded in the physical reality, which is also the space where we breathe. This experience is familiar to those who engage in “virtual skiing” or other sports in their living rooms. So how different is virtual dance as provided through motion-capture physical modeling and character animation from dance or other dance-related technological forms?
A legitmate answer is that with virtual dance through motion-capture, the traditional link between body and movement has been severed, since the movement originally produced by a live body can be mapped onto any virtual body. The passage from optical capture to motion-capture represents a quantum leap in new media arts. It will take more time and effort to begin to understand the wide range of theoretical implications of this leap into the digital.
While the term ‘virtual dance’ will certainly remain in use in its very broad sense (especially as a synonym for telematic or internet), I think its use is more legitimate in instances when a process of actualization is going on; “where the actual answers to the virtual” as Lévy writes. In other words, something has to be happening hic et nunc, here and now, each instance as unique. There is of course no live performer but a user who can actualize the dance that virtually exists in the computer’s memory as data.
Ghostcatching (Bill T. Jones) does not meet this criterion since it replays itself over and over again, identically, like some video loop: repetition, not actualization. DO (not) DISTURB (Cécile Babiole) exhibits the hic et nunc quality, but its array of options is quite limited; what is made manifest happens in a way akin to sampling and DJing, since the displayed dance was not motion-captured but optically captured. There is a degree of interaction which enables actualization, but what is on the virtual side of the device, so to speak, is not nearly as rich as what motion-captured dance could provide.
Motion-captured virtual dance is distinct from computer animation and dance film since each actualization depends on input from a user which makes it unique. To illustrate the ambiguous character of motion-captured virtual dance, imagine that it results from the presupposition that dance is proximate to physics (mechanics), cinema and video to mathematics (geometry), and virtual reality to metaphysics. To conclude on a more serious note, virtual dance as a product of digital technology must be understood both in technological and philosophical terms, and in relation to other multimedia forms where embodiment is a major issue.
Marc Boucher is an interdisciplinary artist who has published papers on synaesthesia, peripheral vision, presence, immersion and other topics related to dance and media arts. He is Associate Professor at Université du Québec à Montréal.
Published on March 17, 2011.
Performed by Annabelle Whitford Moore imitating Loïe Fuller’s original dancing in a film produced by the Edison Company. Accessed on 7 January 2011. http://www.kino.com/edison/qt3.html.
In optics, a virtual image is a reflected image which seems to lie behind a mirror’s surface. In holography, a virtual image is not a reflected image, though it may appear to lie behind the support’s surface.
For Deleuze, virtual is “is real without being actual, ideal without being abstract.” Gilles Deleuze, Bergsonism, trans. H. Tomlinson and B. Habberjam (New York: Zone Books, 1988), pp. 97-98. See also Gilles Deleuze, and Claire Parnet, Dialogues (Paris: Flammarion, 1996), annex to Chapter Five.
Steve Dixon, “A History of Virtual Reality in Performance,” International Journal of Performance Arts and Digital Media, 2.1 (2006), 23-54, ref. on p. 24.
Antonin Artaud is quoted in the same passage: “theatre’s virtual reality develops (...) [on the] dreamlike level on which alchemist signs are evolved.”
For Deleuze-inspired investigations into the virtual and “virtual realities” (other than its technological forms), see: Andrew Murphie, “Putting the Virtual Back into VR” in A Shock To Thought: Expression after Deleuze and Guattari, ed. B. Massumi (London: Routledge, 2002), pp. 188-214.
One could maintain that perception is always mediated, that an image always results from some form of mediation. While it is true that all images are constructed in the visual cortex, according to how photoreceptors are stimulated in the eye’s retina, mediation involves many processes; biological, psychological, cognitive, technological, and so forth. Perception, mediation and image are not understood the same way across disciplines.
Emru Townsend, “Along the Banks of the St. Lawrence...” Animation World Magazine 3.12 (1999) accessed on 7 January 2011; available from www.awn.com/mag/issue3.12/3.12pages/townsendcanada.php3.
"In striking contrast to the awkward, robot-like characters in earlier computer films, De Peltrie looks and acts human; his fingers and facial expressions are soft, lifelike and wonderfully appealing. In creating De Peltrie, the Montreal team may have achieved a breakthrough: a digitized character with whom a human audience can identify.” Time Magazine, August 5, 1985, accessed on 7 January 2011; available from www.time.com/time/magazine/article/0,9171,1048473-3,00.html.
Jeff Kleiser, ‘Synthespianism” accessed on 7 January 2011; available from www.KurzweilAI.net.
See also: Landon Brooks, ‘Synthespians, Virtual Humans, and Hypermedia" in Edging Into the Future: Science Fiction and Contemporary Cultural Transformation. (University of Pennsylvania Press, 2002) pp. 57–59, and Barbara Creed, “The Cyberstar” in The Film Cultures Reader (London: Routledge, 2002).
"Ghostcatching is a digital art installation that fuses dance, drawing, and computer composition. Paul Kaiser and Shelley Eshkar created the visual and sound composition; Bill T. Jones created and performed the dance and vocal phrases.” Accessed on 7 January 2011; available from www.openendedgroup.com/index.php/artworks/ghostcatching/.
Edilson de Aguiar, “Character Animation from a Motion-capture Database” (Master Thesis, University of Saarland, Max-Planck-Institut für Informatik, 2003) p. 77, accessed on 7 January 2011; available from
Bill T. Jones, Paul Kaiser, and Shelley Eshkar, ‘Statement,” (n.d.), accessed on 7 January 2011; available from http://openendedgroup.com/index.php/artworks/ghostcatching/.
Jacques Tisseau, “Virtual Reality: in virtuo autonomy,” (Accreditation to Direct Research Thesis, University of Rennes 1, 2001,) p. 16-17; accessed on 7 January 2011; available from http://www.enib.fr/~tisseau/doc/hdrJTuk.pdf.
Accessed on 7 January 2011; available from www.medienkunstnetz.de/works/the-legible-city/.
Accessed on 7 January 2011; available from www.medienkunstnetz.de/works/the-virtuel-museum/.
21. Accessed on 7 January 2011; available from www.medienkunstnetz.de/works/home-of-the-brain/.
Accessed on 7 January 2011; available from www.mediaport.net/CP/CyberScience/BDD/fich_054.en.html.
Accessed on 7 January 2011; available from www.youtube.com/watch?v=EIQw_QTshLU.
See: Diane Gromala, and Yacov Sharir, “Dancing with the Virtual Dervish: Virtual Bodies” in Immersed in Technology (Cambridge, MA: MIT Press,1996) pp. 281–286. Also, Diana Gromola, Yakov Sharir, and Markos Novak, Dancing with the Virtual Dervish (Austin, University of Texas, 1993).
Accessed on 7 January 2011; available from www.immersence.com/osmose/index.php.
“Common to most virtual reality/performance work is the notion of building a customized input device that becomes a part of the work itself. The computer takes the input information and more or less immediately calculates a perspective within the 3-D environment and renders and displays this as ‘output’ to the user/viewer/audience member via projection devices.” Scott deLahunta, “Virtual Reality and Performance,” Performing Arts Journal. 24.1 (2002), 105–114, ref. on p.105.
Accessed on 7 January 2011; available from www.babiole.net/spip.php?page=english&id_article=38.
Accessed on 7 January 2011; available from www.ycam.jp/en/art/2009/04/steve-paxton-phantom-exhibitio.html.
Yamaguchi Center for Arts and Media. April 6 Press release, (2009), accessed on 7 January 2011; available from www.ycam.jp/en/pdf/Paxton_pr_en.pdf.
Zhenyu Yang, Bin Yu, Ross Diankov, Wanmin Wu and Ruzena Bajcsy “Collaborative Dancing in Tele-immersive Environment” (2006) accessed on 7 January 2011; available from cairo.cs.uiuc.edu/teleimmersion/files/MM_sp65_yang.pdf.
Archived promotional material, accessed on 7 January 2011; available from http://netzspannung.org/cat/servlet/CatServlet?cmd=netzkollektor&subCommand=showEntry&lang=en&entryId=70089. Also available on artist’s website accessed on 7 January 2011; www.rebeccaallen.com/v2/work/pop.mov.php?ID=24.
Y. Sharir, “What Tools do You Use?,” (n.d.), accessed on 27 January 2009; available from [formerly http://internetv.com/sharir.htm].
On the topic of “The bifurcated self – existing isochronically in both the real and the virtual worlds” see: Jacquelyn Ford Morie, “Performing in (Virtual) Spaces: Embodiment and Being in Virtual Environments” International Journal of Performance Arts and Digital Media, 3.2&3 (2007), 123–138.
"...Life Forms is not revolutionizing dance but expanding it, because you see movement in a way that was always there -- but wasn’t visible to the naked eye.” Merce Cunningham quoted on Credo Interactive website, makers of 3D character animation software Life Forms. Accessed on 7 January 2011; available from www.credo-interactive.com/.
Technische Universität München. “First Steps in Virtual Dancing – RECORD & REPLAY of the Disco Fox", accessed on 7 January 2011; available from www.immersence.info/Content_Results/Articles/Article_TUM_WP5_M30.htm.
See: Kim Vincs and John McCormick, “Touching Space: Using Motion-capture and Stereo Projection to Create a ‘Virtual Haptics’ of Dance,” Leonardo, 43.4, (2010), 359–366. Abstract: “This paper describes the work of a group of artists in Australia who used real-time motion-capture and 3D stereo projection to create a large-scale performance environment in which dancers seemed to ‘touch’ the volume. This project re-versions Suzanne Langer’s 1950s philosophy of dance as ‘virtual force’ to realize the idea of a ‘virtual haptics’ of dance that extends the dancer’s physical agency literally across and through the surrounding spatial volume. The project presents a vision of interactive dance performance that ‘touches’ space by visualizing kinematics as intentionality and agency. In doing so, we suggest the possibility of new kinds of human-computer interfaces that emphasize touch as embodied, nuanced agency that is mediated by the subtle qualities of whole-body movement, in addition to more goal-oriented, task-based gestures such as pointing or clicking.”
Max Fleischer, US patent 1242674: Method of producing moving-picture cartoons, issued 1917-10-09. See Richard Fleischer, Out of the Inkwell: Max Fleischer and the Animation Revolution. (University Press of Kentucky, 2005).
David J. Sturman, “A Brief History of Motion-capture for Computer Character Animation” accessed on 7 January 2011. Available from www.siggraph.org/education/materials/HyperGraph/animation/character_animation/motion_capture/history1.htm See also artist’s website accessed on 7 January 2011. www.rebeccaallen.com/v2/work/pop.mov.php?ID=24.
Norman Badler of the Center for Human Modeling and Simulation quoted in Maureen Furniss, “Motion-capture,” [paper on-line] (1999), accessed on 7 January 2011; available from http://web.mit.edu/comm-forum/papers/furniss.html.
Article accessed on 7 January 2011; available from www.oscars.org/press/pressreleases/2010/20100708.html.
"Dance should not drag behind because it is a prisoner of the body.” ("Il ne faudrait pas que la danse tire de l’arrière parce qu’elle est prisonnière du corps.”), attributed to Martine Époque (LARTech) by Frédérique Doyon, “L’entrevue - Danser sans corps,” Le Devoir 27 August 2007. Newspaper article accessed on 7 January 2011; available from www.ledevoir.com/2007/08/27/154744.html.
More from LARTech website: “In their recent (Tabula Rasa: sequel, 2003) and present (NoBody Dance…) work, the digital actors, free from their physical appearance, offer a dance in which “Human" manifests itself through a dynamic print carrying the signature of its motion. Liberated from the traditional reference to body, this new work magnifies the danced movement and its energetic expression.” Article accessed on 7 January 2011; available from www.lartech.uqam.ca/about.htm.
Accessed on 7 January 2011; available from www.openendedgroup.com/index.php/artworks/biped/.
Graham Walters, “The story of Waldo C. Graphic,” Course Notes: 3D Character Animation by Computer, ACM SIGGRAPH 89, Boston, July 1989, pp. 65-79. Quoted in “A Brief History of Motion-capture for Computer Character Animation” (see note 43).
For example, Gongbing Shan, Peter Visentin, and Tanya Harnett, “A Novel Use of 3D Motion-capture: Creating Conceptual Links between Technology and Representation of Human Gesture in the Visual Arts,” Leonardo, 43.1 (2010), 34-42. “As an unfolding of time-based events, gesture is intrinsically integrated with the aesthetic experience and function of the human form. In historical and contemporary visual culture, various approaches have been used to communicate the substance of human movement, including use of science and technology. This paper links the understanding of human gesture with technologies influencing its representation. Three-dimensional motion-capture permits the accurate recording of movement in 3D computer space and provides a new means of analyzing movement qualities and characteristics. Movement signatures can be related to the human form by virtue of trajectory qualities and experientially and/or culturally dependent interactions.”
The Ohio State University, ‘Synchronous Objects Media Release,” (2009), accessed on 7 January 2011; available from http://synchronousobjects.osu.edu/media/inside.php?p=somedia.
M. Vasilescu, O. Alex “Research Projects,” (n.d.), accessed on 7 January 2011; available from http://web.media.mit.edu/~maov/research_index.html.
"In a majority of psychological or physiological studies on perception of biological motion, the stimulus is often limited to a set of dots supposedly attached to the joints of a person. In spite of this drastic information degradation, the human visual system organizes the dots pattern in an undeniable percept of a biological creature. Various techniques were used in order to generate the group of dots, from video recording to pure simulation, including motion-capture. If the first leaves little possibility of deterioration of the signal, a combination of the two last makes it possible to obtain data realistic enough and to handle them on computer, with the aim of identifying pertinent information.” Jean-Louis Vercher, “Perception and Synthesis of Biologically Plausible Motion: From Human Physiology to Virtual Reality,” Gesture in Human-computer Interaction and Simulation, ed. S. Gibet, N. Courty, J. F. Kamp (Berlin and Heidelberg: Springer, 2006) pp. 1-12, ref. on p. 8.
“We investigated quantification of dance movement by simultaneous measurement of body motion and biophysical information. For the above objective, we constructed a simultaneous measurement system with optical motion-capture and EMG equipment. This system enabled us to record basic movements of legs while both the motion data and EMG are visualized on CG character animation. We can expect that our research will help dancers and researchers on dance through giving new information on dance movement which cannot be analyzed with motion-capture alone.” C. Woong, I. Tadao, S. Mamiko, T. Seiya and H. Kozaburo, “Quantification of Dance Movement by Simultaneous Measurement of Body Motion and Biophysical Information,” International Journal of Automation and Computing, 4,1, (2007), 1-6, ref. on p.6.
“En quelque sorte, son travail sur la lumière et le mouvement était déjà trop essentiellement chorégraphique pour être cinématographisé. Nouveau type de spectacle reposant sur l’avènement de l’éclairage électrique et participant au rêve d’une scénographie révolutionnaire formulée par les théoriciens de la scène moderne et de l’art de la mise en scène, la danse serpentine présentait le mouvement dansé comme une énergie désincarnée dépassant les frontières définitoires du corps humain, annihilant même l’idée de corporalité, transformant la présence scénique en une pure image animée.” Dick Tomasovic, Kino-Tanz L’art chorégraphique du cinéma (Paris, Presses universitaires de France, 2009), p. 65.
Annick Bureaud, Kitsou Dubois, “The Embodiment of (Micro)Gravity. Kitsou Dubois’s Analogies: an Artistic and Aesthetic Experience” accessed on accessed on 7 January 2011; available from www.olats.org/space/13avril/2005-2/te_kDuboisBureaud.php.