This text explores the integration of live-action performance within the realm of animation by highlighting the transformative journey from traditional methods to the incorporation of advanced technologies such as AI. Live-action performance has been an essential component in the production of animation since its early days. From rotoscoping to motion capture, and now to AI-driven animation, each technological leap has introduced new challenges and opportunities for animators in how they capture and present performances. Through in-depth studies of live-action performances and leveraging their creativity, animators transcend the limitations imposed by the physical laws governing our reality. But despite the technological shifts, the essence of animation remains rooted in its ability to bring fictional characters to life, capturing the subtleties of human emotion and movement. It posits that the future of animation lies in a symbiotic relationship between technology and traditional artistic skills, where animation artists leverage new tools to enhance storytelling and emotional depth.
Thanks to Max Fleischer and his invention of Rotoscope in 1915 [1], the introduction of rotoscoping in the early 20th century marked a pivotal moment in animation history when it served as a bridge between live-action performances and animated worlds to make possible the potential of combining human performances with animated creativity. Rotoscoping opened a new realm of possibilities, enabling animators to achieve unprecedented levels of realism and emotional depth in their characters. The rotoscoping technique is a method whereby animators conduct frame-by-frame analysis of real-life acting footage, tracing on top of the footage the outlines of objects or characters to create animations based on these traces. This approach can produce extremely realistic and fluid motion effects, easing the animator’s burden in understanding human anatomy and dynamic transformations, while also enhancing the accuracy and quality of movements. This effectively also saved production costs and time. Nevertheless, unlike the traditional drawing method of setting key poses and then filling in the in-between frames known as Pose to Pose from Thomas and Johnston (1981) principles of animation, rotoscoping adopts a technique more similar to drawing each frame sequentially referred to as Straight Ahead by Thomas and Johnston [2]. This approach inevitably results in the animation capturing not only the appearance of the performer but also their nuanced movements. Therefore, selecting actors and actresses who not only closely resemble the animated characters, but also demonstrate similar expressiveness, is crucial for enhancing the animation’s quality. Initially, the performers were often not formally recognized within the production team, partly due to an underappreciation of the live-action performance that lay the foundation for the animation. For examples, during the production of Disney Studios’ Snow White and the Seven Dwarfs (1937), the animators tracked and traced the movements of Marge Champion who served as a dance model, making the traits and movements of the fictional character appear both graceful and fluid. Despite this contribution, Champion’s name was not listed in the film’s credits [3]. Nowadays, voice actors for animated characters also often serve as the performers of those characters, becoming an indispensable part of the animation production. However, the classification of rotoscoping animation — whether it belongs to the category of animation or performance — sparks debate. When watching such animations, viewers might strongly sense the presence of both the actors and the animators [4], see figure 1.
Entering the 1940s, the animation industry began to explore the potential of computer graphics, opening the door to the digital era. Lee Harrison III was in 1959 at the forefront of motion capture technology, when he introduced a bodysuit equipped with potentiometers that facilitated the real-time animation of 3D character rigs. His innovation, named ANIMAC, utilised a programming patch panel [5], see figure 2, setting a new benchmark in the animation technology landscape. The main innovation of ANIMAC lay in its capability to animate 3D characters in real-time through motion capture suits. Despite the early motion capture (mocap) suits being cumbersome to use and the animation results being somewhat rough, these technologies laid the groundwork for the development of full-body motion capture techniques. These motion capture systems represented a leap forward in capturing the intricate details of human movement, thereby providing animators with a rich palette of real-life gestures and expressions to draw from. These advancements shifted the animator’s role from merely drawing and animating characters into being curators of the performance, where selecting the right actors and capturing their essence becomes crucial. The data captured by motion capture technology is typically represented as a series of keyframes. Unlike rotoscoping, which primarily records the performer’s outline, motion capture focuses on capturing the character’s performance without the actor’s physical appearance, allowing the same set of motion data to be applied to different characters. This provides considerable flexibility for creative work, making the actor’s performance a key factor in determining the animation’s appeal. Nonetheless, Steven Paul Leiva argues in the Los Angeles Times that performance capture should not be considered true animation because it documents real motion rather than creating the illusion of life frame-by-frame, thus lacking the unique creativity and meticulous design inherent in traditional animation [6]. The Adventures of Tintin (2011) became the first all-digital motion-captured animated film [7]. However, due to its extensive use of motion capture technology, it was not considered a traditional “animation” by the Academy for The Oscars and was thus not nominated for Best Animated Feature.
The global interest in Artificial intelligence (AI) during the 2020s experienced a significant growth due to the successful application of machine learning. AI-driven animation presents the latest frontier in the evolution of live-action supported animation performance, offering animators tools that can automate aspects of the animation process. The progress in AI animation technology marks the advent of a new era in animation production, signalling a comprehensive overhaul of the production workflow. With the integration of sophisticated machine learning algorithms, AI has revolutionized animation production by reducing production expenses and minimizing the costs associated with trial and error. An example of this innovation is Warpfusion technology, which, when combined with various model rendering styles and prompt scripts, is capable of autonomously generating intricate animation sequences. This technology breathes new life into live action footage by automatically applying unique and unpredicted animated effects, opening the door to limitless creative opportunities. In this dynamic era of modern animation, where live-action performance melds with cutting-edge technologies like AI and motion capture, animators are ushered into a new epoch that transforms not just how animated works are produced, but also what the essence of being an animator is. This transformative journey signifies a shift from traditional techniques to a future where animation is achieved by a deep, collaborative synergy between animators, actors, and technology. It is a collaboration that, coupled with a commitment to artistic excellence and technological innovation, sets the stage for a future where animation enchants not just with visual splendour but also with the depth and authenticity of real-life performances. As animators navigate this landscape, they do not merely practice their craft; they become architects of wonder, exploring new territories in the ever-evolving narrative of animated storytelling, where stories are not just told but are deeply felt. This evolution challenges animators to innovate and push beyond the boundaries of traditional animation. In this way, animators can retain the core of animation — the storytelling and emotional connection that captivates audiences — while exploring new narrative possibilities and visual styles enabled by AI, see figure 3.
References
[1] fleischerstudios.com, “100 Years of Rotoscoping” (no date). [online]. Available at: https://www.fleischerstudios.com/mRotoscope.html (Accessed: 26 May 2024).
[2] Thomas, F. and Johnston, O., Disney Animation: The Illusion of Life. New York: Abbeville Press Inc.,U.S, 1981.
[3] independent.co.uk, “Marge Champion: Dancer who brought Disney’s Snow White to life” by Emily Langer, 2020. [online]. Available at: https://www.independent.co.uk/news/obituaries/marge-champion-dead-dancer-disney-snow-white-obituary-b1252586.html (Accessed: 26 May 2024).
[4] Undone, Hisko Hulsing. Season 1 Episode 2: The Hospital Recap, 2019.
[5] Carlson, W. E. “12.2 ANIMAC / Scanimate” in Computer Graphics and Computer Animation A Retrospective Overview. Columbus: The Ohio State University, 2017. [online]. Available at: https://ohiostate.pressbooks.pub/graphicshistory/chapter/12-2-animac-scanimate/ (Accessed: 26 May 2024).
[6] latimes.com, “Perspective: An animation producer says performance capture is not true animation” by Steven Paul Leiva, 2011. [online]. Available at: https://www.latimes.com/entertainment/la-xpm-2011-jun-05-la-ca-animation-20110605-story.html (Accessed: 26 May 2024).
[7] wikipedia.org, The Adventures of Tintin (film) (2011). [online]. Available at: https://en.wikipedia.org/wiki/The_Adventures_of_Tintin_(film) (Accessed: 26 May 2024).
Emily Tang is an Associate Lecturer at UNSW’s School of Art & Design and a member of the 35th Society for Animation Studies Conference Committee. Her expertise spans various creative visual domains, including watercolour painting, sculpting, life drawing, and 2D/3D animation. With a strong interest in AI animation, she explores cutting-edge techniques to animate imagination on a global scale, and over the past decade she has made significant contributions to the professional animation industry, shaping game objects, designing TV graphics, conceptualising ads, and creating digital learning content. Notable achievements include her work for Prime 7 in the Australian TV industry and animations for the Australian War Memorial. Her short film garnered recognition at the Lights! Canberra! Action! Film festival, receiving the DIRECTOR’S CHOICE honour.