Materials of the Synesthetic. Rethinking the Formal Elements of Animation

“A strange multiplicity of sensations seized me, and I saw, felt, heard, and smelt at the same time; and it was, indeed, a long time before I learned to distinguish between the operations of my various senses.” —Mary Shelly, Frankenstein

Norman McLaren, in a stunning intellectual move and act of synesthetic endeavor, drew sound. By scratching and painting into the optical soundtrack of film, he kicked open the door for the renewed exploration of the value of connecting discrete disciplines and demonstrated yet again that synesthetics is a field that could actually be practiced and perceived.

Figure 1. Musical Score/Jazz Cityscape, student Sketch by the author, 1987.

I chose to study animation because it seemed to be a way to combine into practice many creative interests, from drawing to acting, to storytelling, to sound and music. However, what I found upon graduation was an industry that required specialization in one aspect of the production process. I often did not meet the people responsible for recording the sound, editing the scenes, or painting the backgrounds.

Many introductory undergraduate classes teach animation as a craft based on “Disney’s Twelve Principles.” There is no need to revisit these principles here. However, do these so-called ‘principles’ actually constitute an accurate conceptual framework for animation or do they reinforce a particular, limited view of the stylized practice of animation?

These twelve ‘principles’ shape a certain mode of thinking about animation as a particular stylistic tradition and define a set of ideals. While these may be suited for the mimetic approaches to anthropomorphic character animation, they seem to limit the establishment of a set of more fundamental ‘principles’ that address multiple modalities and may limit the development of new ideas, particularly in students. How would these twelve ‘principles’ account for the brilliant work of animators like McLaren, Reiniger, or Barsamian? In what ways do they limit the advancement of the form?

Can there exist a set of principles that more accurately embodies the formal elements, synesthetic possibilities, and materials of animation and the arts? Such elements would need to be conceived of as multimodal elements or building blocks. Systems such as the ‘twelve principles of animation,’ or academic notions of the ‘seven elements of music’ (rhythm, melody, harmony, timbre, form, texture, dynamics), or ‘seven elements of art’ (color, form, line, shape, space, texture, and value) reify the delineative structures. Instead, perhaps a list of elements could be devised to account for synesthetic practice and open up design thinking to embody a multimodal approach that recognizes the interconnectedness in all the arts.

Below is what is hoped to serve as the beginnings of a community discussion about possible ways of redefining the materials and principles of animation so as to reframe it as an inherently synesthetic art.

It is meant to be food for thought, rather than a declaration. The idea is to formulate a list of attributes in common to all the practices found in the arts as a means of excavating commonalities rather than maintaining differences.

Proposed Revision Towards as Set of Formal Materials of Synesthetic Practice and Design

(A conceptual ‘periodic table’ of materials that could hold true across disciplines and modalities)

Here are four ‘elements’ that appear common to all art practice:

1. Time. It is present in all the art forms and, as such, its materiality might be given a primary consideration. Certainly time is present and critical to music, dance, and film, but it is also a core element of photography, writing, storytelling, and in perhaps less obvious ways, painting. Time encompasses epochs, periods, cultural contexts as well as meter, rhythm, and durations of experience. We “read” a canvas often in a sequential order that implies a linear experiencing of a work, although some artists invite us to absorb a work in a more circular time pattern—or even within an absence of time. Time is a critical element to music as well as film (and to other ‘discrete’ disciplines), not only in the sense of beats, frames, space, footage, measures, notes but also in the possibility of experiencing a suspension of clock time towards a more experiential time. Animation can be thought of as compositions in time.

Figure 2. The frame may be considered a formal material.

2. The Frame: a Contextualizer. The frame constitutes the contextual embodiment of the work of art, be it a physical picture frame, a page, embodied space, a screen, a performance space, or theater. Time itself can be a frame. The frame also accounts for the cultural, theoretical, temporal aspects of a work. In addition, frames may operate within frames, such as two notes within a measure surrounding a third note to create a chord, or a silence between notes of music, which all operate within a performance space, within a culture, within an epoch. As such, frames could be considered as a prime element to art practice.

3. Decision. Decision might be a core element of all art and as such, the artist may wish to consider decisions as material, and also consider decisions about decisions. Decisions are a core element in interactive media such as games. Thinking about decisions also invites the artist to consider the audience. If there is indeed art separate from lived experience (I am not sure there is), it is made discrete, perhaps, by the action of making a decision to frame time and relation.

4. Relation. Assuming the semiotic code of figure/ground represents the linguistic basis for understanding and thought, then the relationships of objects, lines, tones, spaces, audiences, and so on could be considered a primary element in art practice. This relational element is present in time and frames. For example, we can experience the same color differently depending on its relation to other colors (Albers). Similarly, notes and chords are given meaning based on their relation to one another. This invites the question: is it the color or a note itself transmitting an experience or our perception of the relationship of the colors? If it is indeed the relation, then it may be worth exploring the idea that relation itself may be an elemental material, not color.

These concepts propose four elements that could be common to all art practices. Of course, there may be more or fewer, but it is hoped that such an exploration can expand a discussion about the synesthetic opportunities in art practice and move discussions away from style, surface elements, and the temptation of categorization, particularly in an art form that is comprised so heavily of multiple elements and modalities as animation. It is hoped that such exploration may bring the conversation about animation’s opportunities into a deeper conceptual realm that is accessible to artists and students and provides a framework whereby the synesthetic opportunity of animation can be more thoroughly assessed and discussed.

References

Albers, Josef (2013). Interaction of Color. New Haven: Yale University Press.

Braun, Marta (1994). Picturing Time: The Work of Etienne-Jules Marey (1830-1904). Chicago: U of Chicago.

Burgun, Keith (2017). Game Design Theory: A New Philosophy for Understanding Games. Natick, MA.: CRC.

Cytowic, Richard (2018). E. Synesthesia. Cambridge, MA: MIT.

Dobson, Terence (2017). Film Work of Norman McLaren. Eastleigh: John Libbey & Co.

Epstein, David (2020). RANGE: How Generalists Triumph in a Specialized World. New York: MacMillan.

Glissant, Edouard (2009). Poetics of Relation. Ann Arbor: U of Michigan.

Haverkamp, Michael (2013). Synesthetic Design: Handbook for a Multisensory Approach. Basel: BirkhaÌuser.

Redberg, Rita F. (2015). “Cardiac Patient Outcomes During National Cardiology”, MeetingsJAMA Intern Med, 75(2): 245.

Wollstonecraft Shelley, Mary (1977). Frankenstein. London: Dent.

Robby Gilbert is an Assistant Professor of Animation and chair of the Visual Arts Department at Northern Vermont University. He has worked extensively as an illustrator and an animator, and views animation as the synthesis of a variety of practices such as music, painting, drawing, dance, acting, storytelling, and technology. You can see some of his work by visiting www.robbygilbert.com.