The Representation and Realization of Patterns with Animated Notation
In a generative or algorithmically-created musical work the logic of the compositional system may be clear to the creator, and certainly the system itself (if it were self-aware), but might challenge the listener in the identification of a macro pattern of micro patterns. In fact, it may be the case that in some works, perceptible patterns are purposefully avoided or only perceptible with significant analysis and/or an advanced formal education. With a traditionally-notated score a pattern, be it rhythmic, melodic or formal, may be revealed by the trained eye which may or may not be the case even with a trained ear. Relatively new approaches to music notation, specifically Animated Notation, may provide not a solution (as traditional and graphic notations are well-equipped for most situations) but possibilities for different types of pattern representation and realization. Given the unique visual qualities of these scores and the common practice of projecting them, a performance provides the audience member with the opportunity to see these patterns as they unfold in the score. Following an introduction of patterns and an explanation of the primary components of an Animated Score, four compositions by the author will be examined with details provided as to the representation and realization of various micro and macro patterns.
From a young age, simple musical patterns are used to teach the fundamentals of music and Geist, Geist and Kuznik note that “elements such as steady beat, rhythm, melody, and tempo possess inherent mathematical principles such as spatial properties, sequencing, counting, patterning, and one-to-one correspondence.”1 Depending on the type of music being considered, some of these elements may be more or less perceptible, even from a casual listen. But if the relationship between these elements are not discernible, then many of the elements described above will be difficult to identify. Tillmann suggests that, like speech, “music perception requires auditory sequencing. The incoming stream must be segmented (chunked) into events and phrases … and each incoming event must be integrated (also using knowledge about the relevant system) into the structure of the context.”2 Without this segmenting or chunking the formal elements of a piece of music, both micro and macro, may blur together and diminish the possibility to recognize the compositional identity of a work3. While approaches to music pedagogy may at times assume no prior experience, as is often the case in 100 level survey courses at post secondary institutions or preschool music classes, it is also assumed that one’s surroundings will have an impact on one’s knowledge creation through a process called implicit learning: “The cognitive capacity of implicit learning enables perceivers to learn regularities in their environment through mere exposure to materials that obey the rules of a given system.”4 In the context of pattern recognition in music, the particulars of one’s environment, including what musical styles are present, is likely a significant factor. In the Western world the rhythmic, melodic and harmonic structures associated with tonality are so dominant as to be nearly inescapable.5 In popular music, from Hank Williams to Harry Styles, the simple and repetitive melodic contours outlining child-like rhyme schemes represent a musical context that is easy to assimilate, and the recognition of patterns within works such as these is relatively easy, even to the untrained ear. Think Mary Had a Little Lamb or other songs learned by rote as a child. In other words, if the system within which patterns are being developed is not taught then it might be implicitly learned: “prior experience can influence current behavior without awareness of the information previously acquired.”6 Tillmann and Bigand call attention to these prior experiences in the context of Western music:
“By mere exposure to musical pieces, Western listeners acquire sensitivity to the regularities underlying tonal music. This implicitly acquired knowledge allows listeners to perceive subtle relations between musical events…” but is more challenging “when short time structures have to be integrated in larger structures.”7
In a generative or algorithmically-created musical work the logic of the compositional system may be clear to the creator, and certainly the system itself (if it were self-aware), but might challenge the listener in the identification of a macro pattern of micro patterns. In fact, it may be the case that in some compositions perceptible patterns are purposefully avoided or only perceptible with significant analysis and/or advanced formal education. For instance, the patterns found in a work like Xenakis’ Psappha may require a more thorough investigation8 whereas the patterns that emerge in Terry Riley’s In C are likely quite easy to hear and from a notational and visual perspective, well-represented by the score. In Pauline Oliveros’ Rock Piece, repetitive patterns emerge and dissipate at irregular intervals as determined by the text instructions and their application in performance. Each performer “establishes an independent pulse with the rocks. The pulse is to be maintained steadily without any rhythmic interpretation or accents.”9 Local patterns, one or two performers for instance, can be quite easy to hear while the complex patterns that emerge from the ensemble, a pattern of patterns, may begin to exceed the possibility for a casual aural analysis. A very different work, Tenney’s Having Never Written a Note for Percussion does not explicitly represent a rhythmic pattern as the onset of one’s mallet strikes are absorbed by the resonance of the instrument, which is often a gong. However, if the instrument used were less resonant, a wood block for instance, a clear and simple repetitive pattern emerges; likely a roll alternating between the right and left hands. At the macro level a pattern isn’t necessarily perceptible in this work as it is a single, drawn out event. But these works, and many others, represent a relatively small portion of the music that is being consumed, and may or may not favor the implicitly-learned listener. Given that a large majority of works in the Western canon present patterns of a perceptible nature to the implicitly-learned listener, the composer of tonal music (implying the presence of the other structural elements covered earlier) is likely unconcerned with such baggage. Still, for compositions that extend beyond these more common or traditional structures, an alternative approach to notational representation, as well as its realization and presentation may improve the potential one’s perception of musical patterns.
In this paper, the representation and realization of rhythmic patterns using Animated Notation will be discussed. In particular, the focus will be on the author’s attempt to create rhythmic patterns that are clearly perceptible aurally and visually despite existing beyond the context of metered frameworks commonly associated with tonal structures. Animated Notation is used as a method to represent these musical concepts in such a way that performers can realize these works with little to no training. To accomplish this, the notational system must be legible and its functionality clearly perceptible as it may at first fall beyond the performer’s or audience member’s frame of reference. As the title of this paper suggests, instructing performers as to What To Do and When To Do It is accomplished by highlighting the rhythmic patterning in a visually explicit way and clarifying expectations for performer actions in the score. In so doing, the performative behaviors associated with an Animated Score that may or may not include a performer’s prior or implicit knowledge are distilled so as to be relatively simple to execute. It is important to note that for the purposes of this paper the author will focus on their own notational approaches and that these approaches are not global solutions but singular to the author’s work.
Animated Notation is “a subset of dynamic scoring that features perceptible contact and intersection between elements in a notational fashion.”10 As with the previous sentence, many of the concepts discussed in this paper are drawn from several of the author’s previous writings1112 including the discussion of various terms relevant to this field of notational practice.13 As a point of reference, traditional music notation relies on a set of symbols positioned in an immutable fashion. These symbols and their relationships to one another are legible to anyone who has had the opportunity to learn the particulars of the system, and their fixedness requires an educated evaluation of their various functions. In other words, pitch and rhythmic relationships are generally not proportional but represent specific frequencies and durations based on their shape and relative position. In an Animated Score there is also a collection of symbols, a mini-language of sorts, that functions in a dynamic way in order to represent performer actions. This dynamism is an essential component for without it these symbols have no real meaning: perceptible interaction is essential. These symbols often include nodes, attack cursors and other symbolic elements, and the functionality of these elements can generally be distilled down to three actions: contact, intersection and convergence. The term aggregate is used to describe the collection of elements and functionalities assigned to a single performer. The following figures represent each of these actions.
The author began working with Animated Notation in 2011 in order to discover new ways to represent complex rhythmic concepts and patterns in a legible way that enabled performer engagement regardless of experience. Rather than requiring performers to learn a traditional or non-traditional notational language, each of these scores is built around a mini-language of sorts, as mentioned above. These notational mini-languages are designed to be easily understood and often require little to no interpretational background.14 While there are a wide variety of notational approaches in this field of practice15 this paper will focus on the author’s scores. Many of these scores are inspired by the early phase pieces of Steve Reich16 and the sonic patterns produced in those works. In these works, one human or machine performer (i.e. tape machine) maintains a steady tempo while the other slowly increases in speed. This blurring of measured time is a phenomenal effect and one that sits on the fringes of traditional notational strategies. While it is of course possible to notate virtually anything it is far from realistic to assume that multiple tempi could be easily maintained by an ensemble, let alone tempi that slowly change over time.17 Like Reich, the work of Sol LeWitt has also had a major impact on the development of these scores. In LeWitt’s work, patterns, or patterns of actions, are an integral component and are described by the artist in a way not dissimilar to pseudo-code; i.e. the instructions are provided which leads to the rendering of the work. For Wall Drawing no. 49 (first installed in 1970) the instructions are as follows:
“A wall divided vertically into fifteen equal parts, each with a different line direction and color, and all combinations. Red, yellow, blue, black pencil”18
These instructions are often accompanied by a diagram that provides a visual representation of the work on a smaller scale. The diagram for Wall Drawing no. 49, for instance, serves as the model upon which the full-size rendering of the work will be based. The instructions, an algorithm in its own right, describes the specifics of the procedures to be taken in order to represent the diagram. For other works, 1974’s Incomplete Open Cubes for example, a serial pattern is produced through the physical instantiation of a process unfolding. In this case, the discovery and realization of all variations of a cube missing one or more edges “through the application of premeditated rules or plans.”19
Although incredibly brief, the instructions and diagrams created by LeWitt provide the artist with exactly the right amount of information to realize the work. The patterns that result from these instructions are simple in their local or singular instantiation (a straight or curved line for example) but quite complex in the final amalgamated whole. The simplicity of this conceptual process is fundamental to LeWitt’s work: “The ideas need not be complex. Most ideas that are successful are ludicrously simple.”20 Like LeWitt, beginning with a concept, musical or otherwise, is common for many composers. Depending on what that concept is, some alternative approach to its notational system and how it is rendered, including its design characteristics, may be required. Whatever this notational mini-language ends up being should be led by simplicity and legibility, assuming the intentions of the composer are to produce an accurate realization of the concept.21
The concepts of each of the following works generally rely on the creation of possible patterns that may or may not emerge over the course of a performance. In other words, the patterns inherent in a particular piece represent the initial concept but how they are instantiated rests with the possibilities of the score application. The patterns that do emerge are an amalgamation of each performer’s independent actions: performer-specific micro-patterns, often a repeating phrase or collection of quasi-repetitive sonic events. These localized patterns often function independently, but the general model is similar. For instance, in Study no. 40.3 [pulseven], each performer’s local pattern contains a sequence of three events, two short and one long, and their moments of attack are indicated by the intersection of the three rotating attack cursors with the node at 12 o’clock. That model is then modified by increasing or decreasing the tempo for each performer independently of all others over some period of time. Since each performer’s aggregate is functionally-independent of all others, the global pattern, which is the sum of these variably-interlocking parts, will likely not be repetitive. This is due to the fact that this interlockedness is not prescribed beforehand but the result of quasi-random generative processes. These processes inject a malleability into the representation of the local patterns while maintaining a fixed range of possible speeds.
There is also a larger formal pattern in which the aforementioned section, let’s call it “A”, is followed by a “B” section. In the B section, hocket spokes appear which instruct the performers to follow the aggregate in the center of the screen. Due to the spacing of the nodes and attack cursors in this aggregate, a hocket pattern is produced. A hocket pattern is a rhythmic and melodic device with an extraordinary history22 and fairly simple implementation: “a spasmodic or interrupted effect in medieval and contemporary music, produced by dividing a melody between two parts, notes in one part coinciding with rests in the other”.23 Here the relationship between performers is fixed in order to create this particular pattern. While the formal and local patterns are purposefully simple and the expectations of the performers are reasonable, the amalgamation of these patterns, particularly in the A section, is fairly complex yet sightreadable.
Short video recording of Study no. 40.3 [pulseven] in performance: https://www.youtube.com/watch?v=fLY_aHyM_5M
Study no. 8 [15 percussionists] was composed for 15 percussionists and premiered at Mills College in 2012. The initial concept was to create a phase piece for a large ensemble that would require little to no rehearsal. The rhythmic phasing of these 15 parts creates an evolving pattern that moves toward and away unison events in similar fashion to the functionality of a sine wave. The specifics of the work are described below.
Each percussionist has a small collection of instruments including:
Two unpitched, slightly resonant metal objects. These might include brake drums, anvils, soup cans, etc. Tuned instruments like gongs or metallophones are not permissible.
Two slightly resonant wooden objects, such as woodblocks, sections of 2x4, etc. Tuned instruments like bars of a xylophone or marimba are not permissible.
Two resonant, pitched metal objects such as glockenspiel bars, desk bells, etc. Ideally these instruments would all fall within the same key.
In a performance setting the performers stand side by side facing the screen upon which the score is projected, enabling the audience members to see what the performers are seeing. In most cases, at least for the author, this method of presenting the score is typical, and not only enlarges the score to improve legibility but provides a “behind the scenes” view of the score for the audience. This approach enhances one’s ability to perceive patterns and other relationships during a performance.
Each performer follows a single aggregate which includes all of the notational information they require: two nodes and one attack cursor.
The functionality of each aggregate is not dissimilar to that of a metronome: the attack cursor, which is connected by a line to a central node, swings back and forth between the two nodes. Each time it makes contact with a node on the right and left side the performer strikes their instrument with the corresponding hand, i.e. when it hits the right node they play with their right hand and when it hits the left node they play with their left hand. The color of these nodes indicate which instrumental group to play: Gray for unpitched metals, brown for wood, and blue for tuned metals.
From a global perspective the piece follows a simple form (see figure 10). In the first section, unison and descynchronization, the performers begin together and slowly speed up at slightly different rates over approximately 5 minutes. Performer 15, represented by the bottom right aggregate, speeds up faster than player 14 who speeds up faster than player 13, and so on. During this acceleration the ensemble undergoes a phasing process that creates a series of overlapping patterns amongst the ensemble. Toward the end of this section performers are instructed to move to their wooden instruments, indicated by the nodes turning brown. The rhythmic relationship between player 15 (fastest) and player 1 (slowest) during this sustain section is approximately 25:10 with all other performers scaled evenly between. This can be seen and heard from approximately 6:01 to 6:17 in the linked video. During this section each performer maintains their specific tempo, and by sustaining these multiple tempi, a multitude of patterns is created as the ensemble moves toward and away unison or near-unison attacks. The following figure first appeared in Plain and Simple.24
The final section, decelerando, which is performed on tuned metals, is a reversal of the first section: each performer slows down at a different rate to finish at approximately the same time. During this section an evolving pattern emerges that is different from the tight layering at the faster tempi in the second section to a stretched and sparse pattern as the piece comes to a close. In addition to the global or ensemble patterns that emerge it is also possible to hear and see patterns develop amongst smaller collections of players.
Video recording of Study no. 8 [15 percussionists] in performance:
https://www.youtube.com/watch?v=bhhCu_QA3e4
Study no. 30 was designed as an installation for participation by virtually anyone regardless of their musical experience. In its initial installation at EMPAC in 2013, 15 circular platters made of wood were mounted on microphone stands in a circle with a tuned desk bell at the center of each. The score is projected from the ceiling and includes a converging ring as the only notational symbol and functionality. The moment of attack, i.e. pressing the bell, is indicated by the moment the converging ring reaches the bell.
The patterning in this piece is of a different flavor than in Study no. 8 as it relies on weighted randomness to determine the moment(s) of attack within a timeframe of two minutes. During each two minute timeframe the possibility for attack indications (i.e. the converging ring) increases. Thus the patterns that may emerge will become increasingly dense over the course of each two-minute section before returning to a more sparse texture. However, unlike Studies no. 8 or 40.3 the patterns often lack any perceptible repetition. This is not to say that the ear won’t find moments in which a pattern seems to emerge during shorter sections but that these patterns are unlikely to repeat in any perceptible way.
Video recording of Study no. 30:
https://www.youtube.com/watch?v=0SjGwt9Ro9Y
Like Study no. 40.3 [pulseven], Study no. 11 [4x4] explores the musical patterning device known as hocket. In Study no. 11, there is not a specific melodic figure being passed around but rather a disjunct hocket of sorts in which the individual tempi of each performer aggregate creates a malleable but potentially-repeating pattern.
Like Study no. 8, the performers begin in unison, playing one of four sounds depending on what is indicated by the attack cursor when it intersects with the node at 12 o’clock. At 0:31 in the referenced recording, a random process determines what the next tempo will be for each aggregate independently of the others, similar in function to Study no. 40.3. This may be slower or faster than the current tempo and each aggregate slowly speeds up or slows down to reach it. This section effectively blurs the rhythmic gestalt of the ensemble, creating an evolving counterpoint pattern amongst the performers. After approximately 30 seconds each aggregate slows down or speeds up to reach a new tempo shared by each performer. Since each aggregate had previously moved to different tempi, their attacks will likely now be offset as the positions of the attack cursors relative to the other performers will have been altered. While it is possible that some events may occur simultaneously it is unlikely: the pattern is evenly distributed across the ensemble with only one event being played at a time. Even though the duration between each event is not be consistent (for example, the duration between player one’s attack and player two’s may be different from player two’s and player three’s), the sequence of durations will be, creating a repetitive and perceptible pattern: a disjunct hocket. After maintaining the same tempo for some time the cycle repeats and new tempi are assigned to each player’s aggregate. Following this formal pattern, new repeating patterns are created during each section but while similar, are unlikely to be the same at any future occurrence.
Video recording of Study no. 11 [4x4:
https://www.youtube.com/watch?v=EijpjIO77dU
Each score is produced using an application built in either Processing (Studies no. 8 and 11) or OpenFrameworks (Studies no. 30 and 40.3) and while there are similarities, each application is unique to the score it produces. Each score generally relies on a set of variables indicating current speed, old speed, new speed, as well as the timing of each of these changes and any code required for its rendering. The ranges of the speed variables were determined based on both the playability of the piece and the creation of the various patterns or event densities detailed above. The appearance and functionality of each score is designed with legibility in mind and in consideration of how these scores would appear to performers as well as an audience member.
It is the author’s belief that by approaching notational design and functionality from a What To Do and When To Do It standpoint, each score points to an implicit understanding of how to physically recreate a visual action or indication. In other words, the notational approaches in these scores represent simple patterns of movements that are easy to mimic, and while the instructions for the works included in this paper might be generalized as “speed up and slow down at different times over the course of a performance” the creation of patterns specific to each work requires an explicitness in the notation: each event must be sounded at the moment it is indicated in order for the concept of the work to be properly realized. This requires a distillation of the notational language and clarity in its representation. When the expectations of the performers are clearly indicated, the realization of relatively simple patterns by each performer is streamlined, and the creation of complex patterns through their layering is straightforward. Projecting these animated, visual representations of notational patterns to represent musical actions also enables a unique kind of participatory experience: the audience member is privy to the notational functionality of the score that is traditionally reserved for the performers alone.
New works relevant to those detailed above include The Animated Notation Workshop (2023) and the tentatively-titled Study no. 60 [Full Circles] (This work follows a similarly exploratory line of enquiry and at the time of the writing of this paper is still in the early stages of development.). The scores for these works are created with p5.js25 and are designed as notational sandboxes for experimentation in rehearsal and composition/performance. With The Animated Notation Workshop the composer/conductor/director can create patterns by placing a series of symbols (the images on the right side of the screen) within performer frames (the numbered rectangles on the left side of the screen) that indicate their execution by their intersection with a dynamic attack line. These frames can then be resized and overlapped with other performer frames to create additional musical patterns.
To try out the beta version of this project and make your own patterns please visit: http://ryanrosssmith.com/workshop/index.html Please note that at this point Chrome is recommended.