Parameterizing Patterns for Generative Art and Live Coding

Jessica Stringham

doi:doi:10.21428/108765d1.54ba7255

Abstract

Patterns can provide an interesting backbone to generative art and live-coded visuals. Coupling patterns with comprehensive parameterization enables interactive exploration of generative art states to find interesting pieces or to tour a range of parameters in a real-time performance. I'd like to share an overview of the software I've been developing in Rust to support my creative code work, including how it uses parameterization to create and explore patterns.

I’d like to share an overview of the software I’ve been developing in Rust to support my creative code work, including how I use parameterization to make patterns. This is my first time writing about the software in any detail, and it’ll also include a demo of writing a simple system that can be used to live code visual patterns.

Before I get too into the weeds, here are a few examples created by similar code. First is a 48 by 48 grid of squares, where the pattern in the visibility of a square is determined by a bitwise operation, and the size of a square is determined by a formula.

An image of pattern of a grid of rectangles of varying size (they're smaller in the middle). A repeating pattern of blacked out squares also exists. — **Figure 1**
The original pattern.

A photo of a sheet of paper with the pattern above cut out from it. — **Figure 2**
The pattern after going through a vinyl cutter.

Using the same software packages, I can create another system that feeds the grid through shaders to apply effects like distortion, colors, and feedback.

Blobby green shapes that were maybe once a bunch of squares. — **Figure 3**
Similar grid pattern (but inverted) is still visible after going through a lot of filters.

I perform live coded visuals, where I control real-time graphics to go with an audio performance. When live coding visuals, additional patterns can be introduced along the time dimension to complement audio’s temporal patterns.

**Figure 4**
The grid pattern is still visible while the author makes a funny face, performing at a livecode.nyc event with Hardcore Software on audio. (Photo credit Doug Linse).

Overview

I have been writing several connected libraries in Rust that enable my live coded visuals, generative art, and many other creative coding endeavors. The paradigm works like this: I create a new system for a given performance, project, or doodle and import the libraries. A given system might have dozens of parameters, like the color and position of an object, or constants for a physics simulation or an L-system grammar. I can then explore the system interactively by adjusting numerical and categorical parameters by setting the value directly or using midi controllers, audio-reactivity, time normalized by the tempo, or an expression combining all of those. I have a lot of fun with this. It encourages me to crank up parameters beyond what I intuitively thought was interesting, and that’s where I find the coolest outputs.

I make use of nannou, a package written for Rust that is very similar to other creative coding libraries like Processing, p5js, and OpenFrameworks. These packages generally provide things like a way to draw and transform shapes, handle colors, and take care of the event loop to display things on a screen. They are open-ended and flexible, so you can pull in additional packages from their language’s environment, and that’s where my libraries sit.

I also draw inspiration from other software used for live coding. One is Tidal Cycles, a live coding environment commonly used for audio with an emphasis on patterns over time. And while I have a different approach than Hydra, a live coding video synth environment, I am inspired by what people can do with it and other visual libraries. Unlike these other live coding environments, I do need to write and compile the Rust code ahead of time that defines the system and its parameters.

Demo

For this demo, we will once again draw a grid of squares through the configuration. Depending on the goals of the performance, one way to parameterize would be to specify how the squares are arranged on the screen, the number of squares, the squares’ positions, sizes, rotations, and colors/designs. Different systems are capable of producing the the same results as a grid of squares; for example, one could parameterize based on the ratio of a parallelograms’ width/height and its angle, or using the number of sides of a regular polygon. The choice depends on the performance. For the following example, we’ll restrict the program to produce squares.

A starting example

Let’s work up to the grid. Here’s how I could start setting up the parameters and using that to draw the system: I manually specify the location, size, and the color for three squares.

# square.yaml

# draw three squares with different colors
squares:
 - loc: [-200.0, 0.0]
   size: 50.0
   color: [0.1, 1.0, 0.8, 1.0] # hsva
 - loc: [0.0, 0.0]
   size: 100.0
   color: [0.5, 1.0, 0.8, 1.0]
 - loc: [200.0, 0.0]
   size: 50.0
   color: [0.8, 1.0, 0.8, 1.0]

Below is the Rust code that I use. If you’re not familiar with the Rust code, it’s okay to skim this code! In Rust, the #[derive(…)] is a custom proc-macro, a way to write code to write code. The Livecode library I’ve written provides the proc-macro which creates code that can parse the configuration, fill in midi/time/audio, and compute expressions. So in Rust, I only need to write the code to draw the shape given the already-computed values.

# squares.rs
#[derive(Livecode)]
struct Square {
  size: f32,
  loc: Vec2, // yaml is a list of two numbers
  color: LinSrga // a color, the corresponding yaml is a list of 4 items representing HSVA
}
impl Square {
  fn draw(draw: &Draw) {
    // nannou code to draw the square
    draw.rect()
      .xy(self.loc)
      .w_h(self.size, self.size) // we made a choice to only draw a square
      .color(color);
  }
}

#[derive(Livecode)]
struct LivecodeConf { // this is the full configuration
  appConfig: AppConfig, // some common useful things (e.g. real-time or fps)
  squares: Vec<Square>,
}
impl LivecodeConf {
  fn draw(draw: &Draw) {
    self.squares.for_each(|s| s.draw(draw))
  }
}

// other boilerplate for running app

The Rust code above is created in advance before the performance to define the system, and the config above is how I modify the program live. The framework I wrote could be thought of as enhancing the Rust system to be able to make sense of a rich configuration: in the above example it converts a list of two numbers into a 2d-vector and converts four numbers into a color. The next section will describe how this code can inject useful interactive parameters.

Using interactive variables

Now we can start to pull in other ways to control the configuration.

Modifying the above configuration, we can start to make use of MIDI controllers, time-based parameters, and audio values.

For example, MIDI values are accessible as m0, m1, and so on. This can be scaled using a built-in function s(x, min, max which scales x (which is assumed to be between 0 and 1) linearly to between min and max. The audio parameter ac gives access to an audio-amplitude-based value, which is also scaled between 0 and 1. The time parameter t gives you access to a time parameter, which is adjusted based on a global variable bpm and can be reset. This one is not scaled between 0 and 1 and continues indefinitely, but we can use this directly for a hue parameter since the program already modulos the value.

Note that the Rust code did not need to change to take advantage of this richer configuration.

ctx: |
 // use the first midi dial for the loc, scaled from -200 to 200
 let y_loc = s(m0, -200, 200); 
 // slowly rotate the hue over time
 let color_hue_offset = t / 20;
 // set the size based on % of max audio input received
 let size_scale = ac;

squares:
 - loc: [-200.0, y_loc]
   size: mix(20.0, 50.0, size_scale)
   color: [color_hue_offset + 0.1, 1.0, 0.8, 1.0]
 - loc: [0.0, y_loc]
   size: mix(50.0, 100.0, size_scale)
   color: [color_hue_offset + 0.5, 1.0, 0.8, 1.0]
 - loc: [200.0, y_loc]
   size: mix(20.0, 50.0, size_scale)
   color: [color_hue_offset + 0.8, 1.0, 0.8, 1.0]

Patterns with UnitCell

While this is already a good start to pattern-making, I wanted to share another proc-macro I wrote called UnitCells. (The name is borrowed from what you call the repeating units in a crystal’s lattice.)

**Figure 5**
A sequencer can also be configured to represent the symmetries and reflections needed to represent Wallpaper Groups. Here, a unitcell consists of three parallel lines, an arc, a pink, and a blue-green circle. The elements are influenced by time-based expressions which control the sizes and colors of the circles, and the location and rotation of the arc. The sequencer repeats the cell according to the P31M Wallpaper Group.

The way I have it set up breaks down repeating patterns into two parts: the sequencer and the unitcells.

The sequencer configures how many unitcells will be drawn, along with how they are scaled and positioned: for example, a 3D lattice, a grid, a hexagonal grid, or a spiral are all things I’ve used.

The unitcell is responsible for what is drawn and only needs to worry about its own contents. It has access to its indices in the sequencer to use directly or to use to seed randomness when drawing the cell. In the Rust code, I just need to define how to draw a single instance to a reference area given its variables.

The code doesn’t change much:

#[derive(UnitCellExpr)] // update the proc-macro to pull more variables into the context
struct Square {
  size: f32,
  loc: Vec2,
  color: LinSrgba,
}
impl Square {
  // nannou code to draw the square
  draw.rect()
    .xy(self.loc)
    .w_h(self.size, self.size)
    .color(self.color);
 }
}

#[derive(Livecode)]
struct LivecodeConf {
  #[livecode(sequencer="tiler")]
  squares: UnitCells<Square>,
  tiler: TexTiler, // one type of tiler: this one makes various 2D grids
}
impl LivecodeConf {
  fn draw(draw: &Draw) {
    for square in self.squares.iter() {
      squares.node.draw(&draw.transform(square.get_transform()))
    }
  }
}

Now instead of needing to define a list of squares, I can define how each square should be drawn. The x and y represent the coordinates given as the percent of the total width and height. Alternatively, x_i and y_i will give the index, which is useful for exploring patterns with bitwise operators.

ctx: |
 let y_loc = mix(-200, 200, m0);
 let color_hue_offset = t / 20;
 let size_scale = ac;

# draw 144 squares with different colors
tiler:
 rows: 12
 cols: 12
 style: Grid
 size: 20.0

squares:
 loc: [0.0, y_loc]
 color: [color_hue_offset + s(y, 0.3, 0.6), s(x, 0.3, 0.8), 1.0]
 size: |
   size_scale * (40.0 - clamp(40.0 - ((x - 0.5) * 15.0)^2 - ((y - 0.5) * 15.0)^2, 0, 40))

And that’s it! With this, I have a lot of control on how that grid of squares is displayed through the configuration.

Additional Examples

As previously shown, the framework enables many parameters to be manipulated live. The following figures show outputs from the same program which uses the a unit cell of a circle with a parameterized radius. These outputs can be achieved by a single run of the program and modifying the configuration live.

Three images, a grid of circles, a grid of circles offset slightly, and a spiral of circles. — **Figure 6**
An example of different configurations. The left is a 16 by 16 grid pattern (which also supports wallpaper groups for symmetry), the center is a 16-by-16 grid in a hexagonal layout, and the right is based on phyllotaxis (the circles are also rotated correctly).

Three images, the right is once again the grids from above, the center is that grid but the circles get progressively smaller the lower on the image, and the right is the grid but the circles vary in size. — **Figure 7**
Three examples of modifying the size of the unit cell. The left image uses a uniform value as the size ( $\texttt{REFERENCE} \times 0.8$ ), the middle image varies the size based on the y-position ( $\texttt{REFERENCE} \times 0.8 \times y)$ , and the right image varies the size based on an expression $\texttt{REFERENCE} \times y \times ((y_i \mod 2) + 0.5)$

While the above examples used relatively simple shapes, the unit cells can be used to parameterize arbitrarily complex shapes. Below has a unitcell that draws a stylized sphere, with parameters that manipulate the viewing angle and the number of circles used to draw the sphere. The code to draw a sphere based on its parameters is coded ahead of time in Rust. Like the example above, as the program runs, one can manipulate the pattern used to determine the viewing angle and density, as well as the tiling pattern live.

Three example images. The first is a hexagonal grid of spheres, where ones near the center are more detailed and rotated. The second is also a grid, but the only spheres shown are near the center. The last image is a grid of spheres, where ones near the horizontal center are more rotated and dense. — **Figure 8**
Example output of a sphere-drawing program controlled by Unit Cells. Spheres are represented by arcs and are arranged in a hexagonal grid (left and center) or a square grid (right). The attributes of the sphere, such as the number of arcs that represent it and the orientation, are controlled using equations based on the unit cell’s location.

Conclusion

This structure has been extremely useful for my own personal work. I hope to share the Rust code publicly eventually, though in the meantime I hope that write-ups like this can share the (possibly-more-useful-than-the-implementation-itself) ideas behind it. I also hope that sharing it can introduce me to other folks thinking about creative coding in similar ways.