Clint Enns

Charting the Montage: The Roots of Algorithmic Cinema

“Charting the Montage: The Roots of Algorithmic Cinema.” In Marshall McLuhan + Vilém Flusser’s Communication + Aesthetic Theories Revisited, edited by Melentie Pandilovski and Tom Kohut (Winnipeg: Video Pool Media Arts Centre, 2015) 155-62.

…[W]e will see that many of the principles … [of new media] are not unique to new media, but can be found in older media technologies as well.1

Lev Manovich, The Language of New MediA

Algorithmic editing is a term that was first coined by Lev Manovich in an artist statement for Soft Cinema (2002), a collaborative project with Andreas Kratky, which attempts to navigate databases in new and innovative ways. In his statement, Manovich theorizes about algorithmic editing without providing a precise definition. Explicitly, algorithmic editing refers to any method of editing based on direct procedural approaches. In other words, this type of editing can be seen as a technique for cutting and reassembling raw footage by following a schema or score. Algorithmic editing, like most new media, is not new, and its roots can be seen in the earliest attempts to formalize/theorize the practice of cinematic editing.

I intend to argue that algorithmic editing traces back to Soviet montage theory and was further developed through the work of structural filmmakers in the late 1960s and early 1970s. By treating film as a countable and measurable entity, filmmakers such as Sergei Eisenstein and Dziga Vertov used rhythm to develop simple editing structures, in essence, creating early examples of algorithmic editing. In the late 1960s and early 1970s, many experimental filmmakers began to use simple schemata to edit their films and began to experiment with the optical printer, a device which allowed for the creation of slightly more complex schemata through the use of a programmable sequencer.

Dziga Vertov, page from his notebook.

One of the earliest attempts to theorize about algorithmic editing occurs in Vertov’s essay “Kino-Eye to Radio-Eye,” where he describes one stage of editing as a “numerical calculation of the montage groupings.”2 Vertov explains that editing is “…[t]he combining (addition, subtraction, multiplication, division, and factoring out) of related pieces.”2 By describing editing in terms of mathematical process, Vertov is, in essence, invoking the language of algorithms. Moreover, Vertov implied that every well-equipped editing table should contain “definite calculations, similar to systems of musical notation, as well as studies in rhythm, ‘intervals’ etc. …”3 and that it is the editor’s job to “reduce this multitude of ‘intervals’ (the movements between shots) to a simple visual equation”4 To this end, Vertov often experimented with graphically charting or scoring a montage. By frame-counting and viewing the work as an equation, Vertov demonstrated his interest in algorithmic editing. In practice, by editing according to a schema, Vertov was able to create film poems by structuring the montage according to rhyming schemes similar to those found in poetry and music. As explained by film critic Carloss James Chamberlin, Vertov was primarily interested in “… [t]he gaps between shots which, properly handled, yielded a beautiful pattern of variation – a tactility – a new sonically inspired aesthetic.” 5

In “Methods of Montage,” a 1929 essay by Sergei Eisenstein, another early formulation of algorithmic editing is introduced. In this essay, Eisenstein introduced metric montage as an editing technique fundamentally concerned with “the absolute lengths of the pieces.”6 The technique is created by editing sequences together according to their lengths in “a formula-scheme corresponding to a measure of music.”7 In essence, this technique involves counting and the application of a simple procedure, or algorithm, to these frames. Eisenstein theorized that “… [t]ension is obtained by the effect of mechanical acceleration by shortening the pieces while preserving the original proportions of the formula.”7 That is, metric montage could be used to intensify a sequence; however, if the pattern became too complex, then the use of metric montage produces a “chaos of impressions, instead of a distinct emotional tension.” 7 At this point, it is possible to observe the difference between Eisenstein and Vertov’s use of this editing schema: namely, Eisenstein believed that it was a technique that could intensify the montage, whereas Vertov believed that it was a poetic device.

A classic example of metric montage occurs in Eisenstein’s October (1929). A long shot of a large crowd of protesting Bolsheviks is interrupted by a series of two alternating shots, each one or two frames in length, one of a machine-gun and the other of a gunner’s face. Eisenstein uses metric montage to heighten the tension, and with the use of this technique, one can almost hear the pounding of the machine-guns firing. Produced the same year, Vertov’s The Eleventh Year (1929) also made use of metric editing.8 Eisenstein described the mathematical complexity of Vertov’s editing used to create the film as “so complex in the way its shots are juxtaposed that one could establish the film’s structural norm only with a ‘ruler in hand,’ that is, not by perception but only by mechanical [metric] measure.”9 Although Vertov’s film might not fit Eisenstein’s personal tastes and preferences, through this description he establishes it as an early algorithmically edited film.

In the late 1960s and early 1970s, there was a renewed interest in algorithmic editing which stemmed from an intellectual and aesthetic preoccupation with %lmic structure by experimental filmmakers in the United States. In Dreams of Chaos, Visions of Order, James Peterson observed that many structural filmmakers produced films by using simple schemata; in other words, many of these filmmakers were producing films using algorithmic editing. Peterson introduced the term simple schematic films to describe a subset of structural films “whose global template schemata – those that structure the film as a whole – are exceedingly simple and very predominate.”10 and described two types of these global templates used to produce simple schematic films. The first schema, simple numerical schema, involves enumeration, whereas the second schema, simple permutational schema, involves the unordered rearrangement of an image set. It can easily be argued that employing a simple schema does not necessarily limit the scope of the work. For instance, consider Peter Greenway’s first feature length film The Falls (1980), a fascinating work based on a simple numerical schema following lexicographical ordering. The work systematically examines ninety-two people whose surnames begin with the letters “FALL-.” Despite the relatively simple structure, the stories intertwine, and the film slowly reveals the idiosyncratic nature of the bureaucracy that produced the directory entries upon which the !lm is based.

Peter Kubelka’s Arnulf Rainer (1960) and Takahiko Iimura’s 24 Frames per Second (1975; revised 1978) are two elegant, algorithmically edited films that are completely determined by their editing schema. In Arnulf Rainer, Kubelka used an editing chart to produce a film consisting solely of black and white frames. Kubelka’s film takes this idea to its ultimate extreme by reducing cinema to its purest form – black and white frames, silence and noise. Originally, the film was commissioned by painter Arnulf Rainer to document his practice. When Kubelka was unsatis!ed with the footage he shot of Rainer, he made the ultimate homage – a film which would “survive the whole of film history because it is repeatable by anyone. ”11 Kubelka even proclaimed he would commit the script to stone so that the !lm would “last 20,000[years], if it is not destroyed.”11 Iimura’s 24 Frames per Second is also a film consisting solely of black and white frames whose order is completely determined by a simple permutational schema. The film is divided into twenty-four sections, and each section begins with a title of the form N/24, where the numerator, N, increases in each cycle beginning with 1 and ending with 24. In the Nth section, N frames are placed among 24 − N black frames, followed by N black frames placed among 24 − N white frames. This process continues until each of the possible permutations occurs and all of the possible permutations have been exhausted. With each cycle, white frames become black, and black frames become white. In an interview with Scott MacDonald, Iimura explains, “I related it [24 Frames per Second] to the ancient Chinese yin-yang symbol, which I tried to translate into filmic time.”12

Kurt Kren’s 6/64 Mama und Papa (1964) is another film algorithmically edited using a simple numerical schema. Peter Tcherkassky provides a wonderful anecdote about the editing of this film:

In 1964, Wien Film Laboratories refused to print 6/64 Mama und Papa. When Kurt Kren submitted the original, the film grader said with an undertone of sympathy that, given the number of cuts, one would not be able to make out anything. His worries were groundless; when Kren came to pick up the print, some technicians with flushed faces left the projection room, telling him to get out and never come back again.13

One can observe that the film was cut by hand, since the splice lines are visible, adding a violent vibrating line to the foreground against the backdrop of Otto Mühl throwing blood and urine on another performing artist. The violent and systematic cutting of the film elegantly blends two of the ideas that were predominant in the Austrian experimental art scene at the time: namely, Viennese Actionism, an art movement that exchanged the canvas for the human body, and systematic art, art made by employing mathematical structures and rigour.

One of the most interesting examples of algorithmic editing can be found in Hollis Frampton’s Zorns Lemma (1970). By introducing the perfect amount of ambiguity and structure, the film both consciously engages the viewer and seeps into the viewer’s subconscious imagination. As Peterson explains, while the underlying schema itself is clear, the principles that “determine the relationships between them are less apparent”14 Zorns Lemma itself is divided into three parts. The first part of the film consists of a female narrator reading verses from the Bay State Primer (an eighteenth century grammar textbook) set to a black screen. Each verse focuses on a word beginning with a letter from the Roman alphabet – a 24-letter predecessor to the contemporary English alphabet where I/J and U/V are considered equivalent. The second part of the folm, the “main section”15 and the portion that uses algorithmic editing, is set in silence and consists of “2700 one-second cuts, one-second segments, twenty-four frame segments.”15 The section begins by cycling through one iteration of the Roman alphabet “typed on tinfoil and photographed one-to-one close-up.”15 In the following iterations, each letter sequentially is replaced by a word that begins with the same letter, selected using a chance operation.16 Finally, each letter is gradually replaced by an active image from a database of other active images, each playing at a rate of one second per iteration, until all of the letters are replaced, thereby concluding the second section. Finally, in the third section of the film, a man, a woman, and a dog walk across a snow-covered field from foreground to background, while six women read sections from Robert Grossetestes’s On Light, or the Ingression of Forms (an eleventh century work of metaphysics) at a rate of one word per second, allowing the viewer the time and space to contemplate the structure of the film.

In the late 1960s and early 1970s, increasing artistic access to the optical printer – a device which up until that time had been “more firmly associated with Hollywood special effects and industrial optical work than with experimental filmmaking”17 – a key component in the development of algorithmic editing. One of the printer’s strengths is the sequencer, a programmable mechanism which controls the communication between the camera and the projector, transforming the printer into a compositional device that can be programmed to perform much more complex algorithms. In fact, using the optical printer as a compositional device can be seen as the filmic predecessor of the computer.

The avant-garde has always been ahead of its time; therefore, in order to understand the present condition, it is often beneficial to understand the avant-garde of the past. Currently, we are at the point where computer users have the ability to retrieve multi-media information from enormous, well-indexed databases, and one of the ways to access these databases is through algorithmic editing. By establishing the origins of algorithmic editing, it is possible to better understand its contemporary aesthetics and sociocultural aspirations.

  1. Lev Manovich, The Language of New Media (Cambridge:MIT Press, 2001), 50. [↩]
  2. Dziga Vertov, Kino-Eye: The Writings of Dziga Vertov, ed. Annette Michelson, trans. Kevin O’Brien (Berkeley: University of California Press, 1985), 90. [↩] [↩]
  3. Ibid., 100. [↩]
  4. Ibid., 90. [↩]
  5. Carloss James Chamberlin, “Dziga Vertov: The Idiot.” Senses of Cinema 41 (November 2006). [↩]
  6. Sergei Eisenstein, “Methods of Montage,” in Film Form: Essays in Film Theory, ed. and trans. Jay Leyda, 72–83 (New York: Harcourt, Brace & World, 1949), 72; Emphasis added. [↩]
  7. Ibid., 72. [↩] [↩] [↩]
  8. Vlada Petrić, Constructivism in Film: The Man with the Movie Camera, A Cinematic Analysis (Cambridge: Cambridge University Press, 2012), 183. [↩]
  9. Ibid. 59; This translation is slightly from Leyda’s translation in Film Form, 73. [↩]
  10. James Peterson, Dreams of Chaos, Visions of Order: Understanding the American Avante-Garde Cinema (Detroit: Wayne State University Press, 1994), 93. [↩]
  11. Peter Kubelka, “The Theory of Metrical Film,” in The Avant-Garde Film: A Reader of Theory and Criticism, ed. P. Adams Sitney 139-159. (New York: 1978), 159. [↩] [↩]
  12. Scott MacDonald, A Critical Cinema: Interviews with Independent Filmmakers (Berkeley: University of California Press, 1988), 132. [↩]
  13. Peter Tscherkassky,“Lord of the Frames: Kurt Kren,” trans. Elizabeth Frank-Groeßbne, Millennium Film Journal 35/36 (Fall 2000): 147. [↩]
  14. Peterson, Dreams of Chaos, Visions of Order, 115. [↩]
  15. Peter Gidal, “Interview with Hollis Frampton,” October 109 (Summer 2004), 94. [↩] [↩] [↩]
  16. Gidal, “Interview with Hollis Frampton,”96. [↩]
  17. Julie A. Turnock, “Plastic Reality: Special Effects, Art, and Technology in 1970s US Filmmaking,” Dissertation University of Chicago (2008), 68. [↩]