Interpretation in human-machine improvisatory performance

Interactive music, musical meta-creation, electro-acoustic improvisation, human-machine performance… each of these labels refers to a type of performance that involves the improvised collaboration between humans and computer music systems. I have recently been fascinated by the contradictions that abound in this space. For me, placing an instrumentalist in a performative context alongside an autonomous or responsive software system raises significant questions about performative agency, as well as the notion of artistic authorship. This has led me to begin thinking about how the term musical ‘interpretation’ might apply to this context, in addition to improvisation, which is undoubtedly fundamental to this type of performance practice.

At first, speaking of musical interpretation might seem like somewhat of an anathema in the context of free or non-idiomatic improvisation. Of course, by definition in a freely improvised performance musicians develop a musical structure in real-time without the presence of an overarching compositional framework. The notion of interpretation as requiring something to interpret – that is, a pre-existing ‘work’ in the form of performance instructions typical of a musical score – does not exist in any traditional sense. However, despite the best efforts of programmers/system designers to seamlessly embed such systems into previously human-only musical performance, it may be argued that the introduction of autonomous systems into this performance scenario does challenge traditional understandings of improvised performance practice. As such, I believe the term interpretation warrants some attention here.

In the broadest sense, interpretation in performance might be defined as the process of making sense of the artistic ideas of one or more authors, and bringing them to life in the form of a real-time performance. Such an understanding of interpretive performance is of course common to all forms of the performing arts, including dance and its connection to choreographers, theatre and film’s playwrights and screenwriters, and finally to music and its composers. For music as for other art forms, integral to the notion of interpretation is the medium in which artistic ideas are transmitted outside of the performative context itself, what we might call the musical text. It is the interpreter’s job to make decisions about how to most successfully bring to life a composer’s ideas within the constraints communicated by their text. A composer might be fastidiously prescriptive by offering many detailed symbolic instructions for the performer to follow (as in the complex traditional notation of Brian Ferneyhough, or the tablature notation of Aaron Cassidy), or they might outline a framework that guides a performer’s interpretive decisions (“Draw a straight line and follow it” – La Monte Young; or “Make a sound in a middle place, in some respect, of the sounds around it” – Wolff).

Regardless of the medium through which a composer’s ideas are expressed, however, the contents of the musical text not only outline a composer’s musical instructions to be executed, they also embody and express context-specific assumptions about the role of the performer in bringing the work to life. To give a few examples, figured bass notation assumes a level of contextual understanding about harmony on the part of the performer; tablature notation directs the performer towards the execution of physical action before sounding result; graphic notation often relies heavily upon the more subjective interpretive faculties of musicians; and textual instructions like those of La Monte Young and Wolff focus upon providing performers with a musical and interactive framework with which to construct a musical happening. Despite their disparate manifestations, all of the above types of musical text articulate the desired boundaries of a musical performance, and imply various levels of musical and interactive freedom and constraint for the performer. These examples all involve some combination of explicit and implicit constraints upon the performer, and most of them act as a form of instruction. It is within these sets of explicit instructions and implicit boundaries that the musician is said to interpret a musical work.

So what is it that qualifies a musical performance as interpretive as opposed to it being completely improvised? And how might we understand the role of implicit musical constraints in the face of the more explicit requirements found in a set of instructions? The distinction between the concepts of instruction and constraint is an issue that dominates discussions surrounding the relative openness of musical texts, and therefore their ultimate interpretation by performing musicians. Understanding a musical text as a set of explicit instructions provided to performers, the term interpretation implies a balance between fidelity to these instructions, and the injection of significant performative and stylistic understandings of the composer’s intentions by the musician. In addition, the relationship between adhering to and interpreting instructions is often heavily dependent upon the stylistic norms of the musical period in which the work was written (as is often the case in the performance of ‘classical’ music). In this context, interpretation defines the process by which any gaps in communication between composer and performer are made sense of in light of accumulated cultural and stylistic norms. In this respect, the musical text (score) communicates the requirements of the work as articulated by the composer, whilst the musical and cultural context in which the work resides provides further constraints on any reasonable and/or correct interpretation of it.

It follows from this that the more unequivocal a text appears as a form of instruction, and the more significant the weight of cultural conventions associated with that text, the less interpretive flexibility is offered to the performer in creating their own personal ‘reading’. As more ambiguous forms of musical text, graphic scores are often provided to performers with no accompanying instructions or performance direction. Although such scores may not communicate explicit performance requirements of the work to the performer, given their identity as scores these objects still exist as forms of musical text to be interpreted. Graphic scores range from a set of symbolic instructions that leave levels of freedom within certain musical parameters, to objects of immense interpretive flexibility (such as in Cornelius Cardew’s Treatise, for example). In the latter case, the notion of interpretation rests upon the performer developing explicit rules and procedures in order to realise the musical work. Arrangements of graphic symbols such as those found in Treatise therefore become the catalyst for the creation of a personalised musical and sonic grammar in response to the text. Rather than acting as direct instructions, such scores provide a set of loose constraints that anchor a musician’s interpretive decisions; choices that are both guided by their own personal musical vocabulary, and the cultural and stylistic context of experimental musical performance.

Discussing the role of the score and of the composer in contemporary music-making, saxophonist Evan Parker posits that a musical score might be considered as either one of two contrasting types of text: in some cases as a representation of an ‘ideal’ performance, and in others as a ‘recipe for possible music-making’ (Bailey 1993, pp. 80-1). Speaking as an improviser, Parker’s opinion on the role of the composer and the prescriptive nature of some musical scores is articulated in his characterisation of the composer’s role as score-maker: ‘… if anyone in the production of a music event is dispensable, it is the score-maker, or the ‘composer’ as he is often called.’ (Bailey 1993, p. 81) Although the perspective articulated here is largely negative, Parker notes that the more musical scores resemble the ‘recipes’ he describes, the more there is a gap between the score as ‘ideal’ and any live performance of it. It is from within this gap that the real-time act of performance resides, leaving open for consideration further context-specific ‘ingredients’ of musical performance in addition to the instructions outlined by the composer. These may include the choice of specific musicians to perform the work, where the work will be played as well as how the audience might react.

Speaking of the difference between requirements and constraints in the context of musical performance, Bown has noted that performances of graphic scores may also include forms of extra-musical constraint that should be considered either directly curatorial, or at least requiring a form of interpretation by a group of musicians (Bown, pers. comm., 28 September 2014). Such constraints might include the decisions and wishes of a concert promoter (venue decision and therefore acoustics, performers asked to ‘keep it short’), discussions regarding musical/sonic materials between players, to the presence of ‘garish paintings’ hanging on the walls of the concert space. Such instructions and environmental constraints will of course require a form of interpretation on the part of the musicians that will directly or indirectly affect the resultant musical outcome of the performance. As Bown implies, such constraints often also apply to freely improvised music, no doubt affecting the way in which musicians interact with each other on stage, despite the lack of a unified performance framework as embodied in a score.

If such extra-musical constraints may act as objects of interpretation to a group of improvising musicians, might they therefore be considered as forming an interpretive framework on par with the ambiguous instructions of a graphic score in which musicians engage? My answer to this question rests upon an understanding of composition and curation as intentional activities of humans acting outside of the real-time context of musical performance. The framing of a performance context might be achieved through the existence of explicit requirements (notes on a score, imposition of formal boundaries etc.), but equally through placing musicians in a performance context that might constrain or alter their natural performance dynamic in some intentional way. However, the inherent collectivity and self-organisational properties of improvised music-making, free of intentional constraints on musical structure and materials, preclude it from being understood as an interpretive activity.

Because of a decided lack of musical text to be interpreted in this context, any extra-musical constraints that might influence an improvised performance should be considered part of the social and cultural context of improvised performance itself, and not intentionally imposed as objects of interpretation on an improvised performance. Musicians that engage in improvised performance bring with them their own personal histories of engagement with the practice, and although the boundaries and constraints may change from performance to performance, these form part of the context of improvised performance that has developed over time. This context both normalises and frames the performance activity itself. As such, although external constraints may be ‘interpreted’ by a group of improvising musicians – and may therefore affect the musical discourse – these constraints should be seen as manifestations of the context in which an improvisation takes place, and not part of a set of intentionally curated decisions in which the musicians must engage. The social and cultural context enables improvisatory music-making to occur, rather than directing it as such.

Of course, the most fundamental and defining constraint of improvised performance is the contribution of the individual performers to the musical outcome. From a freely improvised perspective, the most important ingredient of any performance is the ability for improvising musicians to be free to make their own decisions on how to affect the musical discourse, without having to defer to a set of instructions that might constrain these decisions. A form of musical activity that Parker has outlined as his ‘ideal music’, free improvisation therefore not only eschews the presence of a pre-defined musical text, it also places specific emphasis on the reliance upon the interpersonal relationships between like minded musicians improvising ‘freely in relation to the precise emotional, acoustic, psychological and other less tangible atmospheric conditions in effect at the time the music is played’. (Bailey 1993, p. 81)

So, acknowledging these specifics of improvised performance practice, what of the inclusion of autonomous systems to improvisatory practice? Can we assume that an improvised duo or ensemble performance would function in an identical way with the added presence of an autonomous musical system? To answer this question we must concern ourselves with the ontology of such systems, and the way in which musicians may perceive them in performance. With the ability to generate independent musical material as well as to adapt to their performance environments, it might not be so unusual to think of such systems as ‘virtual performers’ in their own right. In some respects, one might argue that the presence of such systems in performance, although synthetic, would not change the nature of the musical context given that a successfully cognate system would act in the same way as another musician. Successful systems of this type would therefore share authorship of a musical performance with their co-contributing human improvisers. From this perspective, the presence of such an autonomous machine might simply form part of any ‘less tangible atmospheric conditions’ Parker has referred to – its status as a machine contributor resting as a mere technicality in the context of freely improvised performance.

Contrary to this assumption however, it is my opinion that the introduction of such a system does alter the way in which the performance context is perceived, thereby changing the nature of the practice itself. The presence of a generative, responsive or autonomous musical system in performance brings into focus the relationship between system designer and any human musician engaged in this practice. It is my view that such types of human-machine performance take place as part of a unique form of socio-technical curation. The autonomous machine significantly affects improvisatory performance practice due its existence as a programmed entity. It is precisely because of the specific creation of a non-human actor (the software) to be engaged with in performance – a context traditionally reserved for human musicians – that the dual notions of interpretation and constraint in performance become relevant.

However, at issue when relating these concepts to improvisation-centred human-machine performance is the very notion of a ‘work’, the detachment of the developer/designer from the instantiation of the work in performance, and the lack of a perceivable musical text in either traditional or non-traditional terms. Questions we may ask therefore are where the work resides in human-machine performance, whether or not we can attribute the proposed musical framework (inclusive of the software) to a single author, and whether or not the software itself may be considered a musical text. Such questions lie at the heart of the concept of authorship in human-machine performance, problematising the notion of autonomy of both a software system and the human improviser in such scenarios.

Firstly, regardless of any manifestation of autonomy on the part of a musical system, one cannot discount the fact that such systems are designed by human actors – namely their programmers or system designers. Taking a broad look at the practice, it is first and foremost these actors that propose the musical scenario of human-machine performance to improvising musicians, either explicitly (via invitation) or implicitly (through software distribution). Regardless of how this proposition is made, in this context the system designer, acting as author, has proposed a musical framework to be navigated and explored in performance. The placement of one or more human improvisers in such a performance context should therefore be understood as a non-trivial act of curatorial authorship. The programmer is no longer simply the author of a piece of software, but is also partly responsible for the musical and interactive framework in which both human and machine engage. From this perspective, although a system may act in an unsupervised manner, the programmer shares authorship over the musical discourse through their software’s interactive behaviour, acting as a part of this curated performance framework.

Secondly, neither this broad curated framework nor any musical system itself exists in an aesthetic vacuum. Human actors developing such systems indelibly hold personal musical goals and preferences, many of which are subtly (and sometimes overtly) manifested in the behaviour of their systems in performance. Although our musical systems are often capable of surprising yet musically coherent results, their capabilities are inevitably understood as the results of the programmed decisions of some human author external to the present interaction. Whilst neither the programmer nor the machine may have made specific requirements of a musician’s performance, the programmed dynamics of such software systems will ultimately contribute to framing the musical and interactive boundaries of any human-machine improvisation.

Designer goals and preferences are also manifest in the manner in which improvising musicians are integrated into interactions with such software systems. In some cases, software behaviours and dynamics may have been explained to a musician prior to an interaction, whilst in others they may be left to the improviser to be discovered during performance. However, even if a musician is asked simply to ‘play normally’ as if improvising with another human, or advised not to try and provoke a direct response from the software, certain interactive preferences are revealed to the performer prior to an interaction. In the context of freely improvised musical performance, spontaneity and the ability to provoke and surprise are important elements of the performance practice. The choice not to reveal the underlying capabilities of an autonomous system can be seen as a manifestation of such an attitude to performance. As such, assumptions and expectations of both improvisation style, as well as a musician’s ability to act as if the machine is human are further revealed.

Another important factor in considering the nature of human-machine improvisation is the awareness and understanding of a machine’s capabilities. Just what abilities the machine has, whether directly responsive, analytical, generative or otherwise will no doubt affect a musician’s perception of their relationship to the machine in performance. In this respect, initially unfamiliar performance paradigms encountered as a result of human-machine performance context can be conceptualised as a form of material with which the human improviser brings their own musical self to bear. In addition, whereas human musicians conceivably approach an ensemble improvisatory context aware of the both the musical potential and cognitive faculties of their fellow human interlocutors, the same cannot reasonably be said of human-machine contexts.

In human-human improvisational contexts, although the possibility of encountering surprising musical materials and forms is always apparent, such materials are subconsciously rationalised and understood by musicians in terms of the skillset and abilities of their decidedly human counterparts. By contrast, any relationship that develops between a human and a computer system during performance might be characterised as one of navigation, exploration and discovery. The very programmed nature of a system’s interactive and sonic behaviour promotes a mode of interaction whereby the constraints of any given system are revealed during the act of performance. Programmed constraints, as experienced by the musician in performance, manifest themselves as both musical and interactive constraints on improvised performance. The more a musician spends time with such a system, the more such constraints are revealed. As such, these flexible and relational musical systems promote exploratory modes of interaction due to their inherent curiousness as programmed, interactive entities.

So does this mean that the software system itself becomes a type of musical text that the musician interprets? Although a musical text in the form of explicit requirements discussed previously might not exist, as we have seen, in communicating constraints upon a performance scenario and by embodying design decisions of human authors, these artefacts certainly contribute to the creation of an interpretive framework in which the musician engages. Even if one disregards some of the more obvious compositional decisions made by a system developer (such as parameter mapping, choice of sonic vocabulary etc.), we can also understand the choice of a particular generative algorithm as well as the type of audio analysis and feature extraction used in such software as aesthetically motivated. Although these decisions may seem fine-grained and technical in nature, they can reveal some of the context-specific assumptions of a system designer, and therefore further point towards the software system as providing constraints on performance.

It may therefore be argued that the development of the software itself is akin to the creation of a type of musical text not dissimilar from the graphic and open-scored works discussed previously. However, rather than embodied in an external graphical or textual representation, the constraints on a musician’s improvised performance with such machines are interactively instantiated through performance with the software itself. Certain modes of performance will ultimately reveal themselves as implicitly more ‘reasonable’ or ‘sensible’ than others, and may also have been communicated as such by the developer prior to an interaction. As such, imbedded in the software as text are the context-specific assumptions and expectations of a type of performance practice envisaged by the author of the software, as revealed by both the sonic and interactive constraints experienced in real-time by the human performer.

In such musical contexts, whilst the musician is certainly improvising with the machine, I would also argue that they are also working within a type of interpretive framework. Any musical performance with such systems may be seen as an instantiation of the combined musical ideas of the system developer, the musician navigating this space of ideas, and the interactively instantiated responses of a machine to the actions of both the performer and the overall musical environment. It is precisely because such systems are imbedded with these subjective attributes that performers in this context are necessarily engaging in a form of interpretation. Although their moment-to-moment performance may be freely improvised, the framing of the interactive context cannot suggest anything other than an interpretive framework. From this perspective, any consideration of a musical text must take into account the entire performance scenario.

The musical text is, in effect, the boundaries and constraints of such a human-machine musical interaction as influenced by the machine’s perceived capabilities. Navigating these possibilities in a truly interactive sense is the task laid out for the musician. Interpretation can therefore be characterised as the navigation a space of potential relationships between human and machine agency, a context envisaged and brought forward by a system developer to a live performance context.


The thoughts above have been galvanised by some fruitful discussions with a number of fantastic musicians and researchers recently. Many thanks to Owen Green, Ollie Bown, Bill Hsu, Mark Summers, Aengus Martin, Arne Eigenfeldt and Pierre-Alexandre Tremblay for taking the time to discuss this topic.


Bailey, D. 1993, Improvisation: Its Nature And Practice In Music, Paperback, Da Capo Press, Cambridge, MA.


One thought on “Interpretation in human-machine improvisatory performance

  1. Pingback: Revisiting Symbiosis: a tentative manifesto? | :: ben carey ::

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s