Working generatively – some thoughts and questions…

So I’ve been thinking recently about drawing a distinction between a generative ruleset or process, and its eventual use in a working creative system. What it seems to me is that perhaps this curatorial, compositional process is sometimes not given enough credit – and that this process extends much further than one of ‘mapping’. Lately I’ve been asking myself some questions along these lines. Here are just a few of these:

– Can algorithms be neutral, that is, can they easily be transferred from one system to another?

– Are transferable, generalisable algorithms preferable in generative arts practice?

– What kinds of aesthetic decisions are made when one makes makes creative use of a generative process? Are there recognisable, common approaches?

– What can we learn about our art practices by looking at the process of translation between an algorithm and its instantiation in a working system?

– If a generative algorithm is tightly bound to an individual usage case, where does the algorithm end, and the artwork begin?


Although I like the idea that employing a generative algorithm in a system might be seen as a curatorial act for the artist, at the same time, I think it is also much more than this – I think it is a bi-directional process of translation. For me, understanding how this process of translation twists and bends our creative practice towards the idiosyncratic is crucial to understanding the power of generative approaches. Perhaps viewing generative art this way means that we can take stock of what’s really driving the decisions we make when creating these systems using generative algorithms.

Interpretation in human-machine improvisatory performance

Interactive music, musical meta-creation, electro-acoustic improvisation, human-machine performance… each of these labels refers to a type of performance that involves the improvised collaboration between humans and computer music systems. I have recently been fascinated by the contradictions that abound in this space. For me, placing an instrumentalist in a performative context alongside an autonomous or responsive software system raises significant questions about performative agency, as well as the notion of artistic authorship. This has led me to begin thinking about how the term musical ‘interpretation’ might apply to this context, in addition to improvisation, which is undoubtedly fundamental to this type of performance practice.

At first, speaking of musical interpretation might seem like somewhat of an anathema in the context of free or non-idiomatic improvisation. Of course, by definition in a freely improvised performance musicians develop a musical structure in real-time without the presence of an overarching compositional framework. The notion of interpretation as requiring something to interpret – that is, a pre-existing ‘work’ in the form of performance instructions typical of a musical score – does not exist in any traditional sense. However, despite the best efforts of programmers/system designers to seamlessly embed such systems into previously human-only musical performance, it may be argued that the introduction of autonomous systems into this performance scenario does challenge traditional understandings of improvised performance practice. As such, I believe the term interpretation warrants some attention here.

In the broadest sense, interpretation in performance might be defined as the process of making sense of the artistic ideas of one or more authors, and bringing them to life in the form of a real-time performance. Such an understanding of interpretive performance is of course common to all forms of the performing arts, including dance and its connection to choreographers, theatre and film’s playwrights and screenwriters, and finally to music and its composers. For music as for other art forms, integral to the notion of interpretation is the medium in which artistic ideas are transmitted outside of the performative context itself, what we might call the musical text. It is the interpreter’s job to make decisions about how to most successfully bring to life a composer’s ideas within the constraints communicated by their text. A composer might be fastidiously prescriptive by offering many detailed symbolic instructions for the performer to follow (as in the complex traditional notation of Brian Ferneyhough, or the tablature notation of Aaron Cassidy), or they might outline a framework that guides a performer’s interpretive decisions (“Draw a straight line and follow it” – La Monte Young; or “Make a sound in a middle place, in some respect, of the sounds around it” – Wolff).

Regardless of the medium through which a composer’s ideas are expressed, however, the contents of the musical text not only outline a composer’s musical instructions to be executed, they also embody and express context-specific assumptions about the role of the performer in bringing the work to life. To give a few examples, figured bass notation assumes a level of contextual understanding about harmony on the part of the performer; tablature notation directs the performer towards the execution of physical action before sounding result; graphic notation often relies heavily upon the more subjective interpretive faculties of musicians; and textual instructions like those of La Monte Young and Wolff focus upon providing performers with a musical and interactive framework with which to construct a musical happening. Despite their disparate manifestations, all of the above types of musical text articulate the desired boundaries of a musical performance, and imply various levels of musical and interactive freedom and constraint for the performer. These examples all involve some combination of explicit and implicit constraints upon the performer, and most of them act as a form of instruction. It is within these sets of explicit instructions and implicit boundaries that the musician is said to interpret a musical work.

So what is it that qualifies a musical performance as interpretive as opposed to it being completely improvised? And how might we understand the role of implicit musical constraints in the face of the more explicit requirements found in a set of instructions? The distinction between the concepts of instruction and constraint is an issue that dominates discussions surrounding the relative openness of musical texts, and therefore their ultimate interpretation by performing musicians. Understanding a musical text as a set of explicit instructions provided to performers, the term interpretation implies a balance between fidelity to these instructions, and the injection of significant performative and stylistic understandings of the composer’s intentions by the musician. In addition, the relationship between adhering to and interpreting instructions is often heavily dependent upon the stylistic norms of the musical period in which the work was written (as is often the case in the performance of ‘classical’ music). In this context, interpretation defines the process by which any gaps in communication between composer and performer are made sense of in light of accumulated cultural and stylistic norms. In this respect, the musical text (score) communicates the requirements of the work as articulated by the composer, whilst the musical and cultural context in which the work resides provides further constraints on any reasonable and/or correct interpretation of it.

It follows from this that the more unequivocal a text appears as a form of instruction, and the more significant the weight of cultural conventions associated with that text, the less interpretive flexibility is offered to the performer in creating their own personal ‘reading’. As more ambiguous forms of musical text, graphic scores are often provided to performers with no accompanying instructions or performance direction. Although such scores may not communicate explicit performance requirements of the work to the performer, given their identity as scores these objects still exist as forms of musical text to be interpreted. Graphic scores range from a set of symbolic instructions that leave levels of freedom within certain musical parameters, to objects of immense interpretive flexibility (such as in Cornelius Cardew’s Treatise, for example). In the latter case, the notion of interpretation rests upon the performer developing explicit rules and procedures in order to realise the musical work. Arrangements of graphic symbols such as those found in Treatise therefore become the catalyst for the creation of a personalised musical and sonic grammar in response to the text. Rather than acting as direct instructions, such scores provide a set of loose constraints that anchor a musician’s interpretive decisions; choices that are both guided by their own personal musical vocabulary, and the cultural and stylistic context of experimental musical performance.

Discussing the role of the score and of the composer in contemporary music-making, saxophonist Evan Parker posits that a musical score might be considered as either one of two contrasting types of text: in some cases as a representation of an ‘ideal’ performance, and in others as a ‘recipe for possible music-making’ (Bailey 1993, pp. 80-1). Speaking as an improviser, Parker’s opinion on the role of the composer and the prescriptive nature of some musical scores is articulated in his characterisation of the composer’s role as score-maker: ‘… if anyone in the production of a music event is dispensable, it is the score-maker, or the ‘composer’ as he is often called.’ (Bailey 1993, p. 81) Although the perspective articulated here is largely negative, Parker notes that the more musical scores resemble the ‘recipes’ he describes, the more there is a gap between the score as ‘ideal’ and any live performance of it. It is from within this gap that the real-time act of performance resides, leaving open for consideration further context-specific ‘ingredients’ of musical performance in addition to the instructions outlined by the composer. These may include the choice of specific musicians to perform the work, where the work will be played as well as how the audience might react.

Speaking of the difference between requirements and constraints in the context of musical performance, Bown has noted that performances of graphic scores may also include forms of extra-musical constraint that should be considered either directly curatorial, or at least requiring a form of interpretation by a group of musicians (Bown, pers. comm., 28 September 2014). Such constraints might include the decisions and wishes of a concert promoter (venue decision and therefore acoustics, performers asked to ‘keep it short’), discussions regarding musical/sonic materials between players, to the presence of ‘garish paintings’ hanging on the walls of the concert space. Such instructions and environmental constraints will of course require a form of interpretation on the part of the musicians that will directly or indirectly affect the resultant musical outcome of the performance. As Bown implies, such constraints often also apply to freely improvised music, no doubt affecting the way in which musicians interact with each other on stage, despite the lack of a unified performance framework as embodied in a score.

If such extra-musical constraints may act as objects of interpretation to a group of improvising musicians, might they therefore be considered as forming an interpretive framework on par with the ambiguous instructions of a graphic score in which musicians engage? My answer to this question rests upon an understanding of composition and curation as intentional activities of humans acting outside of the real-time context of musical performance. The framing of a performance context might be achieved through the existence of explicit requirements (notes on a score, imposition of formal boundaries etc.), but equally through placing musicians in a performance context that might constrain or alter their natural performance dynamic in some intentional way. However, the inherent collectivity and self-organisational properties of improvised music-making, free of intentional constraints on musical structure and materials, preclude it from being understood as an interpretive activity.

Because of a decided lack of musical text to be interpreted in this context, any extra-musical constraints that might influence an improvised performance should be considered part of the social and cultural context of improvised performance itself, and not intentionally imposed as objects of interpretation on an improvised performance. Musicians that engage in improvised performance bring with them their own personal histories of engagement with the practice, and although the boundaries and constraints may change from performance to performance, these form part of the context of improvised performance that has developed over time. This context both normalises and frames the performance activity itself. As such, although external constraints may be ‘interpreted’ by a group of improvising musicians – and may therefore affect the musical discourse – these constraints should be seen as manifestations of the context in which an improvisation takes place, and not part of a set of intentionally curated decisions in which the musicians must engage. The social and cultural context enables improvisatory music-making to occur, rather than directing it as such.

Of course, the most fundamental and defining constraint of improvised performance is the contribution of the individual performers to the musical outcome. From a freely improvised perspective, the most important ingredient of any performance is the ability for improvising musicians to be free to make their own decisions on how to affect the musical discourse, without having to defer to a set of instructions that might constrain these decisions. A form of musical activity that Parker has outlined as his ‘ideal music’, free improvisation therefore not only eschews the presence of a pre-defined musical text, it also places specific emphasis on the reliance upon the interpersonal relationships between like minded musicians improvising ‘freely in relation to the precise emotional, acoustic, psychological and other less tangible atmospheric conditions in effect at the time the music is played’. (Bailey 1993, p. 81)

So, acknowledging these specifics of improvised performance practice, what of the inclusion of autonomous systems to improvisatory practice? Can we assume that an improvised duo or ensemble performance would function in an identical way with the added presence of an autonomous musical system? To answer this question we must concern ourselves with the ontology of such systems, and the way in which musicians may perceive them in performance. With the ability to generate independent musical material as well as to adapt to their performance environments, it might not be so unusual to think of such systems as ‘virtual performers’ in their own right. In some respects, one might argue that the presence of such systems in performance, although synthetic, would not change the nature of the musical context given that a successfully cognate system would act in the same way as another musician. Successful systems of this type would therefore share authorship of a musical performance with their co-contributing human improvisers. From this perspective, the presence of such an autonomous machine might simply form part of any ‘less tangible atmospheric conditions’ Parker has referred to – its status as a machine contributor resting as a mere technicality in the context of freely improvised performance.

Contrary to this assumption however, it is my opinion that the introduction of such a system does alter the way in which the performance context is perceived, thereby changing the nature of the practice itself. The presence of a generative, responsive or autonomous musical system in performance brings into focus the relationship between system designer and any human musician engaged in this practice. It is my view that such types of human-machine performance take place as part of a unique form of socio-technical curation. The autonomous machine significantly affects improvisatory performance practice due its existence as a programmed entity. It is precisely because of the specific creation of a non-human actor (the software) to be engaged with in performance – a context traditionally reserved for human musicians – that the dual notions of interpretation and constraint in performance become relevant.

However, at issue when relating these concepts to improvisation-centred human-machine performance is the very notion of a ‘work’, the detachment of the developer/designer from the instantiation of the work in performance, and the lack of a perceivable musical text in either traditional or non-traditional terms. Questions we may ask therefore are where the work resides in human-machine performance, whether or not we can attribute the proposed musical framework (inclusive of the software) to a single author, and whether or not the software itself may be considered a musical text. Such questions lie at the heart of the concept of authorship in human-machine performance, problematising the notion of autonomy of both a software system and the human improviser in such scenarios.

Firstly, regardless of any manifestation of autonomy on the part of a musical system, one cannot discount the fact that such systems are designed by human actors – namely their programmers or system designers. Taking a broad look at the practice, it is first and foremost these actors that propose the musical scenario of human-machine performance to improvising musicians, either explicitly (via invitation) or implicitly (through software distribution). Regardless of how this proposition is made, in this context the system designer, acting as author, has proposed a musical framework to be navigated and explored in performance. The placement of one or more human improvisers in such a performance context should therefore be understood as a non-trivial act of curatorial authorship. The programmer is no longer simply the author of a piece of software, but is also partly responsible for the musical and interactive framework in which both human and machine engage. From this perspective, although a system may act in an unsupervised manner, the programmer shares authorship over the musical discourse through their software’s interactive behaviour, acting as a part of this curated performance framework.

Secondly, neither this broad curated framework nor any musical system itself exists in an aesthetic vacuum. Human actors developing such systems indelibly hold personal musical goals and preferences, many of which are subtly (and sometimes overtly) manifested in the behaviour of their systems in performance. Although our musical systems are often capable of surprising yet musically coherent results, their capabilities are inevitably understood as the results of the programmed decisions of some human author external to the present interaction. Whilst neither the programmer nor the machine may have made specific requirements of a musician’s performance, the programmed dynamics of such software systems will ultimately contribute to framing the musical and interactive boundaries of any human-machine improvisation.

Designer goals and preferences are also manifest in the manner in which improvising musicians are integrated into interactions with such software systems. In some cases, software behaviours and dynamics may have been explained to a musician prior to an interaction, whilst in others they may be left to the improviser to be discovered during performance. However, even if a musician is asked simply to ‘play normally’ as if improvising with another human, or advised not to try and provoke a direct response from the software, certain interactive preferences are revealed to the performer prior to an interaction. In the context of freely improvised musical performance, spontaneity and the ability to provoke and surprise are important elements of the performance practice. The choice not to reveal the underlying capabilities of an autonomous system can be seen as a manifestation of such an attitude to performance. As such, assumptions and expectations of both improvisation style, as well as a musician’s ability to act as if the machine is human are further revealed.

Another important factor in considering the nature of human-machine improvisation is the awareness and understanding of a machine’s capabilities. Just what abilities the machine has, whether directly responsive, analytical, generative or otherwise will no doubt affect a musician’s perception of their relationship to the machine in performance. In this respect, initially unfamiliar performance paradigms encountered as a result of human-machine performance context can be conceptualised as a form of material with which the human improviser brings their own musical self to bear. In addition, whereas human musicians conceivably approach an ensemble improvisatory context aware of the both the musical potential and cognitive faculties of their fellow human interlocutors, the same cannot reasonably be said of human-machine contexts.

In human-human improvisational contexts, although the possibility of encountering surprising musical materials and forms is always apparent, such materials are subconsciously rationalised and understood by musicians in terms of the skillset and abilities of their decidedly human counterparts. By contrast, any relationship that develops between a human and a computer system during performance might be characterised as one of navigation, exploration and discovery. The very programmed nature of a system’s interactive and sonic behaviour promotes a mode of interaction whereby the constraints of any given system are revealed during the act of performance. Programmed constraints, as experienced by the musician in performance, manifest themselves as both musical and interactive constraints on improvised performance. The more a musician spends time with such a system, the more such constraints are revealed. As such, these flexible and relational musical systems promote exploratory modes of interaction due to their inherent curiousness as programmed, interactive entities.

So does this mean that the software system itself becomes a type of musical text that the musician interprets? Although a musical text in the form of explicit requirements discussed previously might not exist, as we have seen, in communicating constraints upon a performance scenario and by embodying design decisions of human authors, these artefacts certainly contribute to the creation of an interpretive framework in which the musician engages. Even if one disregards some of the more obvious compositional decisions made by a system developer (such as parameter mapping, choice of sonic vocabulary etc.), we can also understand the choice of a particular generative algorithm as well as the type of audio analysis and feature extraction used in such software as aesthetically motivated. Although these decisions may seem fine-grained and technical in nature, they can reveal some of the context-specific assumptions of a system designer, and therefore further point towards the software system as providing constraints on performance.

It may therefore be argued that the development of the software itself is akin to the creation of a type of musical text not dissimilar from the graphic and open-scored works discussed previously. However, rather than embodied in an external graphical or textual representation, the constraints on a musician’s improvised performance with such machines are interactively instantiated through performance with the software itself. Certain modes of performance will ultimately reveal themselves as implicitly more ‘reasonable’ or ‘sensible’ than others, and may also have been communicated as such by the developer prior to an interaction. As such, imbedded in the software as text are the context-specific assumptions and expectations of a type of performance practice envisaged by the author of the software, as revealed by both the sonic and interactive constraints experienced in real-time by the human performer.

In such musical contexts, whilst the musician is certainly improvising with the machine, I would also argue that they are also working within a type of interpretive framework. Any musical performance with such systems may be seen as an instantiation of the combined musical ideas of the system developer, the musician navigating this space of ideas, and the interactively instantiated responses of a machine to the actions of both the performer and the overall musical environment. It is precisely because such systems are imbedded with these subjective attributes that performers in this context are necessarily engaging in a form of interpretation. Although their moment-to-moment performance may be freely improvised, the framing of the interactive context cannot suggest anything other than an interpretive framework. From this perspective, any consideration of a musical text must take into account the entire performance scenario.

The musical text is, in effect, the boundaries and constraints of such a human-machine musical interaction as influenced by the machine’s perceived capabilities. Navigating these possibilities in a truly interactive sense is the task laid out for the musician. Interpretation can therefore be characterised as the navigation a space of potential relationships between human and machine agency, a context envisaged and brought forward by a system developer to a live performance context.


The thoughts above have been galvanised by some fruitful discussions with a number of fantastic musicians and researchers recently. Many thanks to Owen Green, Ollie Bown, Bill Hsu, Mark Summers, Aengus Martin, Arne Eigenfeldt and Pierre-Alexandre Tremblay for taking the time to discuss this topic.


Bailey, D. 1993, Improvisation: Its Nature And Practice In Music, Paperback, Da Capo Press, Cambridge, MA.

Open for interpretation | François Roche’s architectural self-organisation got my brain going


“I’ve heard about something that builds up only through multiple, heterogeneous and contradictory scenarios, something that rejects even the idea of a possible prediction about its form of growth or future typology. Something shapeless grafted onto existing tissue, something that needs no vanishing point to justify itself but instead welcomes a quivering existence immersed in a real-time vibratory state, here and now.” – François Roche – “I’ve heard about”

In 2012, I attended a lecture at UTS by French architect François Roche. It was fascinating. Here was architect tackling themes of emergence, self-generation, growth and decay both conceptually and practically. His philosophy of research as speculation was also fascinating, and something I’m only now fully coming to grips with. I took some notes at the time, and have only recently returned to them given my thoughts about symbiosis in performance. I was aided in making these connections by a brilliant app called Mohiomap, which allowed me to map hundreds of notes from my Evernote library into organic, typological graphs – showing connections in my writing I wasn’t aware of.

Roche’s talk focused upon his conception and use of organic processes in his practice, and how this can be harnessed within architecture. He spoke at length about the way in which tools for architecture need not be specified, and that the way in which we look at the way something is created needs to be taken from another angle. An example of this is rendering a movie of an envisaged biological/cyborg process creating a structure. The way in which the structure is created in the first place comes out of the way the process is defined. It is only after this that the means by which the thing can be constructed can be envisaged.

This is fascinating on a number of levels. Firstly the idea that one might ‘outsource’ the specifics of a design to an semi-unpredictable process, an algorithm, a biological model. Of course there are clear links here to evolutionary systems but also politically a rejection of top-down hierarchical structuring of elements. The process itself is defined, and the interacting of the individual elements is the thing that specifies the final design. He spoke of the need for architecture not to be thought of as something fixed, something as a finished and crystalised rendering of a pre-conceived idea, but (if I understand it correctly), as something that can be a particular snapshot of a process not completely known in advance. Taking this idea further are projects that are not snapshots, but constructions in a permanent state of change and completion. Architecture as something living, something definitively un-fixed. Designing environments that are an escape from the real world through metaphorical associations, but also ones that live and breathe with the environment in which they are placed, they become part of the environment itself. Things like this include his structure that grows through electromagnetism, collecting pollutants and allowing this inherent environmental factor to contribute to the overall design of the structure.

This process focuses upon the relationship between a structure that is fabricated in this way and the world in which it lives. However, the way the building/structure is perceived or experienced by the outside world is not something that he spoke much about at all. To me this is especially important given that we generally consider architecture as fixed, as something that structures and defines everyday existence in space. The fact that a structure might be built to be specifically affected by its environment turns this notion on its head. Despite the focus upon evolutionary typographies, and the distancing of control of the architect over the structure itself, there are still aesthetic questions here of course – there would most certainly be a set of desired attributes for such structures. The question then in this type of work is whether or not generally understood aesthetic criteria are irrelevant, or is loss of control and specification of a beautiful ‘process’ more important? I guess what I’m wondering then is where the interface lies between the human relationship with the environment. There are low-level models of biological activity that are creating these structures, these works of art in livable form, however the interface with human inhabitants/audience members/receivers, is this relevant?

This is where I’m beginning to make the link with musical generation of this kind, specifically in the context of musical performance and improvisation. Self-organisation in such processes are appealing in these contexts because the enable a different conception of what it means to present structures of many kinds – architectural, artistic, musical etc. In addition, the fact that such processes stem from our understanding of living systems provides a concrete link to those observing/interacting with the result of such processes. We recognise our collective selves, our social and biological nature reflected in the design and implementation of these cyborg constructions. My questions in the context of all of this work stem from higher up however. Whist appreciating the autonomous nature of such process, implicitly understood to be derived from the very same organisation as living systems such as ourselves, can we recognise us? Can I recognise me? In interfacing with a technological construction (art, architecture, performance etc) should be able to recognise me amongst the design? Is there a political statement being made in such types of work about the separation between collectivism and individualism?

If I zoom out a little I find that its all relative to context, and also to different levels of agency, interaction and causality. Take for example a building. The design of the skeletal structures can be informed by longer form collective interactions, things that are perceived on a low level – biological interaction between components, networks and the like. These things are models of evolutionary systems, or at least inspired by them. I understand them to represent processes that make up me, only through abstraction and scientific explanation. However, they cannot represent me as I perceive and know me, for practical and social reasons. A construction in space of the scale of a room or a building is a social space, to be perceived and experienced socially. It therefore makes sense that for a structure such as this represent or be inspired by ‘me’ as related to the world, as part of a collective universal me. Breaking me down into components that are universal requires a very low-level approach, a consideration of where the fundamental rules and logic of biology and social organisation comes from. Building a structure such as this reflects then – in the design process and perhaps also in the continual fabrication process (those structures that continue to mutate and change) – the interactions between low level components common to humanity, but also to life in general – to darwinian evolution.

I have lots of questions. My thought here then is that the concept of agency in both the design, generation and appreciation of such work is somewhat of a contradiction. Agency and autonomy as we conceive of them as intelligent beings are much richer than these low level, long-form associations between individual components. Autonomous and self-organising structures are of course fascinating and rich, over many iterations and longer forms, and these things are perfect for exploration and inclusion in social and environmental spaces. Does this mean however that in turning to such ways of generating and presenting work that there is also some kind of mistrust of individual will and intentionality? Or are we simply not willing (pun unintended) to engage with the generative genesis of such fundamental concepts that make us human – our notion of self identification? How can we as designers, artists, musicians and creatives reconcile the inevitable abstraction from biological models (those that exert real autonomy and self-organisation on a low-level), with our inherent understandings of human will and intentionality? Is it possible to acknowledge creative intentionality and at the same time distance ourselves from it? What does this mean for authorship and those who are perceiving the result of such processes? Is the conceptual space in which such approaches present themselves bound to be perennially open for interpretation?

Revisiting Symbiosis: a tentative manifesto?


Parasitic Relationships No.2 (Symbiosis Sate), 2013: Amber Stucke

A little while back I sketched out some rough thoughts about the metaphor of symbiosis in art and performance. These were really just some nascent musings thrown out there on a whim, coupled with the curious discovery of Leonel Moura’s Symbiotic Art Manifesto. I’ve read back on this post with curiosity over the past couple of months, and recently I’ve revisited some of these concepts to try and tease out what I was trying to say in a little more depth.

Moura and performativity:

In the previous post I mentioned Leonel Moura’s symbiotic art manifesto. What I questioned was how his ideas might be able to intersect with real-time performance, and with interactivity. The word ‘symbiotic’ in Moura’s lexicon describes the relationship between the artist and the machine/robot the artist has made to create artworks. From his perspective, the symbiotic relationship that is so vital for this new type of art exists between the artist as designer/creator and the machine/robot developing the artwork in real-time. In this context, the human needs the machine’s agency in order to render artworks, and the machine needs the developmental guidance of the creator (the structure) with which it is to then make artworks. This metaphor of symbiosis therefore relies upon mutual dependencies that exist between the designer/artist and the automaton/artist, the former as the ‘preconceiver’ of a possibility space for machine performativity, and the latter being the machine that navigates this space autonomously. This is a high level view of dependency in human-machine art making, but as I mentioned previously, it lacks a concern for an intersection with human agency during the performative act itself. What is missing therefore is a consideration for the real-time performance aspect of the creation of the artwork – the rendering of the artwork as a performance.

Looking at human-machine symbiosis during the performative act, the real-time act of decision-making by the machine (both with respect to its internal state and influence from the environment) can be viewed as a symbiont connected to its environment. In Moura’s work, as in the work of many procedural and generative artists, the machine’s automation could be said to act as the artist in proxy, with the results of this autonomous procedure being considered the final artwork. For those generative and/or metacreative works that exist in time (music, animation etc), it is the process of negotiation and generation – the performative act – that is presented as an artwork. For those works in which the environment involves the input of a human interlocutor, this performative act becomes more complex, and the notion of a symbiotic relationship extends into the act of performance itself. As such, mutual dependence between the automaton/machine, the human performer and the possibility space proposed by the designer all factor into our understanding of symbiosis in art. Because of the introduction of the performer to the interdependent relationship between artist and machine, the designer’s relationship to the situation is in some ways diminished, and in others it is strengthened by the real-time encounter.

What I am trying to understand is the entangled relationship between human and machine performances, both outside of and influenced by our focus upon making machines that participate in musical performance autonomous in some shape or form. Where symbiosis comes in is through an understanding of the bi-directional nature of adaptation and learning in the performance/development of such technological systems. For me, what is most important is finding ways of conceptualising interactivity in this performance practice that directly acknowledge the potential for mutually adaptive relationships to develop between humans and machines. To date, I feel that many approaches in this area have downplayed the need to understand the system’s affect on the human, and to focus upon developing systems that display autonomous behaviour for musicians to engage with as if the machine was another human player. This is why Moura’s ideas have struck a chord with me. Although in his manifesto he focuses on the machine’s ability to make art autonomously, he is also concerned with the new relationship that exists between these technologies and human creators. When considering this concept of developing artists rather than art, he asks the question: “what do we become ourselves?” In doing so Moura acknowledges how this technology puts this mutualistic relationship in sharper focus. As he states, “The life of the artist/machine is interlinked to the life of the artist/human.”

Designing for the (un)known

Whilst artificial intelligence and the study of autonomy is a perfectly viable field of study for musical interaction, it is my view that in seeking ‘natural’ behaviours for such systems we must also acknowledge our personal biases towards certain types of musical interaction. That is, we must acknowledge exactly what we deem to be natural. The irony in many biologically inspired generative contexts is that whilst we may be seeking to develop new modes of musical interactivity, the modes of generativity that are sought are often justified because of their ability to act in ways that approximate us as humans. This is indeed a curious paradox. What this kind of developmental trajectory presupposes is that human performance is itself predictable, that the machines we build should be immediately understandable, easy to interact with and adaptable to our needs. What I find fascinating however, having developed and performed extensively with such a system myself, is that the musician who begins performing with these systems is not the same after each successive interaction. Musicians adapt to different performance situations, they learn to embrace new performance scenarios and their conceptions of an ‘ideal’ performance practice expand through interacting with our systems over time. This suggests that if we are to be engaging in the development of new forms of performance practice, we should also consider the way in which our machines shape performers, as much as we train our machines to be just like ourselves.

Also curious in the design of such systems is the focus upon modelling, and or approximating human performance dynamics in the machine performer. Some approaches take as their starting point an established performance practice such as free improvisation, then attempt to understand its inner workings and then use these understandings as a basis for design. As an example, some approaches have emphasised that the practice of collective, non-idiomatic musical improvisation is global in nature, with improvising musicians reacting to the overall sonic environment and not responding directly to the material provided by individual musicians. Implicit in such approaches is an overall concern for designing for musical improvisation as a known and understood musical practice. Because such forms of musical improvisation might be thought of as non-referential, and that the global sound environment plays a larger part than the sum of its parts, consequently any musical system designed to take part in such an encounter should adhere to this understanding of the performance practice.

Here I take a broader view of the role of the design of musical systems. Although I understand human improvisatory practice as being the foundation of any human-machine performance dynamic, I also understand that such a dynamic is in itself fundamentally different precisely because of the presence of the machine collaborator. As such, I believe that any artistic practice that seeks to explore novel interaction paradigms should also be questioning how the role of the machine will ultimately alter understood modes of practice. Although we may be comfortable modelling our machines on types of behaviour closest to our own experience, we must also understand that the very context of human-machine performance practice changes that experience, often even before a live performance has actually begun.

Metaphors for interactivity:

One thing that I have found curious in this area of research and artistic work, as well as in other areas of human-computer interaction, is that the adoption of metaphors often plays a central role to both design and evaluation of our musical systems. Metaphors are often used to describe interactive traits of musical systems and environments – control metaphors related to instrumental performance; biological systems metaphors such as stigmergy that describe/define group dynamics; categorisations of interactivity into differing modes such as conversational/ornamental/instrumental interaction; understanding systems and interfaces as prostheses etc. On the one hand such metaphors serve define desired modes of interaction to aid design, but they are also ways of understanding interactive traits perceived when observing humans interacting with machines. For the former, they provide ways of conceptualising approaches to designing, situating a complex developmental project within a broad interactive/relational framework. They become anchor points with which to ground future design aims. For the latter, they enable a retrospective understanding of the interactive traits exhibited by the interaction domain itself, understanding these situations without imposing any preconceived notions of interactivity on the system.

Whilst some of these metaphors of human-machine interaction have been developed through observation of real-world settings, others remain speculative understandings of the space in which humans and machines interact and are borne from personal artistic experiences and/or aesthetic leanings. As such, what fascinates me is that such speculative understandings say as much about the aesthetic preoccupations of the designer/researcher as they do the phenomenon itself, as I believe they do in Moura’s work. It is my view that the choice to develop systems that can participate with humans in musical performance is in itself a speculative position, and so it is from this vantage point that I situate my ideas of symbiotic human-machine performance practice.

Symbiosis – what is it really?

Here I’d like to outline what I understand about the concept of symbiosis, and how I see it relating to human-machine performance. Symbiosis is a biological process that describes the interaction and often interdependence between organisms in the natural world. There are three main types of symbiosis observable in nature: mutualistic, parasitic and commensalistic. Each one of these types describes the relationship that exists between different organisms, and is related to dependence which is either uni or bi-directional:

  • Mutualistic Symbiosis – describes a situation where two organisms are dependent upon each other, with both benefitting from this interdependence
  • Parasitic Symbiosis – is a one-sided relationship where one organism benefits and the other member is harmed by the interaction
  • Commensalistic Symbiosis – where one organism benefits from the another, yet the other is neither harmed nor helped

Adaptation and dependence:

So, how might/do these ideas play out in human-machine performance? As I mentioned in my previous post, I came to an understanding of symbiosis as a metaphor for human-machine musical interaction due to its focus upon adaptation and dependence, in contrast with a strict consideration of machine autonomy. For me this is a fundamental distinction to make. In my view, the context of human-machine performance practice (including the process of developing musical systems that take part in performance) is something so specific and alien to traditional performance practices that it benefits from being understood, described and designed for in and of its own terms. What appeals to me with the metaphor of symbiosis is that it becomes a way of understanding and conceptualising a type of musical performance that is not possible outside of a human-machine context. Although the continued development of the autonomous capabilities of machines is a natural technical and theoretical trajectory for software development for many fields, I believe what this priority misses out in the context of human-machine performance is an overall way of understanding the mutual adaptivity of humans and machines in these hybrid environments.

For me, symbiosis as an interactive metaphor sits decidedly outside of the purely technological, acknowledging the mutual dependency and mutual influence between player and machine, machine and composer/developer, and researchers and their theories. The metaphor is both inherently interactive, but also has its genesis in the intersection between separate and contrasting organisms. As such, I use it to encompass the space in which human-machine performance takes place, as well as helping to define what approaches to developing technological systems that take part in these performances. However, this shift in focus towards dependence in an interactive relationship should not be misconstrued as a move away from developing the machine’s capacity for displaying agency in an interactive exchange. Instead, the process of symbiosis in performance should be understood as encapsulating the complex interaction between two autonomous entities that are both aware of each other, but also dependent upon each other in performance.

Defining symbiotic musical interaction:

So, taking the above definitions of symbiotic relationships, I would like to sketch out what I think characterises symbiotic interaction in human-machine performance practice, and how I see it reflected in my own work in this area.

For me, symbiotic musical performance means:

  • Mutualism – mutual dependence between human and machine in performance
  • That one’s actions may affect the evolution of the another’s material, though not as a direct reaction to or cause of the other’s responses
  • Machine agency and autonomy are an ingredient of such interactions, though not the only ingredient

The development of interactive systems, whilst often focused upon maximising the autonomous decision-making capacity of the machine, should be thought of primarily as engaging human musicians in a mutually influential performance context with a machine. Human engagement with such systems fundamentally differs from human-human interactive contexts. In human-machine improvised performance, in my view performers are in some way dependent upon the interactive capabilities of the musical system. Musical systems are developed with specific stylistic constraints in mind, and performers interacting with such systems learn to interpret the nuances of these systems through repeated exposure to them in rehearsal and performance. They are dependent on the system in a broad conceptual sense, as it is often the case that the musical system’s interactive vocabulary (or indeed, sonic vocabulary) defines the range of possible musical territory to be covered in an interaction. Although a musician is clearly an autonomous entity, their musical input to the performance is shaped and altered by the context provided by the intersection with the musical system. This may in fact be thought of as a form of interpretative performance, not unlike traditionally understood notions of musical interpretation. This last point opens up a range of issues related to ownership and authorship of which I plan to write about in a future post, so I won’t go into these ideas further here.

From a system design perspective, it is possible to conceive of such systems as enabling either a mutualistic or commensalistic symbiotic relationship to occur in performance, as outlined above. The development of such a relationship need not seek to maximise the qualities of musical autonomy, however musical autonomy may emerge through cumulative interaction with the system over a period of time. The ability for systems of this type to exert agency in performance whilst still depending upon the sonic input from a musician provides a challenge for the system designer. In the context of human-computer improvisation systems, one thing that enables a musical relationship to develop between software and a musician’s performance is how a piece of software seeks to deal with information it receives in the present, as well as how it makes sense of the past. In human-machine performance practice, an interactive environment fundamentally shapes the performance of a human musician over time throughout an improvised session, as well as from session to session. Mutual dependence in this instance refers to the change brought about in the human’s behaviour towards the system, and the ability for the system to change along with the performer. One way of conceiving of machine dependence therefore in this context is through the material siphoned from the performer and used to further its own autonomous behaviour. Taking the history of past material as a space of possibilities provides the machine a sense of performative agency that is directly tied to the cumulative history of the human musician.

Symbiosis and _derivations:

My work developing and performing with the _derivations performance system I believe has embodied this conception of symbiotic performance. One of my central concerns for developing musical performance system was that the machine, although not directly controlled by a human performer, should be heavily influenced by a performer’s playing. Although I was interested in developing a system that could contribute to performance un-assisted, this autonomy would need to be emergent and tied to the specific context of the current performance. In much of my work leading up to _derivations I experimented with techniques for capturing both audio and analysis data streams that could then be used for re-generation. Beginning simply by live sampling snippets of my saxophone performance, I soon realised that I was drawn to using recorded data of many kinds as a means to generate complex behaviour using simple underlying rules. The possibility of re-organising, processing and re-purposing sampled data streams (including audio recordings) became a meta-compositional process, whereby my musical decisions were tied to finding ways of coherently generating new material from material siphoned throughout a performance with a human player. Such generative techniques, in comparison to other approaches to developing autonomous behaviour, are of course extremely dependent on outside stimulus. Without having interacted with a human performer, systems of this type have no space of possibilities with which to generate their own material. This is a complex and often contradictory approach for human-machine performance, which from a design point of view provided a very concrete set of constraints with which to work with during development.

As for human dependence on the machine, this is what I am currently in the process of analysing in much of my writing. The development process that led to the creation of _derivations in its current form involved much live performance. My dual role as a performer and developer on the project meant that the line between development and performance was continually blurred. In my development process I would often have a saxophone at hand ready to ‘test’ my latest changes to the system, enabling me to make on the spot decisions about the software as I continued to refine its capabilities. The affect on the system’s overall design is indisputable, and I would attest that the machine is very much ‘composed’ in this respect. However, reflecting on both this testing process as well as the very many performances and full length rehearsal sessions I engaged in with _derivations, what is more subtle is the way in which my performance style has evolved to encompass the capabilities and intricacies of the _derivations environment. The intimate knowledge I have as both developer and performer of the system enables me to shape my performances in a way that is particularly in tune with the system’s character. I cannot say for certain that I would perform the way I do with _derivations with any other human performer. Knowing the system’s ultimate dependence on both my current and past performance input, as well as how it makes connections between these states means that I am able to provoke and predict certain actions, but also to be genuinely surprised by others. My performance has evolved with the design of the system, but at the same time, I am in the moment of performance dependent on the interactive context that the system provides. This provides a set of constraints for improvisation that I find fascinating in performance. A composed space of possibilities that – for want of a better term – enforces a mutually dependent relationship to develop from moment to moment in performance.

Closing thoughts:

So to finish off these thoughts on symbiosis, below I outline what I see as some of my fundamental approaches to creating mutual dependence in human-machine performance. Although I call this a template, I must stress that these were not guiding principles outlined before developing a performance system, but rather tendencies I have observed through retrospective consideration of my own work that point towards the concepts I’ve outlined in this post. Considering mutual dependence as a design principle is sufficiently broad in many respects, but also very constraining in others. The methods one might use to generate relationships between captured and analysed performance material are many and varied. However, what I have found is that focusing attention on capture and storage, navigation and re-generation has provided an intriguing balance between transparency and opacity of a machine’s contribution to a human-machine improvised performance.

Template for the design of a musical symbiont:

  • The machine is an empty vessel – it cannot act without having interacted with the human
  • The machine and the human form a symbiotic system whose musical results are an emergent result of their interactive history
  • The machine’s behaviour and/or content is in some way dependent upon the human’s current and past actions
  • The autonomy of the machine’s actions is in some way derived from the interactive history with the human performer
  • The machine displays agency by making connections between materials siphoned from human interlocutors
  • The machine feeds off the performer, using captured memories of the encounter as fresh material for its contribution
  • The human’s past becomes a space from which the machine draws upon to influence the future musical states
  • The machine also listens to itself, depending equally on its own past as an indicator of its future state as it does outside stimulus (autonomy)
  • The impetus for the machine contribution is tied to some present understanding of the human’s actions

Practice-Based Research in NIME

Last week I attended the NIME 2014 in London. This was my second NIME, having presented a poster at the 2012 event in Ann Arbor.

What really energised me about this gathering was in fact the workshop on Practice-Based Research, which I took part in on the first day of the conference. Practice-Based Research (PBR) is an area which has been a part of the NIME community since its inception, but I think this is the very first workshop on the topic for the community. The workshop was unusual in that it was a symposium-like event, with a full day of paper presentations on the topic, organised by the Creativity and Cognition Studios at UTS. Amongst the great papers at the workshop I drew some particular insights from a few writers whose thoughts I’ll summarise below. My own contribution to this discussion can be found in the paper I contributed to the workshop, ‘Artefact ‘scripts’ and the performer-developer‘ (a screencast presentation of the paper can also be viewed here), and the full list of papers can be found here.

For my part, throughout my research I have often struggled with the messy relationship between practice and research. The development of technical artefacts (software/code), musical outputs (performances/recordings) and sustained reflection has often been a difficult balance to strike in my doctoral degree. Something that was clear in the workshop was that there are many other researchers in the community asking the same questions about the role of the practitioner-researcher in the field, a field that many at the workshop believe is becoming increasingly dominated by results-driven research in the form of technical reports.

As an interdisciplinary community made up of artists, performers, engineers, luthiers and designers, I believe NIME certainly would benefit from continuing this discourse on the value of practice-based research and its diverse outputs. As Michael Gurevich so elegantly argued during the workshop, in order for the various types of research being conducted in NIME to fully reveal themselves, those doing the research need to stand strongly and make clear the value of the kinds of interdisciplinary knowledge they are contributing.

Some paper summaries:

Michael Gurevich – Methodology Soup

Gurevich’s general argument is that the NIME community has been going further towards a scientific route, however NIME has not always been this way. Practice-based research has been around since the inception of NIME, with performer-researchers historically reporting on their idiosyncratic instrument design work from an artistic perspective. Gurevich argues strongly that the way in which the NIME community understands and deals with the diversity of methods is important for the future of its development. In doing so he classifies several types of NIME papers, including practice-based approaches. He finalises his thought by suggesting that we, as practice-based researchers, more clearly delineate our approach to research and practice, so as to legitimise our research approaches in the broader community.

Jan C. Schacher – Investigating Gestural Electronic Music

“How can I convey the practical insights, the experiences, instrumental reflexes and reflections that I have built up in the course of these performances and works? Is what I’m developing not utterly subjective and tied to the specific musical, performative, and social moment where these pieces come into existence? What would a transferable knowledge be, that I could extract from this practice?”

This paper raises many questions, focusing on trying to tease out what kind of knowledge might be communicable from an artist-researcher’s perspective. In doing so it highlights some messy questions on how to do PBR in NIME. Schacher posits that the perspective of the musician/artist/developer should be considered in NIME research, but he also asks rhetorically what kind of knowledge can be gained from this endeavour. The musical results of an interaction, and understanding the philosophical and psychological dimensions of performing with new interfaces is a strong focus throughout. The author also puts a spotlight upon concerns of performance practice in the field, linking this important aspect of artistic research to the artist-centred perspective of the performer.

Owen Green – The Situation of Practice-Led Research around NIME

“PLR (Practice-Led Research) sacrifices some generality of observation in order to contend with music in its socially-entangled, contentious, noisy complexity.”

The main crux of this article was to propose slippery methodologies for ‘musical research’ in the NIME field. Green’s understanding of research in NIME is broader than that of much of the field, encompassing a more open definition of practice and its role in research. His concern is also focused upon the divide between institutional understandings of research method and how they intersect (or don’t) with musical practices more broadly. Green’s is in part a critique of the focus upon functional and implementation issues of NIME design, at the expense of understandings of practice more broadly. This focus deprives discussion of the musical practice of ‘live electronics’ in his view. The author argues that our research and practice are part of assemblages, where our outputs are directed to differing audiences depending on the particular content of these outputs (technical reports for those interested in design, musical performances for general audiences, discussion of aesthetics for musicological audiences etc). Musical sharing and co-practice are areas Green suggests have been under exploited as ‘methods’ of research. How these are to be integrated and coupled with existing research approaches remains an open question, however it’s clear that a consideration of these inherent musical contexts would further enrich our research perspectives NIME by considering the complexities of real-world practice in more depth.

Andrew Johnston – Some Opportunities for Practice-Based Research for NIME

“At the very least I make this point: if we consider musical expression as a stable construct which we can design for without further consideration, we miss opportunities to explore, and document, the continual exploration, even redefinition, of musical expression through practice.”

An article that resonates with some of my major concerns about research in the NIME field. Johnston’s observation about the dynamic quality of musical expression is followed by the necessary questions as to the relevance of evaluation in this equation. If something is to support musical expression, rather than to enter into a dialogue with it, then how can we evaluate its effectiveness? If we then do consider this dialogue, then what is the relevance of evaluation, and what other ways can we go about understanding the entanglement between design and use? The discussion about affordances, and viewing the interactions between instruments/interfaces and performers/audiences as in constant flux makes a case for a broader view of research method.

Ludvig Elblaus – NIME Design and Contemporary Music Practice: Benefits and Challenges

This paper is an argument for more sustained engagement in the NIME community with the intersections between interface design and practice. Elblaus’ main argument here is that often we view ‘evaluation’ as something we do to prove that our proofs-of-concept work, though they are not necessarily tested properly ‘in the wild’. He suggests that we, as researchers and designers, engage more fully with musical practice and how these interfaces stack up. His advocacy for sustained engagement with interfaces in practice is promising, and he points out that it is really our job as designers to be engaged in this process in order to lessen the disconnect between the space of design and that of cultural appropriation and use (the author criticises the delegation this process by default to musicologists, historians etc).

Symbiosis in art and performance


Painting Robots Orchestra – PRO021113 – Leonel Moura


Mutualistic Relationships No.5 (Symbiosis State), 2013 – Amber Stucke

I’ve been thinking about the process of developing machines that themselves can make music. This whole concept of ‘metacreation‘ – of creating something that creates – and how it relates to our conceptions of creation, design and control over artistic output. The metaphor of ‘symbiosis’ has come to mind thinking about my _derivations system. For me adapting the concept of symbiosis (the biological process of mutualistic relationships in nature) to performance practice with _derivations is apt, because it acknowledges mutual influence as being at the core of an interaction, rather than striving for complete autonomy between man and machine.

Autonomy is a fascinating concept, and one that has so much potential for the development of generative and interactive music, but it only tells one part of the story where interaction is concerned. The idea that machine and a human performer might need each other to create is appeals to me far more, that such mutual dependence creates a unique environment for exploration and joint creation. I also think that symbiosis describes the process in a much more nuanced way than arguing the line between reactivity and interactivity would ever do in determining the responsive characteristics of a performance system. Autonomy of human and machine forms part of the equation, but acknowledging their mutual dependence also enables a way of understanding how one is entangled with the other in performance, thereby opening up ways of analysing such creative strategies from the perspective of system design and real-time performance.

Incidentally, doing some research of symbiosis and art it’s hard not to come across the work of Leonel Moura, artist and designer working with robotics and artificial intelligence. His conception of symbiosis in art is provocative yet appealing, if a little peripheral to the area of interactive performance. Creating machines as artists themselves that can autonomously create is an exciting practice that is happening right throughout the arts, and Moura has a beautiful yet resolute way of describing its potential in his ‘Symbitotic Art Manifesto’. My thought that follows this is how his ideas might intersect with the notion of interactivity. Can his appropriation of the concept of symbiosis be extended to real-time performance? How might we understand the process of creating something that not only creates something new, but also changes the notion of performative relationships throughout the act of creation?

“Symbiotic Art Manifesto:

1) Machines can make art
2) Man and machine can make symbiotic art
3) Symbiotic art is a new paradigm that opens up new ways for art
4) It involves totally relinquishing manufacture and the reign of the hand in art
5) It involves totally relinquishing personal expression and the centrality of the artist/human
6) It involves totally relinquishing any moralist or spiritual ambition, or any purpose of representation

[Making the Artists that make the Art]

Art as we know it is dead. This time it is definite and official.

Often declared during the last century though never actually achieved, the death of art is now a fact. Not just out of a mere wish or avant-garde rhetoric, but because the conditions for artistic production have changed brusquely. Suddenly, all of modern art has become ancient art. Because the idea of art as a product exclusively of human creativity has been finally abandoned, to adopt the notion that it is the direct output of non-human artists.

As usual, such a change of paradigm has only been possible through technological evolution. From the analysis of the parts we move on to the mechanics of complexity. By studying living organisms we are now in a position to realize life as it could be.

When robots ceased to merely simulate human behavior, such as walking, playing football or cracking jokes, to start being used to make art, something very radical happened. Robots that make art are not only questioning the idea of art or philosophy, they even cast doubts on our own condition as human beings. Why bother continuing to do something that machines can do better and more consistently? If art has no purpose, as all the modern and post-modern theories declare, then machines are the best creators.

Once having freed ourselves from making art we may now devote our efforts to generate a new type of artist born from the broth of protobiotics, robotics and artificial life. We can build the machines that will make art. This new artist/machine has no predetermined objective, nor aesthetics, morals or intent. He realizes the last of the “pure psychic automatism” as announced by Breton and partly developed by Pollock. Besides, there is no concern about individualism or identity. The action is collective and the World is apprehended as a common territory emerging from a stigmergic behavior.

From a philosophical standpoint the action is relational and the works that are generated are synthetic proposals issuing from the unraveling of collective experimentation. The life of the artist/machine is interlinked to the life of the artist/human.
When we cease to make art to start making artists, what do we become ourselves? We become symbiotic artists! Humans are no longer concerned about the direct production of objects, but dedicate all their knowledge and energy to create and cooperate with an imaginary, non-human life that is devoted to art-making.

In doing so, the symbiotic artist asserts that technology serves creativity and not the destructive military industry or mercantilism.
The role of the symbiotic artist from now on is to create non-human artists and to cooperate with them to produce art. This entails understanding the rudiments of non-anthropocentric life and creating the conditions for experimentation to take place. In other words, art as it could be. Art of the 21st Century.”

(Leonel Moura, Henrique Garcia Pereira, 2004)

Free writing on trust, learning and adaptation

(Below are some unedited, purely speculative, unsubstantiated and open-ended ideas. The post is more of a question-raising exercise than anything. How this relates to music technology and interactivity is still not clear to me yet… but I’m sure it’s not far off!)

I’ve been thinking a little recently trusting technology, beginning with one the most ubiquitous pieces of software we engage with daily: our search engines. We use search engines these days as a way to find out new information, but at the same time, we use these kinds of tools also to outsource our previously held capacity to store information. Now, we are very good at knowing where to find information, but potentially less adept at storing this information ourselves.

The ease of use of our search engines allows us to quickly look up information and have it available to us within a split second. The practice of tracking down information has been so practiced and learned that it may have replaced our capacity to store information the way we used to. To me this signals a real shift to an extended man-machine interface in every day life. We have outsourced this capacity to machines.

So it then becomes a question of trust. If we rely so much on the capabilities of machines to retain the information we need for our daily tasks, should we not be more wary of the trust we put into them? How can we be sure they are providing us with the most relevant and targeted information we are seeking? Most of the time we are looking at only the first few search returns to our queries. When we click on a hyperlink that points towards a source purportedly containing the information we seek, we are taking a blind leap of faith that the search algorithm has captured the most relevant data.

One interesting observation however is that in trying to use the machines to serve our purposes, we end up second guessing our own methods by which we interact with them, thereby honing our own skills over time and through practice. We know our search engines are well-honed and accurate most of the time, which explains the reason why most searches we do will be discarded if they don’t return a relevant result on the first page, or even the first few results. We therefore adapt our behaviour in order to work from within the machine’s own constraints. If a particular search query doesn’t yield the appropriate results, it is up to us to amend our query, rather than the machine to interpret what we want.

In this way there’s a give and take of agency between man and machine. Over repeated use, we implicitly understand the rulesets imposed by the technologies we use, so we adapt our own behaviour to suit their idiosyncrasies and get the optimal results. Looked at from one perspective, this friction might be seen as something that needs to be avoided at all costs, leading to new ideas about ways of teaching the computer to anticipate and interpret our needs and thereby eliminating our need to modify our behaviour. The theory behind this way of thinking is that the machine should know what the human wants, interpreting their desires regardless of the mode of inquiry. This kind of artificial intelligence gathers information on a user in order to accurately predict the context of the next enquiry. But what if this friction was not something that was to empower the human user, but instead was something that caged the user into an already established modes of interaction?

If a piece of software is geared towards capturing as much information about the user’s past as a predictor of its future, as well as being able to cross-reference its past with the histories of what it deems to be like-minded humans, then does this not constrain the user to a definition of both its past and the past of other users, without acknowledging the our abilities to adapt? I’m starting to think that this is a really fundamental issue, and one that is pertinent to all of our interactions with such new technologies. By teaching machines to adapt dynamically to our needs, we are forcing them to predict the future only by our past and the pasts of others. More importantly perhaps, we might be teaching our machines that we ourselves do not adapt and change, which we know to be a fallacy.

The machine may well be considered intelligent, autonomous and adaptive by our standards, but where does this leave the conception of the human by the machine’s standards? Looked at in this light it might see ourselves as unadaptable, atemporal beings that do not learn, change and adapt our behaviour to suit our environment. Perhaps this is the future of our interactions with technology when we privilege technology that changes, adapts and moulds itself to suit us. By designing to empower the user, perhaps, through technology, we are restricting the future of our own abilities by our current conceptions of them, as encoded in our machines.

BCF2000 | Ableton Live | Animoog

I’m writing this little configuration post as I had a lot of trouble getting the setup I wanted exactly right. It’s something that will help me retrace my steps, but hopefully might be of use to anyone using this controller with live and external synths (iPad included.)

Basically, I wanted to be able to use the BCF2000 to control live in Mackie Control Emulation Mode, as well as send MIDI through the BCF out to my iPad from the MIDI out of the BCF2000. Mackie emulation mode with the BCF is EXCELLENT in Ableton, especially if you’re using LC X View to view parameters. I’m also getting excited about using synths on the iPad since recently getting an iRig MIDI interface. The setup seemed simple enough, but there were a couple of small issues to take care of… not the least choosing different MIDI channels for synth and controller! (der).

Anyway, for what it’s worth, here’s how I now have my setup configured:


MIDI Out A of BCF2000 connected to iRig MIDI In via iRig MIDI DIN to minijack cable.

Boot BCF2000 in Mackie Control for Cubase Mode (MC C) – hold the button below whilst powering the unit on.

Screen Shot 2013-10-21 at 2.38.06 PM






Once inside the Edit Global Mode, set the first V-Pot to U-4 USB and then press ‘Exit’ – this is explained in the BCF documentation

Ableton Live:

Set BCF2000 MIDI configuration in Live’s preferences. Set Track on MackieControl Output (BCF2000) to ON:

Screen Shot 2013-10-21 at 3.46.53 PM














On a new MIDI track, insert an External Instrument Device. Select the BCF2000 under MIDI To, and set the MIDI channel to Channel 2:

Screen Shot 2013-10-21 at 3.19.02 PM






Animoog (iPad):

In the Animoog choose iRig MIDI as Midi Input. Change Channel 2 for MIDI Input (NOT OMNI!!!):



Finally connect the iPad’s audio out to two channels on your audio interface, and choose the Audio from channels in the External Instrument Device (in the above screenshot), and tweak hardware latency.


techniques, tools and performance

This blog post is a pretty long-winded musing on my relationship to technology in music creation, through the prism of some of my recent interests and an analysis of previous experiences. Begun in January of 2013, it was initially written for myself as a way of understanding some of the things that have gotten me stuck creatively. I find this kind of writing therapeutic, the circuitous nature of sifting through my own creative process by articulating it helps me make sense of where I’m at, but also I enjoy being embedded in the actual doing of writing (which may become clear throughout the post). Throughout my doctoral research I’ve written a number of reflective memos about my own creative work, how it’s evolving and where the links are in seemingly quickly changing and sometimes disparate interests. This has been in an effort to make the explicit implicit by highlighting tacit knowledge accumulating through doing – a kind of reflection-on-action as Schön would put it. Coming back to my thoughts helps me understand the changes in my thinking over time, and editing and massaging these memos into a coherent analysis is an attempt to clarify ideas that might not have been clear at the time of writing.

: creative recording and tools

Ever since I began working with electronic music, recording has been a major part of my practice. Exploring the possibilities of recording is what got me into working with sound and computer music in the first place, so it’s no surprise that amongst my various experiments I have maintained an interest in capturing sound from a variety of sources. At the beginning of this year I began getting excited about field recording once more, and about recording in general. I began obsessively looking into expanding the modest recording setup that I currently owned, and did a lot of reading about and listening to field and foley recordings of those in the field that I admire.

A number of years ago, my first experiments with any kind of field recording for a creative purpose were with the minidisc recorder, going out and about with my brother recording what we could with the minimal equipment we had. This was mainly to see if we could use the material to create weird and wacky sounds through manipulation in simple software packages. In other words, the intention behind the recordings was to manipulate, transform and re-appropriate – to be immersed in the doing of recording and processing – rather than to document or to fit a conceptual theme. The problem was however, besides naïve manipulations and experiments of one kind or another, I never really made great use of anything I recorded. I recorded material, explored this material a little bit, but things never seemed finished, or ‘finalised’. Perhaps my intentions from the outset of the recording were too narrow in focus, too focused upon experimentation and less focused upon an end goal.

I mention this because I believe that this has been a recurring theme in my creative relationship with technology. The idea of the tool is enticing, the possibilities of the tool, and I relish in the chance to hone the skillset needed to allow me seamlessly integrate the tool into the eventual context envisaged for its use. However, I’ve found that sometimes I have great difficulty in actually putting the tool to good use, potentially rendering the skillset useless. One of the things that frustrates me is that I really embed myself the pre-planning, the testing, the optimisations, making sure that things work the way they should etc. I am really interested in the quality and the efficiency of the tool as an artefact, of making the artefact/tool something that can be easily incorporated into the specific context it needs to be incorporated in, and then spend much less time working in that context itself. It seems that I’ve sometimes been more excited about setting up the potential of technology, rather than actually exploring this potential through action. So to come back to specifics then, I’ll give some examples of the way I approach technology to give a clearer picture of my relationship to it.

I’m the kind of person that likes to know exactly how everything works before beginning to use it. It’s like setting the scene for an encounter with the world – getting prepared so that in the moment in which you are faced with it, you have skills that you can rely on to help you deal with anything that it might throw at you. If I think about it, I believe I can pinpoint two related yet contrasting contexts where perhaps I developed this mindset that is often reflected in my use of music technology. These are: 1) interpretive musical performance, and 2) learning another language – in my case, French. Both of these really large aspects of my life I think have profoundly affected the way in which I approach technology, but also (tellingly) they also include aspects that I feel are lacking in my relationship with technology.

: instrumental technique and language

Learning instrumental technique is a fascinating thing, but most of all because of the way in which technique becomes ingrained, frozen and ready to be accessed at any point. The process of learning technique is, of course, tightly bound to the musical context envisaged for its use. Also, the notion of having technique implies a strong potentiality for exercising it. The technique has been honed and refined through repetitive practice, and so in this sense it is knowledge and skill acquired through the application of previously acquired knowledge and skill in the same practical domain. Technique acquired through classical training is developed and refined in a practical session, it cannot simply be theorised about, it must be practiced to be developed. Technique is honed and improved through doing exactly what would necessarily be required in the circumstance of a performance.

There are some clear similarities with learning another language. The way that one improves spoken and written communication in another language (after learning some basics about the general principles of how the language works, grammar, etc.) is to use it. Refining instrumental technique (in the case of interpretive – not improvised – performance) is about perfecting a skill set that allows a work to be presented as a finished product to an audience. Language however lives in a social sphere, and as communication it doesn’t have an end goal such as performance that necessitates the technique to be accessed and exercised exactly as practiced. The technique of communicating in another language (whether written or spoken) gets improved by being used, and the end goal of improving the language is for it to be put to use. Therefore, the method by which language is improved is identical to the eventual usage context of the language itself. It could then be said that the stage of performative utterance of language is also the practice studio, so to speak.

: practice as pretending to perform

This is different to accessing musical technique in interpretive performance, because in order to perform something well, and for it to be an accurate, musical and technically correct rendition of the work, the musician goes through a serious amount of repetition, of preparation through private practice as ‘mock performance’. I have never thought of practice being like this before, but it is exactly what it is. Looked at from this angle it seems so absurd. Pretending to perform is what a much of practice is all about. Even some of the smallest parts of instrumental practice; practicing the throat position so as to not pitch something incorrectly, playing the same phrase a thousand different ways to develop a repertoire of possible phrases that might work in different performance contexts. What I have always found difficult with practice is that once the obvious technical issues have been refined, I get bored. I don’t want to practice performance, and I don’t think I ever have. I have always found the notion absurd, and have strayed away from it as much as possible. I think in my mind there is a large separation between acquiring/refining technical skills, and exercising those skills. There needs to be a separation otherwise the line gets blurred, and this is impossible in performance of something that exists already as a fixed musical score is concerned. This blurry line is however where language lives and breathes.

: preparation

Knowing that you have all the necessary skills to accomplish a task, or approach a set of tasks that may adapt and change unexpectedly is the stuff of preparation. Language skills and acquired instrumental technique are the same in this respect, though with some fundamental differences. From the point of view of technique, in interpretive performance the aim is to be in complete control over all of the elements of the performance in advance. This is because the real-time unfolding of the work (in most cases) is predictable. It’s like giving a speech as opposed to having a conversation. You prepare the material you have to talk about, and you get through the material. There’s no need to adapt and change if it is completely scripted, you just need to know that you have the skills there to remember all it is that needs delivering. In contrast, having a conversation in a new language is different because the direction of the unfolding dialogue is unknown in advance. There are common points of reference, cultural signifiers and past history between two or more people, but it is the language skills themselves that form the backbone upon which the conversation can take place. In this real-world context, the language skills and the conversational skills are both exercised and developed in the real-world setting.

So, to get back to how these two things link up to my approach to technology, the fundamental constant here (regardless of the differences in modes of use and reception of skill and technique) is that both language and interpretive performance are skills that are developed and exercised performatively, and in real-time. Practicing small parts of a piece of music, and then only putting these together in the moment of performance still necessitates a conception of skill building that is exercised through time, in real-time. The same applies to spoken language; it is learned, refined and exercised in real-time, however, the crucial point of difference is that these occur in a real-world setting, not in a studio. So, where some music technologies are concerned, I find it quite difficult to reconcile what I have learned about acquiring and exercising technique because there is a fundamental conflation of preparation, skill building and skill exercising. Some of these live in real-time, others that lie outside of real-time, and others that float in and out between these. It’s a question of performative context more than anything. Referring back to the idea of potentiality and of preparedness, it’s then difficult to discern between what is honing and refining and what is exercising a skill set, and how the two can work together towards creation.

This is the end game, because at the end of the day it’s about creating music using technology. I see a disconnection between the variety of skills and techniques acquired through working with technology and how these are then exercised. If interpretive performance, improvisatory performance and communicating in a second language require real-time skill building, in addition to real-time skill exercising, then is it any wonder that I see the end goal of performative utterance to be exercised in real-time, through performative action? This is why all of my hard work on my interactive system _derivations goes into enabling a performance paradigm that relies upon real-time development and use of performative skills.

: programming and performance

This leads me to programming. The act of programming itself, getting lost in developing software, solving problems that have been set for oneself, by oneself. The _derivations project that has dominated the majority of my doctoral program can be described as a semi-autonomous system for musical improvisation. One of the goals of the system are to be able to enable a performer the freedom to interact with digital re-interpretations of their performative actions, without having to control the computer’s contribution. It can be viewed as a performance partner, directly influenced yet indirectly controlled by the contributions of the performer, both in performance and rehearsal.

What is ironic about this situation is that so much performative, action-driven   creative decision-making has gone into the creation of something that stands on its own – uncontrolled by a performer. I would suggest that all of the programming, testing, optimisations, and debugging that goes into the development of such a piece of software is extremely performative in nature in and of itself, in a physical sense. When looked at in this light, the number of hours spent engaging in a performative dialogue with the system through code far outweighs the number of times the system has been engaged with on a musically performative level. What does this say about my performative engagement with the system, and what does this say about the separation between the acquisition and refining of technical skills and their manifestation as a creative artefact?

: recording and creation

To come back to my original example of field recording, I would say that there is an almost performative immediacy to the activity of collecting sounds, the act of recording itself. To me, developing skills and technical knowledge and know-how relates to the real-time act of recording. As I mentioned previously, my initial concerns with field recording – or any recording for that matter – was that the sounds be transformed, mangled and twisted post-recording. The field/foley recording itself becomes the object of a transformative process in the studio, a material with which to work. However, I am increasingly feeling disconnected from certain aspects of this practice, because the detached nature of much computer processing seems to detract from the immediacy of the initial act of recording – unless, of course, these recordings find their way into a performative re-appropriation or re-animation.

I believe that the main crux of my issue is that I don’t feel like there is a strong enough link between the acquisition of skills and knowledge and how this is exercised towards a creative end. It is true that recording itself relies upon creative decision making skills related to equipment choice, microphone placement, gain levels, what to frame, how long to record for, whether or not to activate the environment or to remain a passive observer. However, recording is by its very nature a means to an end. Recording itself is relaxing and can be quite a profound experience, and listening back to recordings made becomes an act of learning and navigating through past experience. For me, however, working out what to do with these recordings leaves a big hole in this link between technique and creation. Personally, I feel a disconnection between the creative agency I embody when recording these sounds, and the possible ways of framing and/or re-contextualising them in the studio after the fact. Unless I plan to somehow prepare materials to be accessed/animated in real-time, I’m not really interested any more. As I’m not planning to go out there to record specific things for a specific context, the way in which the material is used is going to differ depending on what it is. All I know is that I would like to be able to use these recordings in some kind of real-time context. If not, I’d prefer not to touch them at all, and to let them be.

What all of this says to me is that, although I value the act of recording and listening as a creative act, I have a fundamental bias towards activating sounds performatively. This is something that most definitely comes from being a performer is the sensation and a perceived need to activate sounds. Technique that is developed through real-time action (practicing, speaking, improvising, conversing, code) is necessarily accessed in a performative manifestation of some kind. It’s no wonder then that the way in which I approach what remains of the performative experience of recording is to reframe it into further performative realisations.