Open for interpretation | François Roche’s architectural self-organisation got my brain going

I'vehe1

“I’ve heard about something that builds up only through multiple, heterogeneous and contradictory scenarios, something that rejects even the idea of a possible prediction about its form of growth or future typology. Something shapeless grafted onto existing tissue, something that needs no vanishing point to justify itself but instead welcomes a quivering existence immersed in a real-time vibratory state, here and now.” – François Roche – “I’ve heard about”

In 2012, I attended a lecture at UTS by French architect François Roche. It was fascinating. Here was architect tackling themes of emergence, self-generation, growth and decay both conceptually and practically. His philosophy of research as speculation was also fascinating, and something I’m only now fully coming to grips with. I took some notes at the time, and have only recently returned to them given my thoughts about symbiosis in performance. I was aided in making these connections by a brilliant app called Mohiomap, which allowed me to map hundreds of notes from my Evernote library into organic, typological graphs – showing connections in my writing I wasn’t aware of.

Roche’s talk focused upon his conception and use of organic processes in his practice, and how this can be harnessed within architecture. He spoke at length about the way in which tools for architecture need not be specified, and that the way in which we look at the way something is created needs to be taken from another angle. An example of this is rendering a movie of an envisaged biological/cyborg process creating a structure. The way in which the structure is created in the first place comes out of the way the process is defined. It is only after this that the means by which the thing can be constructed can be envisaged.

This is fascinating on a number of levels. Firstly the idea that one might ‘outsource’ the specifics of a design to an semi-unpredictable process, an algorithm, a biological model. Of course there are clear links here to evolutionary systems but also politically a rejection of top-down hierarchical structuring of elements. The process itself is defined, and the interacting of the individual elements is the thing that specifies the final design. He spoke of the need for architecture not to be thought of as something fixed, something as a finished and crystalised rendering of a pre-conceived idea, but (if I understand it correctly), as something that can be a particular snapshot of a process not completely known in advance. Taking this idea further are projects that are not snapshots, but constructions in a permanent state of change and completion. Architecture as something living, something definitively un-fixed. Designing environments that are an escape from the real world through metaphorical associations, but also ones that live and breathe with the environment in which they are placed, they become part of the environment itself. Things like this include his structure that grows through electromagnetism, collecting pollutants and allowing this inherent environmental factor to contribute to the overall design of the structure.

This process focuses upon the relationship between a structure that is fabricated in this way and the world in which it lives. However, the way the building/structure is perceived or experienced by the outside world is not something that he spoke much about at all. To me this is especially important given that we generally consider architecture as fixed, as something that structures and defines everyday existence in space. The fact that a structure might be built to be specifically affected by its environment turns this notion on its head. Despite the focus upon evolutionary typographies, and the distancing of control of the architect over the structure itself, there are still aesthetic questions here of course – there would most certainly be a set of desired attributes for such structures. The question then in this type of work is whether or not generally understood aesthetic criteria are irrelevant, or is loss of control and specification of a beautiful ‘process’ more important? I guess what I’m wondering then is where the interface lies between the human relationship with the environment. There are low-level models of biological activity that are creating these structures, these works of art in livable form, however the interface with human inhabitants/audience members/receivers, is this relevant?

This is where I’m beginning to make the link with musical generation of this kind, specifically in the context of musical performance and improvisation. Self-organisation in such processes are appealing in these contexts because the enable a different conception of what it means to present structures of many kinds – architectural, artistic, musical etc. In addition, the fact that such processes stem from our understanding of living systems provides a concrete link to those observing/interacting with the result of such processes. We recognise our collective selves, our social and biological nature reflected in the design and implementation of these cyborg constructions. My questions in the context of all of this work stem from higher up however. Whist appreciating the autonomous nature of such process, implicitly understood to be derived from the very same organisation as living systems such as ourselves, can we recognise us? Can I recognise me? In interfacing with a technological construction (art, architecture, performance etc) should be able to recognise me amongst the design? Is there a political statement being made in such types of work about the separation between collectivism and individualism?

If I zoom out a little I find that its all relative to context, and also to different levels of agency, interaction and causality. Take for example a building. The design of the skeletal structures can be informed by longer form collective interactions, things that are perceived on a low level – biological interaction between components, networks and the like. These things are models of evolutionary systems, or at least inspired by them. I understand them to represent processes that make up me, only through abstraction and scientific explanation. However, they cannot represent me as I perceive and know me, for practical and social reasons. A construction in space of the scale of a room or a building is a social space, to be perceived and experienced socially. It therefore makes sense that for a structure such as this represent or be inspired by ‘me’ as related to the world, as part of a collective universal me. Breaking me down into components that are universal requires a very low-level approach, a consideration of where the fundamental rules and logic of biology and social organisation comes from. Building a structure such as this reflects then – in the design process and perhaps also in the continual fabrication process (those structures that continue to mutate and change) – the interactions between low level components common to humanity, but also to life in general – to darwinian evolution.

I have lots of questions. My thought here then is that the concept of agency in both the design, generation and appreciation of such work is somewhat of a contradiction. Agency and autonomy as we conceive of them as intelligent beings are much richer than these low level, long-form associations between individual components. Autonomous and self-organising structures are of course fascinating and rich, over many iterations and longer forms, and these things are perfect for exploration and inclusion in social and environmental spaces. Does this mean however that in turning to such ways of generating and presenting work that there is also some kind of mistrust of individual will and intentionality? Or are we simply not willing (pun unintended) to engage with the generative genesis of such fundamental concepts that make us human – our notion of self identification? How can we as designers, artists, musicians and creatives reconcile the inevitable abstraction from biological models (those that exert real autonomy and self-organisation on a low-level), with our inherent understandings of human will and intentionality? Is it possible to acknowledge creative intentionality and at the same time distance ourselves from it? What does this mean for authorship and those who are perceiving the result of such processes? Is the conceptual space in which such approaches present themselves bound to be perennially open for interpretation?

Advertisements

Revisiting Symbiosis: a tentative manifesto?

parasitic_relationships_no.2_fungus_180dpi_2

Parasitic Relationships No.2 (Symbiosis Sate), 2013: Amber Stucke

A little while back I sketched out some rough thoughts about the metaphor of symbiosis in art and performance. These were really just some nascent musings thrown out there on a whim, coupled with the curious discovery of Leonel Moura’s Symbiotic Art Manifesto. I’ve read back on this post with curiosity over the past couple of months, and recently I’ve revisited some of these concepts to try and tease out what I was trying to say in a little more depth.

Moura and performativity:

In the previous post I mentioned Leonel Moura’s symbiotic art manifesto. What I questioned was how his ideas might be able to intersect with real-time performance, and with interactivity. The word ‘symbiotic’ in Moura’s lexicon describes the relationship between the artist and the machine/robot the artist has made to create artworks. From his perspective, the symbiotic relationship that is so vital for this new type of art exists between the artist as designer/creator and the machine/robot developing the artwork in real-time. In this context, the human needs the machine’s agency in order to render artworks, and the machine needs the developmental guidance of the creator (the structure) with which it is to then make artworks. This metaphor of symbiosis therefore relies upon mutual dependencies that exist between the designer/artist and the automaton/artist, the former as the ‘preconceiver’ of a possibility space for machine performativity, and the latter being the machine that navigates this space autonomously. This is a high level view of dependency in human-machine art making, but as I mentioned previously, it lacks a concern for an intersection with human agency during the performative act itself. What is missing therefore is a consideration for the real-time performance aspect of the creation of the artwork – the rendering of the artwork as a performance.

Looking at human-machine symbiosis during the performative act, the real-time act of decision-making by the machine (both with respect to its internal state and influence from the environment) can be viewed as a symbiont connected to its environment. In Moura’s work, as in the work of many procedural and generative artists, the machine’s automation could be said to act as the artist in proxy, with the results of this autonomous procedure being considered the final artwork. For those generative and/or metacreative works that exist in time (music, animation etc), it is the process of negotiation and generation – the performative act – that is presented as an artwork. For those works in which the environment involves the input of a human interlocutor, this performative act becomes more complex, and the notion of a symbiotic relationship extends into the act of performance itself. As such, mutual dependence between the automaton/machine, the human performer and the possibility space proposed by the designer all factor into our understanding of symbiosis in art. Because of the introduction of the performer to the interdependent relationship between artist and machine, the designer’s relationship to the situation is in some ways diminished, and in others it is strengthened by the real-time encounter.

What I am trying to understand is the entangled relationship between human and machine performances, both outside of and influenced by our focus upon making machines that participate in musical performance autonomous in some shape or form. Where symbiosis comes in is through an understanding of the bi-directional nature of adaptation and learning in the performance/development of such technological systems. For me, what is most important is finding ways of conceptualising interactivity in this performance practice that directly acknowledge the potential for mutually adaptive relationships to develop between humans and machines. To date, I feel that many approaches in this area have downplayed the need to understand the system’s affect on the human, and to focus upon developing systems that display autonomous behaviour for musicians to engage with as if the machine was another human player. This is why Moura’s ideas have struck a chord with me. Although in his manifesto he focuses on the machine’s ability to make art autonomously, he is also concerned with the new relationship that exists between these technologies and human creators. When considering this concept of developing artists rather than art, he asks the question: “what do we become ourselves?” In doing so Moura acknowledges how this technology puts this mutualistic relationship in sharper focus. As he states, “The life of the artist/machine is interlinked to the life of the artist/human.”

Designing for the (un)known

Whilst artificial intelligence and the study of autonomy is a perfectly viable field of study for musical interaction, it is my view that in seeking ‘natural’ behaviours for such systems we must also acknowledge our personal biases towards certain types of musical interaction. That is, we must acknowledge exactly what we deem to be natural. The irony in many biologically inspired generative contexts is that whilst we may be seeking to develop new modes of musical interactivity, the modes of generativity that are sought are often justified because of their ability to act in ways that approximate us as humans. This is indeed a curious paradox. What this kind of developmental trajectory presupposes is that human performance is itself predictable, that the machines we build should be immediately understandable, easy to interact with and adaptable to our needs. What I find fascinating however, having developed and performed extensively with such a system myself, is that the musician who begins performing with these systems is not the same after each successive interaction. Musicians adapt to different performance situations, they learn to embrace new performance scenarios and their conceptions of an ‘ideal’ performance practice expand through interacting with our systems over time. This suggests that if we are to be engaging in the development of new forms of performance practice, we should also consider the way in which our machines shape performers, as much as we train our machines to be just like ourselves.

Also curious in the design of such systems is the focus upon modelling, and or approximating human performance dynamics in the machine performer. Some approaches take as their starting point an established performance practice such as free improvisation, then attempt to understand its inner workings and then use these understandings as a basis for design. As an example, some approaches have emphasised that the practice of collective, non-idiomatic musical improvisation is global in nature, with improvising musicians reacting to the overall sonic environment and not responding directly to the material provided by individual musicians. Implicit in such approaches is an overall concern for designing for musical improvisation as a known and understood musical practice. Because such forms of musical improvisation might be thought of as non-referential, and that the global sound environment plays a larger part than the sum of its parts, consequently any musical system designed to take part in such an encounter should adhere to this understanding of the performance practice.

Here I take a broader view of the role of the design of musical systems. Although I understand human improvisatory practice as being the foundation of any human-machine performance dynamic, I also understand that such a dynamic is in itself fundamentally different precisely because of the presence of the machine collaborator. As such, I believe that any artistic practice that seeks to explore novel interaction paradigms should also be questioning how the role of the machine will ultimately alter understood modes of practice. Although we may be comfortable modelling our machines on types of behaviour closest to our own experience, we must also understand that the very context of human-machine performance practice changes that experience, often even before a live performance has actually begun.

Metaphors for interactivity:

One thing that I have found curious in this area of research and artistic work, as well as in other areas of human-computer interaction, is that the adoption of metaphors often plays a central role to both design and evaluation of our musical systems. Metaphors are often used to describe interactive traits of musical systems and environments – control metaphors related to instrumental performance; biological systems metaphors such as stigmergy that describe/define group dynamics; categorisations of interactivity into differing modes such as conversational/ornamental/instrumental interaction; understanding systems and interfaces as prostheses etc. On the one hand such metaphors serve define desired modes of interaction to aid design, but they are also ways of understanding interactive traits perceived when observing humans interacting with machines. For the former, they provide ways of conceptualising approaches to designing, situating a complex developmental project within a broad interactive/relational framework. They become anchor points with which to ground future design aims. For the latter, they enable a retrospective understanding of the interactive traits exhibited by the interaction domain itself, understanding these situations without imposing any preconceived notions of interactivity on the system.

Whilst some of these metaphors of human-machine interaction have been developed through observation of real-world settings, others remain speculative understandings of the space in which humans and machines interact and are borne from personal artistic experiences and/or aesthetic leanings. As such, what fascinates me is that such speculative understandings say as much about the aesthetic preoccupations of the designer/researcher as they do the phenomenon itself, as I believe they do in Moura’s work. It is my view that the choice to develop systems that can participate with humans in musical performance is in itself a speculative position, and so it is from this vantage point that I situate my ideas of symbiotic human-machine performance practice.

Symbiosis – what is it really?

Here I’d like to outline what I understand about the concept of symbiosis, and how I see it relating to human-machine performance. Symbiosis is a biological process that describes the interaction and often interdependence between organisms in the natural world. There are three main types of symbiosis observable in nature: mutualistic, parasitic and commensalistic. Each one of these types describes the relationship that exists between different organisms, and is related to dependence which is either uni or bi-directional:

  • Mutualistic Symbiosis – describes a situation where two organisms are dependent upon each other, with both benefitting from this interdependence
  • Parasitic Symbiosis – is a one-sided relationship where one organism benefits and the other member is harmed by the interaction
  • Commensalistic Symbiosis – where one organism benefits from the another, yet the other is neither harmed nor helped

Adaptation and dependence:

So, how might/do these ideas play out in human-machine performance? As I mentioned in my previous post, I came to an understanding of symbiosis as a metaphor for human-machine musical interaction due to its focus upon adaptation and dependence, in contrast with a strict consideration of machine autonomy. For me this is a fundamental distinction to make. In my view, the context of human-machine performance practice (including the process of developing musical systems that take part in performance) is something so specific and alien to traditional performance practices that it benefits from being understood, described and designed for in and of its own terms. What appeals to me with the metaphor of symbiosis is that it becomes a way of understanding and conceptualising a type of musical performance that is not possible outside of a human-machine context. Although the continued development of the autonomous capabilities of machines is a natural technical and theoretical trajectory for software development for many fields, I believe what this priority misses out in the context of human-machine performance is an overall way of understanding the mutual adaptivity of humans and machines in these hybrid environments.

For me, symbiosis as an interactive metaphor sits decidedly outside of the purely technological, acknowledging the mutual dependency and mutual influence between player and machine, machine and composer/developer, and researchers and their theories. The metaphor is both inherently interactive, but also has its genesis in the intersection between separate and contrasting organisms. As such, I use it to encompass the space in which human-machine performance takes place, as well as helping to define what approaches to developing technological systems that take part in these performances. However, this shift in focus towards dependence in an interactive relationship should not be misconstrued as a move away from developing the machine’s capacity for displaying agency in an interactive exchange. Instead, the process of symbiosis in performance should be understood as encapsulating the complex interaction between two autonomous entities that are both aware of each other, but also dependent upon each other in performance.

Defining symbiotic musical interaction:

So, taking the above definitions of symbiotic relationships, I would like to sketch out what I think characterises symbiotic interaction in human-machine performance practice, and how I see it reflected in my own work in this area.

For me, symbiotic musical performance means:

  • Mutualism – mutual dependence between human and machine in performance
  • That one’s actions may affect the evolution of the another’s material, though not as a direct reaction to or cause of the other’s responses
  • Machine agency and autonomy are an ingredient of such interactions, though not the only ingredient

The development of interactive systems, whilst often focused upon maximising the autonomous decision-making capacity of the machine, should be thought of primarily as engaging human musicians in a mutually influential performance context with a machine. Human engagement with such systems fundamentally differs from human-human interactive contexts. In human-machine improvised performance, in my view performers are in some way dependent upon the interactive capabilities of the musical system. Musical systems are developed with specific stylistic constraints in mind, and performers interacting with such systems learn to interpret the nuances of these systems through repeated exposure to them in rehearsal and performance. They are dependent on the system in a broad conceptual sense, as it is often the case that the musical system’s interactive vocabulary (or indeed, sonic vocabulary) defines the range of possible musical territory to be covered in an interaction. Although a musician is clearly an autonomous entity, their musical input to the performance is shaped and altered by the context provided by the intersection with the musical system. This may in fact be thought of as a form of interpretative performance, not unlike traditionally understood notions of musical interpretation. This last point opens up a range of issues related to ownership and authorship of which I plan to write about in a future post, so I won’t go into these ideas further here.

From a system design perspective, it is possible to conceive of such systems as enabling either a mutualistic or commensalistic symbiotic relationship to occur in performance, as outlined above. The development of such a relationship need not seek to maximise the qualities of musical autonomy, however musical autonomy may emerge through cumulative interaction with the system over a period of time. The ability for systems of this type to exert agency in performance whilst still depending upon the sonic input from a musician provides a challenge for the system designer. In the context of human-computer improvisation systems, one thing that enables a musical relationship to develop between software and a musician’s performance is how a piece of software seeks to deal with information it receives in the present, as well as how it makes sense of the past. In human-machine performance practice, an interactive environment fundamentally shapes the performance of a human musician over time throughout an improvised session, as well as from session to session. Mutual dependence in this instance refers to the change brought about in the human’s behaviour towards the system, and the ability for the system to change along with the performer. One way of conceiving of machine dependence therefore in this context is through the material siphoned from the performer and used to further its own autonomous behaviour. Taking the history of past material as a space of possibilities provides the machine a sense of performative agency that is directly tied to the cumulative history of the human musician.

Symbiosis and _derivations:

My work developing and performing with the _derivations performance system I believe has embodied this conception of symbiotic performance. One of my central concerns for developing musical performance system was that the machine, although not directly controlled by a human performer, should be heavily influenced by a performer’s playing. Although I was interested in developing a system that could contribute to performance un-assisted, this autonomy would need to be emergent and tied to the specific context of the current performance. In much of my work leading up to _derivations I experimented with techniques for capturing both audio and analysis data streams that could then be used for re-generation. Beginning simply by live sampling snippets of my saxophone performance, I soon realised that I was drawn to using recorded data of many kinds as a means to generate complex behaviour using simple underlying rules. The possibility of re-organising, processing and re-purposing sampled data streams (including audio recordings) became a meta-compositional process, whereby my musical decisions were tied to finding ways of coherently generating new material from material siphoned throughout a performance with a human player. Such generative techniques, in comparison to other approaches to developing autonomous behaviour, are of course extremely dependent on outside stimulus. Without having interacted with a human performer, systems of this type have no space of possibilities with which to generate their own material. This is a complex and often contradictory approach for human-machine performance, which from a design point of view provided a very concrete set of constraints with which to work with during development.

As for human dependence on the machine, this is what I am currently in the process of analysing in much of my writing. The development process that led to the creation of _derivations in its current form involved much live performance. My dual role as a performer and developer on the project meant that the line between development and performance was continually blurred. In my development process I would often have a saxophone at hand ready to ‘test’ my latest changes to the system, enabling me to make on the spot decisions about the software as I continued to refine its capabilities. The affect on the system’s overall design is indisputable, and I would attest that the machine is very much ‘composed’ in this respect. However, reflecting on both this testing process as well as the very many performances and full length rehearsal sessions I engaged in with _derivations, what is more subtle is the way in which my performance style has evolved to encompass the capabilities and intricacies of the _derivations environment. The intimate knowledge I have as both developer and performer of the system enables me to shape my performances in a way that is particularly in tune with the system’s character. I cannot say for certain that I would perform the way I do with _derivations with any other human performer. Knowing the system’s ultimate dependence on both my current and past performance input, as well as how it makes connections between these states means that I am able to provoke and predict certain actions, but also to be genuinely surprised by others. My performance has evolved with the design of the system, but at the same time, I am in the moment of performance dependent on the interactive context that the system provides. This provides a set of constraints for improvisation that I find fascinating in performance. A composed space of possibilities that – for want of a better term – enforces a mutually dependent relationship to develop from moment to moment in performance.

Closing thoughts:

So to finish off these thoughts on symbiosis, below I outline what I see as some of my fundamental approaches to creating mutual dependence in human-machine performance. Although I call this a template, I must stress that these were not guiding principles outlined before developing a performance system, but rather tendencies I have observed through retrospective consideration of my own work that point towards the concepts I’ve outlined in this post. Considering mutual dependence as a design principle is sufficiently broad in many respects, but also very constraining in others. The methods one might use to generate relationships between captured and analysed performance material are many and varied. However, what I have found is that focusing attention on capture and storage, navigation and re-generation has provided an intriguing balance between transparency and opacity of a machine’s contribution to a human-machine improvised performance.

Template for the design of a musical symbiont:

  • The machine is an empty vessel – it cannot act without having interacted with the human
  • The machine and the human form a symbiotic system whose musical results are an emergent result of their interactive history
  • The machine’s behaviour and/or content is in some way dependent upon the human’s current and past actions
  • The autonomy of the machine’s actions is in some way derived from the interactive history with the human performer
  • The machine displays agency by making connections between materials siphoned from human interlocutors
  • The machine feeds off the performer, using captured memories of the encounter as fresh material for its contribution
  • The human’s past becomes a space from which the machine draws upon to influence the future musical states
  • The machine also listens to itself, depending equally on its own past as an indicator of its future state as it does outside stimulus (autonomy)
  • The impetus for the machine contribution is tied to some present understanding of the human’s actions

bc.multi-sampler – a new Max4Live sampler instrument

bc.multi-sampler is a quad buffer polyphonic sampler instrument. It’s been a large part of my own electro-acoustic arsenal in MaxMSP for a few years now, and I’ve finally gotten around to porting it to Max4Live for others to make use of. Very keen to see what others will make of it, and hopefully use it for!

You can download the device over at my site. Here are some of the features of the device:

– enables the simultaneous playback of four sample buffers of arbitrary length, with up to 16 voice polyphony
– drag and drop samples from the live browser
– an X/Y pad provides control over the overall mix between the four loaded samples
– automated sample start position provides a level of unpredictability and variety to sample playback
– independent loop and pitch bend for each sample
– independent filtered delay lines for each sample
– option to normalise files on import

short sampler sequence – explanation coming soon :)

Hi – recorded this short sequence today using a few of the tools I’ve been building in Max. You’ll find a short description below – I’ll explain a little bit more about each of the components seen (or not seen) in the video and how they work a little later – in the meantime check out the video below

Some info about the video:
GPS data controlling the mix between stereo outputs of a 4 buffer polyphonic sampler;
Sampled iphone accelerometer data controlling movement through the four sound files;
some other algorithms (offscreen) controlling and triggering other aspects of the sampler and effects processing…