portfolio >chaosflöte >AIYA


machine improvisation


AIYA is an algorithmic and AI-driven improvisation machine, originally designed for the Chaosflöte and MIDI instruments, and recently adapted for installation settings.

Using MaxMSP as a starting point for the logic operations of the machine, AIYA also connects to any other program that is capable of receiving OSC messages, for the intention of creating flexible performance setups (e.g. with Touchdesigner, Ableton, Supercollider, etc.). It spins textures and melodies that are comprised of both purely synthesized sounds as well as from live-recordings of the sound and motion behavior of the human performer during the performance. Through an analysis of performance attributes such as frequency, amplitude, density, spectral centroid, spectral flatness, body position, and join rotation, interaction and behavioral rules for the machine are derived. These explicit rules are combined with "black box" AI processes that generate behaviors that may be less predictable, but still intuitive. Important in AIYA's design is to have a balance between familiarity vs. surprise, experimentation, and the expanding of the improvisational pallete.

AIYA is a fundamentally flexible system. Many different adaptations have been made, including versions for MIDI keyboards, PoseNet performer, sensor-augmented flute, human-machine quartet (two humans, two improvisation machines), vocalist, and percussionist. The spirit of the project is such that each different version of AIYA constitutes a completely new work, where the design of the non-linear performance system is considered itself a composition.

AIYA is a central component of my doctoral research project at the Künstuniversität Graz and Zürich University of the Arts.

On this page are projects and samples featuring AIYA.

doctoral research

Research Title:
Developing a Visual Framework for Musical Improvisation Machines: Investigating Impacts on Human-Machine Interaction, Posthuman Identity, and Performance Narrative

The practice of creating and performing with improvisation machines spans several decades, most notably seen through the works of George Lewis, Benjamin Carey, Guy Hoffman, Gil Weinberg, and IRCAM, among others. Arguably, the aforementioned developments have focused largely on the sonic sophistication of the improvisation machine and presents a research gap in the development of more sophisticated visual representations of the machine. My dissertation research aims to address this gap by developing a visual framework for improvisation machines that can be used as a defined, yet modular starting point for other creators of improvisation machines to use for their own visual representations.

The development of such a framework instigates reflections on the visual representations’ effects on the performance setting, specifically addressing the following questions:

- How might a visual representation for a musical improvisation machine be developed that aids in facilitating bidirectional human-machine communication and the blurring of perceived borders between human and machine?
- How does the visual representation contribute to the shifts in perception between the two identities of the improvisation machine, when it appears at times as an extension of the mindbody and at other times a distinct improvisation agent?
- What effects does this have on the perceived narrative of the improvisation?

The research will culminate into three distinct components: a suite of performances with the developed visual framework applied to the improvisation machine, a written dissertation reflecting upon the aforementioned issues raised by the research topic, and the software of the improvisation machine with the applied visual framework available as an open-source project for other creators to use and modify.

other notes:
 AIYA is derived from the Chinese expression "aiya..!" which expresses a mixture of frustration and surprise. Such is my experience with creating this machine, where unexpected behaviors of the machine continually challenge - and therefore develop - my improvisation practice.

LiberAiya | March-May 2024

A collaboration between the German-Australian pianist and composer Max Petersen’s LiberA ensemble and the American transdisciplinary artist Melody Chua’s AIYA improvisation machine, LiberAiya takes the form of an interactive installation and a set of performances.

>>The work considers playing with the improvisation machine as a way of engaging with the unfamiliar and the “othered.” What do we do if AIYA confronts us with the uncomfortable and oppositional? Instead of shutting down, do we have the capacity to continue the dialogue? How can we adapt to new ways of expression, where one is not suppressed by the other?

By interacting with AIYA, indeterminate narratives are told through video footage of protests from around the world and from different political positions, recognizing that holding space for opposing views of social change is vital for a pluralistic society. The footage, paired with texts selected and composed by the artists, offer varied glimpses into the possibilities of peaceful protest, tolerance, and expression beyond reactionism. Not only raising fists, but also holding hands.

Interactive Installation:
March 8th 12:00-18:00 @Les Complices*
March 9th 10:30-14:30 @Les Complices*

March 8th 18:30 Song Yi Jeon (vocals) + AIYA @Les Complices*
March 8th 20:00 Fabian Arends (percussion) + AIYA @Les Complices*
March 9th 20:00 LiberA Ensemble ft. Melody Chua + AIYA @Notenpunkt

LIBERAIYA ::KONSTANZ EDITION brings the LiberAiya installation inside of a 5m-wide clear dome situated by the Lake of Konstanz.

Interactive Installation:
Outside of the Konzilgebäude Konstanz

April 13th all day/night
April 14th morning-16:00
April 15th morning-12:00

Outside of the Konzilgebäude Konstanz

April 13th 18:00 Song Yi Jeon (vocals) + AIYA
April 13th 19:00 Melody Chua (chaosflöte) + AIYA
April 13th 20:00 Max Petersen (keyboard) + AIYA
@Wolkensteinsaal, Konstanz
April 14th 20:00 LiberA Ensemble ft. Melody Chua + AIYA

Interactive Installation:
Dialogplaz, Winterthur

May 2nd 15:00-18:00

@Dialogplatz, Winterthur

May 2nd 18:00 Fabian Arends (percussion) + AIYA 
May 2nd 19:00 Song Yi Jeon (vocals) + AIYA
May 2nd 20:00 Melody Chua (chaosflöte) + AIYA
@Esse Music Bar, Winterthur
May 3rd 20:00 LiberA Ensemble ft. Melody Chua + AIYA

left: vocalist Song Yi Jeon performing with AIYA
right, top: interactive insallation at Les Complices*, Zürich
right, bottom: percussionist
Fabian Arends performing with AIYA

more portfolio examples