Visual communication is a crucial component of musical performance, which, in performances involving only human agents, is something that is trained through years of music education.
The visual medium can aid in developing perceived agency in the machine, namely its “social ability” (Woolridge and Jennings, 1995), which in the improvisation context is its ability to communicate material that contributes to the performance’s aesthetics, to communicate its analysis of the performance scenario, and to communicate anticipations of future decisions with other agents (e.g., humans).
Yet, in the current development of improvisation machines, this aspect is severely under-researched or at least under-documented in the current literature (exceptions including the work of Hoffman and Weinberg and Francois et al).
Current visual “representations” of improvisation machines:
i. Technical equipment: speakers, microphones, laptop, sensors
ii. Scrolling “piano roll” interface (Mimi)
iii. A moving instrument without the physical body of the player (Voyager) (a piano is being played, but one
does not see the player—only the moving keys)
iv. Humanoid moving machine (Shimon)
These examples illustrate the current development of visual representations in improvisation machines, but there is great potential to explore this further.
Disclaimer:
1. I do not try to assert the dominance of the visual medium over the sonic medium, nor do I believe that the sonic medium depends on the visual medium to function. I only see the visual medium as an opportunity that is very much worth investigating when it comes to representing and communicating with improvisation machines.
2. I do not try to imply the eventual replacement of human improvisers with machine improvisers either, but rather see the creation of improvisation machines as a method in which to reflect critically on our own artistic identities and a way to be challenged to find novel ways of interacting and creating sound in a performance setting.