Lex Moakler, Digital Artist (lexmoakler.com)

Prof. Mark-David Hosale, Course Director

Prof. Ebrahim Ghafar-Zadeh, Course Director

I am proposing to build an art installation that explores how ever-evolving technology mediates our perceptions of ourselves. It will build on prior work in the domain of Artificial Life (A-Life), and will incorporate a real-time simulation of evolutionary processes. The interactive artwork will embody the transforming mirror model of interaction proposed by David Rokeby in that it will confront participants with distorted reflections of their own images, through which a participant can simultaneously perceive themself and engage in a dialogue between their self and the world beyond [Rokeby 1996]. It aims to bring the dynamics of media described by Marshall McLuhan to the forefront of participants’ experiences: media technologies extend the reach and character of our senses, reconfigure the relative degrees to which we use each of our senses, and structure the ways in which we relate to the mediated content [McLuhan 1964]. The proposed project will invite subjective interpretations of the diverse ways in which we humans relate to ourselves, to our media technologies, and to the selves we perceive through these technologies.

The installation will be designed for a gallery context, so that participants will be implicitly encouraged to view, contemplate, and discuss what they see. Six outward-facing monitors will be mounted onto the outside circumference of a circular base, which can be suspended from a ceiling or placed on the floor depending on the constraints of the particular gallery (for the purposes of this text we will assume it is attached to the ceiling using a single post). Webcams will be secured underneath each display, oriented outward to capture the images of nearby participants. Hidden behind the monitors, two computers will be securely attached to the central base. Multi-display adaptors will be used to connect three displays to each computer, and similarly the webcams will be connected to the computers using USB hubs and software such that each camera is connected to the same computer as its corresponding monitor. The power cables for the monitors and computer will spiral up around the central base’s supporting post, and be connected with outlets or power supplies away from the main body of the work. Each display/camera entity will appear both separate from yet attached to the other entities. Theoretically each entity could be truly separate—each with a dedicated computer or processing unit—but the number of computers has been limited to conserve resources and reduce power requirements.

The proposed project will incorporate software designed using Max/MSP/Jitter/Gen to create the transforming mirror effect described above. The workflow is as follows: each entity gathers live video of its surroundings through its webcam, this image is analyzed and transformed in the Max-based software on the corresponding computer, and then displayed on the entity’s monitor. The video analysis will use an optical flow algorithm to track whether there are one or more participants in front of each entity, as well as how much these participants are moving [Yilmaz 2006]. The video transformations make up the A-Life component of the installation: the nature of each transformation will be based on its entity’s genetic sequence, where genes can be interpreted as instructions for distorting the source video feed. For example, some genes may produce a ‘sliding’ effect where the image pixels are slow to change—this will result in the participants’ motions creating trail-like visuals that encourage further interaction. In this way, the project will be an evolutionary simulation based on a small population of six individuals. In an effort to show “truth to process”—showing the aesthetics of the evolutionary process itself—there will be no specific fitness function or teleological expectation for the simulation [Galantar 2010]. Instead the relative amount of participant interaction with each entity, inferred from the amount of motion detected, will be used to determine which transformations are the most successful. As such, the participants’ motions are like ‘food’ for the project’s population, and fitness is indirectly determined by the entities competition for this sustenance. More movement will indicate a participant is engaging with a particular entity, and so over time the installation will tend to generate transformations that encourage participants to remain in front of one entity and interact with it through movement.

The proposed art installation uses A-Life techniques to create multiple transforming mirrors, which will create distorted representations of participants’ images. The software system will interpret the participants’ movements as indication of a transformation’s success, and use the methods of genetic algorithms to breed the best transformations and mutate new versions. Participants will be confronted with not only their own image, but also the contribution of the evolutionary system. This discrepancy provides an opportunity for participants to re-interpret their own image in the context of media technologies. Participants that experience the installation in multiple instances over time may see different transformations at each subsequent visit, which reveals how the inner workings are inspired by the biological processes of evolution. Even with one visit, participants are invited to contemplate the ‘how’ and ‘why’ of their reflections’ distortions, encouraging dialogue on the relationships between humans, media, and humans-perceived-through-media. The ways in which we relate to each other and the constructed parts of our environment are as varied and idiosyncratic as ourselves, and the proposed project aims to reflect these relationships back out to the participants with the hope that this can facilitate interdisciplinary discourse on these topics.

1. David Rokeby. 1996. Transforming Mirrors: Subjectivity and Control in Interactive Media. In Critical Issues in Interactive Media, Simon Penny (Ed.) SUNY Press.
2. Marshall McLuhan. 1964. Understanding Media: The Extensions of Man. McGraw-Hill Book Company, New York.
3. Alper Yilmaz, Omar Javed, and Mubarak Shah. 2006. Object Tracking: A Survey. ACM Comput. Surv. 38, 4, Article 13 (Dec. 2006), 45 pages.
4. Philip Galantar. 2010. The Problem with Evolutionary Art Is … In Application of Nature-inspired Techniques for Communication Networks (EvoCOMNET). Springer Berlin Heidelberg. 321-330.