 |
 |
 |
|
|
LATela research lab design and performance
Conceptual Apparatus 2005 readings on transdimensional multimedia and computational environments go back to readings on video and space go forward to conceptual essay on "The Telematic Dress"
essay 1
Sha Xin Wei / Satinder Gill Gesture and Response in Field-Based Performance
We achieve this by considering how gesture spans multiple bodies and how aesthetic design works with this and facilitates it. The paper is in two parts, the first part covers movement studies, focusing on gesture and body movement, drawing on the acting and pragmatics, and the second part develops this with an example of TGarden, a responsive play space for experimental performance augmented by gesturally nuanced computational media.
Body Moves is an analytical method by the second author (4) that deals with the pragmatics of meaning where salient body rhythms span more than one body, and are in relation to each other. In extending this work to the responsive media environment of TGarden, the relation is not limited to the rhythm of one body with another, but of one body with the salient responsive elements in the environment. Learning to master this responsive space is to be skilled in extending one's own body field. As people are creating the TGarden, they are very receptive to fields and conscious of their gestures emerging out of the field through rhythm, with precision, and reflexivity.
BODY MOVES-GESTURE (GILL) Two kinds of Body Moves, having sequential and parallel structures, have been identified and analysed within the engagement space: sequential Body Moves have the structure of action-reaction motion, whilst Parallel Coordinated moves have the structure of parallel motion (1,3, 5). They have different priorities in their functionality. Sequential moves serve to maintain the steady-state communication, whilst parallel moves serve to transform the communication. There is a pulsation in the movement from the sequential to the parallel action that facilitates the process of the building of a common ground or sense-making in the interaction environment. Each person has a body field of engagement, and together, the aggregate of their fields, forms the engagement space. This space is therefore also called, the Body Field of Engagement. It is a variable field and alters with the degrees of comfort and discomfort, expressed in our work as 'contact.' Within the engagement space, persons cooperate to sustain the space that enables them to remain committed to be together. It necessitates that the membranes of the person's body fields are in contact, the degree of which alters with levels of commitment and nature of attitude. Overlap or mergence of the fields occurs when bodies move in parallel coordinated action, where the overlap is complete for the period of that action. However, this overlap is only meta-communicatively shared, and does not denote a common focus of attention. In fact in parallel co-ordinated action, persons are acting autonomously but simultaneously in rhythmic pulsation on different foci of attention, and in doing so they are aware and attending to each other at the same time (5, 3, 4). Space is considered as a resonating space.
TGARDEN When you walk into a TGarden, you choose from a set of sumptuous garments, each with a different unfamiliarity. Some billow around you in clouds of fabric so that you grow three times larger but no heavier. Some add an odd elasticity to your body so you tend to flop as you walk. Some may rip as you walk, or glue to each other or the walls so you must tear yourself free as you disambiguate your body from the ambient matter. You notice that there are no well defined objects in the room, but as you play in it over time (minutes or days of repeated visits) you learn certain ways of playing that characteristically elicit more or less well defined entities, whether they are acoustic or visual, or perhaps socio-psychological objects. In time you discover people who have invented virtuosic ways of playing and engaging this responsive space, and without a word you are able to learn from their deft action or inaction. As you walk past another body, you leave behind material traces of yourself: shadow, hair echoes, and air currents. Even if you do not explicitly and actively acknowledge the passer-by, your residues intertwine with the others and conduct material conversations in your wake. The dynamic physical and symbolic matter of these residues and traces constitute a continuous substrate or field of activity. A particular gesture does not always elicit exactly the same sound; it seems as if you are dragging your fingers and limbs across materials like wool or metal sheet or rubber. As your movements couple to the responsive dynamics of the dragged sounds or visual textures, you learn to intentionally 'bow' or brush calligraphically through the medium. In TGarden, salient rhythms occur within the substrate of the combined activity, indicating particular resonances as body fields move in response to each other. We will illustrate how the TGarden creative space works through some examples of activity that span how an individual and a group are coupled with the environment and with objects in the environment.
ILLUSTRATIVE ANALYSIS OF TGARDEN (SHA AND GILL) Hop-Skip Example This is the salient response change in the environment that cues in turn the response of the person. Why does the flash re-emerge? The 3D graphics is filled in with a 'texture map' and this texture of pale light colour is filled by using 2 rules: a) it is triggered by the person's hop, and b) it is interpolated such that its echo, the echo of the person's hop, goes on. In other words, the texture map is continuous function of both the internal clock of the machine, as well as the rich real-time data from the human body's ongoing physical movement. We look more carefully at what is happening with the dynamic response. There are two kinds of responses in the TGarden environment to a person's movement. The first is the response of the real-time video synthesis software to the person's physical movement, as measured by accelerometer on his chest. In the video, for example, a bright texture map fills in as an immediate response to the person's hop. The other is the system's programmed, retraction of the texture map back to a plain, open mesh, interpolated over a fixed time interval. However, there is a recoil, an echo of the accelerometer on the chest of the human body. The filling in of the mesh by texture comes from prior logic, but the echo of the flash - that recoil - comes from the recoil of the physical body. Hence the responsivity in the TGarden arises from both software dynamics and body dynamics, the intertwining between simulated physics or body physics. Another critical aspect of this responsive system is how it deals with the characteristic time of decay or response. If the characteristic time is too long, the environment begins to feel decoupled from the person and if it is too short the environment responds as a simple discrete series of stimulus-response events. With just the right characteristic time of response, the player imputes a strong sense of elasticity to projected, structured light shining on a hard floor. Since the TGarden is engineered with such low latencies as to produce computed media that the human perceives as concurrent with his or her activity, the human interprets the computed response not as a macroscopic interactive reply but as a tangible quality. In the hop-skip example, this tangible quality derives from the micro-physics of the body intertwined with the synthesized dynamics of the visual texture and the rhythmic sound.
V2 Professional Dancers Example Once positioned the dancers begin to warm up in an improvised rehearsal, sensuously moving with sounds and colours. The shifting shapes on the floor occasionally and very momentarily detach from a dancer who then reaches out and regains contact. As they move in the space the dancers quickly (within 51 seconds) find resonant connections with each other as an engagement field. By the end of their performance, they are fully rhythmically coordinated. This enabled through grounding their coordinations with the environment and each other during discovery and improvisation. In analyzing how they coordinate or attending to each other, it is not essential to know their gaze activity, which is subsumed as background knowledge to their body movement. In the TGarden, gaze is not a core part of 'attention'. In much cognitive work, visual attention or gaze is often used as an explicit indicator of attention. However, to correlate gaze with cognition and attention is to reduce the connectivity in the field space of the TGarden. Our analysis focuses body movements as fields, i.e. of the whole body and thereby avoids reduction to solely geometric data and visual perception.
Slo-mo Example As they do so, there is a change in their rhythmic coordination. The dynamics and tempo of their field space shift from a smoothly coordinated rhythm to a seemingly staccato random tempo, affected by their individual movement with the balls and the physical contact between bodies that comes with rolling the balls to each other. The rhythm alters again as they disperse and their body fields engage in smoother coordinated autonomous choreographies. In analyzing this, one could produce an agent-based description of a sequence of actions, for example, a) the person with the tiger striped costume initiated the movement of the balls in the centre of the play space, and this entrained the others to do likewise. Alternatively, and as we do, one could produce a field-based analysis where the overall movement in the space and the relations of engagement undergoes change. In this example, the 'tiger stripes person…' description is an ego-based and object orientated account, whilst the latter description of rhythm and tempo is an account of the entire scene in terms of rhythmic fields.
TGARDEN: SUBSTRATE AND CONCURRENCY (SHA)
Concurrency
The TGarden as constructed does not interpret movement presuming intentionality or a model of the ego subject. The salient rhythms are essentially resonances of spontaneous actions and non-symbolic, providing an example of an 'a-linguistic semiology of human performance' (10). We extend the concept of the Body Move based on its essential fields of resonant performance to movement based on fields instead of particular human bodies. We consider how the players in a TGarden form tacit awareness in overlapping and autonomous space and gauge elements and patterns of connectivity, and through this tacit learning, shape the media space and are concurrently shapes by it.
References:
2. Gill, S.P., Kawamori, M., Katagiri, Y., Shimojima, A. : The Role of Body Moves in Dialoguie. Interantional Journal for Language and Communication (RASK), 12, (2000), 89-114. 3. Gill, S.P. : The Engagement Space and Parallel Coordinated Movement. Case of a Conceptual Drawing Task. CKIR (Centre for Knowledge and Innovation Research) Working Paper series, CKIR-1. 2002. 4.Gill, S.P.: Body Moves and Tacit Knowing. In: Gorayska, B. and Mey, J., eds., Cognition and Technology. Amsterdam: John Benjamin, 2004, pp. 241-65. Gill. S.P. and Borchers: Knowledge in Co-Action: Social Intelligence in Collaborative Design Activity. AI & Society, 17:3 (2003), 322-39. Harris, R.: Signs of Wriring, London: Routledge, 1997. Salter, C. and Sha, X.W.: "Sponge: A case Study in Practice-based Collaborative Art Research," Creativity and Cognition2005. Scheflen, A.B.: How Behavior Means; Exploring the Contexts of Speech and Meaning, Kinesis Posture, Interaction, Setting and Culture. New York: Anchor Press/Doubleday, 1974. Sponge. Tgarden Project TG2001. http://sponge.org/projectsm3_tg/intro.html Sha, X.W. : "Resistance is Fertile: Gesture and Agency in the Field of Responsive Media," in "Makeover: Writing the Body into the Posthuman Technospace, Configurations, 10:3 (2002), 439-472. Sha, X.W.: "the Tgarden As a Phenomoneological Experiment," (in preparation)
Permisison to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made to or distributed for profot or commercial advantage and that copies bear this notice and the full citation on the first page. Sha Xin Wei / Satinder Gill: "Gesture and Response in Field-Based Performance." Creativity & Cognition2005, London 2005, pp. 102-08. Copyright 2005 ACM 1-59593-025-6/05/0004.
essay 2
Christopher L. Salter/Sha Xin Wei
Introduction: Sponge
Sponge further described its work succinctly in a 2002 interview for the French publication £EcArts: "…thus we are interested in setting up compelling conditions which enable people to make their own meanings out of built spaces and environments (spaces being architectural, symbolic and media)" (14). One review aptly described the group as a "decentralized association of ideas and tactics with countless points of entry and use" while other critics and participants in its work exhibited in more than seven countries and in international media venues such as Ars Electronica, V2 Rotterdam, and SIGGGRAPH, among others, have described it as "hermetic," "groundbreaking," "utopian", "highly cerebral and potentially, wholly practical." (6). Community of practice: What makes Sponge unique enough to warrant these pages goes beyond the group as a fruitful case study of collaborative, practice-based research in creative art and design utilizing media and computational technologies. Rather, we wish to examine the work of such groups (in laboratories, workshops, projects, sci-art collaborations) as a salient example of a specific kind of community of practice (cf. Jean Lave/Etienne Wenger: social groupings involving the sustained pursuit of a common enterprise and a shared repertoire of communal resources developed by its members over time).(4) The practice of the group, its manner of doing and approaching things shared by the members over several years, has been an effort to expand as well as question the boundaries of artistic production with technology. This is done by exploring dissemination avenues normally not associated with artistic contexts such as peer reviewed publication and academic conference settings in areas ranging from computer science, computer music, gaming, and social studies of science and technology, but also by using artifacts associated with the sphere of cultural production (public exhibition and performance) as study objects to rigorously examine the permeable boundaries between aesthetic practice, techno-scientific research and philosophical inquiry. What is also interesting to examine is how such groups (e.g. Sponge), as both a collective entity under a single signature as well as through the personalities of its individual members, has utilized the space of artistic expression as a vehicle for the production and transfer of knowledge from its internal community of practice into the larger public realm. The other aspect of Sponge's work (and the collaborative mechanisms deployed by the group) which is interesting for our purposes here are the theoretical and practical issues the group has explored and encountered in its attempt at maintaining a collaborative, transdisciplinary practice. Here, we use transdisciplinary in Michael Gibbons' sense as "research involving a stronger 'interpenetration of disciplinary epistemologies.'" Effectively, this means new fused horizons become possible, beyond or transcending paradigms existing within single disciplines.(2)n Due in part to the multi-disciplinary backgrounds (mathematics, computer science, theatre, computer music and philosophy), of its core founder members, Sponge from its start has recognized the difficulties inherent in such collective enterprises that initiate play and blurring between different 'epistemic cultures"(3) while simultaneously making such an enterprise an integral part of its practice. It is in this respect that the study of Sponge bears import for the domain of practice based research and continued work in areas such as the social study of science and technology, knowledge formation and production and the burgeoning field of art research: the creation of knowledge from different fields that results in the production of artistic artifacts, events, and practices.
The group: Sponge originated out of a Stanford University Humanities Center co-urricular seminar entitled "Interactive Media: Theories and Technologies of Representation" (code name :IMG: Interaction and Media Group). A course first co-organized by Sha Xin Wei (mathematician and human-computer systems architect). The aim of the seminar was to seek ways of articulating, conceptualizing and working with digital media and more generally, computer-mediated interaction. IMG's strategy was to examine interaction and media paradigms from a number of complementary and contrasting disciplinary perspectives across the humanities, arts, and sciences, and through this, expose the participants to a multiplicity of languages, techniques and approaches. What seemed interested was the way certain fields yield unexpected and fruitful clues for practical development in technology. For example, theatre may provide models for user-interface design, topology and geometry for media structures and urban architecture for 'cyberspace' design.
Shared versus common languages The group sought ways to articulate and conceptualize working with digital media and computer-mediated interaction. More specifically, IMG was engaged in a study of issues related to interactive media, hoping to find a way toward a constructive theory of how people compose and inhabit interactive media A second, and more ambitious goal, lay in the attempt to bring individuals from divergent disciplines together in the hopes of articulating not just a specialist language for media and interaction but a shared, publicly developed one. The aim of creating such a shared language contrasts markedly with the notion of a common language: the presumption that individuals from radically different epistemic cultures can eventually agree on the same connotations, contexts, and meaning of words. While the goal of a shared language was not to be underestimated, the ambition to create a space of discourse based on shared concepts and constructs rather than a collage or dictionary of mutually-alien expertises that had little common epistemological ground (theatre and mathematics, computer science and literature, organizational behavior and art practice) lay at the core of the mode of operation.
Organisational behavior in art practice (The Coal Mine)
Materialized conversation: How do collectives emerge? Sponge emerged as a collective out of both the general IMG discussions as well as a spin off dialogue initiated by Sha, Laura Farabough and Chris Salter, three of IMG's core members. If IMG's stated interest was not only a theoretical exploration of the interplay between interaction and media but also how to compose and inhabit such media, why not then utilize techniques from artistic practice to bring the conversation to a broader public by way of real time experiments? What bears mentioning here is that Sponge derived from the need to create a site of continued discourse for a group of individuals who felt they were pressing up against the epistemological limits of their own disciplines (performance, mathematics, computer science, music) and were searching for new techniques and knowledge from other forms of practice. "Sponge is interested in setting up compelling conditions which enable people to make their own meanings out of built spaces and environments (spaces being architectural, symbolic and media) and for this reason we are looking outside of the domain of art - to fields such as human-computer interaction, ethnography, and information design. These sorts of enterprises use methods very different from the convention of narrative to construct a compelling or meaningful experience."(15) This inadequacy of singular disciplinary approaches to pose broader investigatory questions also marks a second characteristic of the trend towards transdisciplinary research and knowledge production that we spoke of earlier. Of course, artistic production has a firmly established historical tradition of such disciplinary fusions from Brunelleschi to the Oulippo. Yet, what we feel distinguishes Sponge's mode of practice from other 'interdisciplinary' and 'cross-disciplinary' projects is the desire to take the interplay between the epistemic cultures of techno-science and artistic production and to use such cultural production contexts as Gedankenexperimente, a forum for the posing of specific thought-experiments and the production of concepts. What is even more relevant is the role that Sponge serves as a factory creating "boundary objects," Susan Leigh Star and James Griesemer's term for objects that inhabit a space of negotiation and serve to support cooperation between the participants without agreeing on the classification of such objects or their actions.(17) Such boundary objects arise over time from "durable cooperation among communities of practice. They are working arrangements that resolve anomalies of naturalization without imposing a naturalization of categories from one community or from one outside source of standardization."(18) Sponge's boundary objects are actually more boundary events, with definite temporal and social extent, in which people from different communities of discourse and practice, local citizens, media festival goers, school-children, visual artists, performance artists, designers, musicians, philosophers, programmers jointly create and reshape responsive media in a common location using their respective manipulatory techniques.
Micro and Macro Performance Another theme that distinguishes Sponge from many digital media practices concerned with object creation, data representation and virtuality is its emphasis on performance and materiality. This approach is partly rooted in the collective professional and personal histories of the group's members. Yet, it is also based in the desire to utilize performance heuristics and intentions to enlarge the scope of questions normally associated strictly with technology as well as to apply performance experience towards the design of richer human-computer interaction environments. Sponge's use of the word "performance" is interpolated from several different contexts culled from performance theory, architecture, mathematics and philosophy and does not completely subscribe to traditional connotations, either in artistic production or as a term denoting the efficiency and optimization characteristics of technical systems, most often utilized in HCI and human factors contexts. Sponge's notion of performance is first distinguished (in view of their interest in creating sensor driven responsive media environments for the audience-as-player) by micro and macro scales (and a distinction of "gestures"): At the micro level, performance signifies the unintentional or intentional ordinary gesture; drawing, writing, shaping objects, throwing away something, walking.... What is important to note is that such gestures, although enacted either consciously or unconsciously by individuals may not be noticed as being performative - that is, as deliberately enunciating some meaning or action. In fact, such gestures may not be deliberate or signify anything at all at the level of communication. This is the micro level, low-key activity of performance (the making of traces, the making of symbols, the shaping of objects which are temporally-embedded processes... something you can fall into or step out of but usually fall into......). (16) This sort of microperformance is quite different from the normally accepted sense of performance at the macro level: the conscious construction of an event between an "actor" and a spectator. Here one can invoke performance in an accepted theatrical context in order to point out a crucial distinction between micro and macro scales. While the micro scale of gesture may not even be noticed, i.e. may remain under a certain perceptual threshold and thus not depend on a spectator, the macro scale of the performance event assumes a clear cut between looking versus doing or acting versus spectating. The consequences of shifting between (or blending together over time) the micro and macro scale of performance by amplifying or diminishing the thresholds of gestures and actions, however, begins to make the border between performing and observing more permeable. Sponge argues that they are now interested in "dissolving the stage itself." This means saying good bye for the time being to the traditional set relations between performer and viewer. Sponge is not trying to do this in a forced way, as many 1960s theatrical experiments that 'democratized' the stage did by pulling spectators over the proscenium and onto the stage itself. Sponge is after something much subtler, designing situations and events where unpredicted spatial and social conventions emerge out of locally-situated actions."(16)
If performances, then, can be seen as an intertwining of human and material agencies, continually performing and 'dancing' together in a "dialectic of resistance and accommodation," then performance may become a characteristic and quality of numerous material agencies (gestures, bodies, machines, architectures, data). Sponge writes: "Our laboratory, which is partly made of responsive media, is itself not a fixed object - its form is deformed under the action, the impact of these subjects who go through the space. And finally, our subjects may not be subjects at all, human or non human, but rather diffused flows of agencies - fleshly, fabric, computational or media agencies." Performance at this fundamentally other level includes shaping and playing in the material substance and substrate of the world itself and enables performance to diffuse and move among many agencies simultaneously. "The world makes us in one and the same process as we make the world."(9) The account of material agency presented here underscore a strong difference between Sponge's method of approach and many reigning modes of digital art making aesthetics fascinated by the informatic and cybernetic representation of data as knowledge. As Pickering's articulation of human and material agencies constitute an attempt to move away from seeing science as semiotic practice with representations of facts, Sponge's deliberate performance turns aims to shift away from pre-defined, a priori objects of representation that are already given in the world (i.e. schemas, models, etc) outside of experience. Instead, the focus falls on an ongoing process where bodies and subjects emerge through play and performing within the material field of the world itself. It is here where part of Sponge's interest in the potential pliability and responsiveness of computational media technology lies: not in media and data as respresentation but in its material substance. "We are equally fascinated by the agency of the material, the friction of cloth, the decay of data, the elasticity of MIDI-controlled sound, and by the agency of disciplines - grammar, algebra, systems of orthography, legal systems, and so forth. These are all larger than any one of us, and yet they are born out of our own actions."(16) The move to seeing performance as a means of articulating the play and friction among different types of agencies is potentially fruitful for examining the consequences and experiences involved in the interaction of humans and machines. This seems to us particularly useful given the turn in some circles of HCI away from models of representation and towards theories of "situated action" (Suchman), "interactive cognition" (Gedenryd) and "embodied interaction" (Dourish). Ironically, however, much of the influence of performance (macro) that has entered into the HCI dialogue still clings to theories of dramatic representation, through concepts of mimesis, character, identification, and catharsis.(5) We believe that the notions of performance described here may provide more potent ways of thinking of the design and subsequence experience between human and machine systems.
Projects: M1, M2, Tgarden M1 (1997) (1) After an extended period of conversation among its three founders, Sponge's first experiment, which took place in the spring of 1996, was ironically an experiment in investigating performance from the ground up. Beginning a study of gesture, the project sought to research the relationship between intentional and unintentional gesture, and to investigate the potential emergent social patterns that may evolve in a situation fluctuating between scripted and aleatoric events. Questions that provoked the experiment included the following: Where does the threshold lie where an ordinary gesture becomes performative? How are ways we can mark a gesture in such a way that it becomes performative to an 'ideal spectator' that recognizes such a gesture but doesn't call it out so that others will notice it? M1 test experiment (6 performers carrying our a pre-scripted sequences of ordinary gestures and actions for 35 minutes in a crowded public eating area at Stanford university: each of the participants was assigned a series of gestures and actions that were "embedded" into the social climate of the eating area - embedded to such a degree that most, if not all of the gestures would go virtually unnoticed to those inhabiting the site. Everyday gestures such as unwrapping a sandwich, throwing something into a trash container, walking out of a store reading a newspaper, among others, were performed first in a straight sequence and then marked through various techniques to change their "thresholding": repetition, architecture and geometry of bodies, patterns of movement over defined time sequences, and the purposeful ignoring of ambient intentions and environmental feedback. In viewing the videotaped results of the experiment after the fact, one could see a kind of spontaneous choreography of gestures (as seen from without) emerge and spread among the participants over the duration of the thirty-five minutes. What was perhaps more revealing, however, was the fact that such gestures and patterns appeared to be perpetuated and 'picked up" not only from the pre-scripted performers but also from those who were not "officially" part of the scripted performance. The question of whether or not such mirroring or imitation of gestures was deliberately intended by observers in the scene who had "discovered" or recognized the artificial performance in progress, or instead only inferred from Sponge in a post-experiment phase of observation constitutes part of M1's overall inquiry. Gestures and actions in the form of imitation, doubling and further repetitions from "spectators" in the scene were in hindsight perhaps subjected to the question of observer bias (i.e. what kind of pattern one is looking for) or just the result of chance occurrence. Finally, in something that would play a continued role in Sponge's later, more specifically media-compositional investigations, the question of how recognition and propagation of patterns from such gestures contribute to the gradual building up of meaning over time would be fully explored in Sponge's later work of conceiving and building sensor-activated , responsive spaces. M2 (1998) Produced in San Francisco in 1998, Sponge's next project M2 was the result of a one-year concept, design, and implementation period and moved much further into work with digital systems than M1. In what would become a standard approach to developing projects, individual members would work outside of the group on independent projects both for financial as well as creative reasons. Whether conscious or not, such work in more complex (and financially lucrative) projects in the then-burgeoning multi-media industry in san Francisco and Silicon Valley had an indirect influence on Sponge's approach to project development, planning and management. The issues of emergent patterns that were undertaken and studied in the M1 experiment directly transferred into the conceptualization of M2 yet resulted in a decidedly different manifestation. From the start of the process, it was agreed that the site of presentation should be more controlled than the earlier M1 experiment in a public space. Indeed , the contextual shifting between public-outdoors space and controlled environments would become an important hallmark of Sponge's subsequent work. This level of control not only is relevant in terms of the logistical complexity of the physical event that Sponge constructed (a six-room walk-through architectural/media environment through which small groups of spectators were cycled over varying time lengths) but also provides the background horizon for the central question that drove the M2 project. How is it despite the instability of symbolic, linguistic, and representational systems that we make and produce that a fundamental material stability still exists at the deeper, sedimented part of human experience? In other words, despite the instability and anxiety we afford to symbolic systems, there still exists a deep, sedimented stability that is afforded to us by the world of matter. The starting point of M2 attempted to place this relationship into question. In the san Francisco realization of the environment, five individual spaces were architected inside a large empty gallery. Each of these spaces was assigned a particular thematic significance that related to the individual spaces as well as to the overall event. As visitors moved from space to space with the help of assistants carrying specific time instructions, the participants encountered themes of 'waiting" (the waiting room), "control" (the room where performers controlled and played the audio/visual systems), "immersion" (a space of floating screens and projections and physical heat generated from live stove top heating coils suspended inside mesh cages) and "transformation." Cycling through the environments, the density of elements, degree of media overload and ways of interpreting the sequence and meaning of individual elements of the experience fluctuated in both emotional affect and perceptual complexity. Disjunctive images and sounds evoking erotic experience, solitude, abandonment and transformation, instability and deformation of media were played and edited live from prerecorded narratives burned onto laser discs and projected onto floating, miniaturized screens in the second environment (the immersion space). Such emotionally charged media reappeared several times throughout the installation in increasingly mutated forms, while perspective shifts were continually re-introduced throughout the sequence of spaces. Through such a process of meaning accretion, the experience for the visitors was akin to a physical and emotional journey through increasingly layered fields of architectural, media and symbolic affect. While this brief description of the event clearly doesn't suffice in conveying the overall experience of participants (audience members later described physical sensations of vertigo, stillness, confusion, solitude) what is useful in this context is to briefly examine how Sponge's collaborative strategies enabled the transformation of conceptual and philosophical constructs into a material, performative event between bodies, heat, media, and architecture. What is interesting are the ways in which real time, performative process of talking, sketching, drawing and writing were utilized by Sponge to render ideas from multiple perspectives into concrete, material form. In initial concept and eventually, design sessions, ideas brought to the table were consistently subjected to a rigorous process of "translation" from one epistemic culture and form to another. This process of translation marks another key characteristic inherent in Lave and Wenger's understanding of communities of practice. For example, an idea about the instability of matter was translated into several different contexts, ranging from philosophical (interpretative) notion to one rendered in the mathematics of manifolds to one based on the thinking of centers of gravity in the physical performing and dancing body. A further process involved the constant "performing" of ideas between members, where communal writing and sketching served as material for creating ideas and sharing them amongst the group. Indeed, this process of struggle and accommodation in communal sketching, marking and writing is acutely evidenced by the endless series of notebooks and sketches generated during the M2 process featuring page after page of diagrams, flow charts, notes and illustrations on things ranging from the time cycles of the environment to philosophical inquiries on the nature of phenomenal experience and mathematical notation. This theme of writing and performing would later take on further weight, in both the group's work with its long term project M3 as well as with individual members' research. What also bears noting here is the disciplinary diversity of audiences who attended the three-week event in May 1998. Veterans from Silicon valley research labs, video and well known theatre artists, mathematicians, curators, electronic musicians, philosophy students from Stanford and Berkeley as well as computer scientists and literary scholars all were initial participants in Sponge's first attempt at bringing a discourse and set of practices that had originated under IMG's umbrella outside of the confines of the academy and the gallery space and into the broader sphere of its participants.
Sponge's next project, developed in 1999 after an extensive period of evaluation and re-grouping, was to become its most ambitious one, in theoretical and practical terms. It is also in M3 where the notions of threshold performance, materiality and agency set out in M1 and M2 would reach their full fruition in the design of new kinds of sensor-driven responsive media environment. Originally conceived as a set of three large-scale spaces which would take the visitors/players (as the "audience" would from hence be called) through a performative manifestation of Sponge's multivalent research exploration, visitors to M3 would begin with a critique of the informatic world view (Room 1, entitled Puzzle), then segue into a space focusing on the perceptual experience of the world undergone in spaces of immersion (Room 2: Sauna) and, finally, arrive inside a social play space where media and social activity could be collectively and continuously shaped by the participants in real time (Room 3: TGarden). While conceived as a total event, financial, logistical and creative complexities prohibited the simultaneous realization of all three parts of the work. Thus, between 1999-2002 Sponge focused on the production of two components of the project: Sauna and TGarden. The project Tgarden (which we will focus on here), realized between 1999-2002 in an international co-production between premiere arts and technology venues in the US, Canada and Europe (Banff Center, Ars Electronica, SIGGRAPH, V2 in Rotterdam), is arguably Sponge's most ambitious work in technological, aesthetic and philosophical concerns. The project aimed not only to further (and more rigorously) explore the theme of performance and intentional/unintentional gesture whose groundwork was already laid in earlier work but also to develop a suite of sensor-activated, responsive media and computational technologies that would embody Sponge's philosophical concerns. More to the point, TGarden attempted to design what Sponge member Sha Xin Wei has labeled "substrate" technologies, where computational processes at the low level would actually percolate upwards to the highest metaphorical and experiential level for the general participants who would visit the environment. In other words, perhaps through the conscious design of such "substrate" technologies (in Tgarden's case these included wireless and wearable sensing, responsive materials, software for choreographing continuous room state changes, and phsyics-based image and sound instruments), the low level (in conceptual terms) of such substrates would provide the foundation for a different kind of participant experience at the phenomenal rather than the cognitive level. As Tgarden has been written about extensively in other publications (10,11,12), we will provide here only a brief description of the project, Instead, our emphasis will be on the kinds of collaborative methods and work heuristics that arose in the course of the project development.
As visitors enter into the Tgarden environment, they are escorted to private dressing cabinets where they will find various types of clothing. The clothing itself is designed with specific physical and material constraints in order to interfere with the visitors' standard ways of physically relating to the world. The clothing is also embedded with accelerometers that measure the degree of acceleration, tilt and gravity of each person's movement. Such on the body computing enables the visitor's movement, acceleration and balance to be measured and sent, via wireless Ethernet, to a central logic computer that forms the core of the Tgarden system architecture. This logic system interprets data from the sensors (both on the body and room tracking), analyzes what is happening in the overall Tgarden system and sends commands to the sound and computer graphics system based on its judgement. This central system ("Oz") contains the microscopic logic of how the environments responds to visitors' actions over various time scales: thus, the system is designed to operate across multiple time scales (person as well as room). The different sound and image systems then modify their own internal states on the basis of Oz's hints and also on the basis of the continuous output from the sensors themselves. What is essential to note here is the reliance on non-rule based, improvised "on the fly" gestures and movement that provides the compositional and performative material fror Tgarden. Gestures and movements are not predetermined or subjected to a series of rules or behavioral and spatial codifications (i.e. you stand here and this happens), As visitors enter the Tgarden environment and movement initiates responsive processes of sound and image, the participants gradually become conscious not only of themselves but also how other bodies around them effect, shift and shape the environment. Furthermore, in contrast to many projects where a strong separation exists between performer and spectator, Tgarden attempts a dissolution of this distinction in order to bring interaction into the realm of haptic, felt and senses experience. This interaction "close up" can be precisely articulated, for example, in the physics-based models utilized in the Tgarden software. "players who expend effort by jumping, bouncing and dragging themselves in space encounter musical and visual equivalents of this physicality in the lowest levels of software: phantom masses and springs, virtual kinetics, friction and energy. Here, in this software physics "the physicality of the performance interface gives definition to the (musical) modeling process itself," writes Joel Ryan, one of the project's collaborators, suggesting that there must be a resonance between the space, interface (i.e. sensors) and software."(10) The disintegration of the dichotomy between performer and observer is crucial to the overall setup of the Tgarden project so much so that there exists no singular spectator (either spatially or formally) outside of the participants in the event. Tgarden's visitors/players not only engage in an oscillating social game of performing with and watching each other, they also perform with the visual and sonic media that inhabit the environment itself. In this sense, the physical performance space occupied by the public "players" is conceived from the start as a mediated substance that is shapeable by the way of social play. By this definition, Tgarden provides a space to begin investigating how new experiences of spatiality and felt, embodied experience begin to emerge out of computationally augmented environments where the distinction between viewer and participant is interrogated and purposefully blurred. Outside of its role as an experiential artistic event, the complex development process of Tgarden also provides a rich example for examining a specific set of methodological issues involved in transdisciplinary collaboration. In terms of these collaborative issues among the different participants during the Tgarden project, we want to focus here on four pertinent ones: (1) difference in co-present design discussions versus distributed communication in the development-production process, (2) difference of approach in disciplinary languages and individual artist-designer cultures, 93) incorporation of heterogeneous design methodologies and methods, and finally, (4) difficulties encountered in the application of techno-scientific research and development paradigms to a project in cultural production. As in earlier Sponge projects, the extensive development period for Tgarden involved cross-disciplinary discussion, brainstorming and collaboration with individuals of diverse training and expertise. The Tgarden development process, however, reached further than earlier work in geographic and disciplinary terms through its joint collaboration with FoAM, a decentralized network of artists and designers sited in Brussels but spread across Europe. The contrast between Sponge's face to face (i.e. co-presence) practice of concept generation through the consensual domain of verbal and non-verbal language and the distributed, non co-present methods (email, chat, etc) necessary to sustain collaboration with the international team added an additional dimension of complexity to the project. The second issue arising in the Tgarden process involved the heterogeneity of disciplinary languages and epistemic cultures that came together to realize the project. Whereas the three main members of Sponge had continually worked on an evolving conversation, the entrance of a wider range of participants and an even wider range of fields (fashion and textile design, computer graphics, physics, etc) and cultures (scientists trained in the research lab, anti-system hacker artists, artists used to working in large scale infrastructures) increased the level of working complexity. A collaborative design workshop held at the Banff Center for the Arts in the winter of 2001, already in the key year of development for the 2001 presentation of TG in Europe acutely points out the challenges of such heterogeneous, transdisciplinary collaboration as well as the different work cultures (hacker culture versus performance production verfsus students trained in the techno-scientific lab context). Despite the presence of the boundary object of a performance/installation event (Tgarden), in essence, the challenges that the Tgarden team encountered reflect back on the challenges encountered by a so-called community of interest in trying to build a shared understanding and common ground. As Ernesto Arias wrote in 1996, "fundamental challenges facing communities of interest are found in building a shared understanding of the task at hand (which often does not exist upfront, but is evolved incrementally and collaboratively)…Members of communities of interest need to learn to communicate with and learn from others who have a different perspective and perhaps a different vocabulary for describing their ideas. [They need to] establish a common ground and a shared understanding."(1) Aspects of languaging, different professional cultures, and intentions, contextual conflicts and relationships already manifested themselves at the start of the workshop - in essence, a period devoted to construct a set of shared goals for the overall production of the work - and continued throughout the development and presentation process of the project. Such issues point up the complexities inherent in transdisciplinary work like Tgarden, particularly in the merging together of artistic and technsocientific goals and intentions in one context. A further issue arising in the Tgarden production period was the degree of incorporation of so-called iterative and participatory design methods into the artistic process. By now it is evident that the turn toward artistic projects where user/viewer/player participation is a central feature of the work has resulted in the increased incorporation of iterative and participatory design methodologies in the domain of artistic practice. Despite this, the 1;1 transfer of such methodologies is directly complicated by the heterogeneity of disciplinary approaches. In other words, no singular technique or methodology can be said to apply to such complex projects much to the chagrin of those who would see the use of such methods in the conceiving, producing and evaluating of a complex artistic work as comparable to the usability cycle for commercial product development or software design. It goes without saying that the design/production processes for clothing and fashion, engineering and software development and live performance (all of which were incorporated into Tgarden) have radically different life cycles and requirements. For example, introducing so-called iterative design procedures (prototyping, user evaluation), particularly in terms of continually reevaluating and redefining the project utilizing user testing and feedback was continually complicated by institutional obstacles, including lack of user testing time to accurately tune the system. Along the same framework, user evaluation, one of the central facets of participatory design, was also brought into the Tgarden process. Partly based on a grant from the Arts Council of the UK examining the constitution of new audience formations in art science creative production, the Tgarden team undertook an extensive series of user interviews incorporating 1st first methods in the form of video taped interviews. The complexity of evaluating the multidimensional axes of experience in the project was consciously integrated into the Tgarden developmental process due to the project's central reliance on an audience of participants rather than simply observers and the need for real world testing (in the wild) outside of the constraints of the design studio or the lab. This was complicated, however, by institutional conditions.
Sponge and the Topological Media Lab A final issue concerns the complexity of combining techjno-scientific research cycles and cultural production cycles within the same framework. Given that no single framework can sustain such incommensurate practices, in 2001, as faculty in critical studies of technology and media in the School of Literature, Communications and Culture at Georgia Institute of Technology, Sha Xin Wei founded the Topological Media Lab (TML) for art research. Housed in the College of Computing's Graphics, Visualization and Usability Center, the TML has provided a protected space within which students and visiting artists and researchers can build experimental technologies of performance. These experimental works are driven by artistic and philosophical questions, so the heuristics are largely external to techno-science. However, in order to pursue these questions, the TML strategically applies resources at carefully chosen points on the frontiers of engineering and media research. The TML's goals are neither to produce singular works of art nor to produce singular demonstrations of technology, but to create phenomenological understandings of philosophical questions regarding, for example, gesture, agency, and materiality, substantiated by embodied, holistic experiments in a setting free of the compromises imposed by a performance and entertainment calendar. The TML's fusion of organizational-pedagogical practices drawn from art studio, team-based engineering lab, and performance collective has attracted students from visual arts, digital media, architecture, electronic music, computer science, industrial design, and electrical and mechanical engineering.
Conclusion
1. Arias, E. and Fischer, G.: "Boundary Objects: Their Role in Articulating the Task at hand and Making Information Relevant to It." (ICC'2000), ICSC Academic Press, Wetaskiwin, Canada, December 2000, 567-74. 2. Century, Michael. "Pathways to Innovation in Digital Culture." Rockefeller Foundation report (1999). 3. Knorr-Cetina, Karen. Epistemic Cultures: How the Sciences make Knowledge. Cambridge: Harvard UP, 1999. 4. Lave, Jane and Wenger, Etienne. Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge UP, 1999. 5. Laurel, Brenda. Computers as Theater. Reading, Mass.: Addison-Wesley Press, 1990. 6. Le Duc, Aimee. "Sauna 02 At the Lab." Artweek 33:7 (2002), 17-18. 7. Pickering, Andrew. The Mangle of Practice: Time, Agency, and Science. Chicago: University of Chicago Press, 1999, p.144. 8. Pickering, p.6 9. Pickering, p.26 10. Ryan, Joel and Salter, Christopher. "TGarden: Wearable Instruments and Augmented Physicality." Proceedings of the 3rd Annual Conference on New interfaces in Musical Expression (NIME), Montreal (2003). 11. Sha, Xin Wei. "Resistance is Fertile: Gesture and Agency in the Field of Responsive Media," in Configurations, Vol. 10:3 (2003). 12. Sha, X.W., and Gill, S.P. "Gesture and Response in Field-based Performance," Creativity and Cognition 2005. 13. Sponge website: http://www.sponge.org 14. Sponge. "The Surface that Holds the Image is Unstable." Ec/arts #, ed. Eric Sadin. Paris, 2002. 15. Sponge. Ec/arts (2002), 99. 16. Sponge. Ec/arts (2002), 99-100. 17. Star, Susan Leigh and Griesemer, James. "Institutional Ecology, 'Translations,' and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-1939," The Science Studies Reader, ed. Mario Biagioli, London: Routledge, 1999, 5-5-524. 18. Star and Griesemer, 1999. 19. Topological Media Lab, http://topologicalmedia.net 20. Winograd, Terry amd Flores, Fernando. Understanding Computers and Cognition: A New Foundation for Design. Reading, Mass.: Addison-Wesley, 1986, 48-52.
Permisison to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made to or distributed for profot or commercial advantage and that copies bear this notice and the full citation on the first page. Christopher L. Salter/Sha Xin Wei: "Sponge: A Case Study in Practice-based Collaborative Art Research," Creativity & Cognition2005, London 2005, pp. 92-101. Copyright 2005 ACM 1-59593-025-6/05/0004.
Project directors: Johannes Birringer & Michèle Danjoux
|
|||
 |
 |
||
|
|
|||