Gesture: The Living Medium, The First congress of the International Society for Gesture Studies held at the College of Communication, the University of Texas at Austin, June 5-8, 2002.

Cornelia Müller, Freie Universität Berlin

Jürgen Streeck (Austin) organized the inaugural conference of the International Society for Gesture Studies (ISGS) and gave it its specific thematic focus: Gesture as a living medium, a medium leading its own specific life, a medium that is inextricably rooted in different 'life worlds'. To look at gesture as a living medium highlights the embedding of communicative bodily movements in specific natural contexts of occurrence and argues that largely the significance of gesture emerges from its place in a certain communicative environment. Streeck developed his stance towards the field of gesture studies in his opening lecture, which situated it within a historical sketch on the scholarly study of gesture and within contemporary gesture research. In doing things Streeck remembered the foundation of a Society for the Research on Mankind (Société pour les recherches de l'homme) in the French Enlightenment and expressed his hopes that the ISGS will enhance enlightening research on the medium gesture.

The conference was framed by two high level plenary lextures every day, all of which addressed different dimensions of gesture 'as a living medium'. Streeck invited speakers from various disciplines such as anthropology, semiotics, art history, sign language, cognitive psychology, and micro-ethnography and hence inspired a multi facetted perspective on the study of gesture.

Charles Goodwin (Los Angeles) held the second lecture on the first day of the conference in which he vividly introduced the audience to the intricate coupling of gestures to their material, medial, and communicative environment. Based on videotapes of archeologists he showed that gestures used to talk about structure in the dirt display not only a symbiotic relationship to the medium they are connected with but also that they emerge through a mutual elaboration of different media and materials in play.

The second day began with a condensed lecture on semantic structures of co verbal gestures held by Geneviève Calbris (Paris). Calbris developed her semiotic approach to the analysis of gesture. One focus of her talk was on the analogical links that motivate the gestures' form and meaning. These analogical links may establish relationships between single aspects of a gesture and thus lead to compound gesture forms and meanings. Yet these 'natural' anagogic links are not stable across cultures, in contrast, they may vary between cultures just because cultures may choose different analogical links between form and meaning.

With Richard Shiff's (Austin) lecture the audience entered yet another 'life world' of the gestural movement: the painter's gestures. A painter's movements with his brush leave different kinds of visible traces – marks – that may relate to different aspects of the painter's doing. Shiff distinguished marks that function on an iconic base (signs) representing for example the contour of an object in representational art from marks that display the actor’s physical act of moving the paint brush and which are considered to function on an indexical base.

The following day started with a lecture by Scott Liddell (Washington) addressing unprompted and grammatically required directional signs in American Sign Language, hereby uncovering a kind of gestural dimension in ASL – the spontaneous creation of directional signs which require an ad hoc reconstruction of meaning from the part of the addressee as opposed to grammatically required directionals with a conventionally established meaning.
David McNeill's (Chicago) lecture on the 'Dialectic of gesture and language' presented a substantial psychological theory of language and gesture including a scenario of the evolution of language. The core notion in this theory is the 'growth point'. The growth point is a conceptual unit retaining the properties of the whole, encapsulating the core features of a verbal-gestural unit of discourse. The growth point is driven by a dialectic of imagistic (iconic) and linguistic categorical (arbitrary) forms of thought. In a rather fine grained and condensed line of argumentation in which McNeill incorporated recent findings in the functioning of mirror neurons as well as Mead's concept of the significant gesture he developed the idea that the evolution of language must have induced the capability to think in two modes at once – imagistic and linguistic categorical.

On the last day of the conference two closing lectures where held, which added yet further perspectives but also gave hints as to which roads the field of gesture studies may want to travel in the future.

John Haviland (Portland/San Cristobal) presented parts of a larger study on the communicative skills of Tzotzil speakers – corn farmers living in the highlands of Chiapas in Mexico. His talk introduced the audience to an aged man who is a highly recognized person in his local society just because of his particular linguistic skills. He is therefore considered a master talker. And as such he is asked to perform his skills on special, often ritual, occasions. Haviland put forward the argument that this person was recognized as a master talker not only because of his linguistic skills but also because of his mastery of gestures that go along with speech let alone the ways in which both media were used to amplify each other's meaning.

Adam Kendon's (Philadelphia, Naples) closing lecture provided an overarching sketch of the phenomenon of human gesture based on micro-ethnographic records of naturally occurring conversations of Neapolitan, English, and North-American speakers. Kendon gave a systematic account of the various aspects gesture as a medium of communication that need to be considered in the descriptive analysis of gesture. Touching upon the close relationship of phrasal units in gesture and speech, the ways in which gestures contribute to the propositional content of the utterance, the ways in which gestures may operate meta-linguistically as modal gestures, and how these gestures tend to cluster in gesture families, Kendon drew not only a colorful picture of the diversity of the gestural medium of communication but directed the attention to a wide and open field of documentary research on gesture.

The conference was widely attended and attracted about 190 participants. About 150 scholars presented their findings and stances towards the study of gesture within only three days. In order to ensure presenting space for all of these interested persons five parallel sessions had to be set up, which – by not following a coherent time – unluckily reduced the possibility to switch panels and created disadvantages for panels held by younger and less well known scholars, for most of the audience would decide to take advantage of the possibility of listening to the 'big names'. Still the following report of the work presented aims to give a rather broad overview on the subjects touched in the conference. Nevertheless, choices had to be made, and the report to a certain degree reflects the perspective of the author.

The following sections where organized: 'The Lives and Work of Gestures', ' Gesture and Talk', 'Aesthetics, Embodiment, Performance', 'Development', 'Discourse and Semantics', 'Multi-Modality / Sign Language', 'Technology'.

The Lifes and Works of Gestures

In this section various different 'life histories' of gestures were presented. Among them: The development from instrumental action to gesture in varying naturally occurring settings (Curtis LeBaron, Brigham U.). The unfolding of activities in different situational contexts, such as the role of mimetic gesture in the communication about food-stuff (Carlnita Greene, U. of Texas, Austin) or its functioning within origami instructions (Sae Oshima, U. of Texas Austin); The session further included talks on the role of sketching gestures in collaborative concept formation (Santinder Gill, Stanford U.), as well as the use of different kinds of gestures in classical as well as modern dance preparations (Freya Vass-Rhee U. of Californa, Riverside; Michael Bedar, U. of California San Diego). The section concluded with a panel on gesture and musical performance touching upon: the role of gestures in defining musical performance (Richard Ashley, Northwestern U.), functional variations and organization of expressive gestures by a classical orchestral conductor (Shin Maruyama U. of Tokyo & Nobuhiro Furuyama, National Institute of Informatics, Japan), the gestures of flutists (Ronda Mader & Richard Ashley, Northwestern U.), and on improvisatory gestures and the transmission of Western classical music (Laura Lohmann, DePauw U.).

Gesture and Talk

The relation of gesture and talk was one the most frequently addressed topics of the conference. The panels were devoted to ‘Multimodal Communication & Representation’, to ‘Gesture, Deixis and Space’, to ‘The Role of Gestures in Conversational Grounding’, to ‘The Reception and Response of Gestures’, to ‘Gestures in Conversation’, and ‘Gesture in Relation to Different Cultural Contexts’. In the panel on 'Multimodal Communication & Representation' Lluís Payrató (U. Barcelona) presented a project on multimodal communication in relation to linguistic variation and multilingualism conducted in Barcelona with Catalan and Spanish speakers whereas B. Sauer (J. Hopkins U.) & A. Meyer (Carnegie Mellon U.) as well as Craig Martell (U. Pennsylvania) presented different accounts to the computer based representation of multimodal communication. The panel on 'Deixis and Space' brought together studies comparing the spatial information given in speech and gesture with the information given in American Sign Language (Sarah Taub, Pilar Pinar & Dennis Galvan, Gallaudet U.), and research on deictic forms and functions (N.J. Enfield MPI Nijmegen), and on gestural anaphora (Lisa D. Harper, Georgetown U.). The panel on 'The Role of Gestures in Conversational Grounding' stimulated presentations on listener's gestures (Herbert Clark & Meredith Krych, Stanford U.), on gestural repetition (Janet Bavelas, Christine Kenwood & Jennifer Gerwing U. Victoria), on pointing in discussing house plans (Van der Wege & Mija A. Carlton College), and on gesture location as a means to distinguish situation models (Randi Engle U. Pittsburgh) - all considered as different ways of gestural grounding in conversation. 'Reception and Response' focused on listener's participation in gesture production as well as on the self-monitoring of errors in speech and gesture. Christine Kühn (Hokkaido U.) showed how listeners respond to and correct spontaneous gestures, Marianne Gullberg and Sotaro Kita (MPI Nijmegen) examined information uptake and visual fixation by addressees, Claire Maury-Rouan (CNRS, U. de Provence) investigated the impact of listener's facial expressions on the speaker's discourse, while Mandana Seyfeddinipur (MPI Nijmegen) showed when and how speech or gesture react to trouble in the other modality within a speaker. In the panel 'Gestures in Conversation' Lone Laursen (U. of Odense) and Ulrike Bohle (Freie U. Berlin) presented micro analytic studies on the relation of gesture and speech, showing participant orientation towards gesture and revealing a diversity of ways in which gestures contribute to compound turn-constructional units. Ellen Fricke (Technische U. Berlin) argued for a re-conceptualization of a theory of deixis based on the analysis of gesture and speech in route descriptions. André Hatting (Freie U. Berlin) outlined what the communicative functions of gestures might actually be and uncovered that the debate on the communicativity of gesture is rooted in the mis-conceptualization of communicative processes in the psycholinguistic paradigm. The rather heterogeneous panel 'Gesture in relation to different cultural contexts' addressed gestural practices between indigenous people in America, Australia (Jeffrey Davis U. of Tennessee) and in South East Asia (Arnold Groh, Technische Universität Berlin) as well as head gestures used in a group of four Neapolitan adults (M.A. Morel et al., Paris) and the relation of hand gestures to acoustic aspects of Japanese foreign language speakers (Shuichi Nobe, Aoyama Gakuin U.).

Aesthetics, Embodiment, Performance

This section brought together a rather disparate field ranging from 'Performance as Gestural Inquiry' to 'Gestures in the Past', to different 'Aesthetic Genres', to the 'Everyday Performance and Ritualization', and to a 'Dynamic Embodiment in an Anthropological Perspective'. In the panel 'Living the Medium: Performance as Gestural Inquiry' Tessa Carr & Deanna Shoemaker (U. of Texas, Austin), Casey Garcia (U. of Texas, Austin), Angela Kariotis (U. of Texas, Austin), and Chris Koenig (U. of California, Los Angeles) explored the nature of gesture by employing performance as ethnographic study of gesture. The subsequent panel 'Gestures in the Past' addressed different aspects of historical gesture studies: Herman Roodenburg (Meertens Institute, Amsterdam) focused on a cultural historic perspective when describing the search of physical grace in the Dutch Republic; Reinhard Krüger (Technische U. Berlin) added an analysis of gestures in literary dialogues; Joachim Gessinger & Manuela Böhm (U. Potsdam) presented an analysis of Diderot's notion of sign and body within his concept of communication based on theatrical performance; Cornelia Müller & Harald Haferland (Freie U. Berlin) described the semiosis of a Medieval honorific gesture showing that it is based on the instrumental action of tying-up the hands. In the panel on 'Aesthetic Genres' figure drawings were considered as gestures or visual texts (Thérèse Boyle) and the rhetoric figures underlying the gestures used in caricatures (Massimo Serenari, Technische Universität Berlin) were uncovered. The panel treating 'Everyday Performance and Ritualization' ranged from and analysis of the gestural metaphors Bush and Gore used (Alan Cienki, Emory U., Atlanta) to analyses of the cross sign in Serbia (Bojan Zikic U. Belgrad) to a study of the 'clever' gesture used among black urban South Africans (Heather Brookes, Stanford, Pretoria). A somewhat different focus was followed in the panel called 'Theorizing Gesture: Dynamic Embodiment in Anthropological Perspective'. The papers given here presented an anthropological stance arguing for a semasiological approach to the analysis of gesture systems based on the indigenous conceptualizations rather than on external categories. The papers touched upon the sign of the cross in the Greek Orthodox Church (Angela Shand, U. Illinois), on indigenous gestures in Egyptian folkloric dance (Marjorie Franken, U. California, Riverside), on gestures in Hatha Yoga (Kenneth McKandless U. Illinois, Urbana-Champain) as well as on the notation of indigenous conceptions of action and space in Plains Indians Sign Language (Brenda Farnell, U. Illinois, Urbana-Champaign). The subsequent panel 'Embodiment' pursued investigations into culturally varying forms of embodiment. while Julio Cesar de Tavares (U. Federal Fulminense, Rio de Janeiro) reported on a shoulder and hip-movement evoking a non-confrontation logic in an aggressive everyday life, Katharine Young (San Francisco) developed the idea that the charisma of a philosopher may be grounded in his gestural practice.

Development

The section had three foci: First the role of gesture in language acquisition addressed by the panels on: 'The Functions of Gestures in the Development of Speech Communication' and 'Early Childhood'; Second the role of gesture in second language acquisition addressed by the panel on 'Gesture and Second Language Acquisition'; Third the acquisition of quotable gestures or emblems and the etymology of the same group of quotable gestures: , 'Ontogeny and Phylogeny of Gestures. ‘The Berlin Dictionary of Everyday Gestures'. In the panel on 'The Functions of Gestures in the Development of Speech Communication' Jana Iverson (U. Missouri) examined developmental origins of the speech-gesture link in typically developing children and Martha Alibali (U. Wisconsin, Madison) and Sotaro Kita (MPI Nijmegen) also investigated the function of gesture in speaking. In contrast Julia Evans (U. Wisconsin, Madison) and Donna Thal (San Diego State U.) discuss the use of gestures in children showing language delay or impairment. In the panel on 'Early Childhood' the problem of language acquisition was regarded from a broader viewpoint, including the conversational practices in which lexical development is embedded (Patricia Zukow-Goldring, N. de Villiers Rader & T. Cain, Ithaca College), the gestures as pre-cursors to the acquisition of arguments (B.F. Kelly, U. California, Santa Barbara), the close integration of gesture and speech as displayed in 'late talker' children (J. Moore, B. Davis & P. Macneilage U. Texas, Austin), or the tight synchronization of gesture and speech on multiple levels in a 3 1/2 year old (B. Belbas and Amy Sheldon, UMN-Twin Cities). The panel on 'Gesture and Second Language Acquisition' had too many talks for the time slot provided – apparently this is an expanding field within the study of gesture. It approached the topic from various perspectives, showing that the consideration of gesture facilitates the understanding of this process in general (Gale Stam, National Louis U., Chicago), revealing one facet of this process by describing how second language learners establish anaphoric linkages through speech and gesture, and by uncovering yet furthers facets in the use of speech and gesture by second language learners in Japan (Nicholas O. Jungheim Waseda U., Shuichi Nobe Aoyama Gakuin U.) as well as in oral classroom presentation and interaction (Alexis Tabensky, U. of New South Wales). The panel on 'Ontogeny and Phylogeny of Gestures: The Berlin Dictionary of Gestures' presented the organization of a regional dictionary of emblematic gestures, describing not only forms, meanings, and functions but also the distribution of the gestural knowledge and use with regard to age, gender, and origin, i.e. the etymology of a given gesture (Roland Posner, Massimo Serenari, Reinhard Krüger, Thomas Noll, Technische U. Berlin).

Discourse and Semantics

Just like the section on 'Gesture and Talk' the one on discourse and semantics attracted a large amount of interest. Both: with regard to the amount of papers given and with regard to the interest received from the audience. The following panels were organized: 'The Production of Speech and Gesture', 'Acts of Meaning', 'What Do we Mean by Meaning', 'Components of Gesture', 'Experimental Studies of Gestures as Communication', 'Gesture as a Facilitator of Lexical Retrieval', 'Gesture and Lexical Access'.
The first panel addressed the 'Production of Speech and Gesture' from a psycholinguistic point of view. It touched upon the gesture-speech coordination in metonymy (Mika Ishino, U. of Chicago), the regiment of gesture production as ballistically or interactively motivated (Nobu Furuyama, National Institute of Informatics, Tokyo, David McNeill & M. Park-Doob, U. of Chicago), on the temporal correlation of speech and gestures' focal points (L. Valbonesi, U. of Illinois, Chicago), as well as on the relation of intonation, gesture & discourse structure (D. Loehr, Georgetown U.). The second panel 'Acts of Meaning' brought together papers addressing the use of gesture in relation to speech across different speech communities: Mexican Spanish (Rosa Montes, U. Autonoma de Puebla), Indonesian (Juliana Wijaya, U. California, Los Angeles), Japanese: Irene Kimbara (U. of Chicago). In addition Curtis LeBaron (Brigham Young U.) and Timothy Koschman (U. of Southern Illinois) argued that in the tradition of gesture research two different approaches towards the analysis of gesture are to be distinguished: One viewing gesture 'as an act of meaning' the other viewing gesture 'as an act for meaning'. In the subsequent panel of this section 'What Do we Mean by Meaning. Conceptual Integration Theory in Gesture Transcription and Analysis' Susan D. Duncan (National Yoan Ming University), Scott Liddell (Gallaudet U., Washington), and Eve Sweetser (U. California, Berkeley) together with F. Parrill (U. Chicago) discuss the use of the Mental Spaces and Blending framework for a clear representation of gestural and verbally established reference. The panel 'Components of Gesture' addressed the question of a componentiality of conventionalized gestures. Recurring hand-shapes, positions, and movement patterns in gestures have been observed in different cultures suggesting a rudimentary gesture morphology. Adam Kendon (Philadelphia/ Naples) showed that Neapolitan, British and American pragmatic gestures tend to cluster in 'gesture families' which are based on recurring kinesic forms and themes. Cornelia Müller (Freie U. Berlin) presented a micro-analytic case study of the 'Palm-up-open-hand' which uncovered a semantic and formational core of this gesture. This core configuration is combined with varying movement patterns adding new meaningful elements to the core configuration and meaning of the hand. Rebecca Webb (U. Rochester) presented an analysis of American metaphoric gestures displaying a repertoire of recurring forms and meanings, Irene Mittelberg (Cornell U.) presented an analysis of the metaphorical gestures Linguists use in the classroom to explain grammatical models and structures. The panel focusing on 'Experimental Studies of Gestures as Communication' brought together work by a group of researchers headed by Janet Bavelas (U. Victoria) that presented three experiments showing the communicative impact of gesture. The following panel in this section was devoted to Robert Krauss' psycholinguistic work on the relationship between gesture and speech. This work lead Krauss to a rather influential and widely discussed hypotheses on the nature and function of co-verbal gesture. He considers 'Gesture as a Facilitator of Lexical Retrieval'. Krauss presented the recent state of the art of his work including the recent formulation of the lexical retrieval hypothesis. Herbert Clark (Stanford U.) was invited to discuss Krauss' hypotheses in this panel. 'Gesture and Lexical Access' followed up some of the questions raised by Krauss' provoking hypotheses. Several experimental studies were presented that provide evidence for a communicative function of gestures and for a necessary differentiation of the formulated hypothesis. Susan Wagner, Susan Goldin-Meadow, H. Nusbaum (U. Chicago) presented findings that point to an association of gesture to spatial and verbal cognition, Gale Stam (National Louis U., Chicago) showed that different types of gesture occur depending on the activity the speaker is involved in: retrieve a word or elicit a word; Alissa Melinger & Willem Levelt (MPI, Nijmegen) provided evidence from the speaker for a communicative function of gesture, showing that speakers omit critical information from speech when this information is also expressed in gesture.

Multi-Modality and Sign Language

This section only consisted of two panels: 'American Sign Language' and 'Sign-Languages in Comparative Perspective' and had one recognizable focus: the role gestures play within Sign Languages. Talks by Kearsy Cormier (U. Texas, Austin) on the gestural or linguistic use of space in ASL, by Christian Rathman (U. Austin) & Richard P. Meier (Gaurav Mathur U., Connecticut) on the grammaticization of gesture to verb agreement in ASL, by Evelyn McClave (California State U., Northridge) on the nonmanual gestures in ASL, by David Quinto-Pozos (U. Texas, Austin) on the amount of non-linguistic gestures used by Mexican as well as by American signers, as well as by Alejandro Oviedo (U.de los Andes, Venezuela, U. Hamburg) who shows that Venezuelan Deaf signers use emblems along with the sign language.

Technology

Unfortunately just a very small panel was devoted to technological aspect of gesture studies, a fact that somewhat misrepresents the existing interest and intiative of the computer engineering field into gesture. This panel concentrated on 'Gesture as Interface in Human-Computer Interaction' and presented different solutions. Thus, Andrea Corradini (Oregon Health & Science U.) presented two multimodal speech-gesture architectures, P. Kühnlein, M. Nimke, H. Rieser, & J. Stegmann (U. Bielefeldt) presented an HPSG based interface for the generation of the syntax and semantics of verbal utterances including pointing gestures; S. Kopp, T. Sowa, I. Wachsmuth (U. Bielefeldt) present a study focusing around the problems of gesture imitation by a virtual agent; K. Müller (Fraunhofer Institut, Stuttgart) describes the use of bodily communication in collaborative virtual settings. Concluding remarks
Looking retrospectively at the conference it becomes clear that the field of gesture studies as it presented itself at the Austin conference appears to autopoietically structure itself around one central question: gesture and its relation to speech. This implies a rather clear preference for the analysis of co-verbal gesture yet somewhat independent from a given degree of conventionalization of the gestures in question. This also implies a significant shift away from studying gesture as nonverbal communication, which has dominated the field since the 60ies and has created an artificial separation between speech and gesture. Such a separation appears to be obsolete today and the question of how gesture is related to speech is now addressed from a multitude of perspectives such as anthropology, cross-cultural comparison, psychology, cognitive science, linguistics, semiotics, ethnography, sociology, and technology. Another highly remarkable shift concerns an increasing interest of sign-linguists to gesture. For quite a long period of time sign-linguistics insisted on a strict distinction between gestures of the speaking and signs in sign languages. Although for political reasons this undoubtedly has been a useful and probably necessary strategy to follow – i.e. in order to establish the recognition of sign languages as languages – this stance appears to have been overcome by now. Sign languages are recognized as languages by now and as such they may incorporate gestures – just as spoken languages do. Hence the analysis of gestures used along with spoken and signed speech appears to form the core of an emerging field. At least this is the path indicated by a majority of contributions to the conference.