Multimodality in aphasic conversation
Why gestures sometimes do not help
DOI:
https://doi.org/10.1558/jircd.v2i2.215Keywords:
Gesture, aphasia and conversation, aphasia therapy, speech/gesture modelAbstract
The use of gestures is one of the reasons for which some aphasics communicate better than they speak. For these aphasics as well as their partners, multimodality becomes a resource for the successful management of interaction. However, the successful employment of gesture implies multiple practices of adaptation, since there are important differences between gesture use in non-impaired and impaired aphasic interaction. In non-impaired faceto- face interaction, gestures mostly accompany language and support and influence its interpretation in various ways. Only rarely do gestures, body movements and the manipulation of artifacts take the lead, and even more rarely do they replace language entirely (Goodwin 1986, Heath 1982, 1992, Melinger & Levelt 2004). Under conditions of severe non-fluent aphasia (as described here), gestures therefore are used for functions which they cannot easily fulfill. In the absence of language, gestures become a substitute (Goodwin 2000, 2003 and 2006a, b). This uncoupling of gestural communication from language usually results in complex and often lengthy collaborative sequences in which the participants set out to co-construct what the aphasic “gesturer” means, with the non-impaired speakers providing, in a way, the missing speech for the linguistically impaired speaker (cf. Damico et al. 2008, Goodwin 2000, 2003, 2006). As we will show, these processes of coconstruction are encumbered not by the aphasics’ deficient gestures, but rather the semiotic nature of gesturing itself. We will discuss examples of successful and unsuccessful gestural communication in aphasics and suggest some reasons why gestural communication fails sometimes, but why it helps in other cases. Our observations lend support to the idea that therapy has to respect the semiotic characteristics of gesturing and the interactional practices necessary for their management in order to enhance the successful employment of the visual modality.
References
Auer, P. (1988). On deixis and displacement. Folia Linguistica XXII (3?4): 263?292. doi: 10.1515/flin.1988.22.3-4.263
Auer, P. (2005). The limits of cooperation – speakership in conversations with aphasics, unpublished MS, University of Freiburg.
Bauer, A. (2009). Miteinander im Gespräch bleiben. Partizipation in aphasischen Alltags gesprächen. Radolfzell: Verlag fur Gesprachsforschung (http://www.verlag-gespraechsforschung.de/2009/bauer.htm).
Bauer, A. and Auer, P. (2009). Aphasie im familiären Alltag. Stuttgart: Thieme.
Bavelas, J. B., Gerwing J., Sutton, C. and Prevost, D. (2008). Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58 (2): 495?520. doi:10.1016/j.jml.2007.02.004
Damico, J. S., Wilson, B. T., Simmons-Mackie, N. N. and Tetnowski, J. A. (2008). Overcoming unintelligibility in aphasia: The impact of non-verbal interactive strategies. Clinical Linguistics & Phonetics 22 (10?11): 775?782. doi:10.1080/02699200801950518
Cohen, A. and Harrison, R. (1973). Intentionality in the use of hand illustrators in faceto-face communicative situations. Journal of Personality and Social Psychology 28 (2): 276?279. doi:10.1037/h0035792
Ekman, P. and Friesen, W. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1 (1): 49?89.
Feyereisen, P. (1993). Communicative behaviour in aphasia. In G. Blanken, J. Dittmann, H. Grimm, J. Marshall and C. W. Wallesch (eds) Linguistic Disorders and Pathologies. An International Handbook, 288?303. Berlin: Thieme.
Fillmore, C. (1997). Lectures on Deixis. Stanford, CA: CSLI Publications.
Goffman, E. (1979). Footing. Semiotica 25 (1?2): 1?29. doi:10.1515/semi.1979.25.1-2.1
Goodwin, C. (1986). Gestures as a resource for the organization of mutual orientation. Semiotica 62 (1?2): 29?49. doi:10.1515/semi.1986.62.1-2.29
Goodwin, C. (1995). Co-constructing meaning in conversations with an aphasic man. Research on Language and Social Interaction 28 (3): 233?260. doi:10.1207/s15327973rlsi2803_4
Goodwin, C. (2000). Gesture, aphasia and interaction. In D. McNeill (ed.) Language and Gesture, 85?95. Cambridge: Cambridge University Press.
Goodwin, C. (2003). Conversational frameworks for the accomplishment of meaning in aphasia. In C. Goodwin (ed.) Conversation and Brain Damage, 90?116. Oxford: Oxford University Press.
Goodwin, C. (2004). A competent speaker who can´t speak: The social life of aphasia. Journal of Linguistic Anthropology 14 (2): 151?170. doi:10.1525/jlin.2004.14.2.151
Goodwin, C. (2006). Human sociality as mutual orientation in a rich interactive environment: Multimodal utterances and pointing in aphasia. In N. Enfield and S. Levinson (eds) Roots of Human Sociality: Culture, Cognition and Human Interaction, 96?125. London: Berg.
Goodwin, C. (2007). Interactive footing. In E. Holt and R. Clift (eds) Reported Talk: Reported Speech in Interaction, 16?46. Cambridge: Cambridge University Press.
Hadar, U. and Butterworth, B. (1997). Iconic gestures, imagery and word retrieval in speech. Semiotica 115 (1?2): 147?172. doi:10.1515/semi.1997.115.1-2.147
Heath, C. (1982). Interactional participation: The coordination of gesture, speech and gaze. In V. D’Urso and P. Leonardi (eds) Discourse Analysis and Natural Rhetorics, 85?97. Padua: Cleup Editore.
Heath, C. (1992). Gesture´s discreet tasks: Multiple relevancies in visual conduct and the contextualisation of language. In P. Auer and A. Di Luzio (eds) The Contextualisation of Language, 101?127. Amsterdam: Benjamins.
Herrmann M., Reichle T., Lucius-Hoene G., Wallesch C. W. and Johannsen-Horbach H. (1988). Nonverbal communication as a compensative strategy for severely nonfluent aphasics? A quantitative approach. Brain and Language 33 (1): 41?54. doi:10.1016/0093-934X(88)90053-3
Holle, H., Gunter, T. C., Ruschemeyer, S. A., Hennenlotter, A. and Iacoboni, M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage 39 (4): 2010?2024. doi:10.1016/j.neuroimage.2007.10.055
Kendon, A. (2004). Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press.
Kendon, A. (2009). Language’s Matrix. Gesture 9 (3): 355?372. doi:10.1075/gest.9.3.05ken
Kita, S. (ed.) (2003). Pointing. Hillsdale, NJ: Lawrence Erlbaum.
Krauss, R. M. and Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In L. S. Messing and R. Campbell (eds) Gesture, Speech, and Sign, 93?116. Oxford: Oxford University Press.
Laakso, M. and Klippi, A. (1999). A closer look at the ?hint and guess’ sequences in aphasic conversation. Aphasiology 13 (4&5): 345?363. doi:10.1080/026870399402136
Le May, A., David, R. and Thomas, A. P. (1988). The use of spontaneous gesture by aphasic patients. Aphasiology 2 (2): 137?145. doi:10.1080/02687038808248899
Levelt, W. (1989). Speaking. Cambridge, MA: MIT Press.
McNeill, D. and Duncan S. (2000). Growth points in thinking-for-speaking. In D. McNeill (ed.), Language and Gesture, 141?161. Cambridge: Cambridge University Press.
Melinger, A. and Levelt, W. (2004). Gesture and the communicative intention of the speaker. Gesture 4 (2): 119–141. doi:10.1075/gest.4.2.02mel
Miller, N. (2006). Utility of arm and hand gestures in the treatment of aphasia: Some comments and expansions. International Journal of Speech-Language Pathology 8 (2): 143?148. doi:10.1080/14417040600667319
Milroy, L. and Perkins, L. (1992). Repair strategies in aphasic discourse. Towards a collaborative model. Clinical Linguistics and Phonetics 6 (1?2): 27?40. doi:10.3109/02699209208985517
Ozyurek, A., Willems, R. M., Kita, S. and Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience 19 (4): 605?616. doi:10.1162/jocn.2007.19.4.605
Power, E. and Code, C. (2006). Waving not drowning: Utilising gesture in the treatment of aphasia. Advances in Speech-Language Pathology 8 (2): 149–152.
Rose, M. L. (2006). The utility of arm and hand gestures in the treatment of aphasia. International Journal of Speech-Language Pathology 8 (2): 92–109. doi:10.1080/14417040600657948
Rose, M. and Douglas, J. (2001). The differential facilitatory effects of gesture and visualisation processes on object naming in aphasia. Aphasiology 15 (10&11): 977?990. doi:10.1080/02687040143000339
Rose M. and Douglas J. (2003). Limb apraxia, pantomime and lexical gesture in aphasic speakers: Preliminary findings. Aphasiology 17 (5): 453?464. doi:10.1080/02687030344000157
de Ruiter, J. (2000). The production of gesture and speech. In D. McNeill (ed.), Language and Gesture, 284–311. Cambridge: Cambridge University Press. doi:10.1080/14417040600667285
de Ruiter, J. (2006). Can gesticulation help aphasic people speak, or rather, communicate? Advances in Speech-Language Pathology 8 (2): 124–127.
Scharp, V. L., Tompkins, C. A. and Iverson, J. M.. (2007). Gesture and aphasia: Helping hands? Aphasiology 2 (6–8): 717–725. doi:10.1080/02687030701192273
Schegloff, E. A. (1984). On some gestures’ relation to talk. In J. M. Atkinson and J. Heritage (eds) Structures of Social Action, 266?296. Cambridge: Cambridge University Press.
Selting, M., Auer, P. et al. (2009). Gesprachsanalytisches Transkriptionssystem 2 (GAT 2). Zeitschrift für Gesprächsforschung 10 (http://www.gespraechsforschung-ozs.de/heft2009/heft2009.htm).
Simmons-Mackie, N. and Damico, J. S. (1997). Reformulation the definition of compensatory strategies in aphasia Aphasiology 11 (8): 761?781. doi:10.1080/02687039708250455
Streeck, J. (2009). Gesturecraft: The Manufacture of Meaning. Amsterdam: John Benjamins. doi:10.1075/gs.2
Stukenbrock, A. (2009). Referenz durch Zeigen: Zur Theorie der Deixis. Deutsche Sprache 37: 289?316.
Tomasello, M. (2008). Origins of Human Communication. Cambridge, MA: MIT Press.
Tompkins, C.A. and Scharp, V. L. (2006). Communicative value of self cues in aphasia: A re-evaluation. Aphasiology 20 (7): 684?704. doi:10.1080/02687030500334076
Willems, R. M. and Hagoort, P. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain & Language 101 (3): 278?89. doi:10.1016/j.bandl.2007.03.004
Auer, P. (2005). The limits of cooperation – speakership in conversations with aphasics, unpublished MS, University of Freiburg.
Bauer, A. (2009). Miteinander im Gespräch bleiben. Partizipation in aphasischen Alltags gesprächen. Radolfzell: Verlag fur Gesprachsforschung (http://www.verlag-gespraechsforschung.de/2009/bauer.htm).
Bauer, A. and Auer, P. (2009). Aphasie im familiären Alltag. Stuttgart: Thieme.
Bavelas, J. B., Gerwing J., Sutton, C. and Prevost, D. (2008). Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58 (2): 495?520. doi:10.1016/j.jml.2007.02.004
Damico, J. S., Wilson, B. T., Simmons-Mackie, N. N. and Tetnowski, J. A. (2008). Overcoming unintelligibility in aphasia: The impact of non-verbal interactive strategies. Clinical Linguistics & Phonetics 22 (10?11): 775?782. doi:10.1080/02699200801950518
Cohen, A. and Harrison, R. (1973). Intentionality in the use of hand illustrators in faceto-face communicative situations. Journal of Personality and Social Psychology 28 (2): 276?279. doi:10.1037/h0035792
Ekman, P. and Friesen, W. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1 (1): 49?89.
Feyereisen, P. (1993). Communicative behaviour in aphasia. In G. Blanken, J. Dittmann, H. Grimm, J. Marshall and C. W. Wallesch (eds) Linguistic Disorders and Pathologies. An International Handbook, 288?303. Berlin: Thieme.
Fillmore, C. (1997). Lectures on Deixis. Stanford, CA: CSLI Publications.
Goffman, E. (1979). Footing. Semiotica 25 (1?2): 1?29. doi:10.1515/semi.1979.25.1-2.1
Goodwin, C. (1986). Gestures as a resource for the organization of mutual orientation. Semiotica 62 (1?2): 29?49. doi:10.1515/semi.1986.62.1-2.29
Goodwin, C. (1995). Co-constructing meaning in conversations with an aphasic man. Research on Language and Social Interaction 28 (3): 233?260. doi:10.1207/s15327973rlsi2803_4
Goodwin, C. (2000). Gesture, aphasia and interaction. In D. McNeill (ed.) Language and Gesture, 85?95. Cambridge: Cambridge University Press.
Goodwin, C. (2003). Conversational frameworks for the accomplishment of meaning in aphasia. In C. Goodwin (ed.) Conversation and Brain Damage, 90?116. Oxford: Oxford University Press.
Goodwin, C. (2004). A competent speaker who can´t speak: The social life of aphasia. Journal of Linguistic Anthropology 14 (2): 151?170. doi:10.1525/jlin.2004.14.2.151
Goodwin, C. (2006). Human sociality as mutual orientation in a rich interactive environment: Multimodal utterances and pointing in aphasia. In N. Enfield and S. Levinson (eds) Roots of Human Sociality: Culture, Cognition and Human Interaction, 96?125. London: Berg.
Goodwin, C. (2007). Interactive footing. In E. Holt and R. Clift (eds) Reported Talk: Reported Speech in Interaction, 16?46. Cambridge: Cambridge University Press.
Hadar, U. and Butterworth, B. (1997). Iconic gestures, imagery and word retrieval in speech. Semiotica 115 (1?2): 147?172. doi:10.1515/semi.1997.115.1-2.147
Heath, C. (1982). Interactional participation: The coordination of gesture, speech and gaze. In V. D’Urso and P. Leonardi (eds) Discourse Analysis and Natural Rhetorics, 85?97. Padua: Cleup Editore.
Heath, C. (1992). Gesture´s discreet tasks: Multiple relevancies in visual conduct and the contextualisation of language. In P. Auer and A. Di Luzio (eds) The Contextualisation of Language, 101?127. Amsterdam: Benjamins.
Herrmann M., Reichle T., Lucius-Hoene G., Wallesch C. W. and Johannsen-Horbach H. (1988). Nonverbal communication as a compensative strategy for severely nonfluent aphasics? A quantitative approach. Brain and Language 33 (1): 41?54. doi:10.1016/0093-934X(88)90053-3
Holle, H., Gunter, T. C., Ruschemeyer, S. A., Hennenlotter, A. and Iacoboni, M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage 39 (4): 2010?2024. doi:10.1016/j.neuroimage.2007.10.055
Kendon, A. (2004). Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press.
Kendon, A. (2009). Language’s Matrix. Gesture 9 (3): 355?372. doi:10.1075/gest.9.3.05ken
Kita, S. (ed.) (2003). Pointing. Hillsdale, NJ: Lawrence Erlbaum.
Krauss, R. M. and Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In L. S. Messing and R. Campbell (eds) Gesture, Speech, and Sign, 93?116. Oxford: Oxford University Press.
Laakso, M. and Klippi, A. (1999). A closer look at the ?hint and guess’ sequences in aphasic conversation. Aphasiology 13 (4&5): 345?363. doi:10.1080/026870399402136
Le May, A., David, R. and Thomas, A. P. (1988). The use of spontaneous gesture by aphasic patients. Aphasiology 2 (2): 137?145. doi:10.1080/02687038808248899
Levelt, W. (1989). Speaking. Cambridge, MA: MIT Press.
McNeill, D. and Duncan S. (2000). Growth points in thinking-for-speaking. In D. McNeill (ed.), Language and Gesture, 141?161. Cambridge: Cambridge University Press.
Melinger, A. and Levelt, W. (2004). Gesture and the communicative intention of the speaker. Gesture 4 (2): 119–141. doi:10.1075/gest.4.2.02mel
Miller, N. (2006). Utility of arm and hand gestures in the treatment of aphasia: Some comments and expansions. International Journal of Speech-Language Pathology 8 (2): 143?148. doi:10.1080/14417040600667319
Milroy, L. and Perkins, L. (1992). Repair strategies in aphasic discourse. Towards a collaborative model. Clinical Linguistics and Phonetics 6 (1?2): 27?40. doi:10.3109/02699209208985517
Ozyurek, A., Willems, R. M., Kita, S. and Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience 19 (4): 605?616. doi:10.1162/jocn.2007.19.4.605
Power, E. and Code, C. (2006). Waving not drowning: Utilising gesture in the treatment of aphasia. Advances in Speech-Language Pathology 8 (2): 149–152.
Rose, M. L. (2006). The utility of arm and hand gestures in the treatment of aphasia. International Journal of Speech-Language Pathology 8 (2): 92–109. doi:10.1080/14417040600657948
Rose, M. and Douglas, J. (2001). The differential facilitatory effects of gesture and visualisation processes on object naming in aphasia. Aphasiology 15 (10&11): 977?990. doi:10.1080/02687040143000339
Rose M. and Douglas J. (2003). Limb apraxia, pantomime and lexical gesture in aphasic speakers: Preliminary findings. Aphasiology 17 (5): 453?464. doi:10.1080/02687030344000157
de Ruiter, J. (2000). The production of gesture and speech. In D. McNeill (ed.), Language and Gesture, 284–311. Cambridge: Cambridge University Press. doi:10.1080/14417040600667285
de Ruiter, J. (2006). Can gesticulation help aphasic people speak, or rather, communicate? Advances in Speech-Language Pathology 8 (2): 124–127.
Scharp, V. L., Tompkins, C. A. and Iverson, J. M.. (2007). Gesture and aphasia: Helping hands? Aphasiology 2 (6–8): 717–725. doi:10.1080/02687030701192273
Schegloff, E. A. (1984). On some gestures’ relation to talk. In J. M. Atkinson and J. Heritage (eds) Structures of Social Action, 266?296. Cambridge: Cambridge University Press.
Selting, M., Auer, P. et al. (2009). Gesprachsanalytisches Transkriptionssystem 2 (GAT 2). Zeitschrift für Gesprächsforschung 10 (http://www.gespraechsforschung-ozs.de/heft2009/heft2009.htm).
Simmons-Mackie, N. and Damico, J. S. (1997). Reformulation the definition of compensatory strategies in aphasia Aphasiology 11 (8): 761?781. doi:10.1080/02687039708250455
Streeck, J. (2009). Gesturecraft: The Manufacture of Meaning. Amsterdam: John Benjamins. doi:10.1075/gs.2
Stukenbrock, A. (2009). Referenz durch Zeigen: Zur Theorie der Deixis. Deutsche Sprache 37: 289?316.
Tomasello, M. (2008). Origins of Human Communication. Cambridge, MA: MIT Press.
Tompkins, C.A. and Scharp, V. L. (2006). Communicative value of self cues in aphasia: A re-evaluation. Aphasiology 20 (7): 684?704. doi:10.1080/02687030500334076
Willems, R. M. and Hagoort, P. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain & Language 101 (3): 278?89. doi:10.1016/j.bandl.2007.03.004
Published
2011-08-29
Issue
Section
Articles
How to Cite
Auer, P., & Bauer, A. (2011). Multimodality in aphasic conversation: Why gestures sometimes do not help. Journal of Interactional Research in Communication Disorders, 2(2), 215-243. https://doi.org/10.1558/jircd.v2i2.215
