Exploring the Stability of Second Language Speech Ratings through Task Practice in Bilinguals’ Two Languages


  • Kym Taylor Reid Concordia University, Montreal
  • Mary Grantham O’Brien University of Calgary
  • Pavel Trofimovich Concordia University, Montreal
  • Aki Tsunemoto Concordia University, Montreal




speech rating, perspective taking, assessment, L2 English, rating stability, task practice


This study examined the stability of second language (L2) speech ratings as a function of bilingual raters engaging in perspective taking through practice of the target speaking task. Thirty English-dominant English–French bilinguals evaluated narratives from 40 French speakers of L2 English for segmental errors, intonation, and flow. Before providing the ratings, 20 raters practiced the task in their more or less dominant language (10 per group), while 10 baseline raters performed no practice. Whereas the English practice enhanced raters’ evaluations for two of the three dimensions relative to baseline raters’ assessments, the French practice did not impact rating stability.


Baquiran C. L. C. & Nicoladis, E. (2020). A doctorâ??s foreign accent affects perceptions of competence. Health Communication, 35, 726â??30. https://doi.org/10.1080/10410236.2019.1584779

Beukeboom, C. J. & Semin, G. R. (2006). How mood turns on language. Journal of Experimental Social Psychology, 42, 553â??66. https://doi.org/10.1016/j.jesp.2005.09.005

Boland, R. J. & Tenkasi, R. V. (1995). Perspective making and perspective taking in communities of knowing. Organization Science, 6, 350â??72. https://doi.org/10.1287/orsc.6.4.350

Campbell, K. S., Mothersbaugh, D. L., Brammer, C. & Taylor, T. (2001). Peer versus self assessment of oral business presentation performance. Business Communication Quart­erly, 64, 23â??40. https://doi.org/10.1177/108056990106400303

Campbell-Kibler, K. (2007). Accent, (ING), and the social logic of listener perceptions. American Speech, 82, 32â??64. https://doi.org/10.1215/00031283-2007-002

Davis, L. (2016). The influence of training and experience on rater performance in scoring spoken language. Language Testing, 33, 117â??35. https://doi.org/10.1177/0265532215582282

Deprez-Sims A.-S. & Morris, S. B. (2010). Accents in the workplace: Their effects during a job interview. International Journal of Psychology, 45, 417â??26. https://doi.org/10.1080/00207594.2010.499950

Derwing, T. M. & Munro, M. J. (2009). Comprehensibility as a factor in listener interaction preferences: Implications for the workplace. Canadian Modern Language Review, 66, 181â??202. https://doi.org/10.3138/cmlr.66.2.181

Derwing, T. M. & Munro, M. J. (2015). Pronunciation fundamentals: Evidence-based perspectives for L2 teaching and research. Amsterdam: John Benjamins. https://doi.org/10.3138/cmlr.418

Derwing, T. M., Rossiter, M. J., Munro, M. J. & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54, 655â??79. https://doi.org/10.1075/lllt.42

Dewaele, J.-M. (2002). Psychological and sociodemographic correlates of communicative anxiety in L2 and L3 production. International Journal of Bilingualism, 6, 23â??38. https://doi.org/10.1177/13670069020060010201

Gass, S. & Varonis, E. M. (1984). The effect of familiarity on the comprehensibility of non­native speech. Language Learning, 34, 65â??89. https://doi.org/10.1111/j.1467-1770.1984.tb00996.x

Gatbonton, E. & Trofimovich, P. (2008). The ethnic group affiliation and L2 proficiency link: Empirical evidence. Language Awareness, 17, 229â??48. https://doi.org/10.1080/09658410802146867

Halim, M. L., Moy, K. H. & Yoshikawa, H. (2017). Perceived ethnic and language-based discrimination and Latina immigrant womenâ??s health. Journal of Health Psychology, 22, 68â??78. https://doi.org/10.1177/1359105315595121

Hansen, K., RakiÄ?, T. & Steffens, M. C. (2014). When actions speak louder than words: Preventing discrimination of nonstandard speakers. Journal of Language and Social Psychology, 33, 68â??77. https://doi.org/10.1177/0261927x13499761

Holliday, A. (2006). Native-speakerism. ELT Journal, 60, 385â??7. https://doi.org/10.1093/elt/ccl030

Isaacs, T. & Thomson, R. I. (2013). Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly, 10, 135â??59. https://doi.org/10.1080/15434303.2013.769545

Isaacs, T. & Trofimovich, P. (2011). Phonological memory, attention control, and musical ability: Effects of individual differences on rater judgments of second language speech. Applied Psycholinguistics, 32, 113â??40. https://doi.org/10.1017/s0142716410000317

Kang, O. (2008). Ratings of L2 oral performance in English: Relative impact of rater characteristics and acoustic measures of accentedness. Doctoral dissertation, University of Georgia, Athens, GA. Retrieved from https://getd.libs.uga.edu/pdfs/kang_okim_200805_phd.pdf.

Kang, O. & Rubin, D. L. (2009). Reverse linguistic stereotyping: Measuring the effect of listener expectations on speech evaluation. Journal of Language and Social Psychology, 28, 441â??56. https://doi.org/10.1177/0261927x09341950

Luoma-aho, V., Pirttimäki, T., Maity, D., Munnukka, J. & Reinikainen, H. (2019). Primed authenticity: How priming impacts authenticity perception of social media influencers. International Journal of Strategic Communication, 13, 352â??65. https://doi.org/10.1080/1553118x.2019.1617716

MacIntyre, P. D. & Gardner, R. C. (1994). The subtle effects of language anxiety on cognitive processing in the second language. Language Learning, 44, 283â??305. https://doi.org/10.1111/j.1467-1770.1994.tb01103.x

Pérez-Ramón, R., Cooke, M. & García Lecumberri, M. L. (2020). Is segmental foreign accent perceived categorically? Speech Communication, 117, 28â??37. https://doi.org/10.1016/j.specom.2020.01.003

Plonsky, L. & Oswald, F. L. (2014). How big is â??big?â?? Interpreting effect sizes in L2 research? Language Learning, 64, 878â??912. https://doi.org/10.1111/lang.12079

Saito, K., Trofimovich, P. & Isaacs, T. (2017). Using listener judgements to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and gen­eralization study. Applied Linguistics, 38, 439â??62. https://doi.org/10.1093/applin/amv047

Salmanowitz, N. (2016). Unconventional methods for a traditional setting: The use of virtual reality to reduce implicit racial bias in the courtroom. UNHL Review, 15, 117â??60.

Sereno, J., Lammers, L. & Jongman, A. (2016). The relative contribution of segments and intonation to the perception of foreign-accented speech. Applied Psycholinguistics, 37, 303â??22. https://doi.org/10.1017/s0142716414000575

Staples, S., Kang, O. & Wittner, E. (2014). Considering interlocutors in university discourse communities: Impacting US undergraduatesâ?? perceptions of ITAs through a structured contact program. English for Specific Purposes, 35, 54â??65. https://doi.org/10.1016/j.esp.2014.02.002

Steele, C. M. & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797â??811. https://doi.org/10.1037/0022-3514.69.5.797

Stewart, M. A. (2010). Walking in my studentsâ?? shoes: An ESL teacher brings theory to life in order to transform her classroom. Networks, 12, 1â??6. https://doi.org/10.4148/2470-6353.1095

Taylor Reid, K., Trofimovich, P. & Oâ??Brien, M. G. (2019). Social attitudes and speech ratings: Effects of positive and negative bias on multiage listenersâ?? judgments of second language speech. Studies in Second Language Acquisition, 41, 419â??42. https://doi.org/10.1017/s0272263118000244

Taylor Reid, K., Trofimovich, P., Oâ??Brien, M. G. and Tsunemoto, A. (under review). Using task practice to reduce effects of social priming on ratings of second language accent and comprehensibility. Manuscript submitted for publication.

Timming, A. R. (2017). The effect of foreign accent on employability: A study of the aural dimensions of aesthetic labour in customer-facing and non-customer-facing jobs. Work, Employment and Society, 31, 409â??28. https://doi.org/10.1177/0950017016630260

Weyant, J. M. (2007). Perspective taking as a means of reducing negative stereotyping of individuals who speak English as a second language. Journal of Applied Social Psychology, 37, 703â??16. https://doi.org/10.1111/j.1559-1816.2007.00181.x

Zhang, Y. (2017). Walking a mile in their shoes: Developing pre-service music teachersâ?? empathy for ELL students. International Journal of Music Education, 35, 425â??34. https://doi.org/10.1177/0255761416647191





Research Notes

How to Cite

Taylor Reid, Kym, Mary Grantham O’Brien, Pavel Trofimovich, and Aki Tsunemoto. 2020. “Exploring the Stability of Second Language Speech Ratings through Task Practice in Bilinguals’ Two Languages”. Journal of Monolingual and Bilingual Speech 2 (2): 315–329. https://doi.org/10.1558/jmbs.17714.