The Effectiveness of Duolingo English Courses in Developing Reading and Listening Proficiency

Authors

  • Xiangying Jiang Duolingo
  • Ryan Peters Duolingo
  • Luke Plonsky Northern Arizona University
  • Bozena Pajak Duolingo

DOI:

https://doi.org/10.1558/cj.26704

Keywords:

Duolingo, efficacy, English, reading proficiency, listening proficiency, MALL

Abstract

This study evaluated the reading and listening proficiency levels of 245 English language learners who completed the Basic content (through Common European Framework of Reference for Languages [CEFR] level A2) of one of the following three Duolingo English courses: English for Japanese, Spanish, or Portuguese speakers. Participants self-reported having little to no prior proficiency in English and used Duolingo as their only learning tool. Their language skills were measured using the reading and listening sections of the STAMP 4S English Test from Avant Assessment. The results show that, on average, learners at the end of the Basic content (A2) scored Intermediate High in both reading and listening in all three courses. Given that Duolingo’s English courses are CEFR-aligned with an expected proficiency outcome of Intermediate Mid upon completion of the A2 content, it is noteworthy that participants across the three studied courses surpassed these expectations by achieving an additional American Council on the Teaching of Foreign Language (ACTFL) sublevel in both reading and listening. Such results present a robust and coherent body of evidence affirming the efficacy of Duolingo’s English courses in enhancing learners’ competencies in reading and listening comprehension.

Author Biographies

  • Xiangying Jiang, Duolingo

    Xiangying Jiang is a lead learning scientist at Duolingo. She works on learning assessment and efficacy research at Duolingo. She has a PhD in Applied Linguistics (Northern Arizona University, 2007). Prior to joining Duolingo in 2019, she was a faculty member at West Virginia University. Her work has appeared in journals such as The Modern Language Journal, Reading in a Foreign Language, Reading Psychology, and Foreign Language Annals.

  • Ryan Peters, Duolingo

    Ryan Peters is a learning scientist at Duolingo. He has a PhD in Speech, Language and Hearing Sciences (Purdue University, 2019). Prior to joining Duolingo in 2022, he held dual positions as a Postdoctoral Researcher in the Department of Psychology at the University of Texas at Austin and as a Museum Researcher at Thinkery, a children’s science museum in Austin. His research focused primarily on the interplay between attention and learning, with an emphasis on first and second language learning.

  • Luke Plonsky, Northern Arizona University

    Luke Plonsky is Associate Professor of Applied Linguistics at Northern Arizona University. His work, focusing primarily on second-language acquisition and research methods, has appeared in over 80 articles, book chapters, and books. Luke is Associate Editor of Studies in Second Language Acquisition, Managing Editor of Foreign Language Annals, and Co-Director of the IRIS Database.

  • Bozena Pajak, Duolingo

    Bozena Pajak holds a PhD in Linguistics (University of California, San Diego, 2012). Before joining Duolingo in 2015, she was a Research Associate and a Lecturer in Linguistics at Northwestern University, Illinois. Her research focused primarily on the acquisition of additional languages in adulthood. She is currently the Vice-President of Learning and Curriculum at Duolingo.

References

ACTFL. (2012). ACTFL proficiency guidelines. Retrieved June 2, 2023 from https://www.actfl.org/resources/actfl-proficiency-guidelines-2012

ACTFL. (n.d.). Assigning CEFR ratings to ACTFL assessments. Retrieved June 2, 2023 from https://www.actfl.org/uploads/files/general/Assigning_CEFR_Ratings_To_ACTFL_Assessments.pdf

Ajisoko, P. (2020). The use of Duolingo apps to improve English vocabulary learning. International Journal of Emerging Technologies in Learning, 15(7), 149–155. https://doi.org/10.3991/ijet.v15i07.13229

Ajisoko, P. (2022). Using Duolingo apps to improve English reading comprehension of engineering students in Universitas Borneo Tarakan. Exposure: Jurnal Pendidikan Bahasa Inggris [English Language Education Journal], 11(1), 1–6. https://doi.org/10.26618/exposure.v11i1.6452

Avant Assessment. (2023). STAMP scaled scores guide. Retrieved June 2, 2023 from https://avantassessment.com/stamp-scaled-scores-guide

Brown, K. (2023). CALL as a driver of equity, access, and outcomes in an increasingly connected world. Opening plenary at CALICO Annual Conference, June 7, 2003, University of Minnesota, Minneapolis.

Burston, J., & Giannakou, K. (2022). MALL language learning outcomes: A comprehensive meta-analysis 1994–2019. ReCALL, 34(2), 147–168. https://doi.org/10.1017/S0958344021000240

Bustillo, J., Rivera, C., Guzman, J. G., & Acosta, L. R. (2017). Benefits of using a mobile application in learning a foreign language. Sistemas Y Telematica, 15(40), 55–68. https://doi.org/10.18046/syt.v15i40.2391

Carpenter, S. K. (2020). Distributed practice/spacing effect. In L. Zhang (Ed.), Oxford research encyclopedia of education (pp. 1–20). New York: Oxford University Press. https://doi.org/10.1093/acrefore/9780190264093.013.859

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. New York: Cambridge University Press.

DeKeyser, R. M. (2015). Skill acquisition theory. In J. Williams & B. VanPatten (Eds.), Theories in second language acquisition: An introduction (2nd ed., pp. 94–112). New York: Routledge.

Fatmawati, I., Sudirman, A., & Munawaroh, T. (2023). The effect of using Duolingo application towards students’ grammar mastery in simple present tense. Journal of English Pedagogy and Applied Linguistics, 3(2), 13–21. https://doi.org/10.32627/jepal.v3i2.576

Heift, T., & Vyatkina, N. (2017). Technologies for teaching and learning L2 grammar. In C. Chapelle & S. Sauro (Eds.), The handbook of technology and second language teaching and learning (pp. 26–44). New York: John Wiley & Sons. https://doi.org/10.1002/9781118914069.ch3

James, K. K., & Mayer, R. E. (2019). Learning a second language by playing a game. Applied Cognitive Psychology, 33(4), 669–674. https://doi.org/10.1002/acp.3492

Jiang, X., Rollinson, J., Plonsky, L., Gustafson, E., & Pajak, B. (2021). Evaluating the reading and listening outcomes of beginning-level Duolingo courses. Foreign Language Annals, 54(4), 974–1002. https://doi.org/10.1111/flan.12600

Kessler, M., Loewen, S., & Gonulal, T. (2023). Mobile-assisted language learning with Babbel and Duolingo: Comparing L2 learning gains and user experience. Computer Assisted Language Learning, 36. https://doi.org/10.1080/09588221.2023.2215294

Kim, S. K., & Webb, S. (2022). The effects of spaced practice on second language learning: A meta-analysis. Language Learning, 72(1), 269–319. https://doi.org/10.1111/lang.12479

Krashen, S. (2014). Does Duolingo “trump” university-level language learning? International Journal of Foreign Language Teaching, 13–15. Retrieved June 2, 2023 from https://sdkrashen.com/content/articles/krashen-does-duolingo-trump.pdf

Lin, C.-H., & Warschauer, M. (2015). Online foreign language education: What are the proficiency outcomes? Modern Language Journal, 99(2), 394–397. https://doi.org/10.1111/modl.12234_1

Loewen, S., & Hui, B. (2021). Small samples in instructed second language acquisition research. Modern Language Journal, 105(1), 187–193. https://doi.org/10.1111/modl.12700

Loewen, S., Crowther, D., Isbell, D., Kim, K., Maloney, J., Miller, Z., & Rawal, H. (2019). Mobile-assisted language learning: A Duolingo case study. ReCALL, 31(3), 293–311. https://doi.org/10.1017/S0958344019000065

Loewen, S., Isbell, D. R., & Sporn, Z. (2020). The effectiveness of app-based language instruction for developing receptive linguistic knowledge and oral communicative ability. Foreign Language Annals, 53(2), 209–233. https://doi.org/10.1111/flan.12454

M Science. (2023). DUOL Q2’23 first look. Retrieved May 12, 2023 from https://mscience.com

Norouzian, R. (2020). Sample size planning in quantitative L2 research: A pragmatic approach. Studies in Second Language Acquisition, 42(4), 849–870. https://doi.org/10.1017/S0272263120000017

Norris, J. M., & Ortega, L. (2012). Assessing learner knowledge. In S. M. Gass & A. Mackey (Eds.), The Routledge handbook of second language acquisition (pp. 573–589). New York: Routledge.

Park, H. I., Solon, M., Dehghan-Chaleshtori, M., & Ghanbar, H. (2022). Proficiency reporting practices in SLA research: Have we made any progress? Language Learning, 72(1), 198–236. https://doi.org/10.1111/lang.12475

Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687. https://doi.org/10.1017/S0272263113000399

Plonsky, L. (2017). Quantitative research methods. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 505–521). New York: Routledge. https://doi.org/10.4324/9781315676968-28

Rachels, J. R., & Rockinson-Szapkiw, A. J. (2018). The effects of a mobile gamification app on elementary students’ Spanish achievement and self-efficacy. Computer Assisted Language Learning, 31(1–2), 72–89. https://doi.org/10.1080/09588221.2017.1382536

Rohrer, D. (2015). Student instruction should be distributed over long time periods. Educational Psychology Review, 27(4), 635–643. https://doi.org/10.1007/s10648-015-9332-4

Rolando, A. P. P., Gabriela, R. E. A., Carolina, M. S. E., & Antonio, A. R. M. (2019). Incidencia de Duolingo en el desarrollo de las habilidades comunicacionales verbales del idioma ingles a nivel de educación superior. European Scientific Journal, 15(16), 29–44. https://doi.org/10.19044/esj.2019.v15n16p29

Santos, V. (2022). The STAMP 4S English test reading and listening sections. Technical report shared by Avant Assessment.

Shortt, M., Tilak, S., Kuznetcova, I., Martens, B., & Akinkuolie, B. (2023). Gamification in mobile-assisted language learning: A systematic review of Duolingo literature from public release of 2012 to early 2020, Computer Assisted Language Learning, 36(3), 517–554. https://doi.org/10.1080/09588221.2021.1933540

Sudina, E., & Plonsky, L. (2023). The effects of frequency, duration, and intensity on L2 learning through Duolingo: A natural experiment. Journal of Second Language Studies, 7(1), 1–43. https://doi.org/10.1075/jsls.00021.plo

Tarone, E. (Ed.) (2015). Online foreign language education: What are the proficiency outcomes? Modern Language Journal, 99(2), 392–415. https://doi.org/10.1111/modl.12220

Thomas, M. (2006). Research synthesis and historiography: The case of assessment of second language proficiency. In L. Ortega & J. M. Norris (Eds.), Synthesizing research on language learning and teaching (pp. 279–298). Philadelphia: John Benjamins. https://doi.org/10.1075/lllt.13.13tho

Winke, P., Gass, S. M., Soneson, D., Rubio, F., & Hacking, J. F. (2020). Foreign language proficiency test data from three American universities [United States], 2014–2017. Inter-University Consortium for Political and Social Research.

Open access logo

Downloads

Published

2024-03-25

Issue

ADVANCE ACCESS TO FORTHCOMING ARTICLES

Section

Articles

How to Cite

Jiang, X., Peters, R., Plonsky, L., & Pajak, B. (2024). The Effectiveness of Duolingo English Courses in Developing Reading and Listening Proficiency. CALICO Journal. https://doi.org/10.1558/cj.26704