Building a City in the Sky
Multiliteracies in Immersive Virtual Reality
Keywords:affordances of IVR, embodiment, immersive virtual reality (IVR), multiliteracies, social VR
The conceptualization of multiliteracies initiated by the New London Group (NLG, 1996) emphasized the situated nature of language use as a socially complex network of multimodal engagement. Inspired by this view of language and literacy, computer-assisted language learning and second language acquisition scholars have advocated for a broader scope of second language learning to include the development of multiliteracies (Reinhardt & Thorne, 2019; Warner & Dupuy, 2018). In this conceptual article, we explore the potential affordances of immersive virtual reality (IVR) for the development of multiliteracies. As we attempt to construct a working theory of IVR as a catalyst for understanding and creating multiple forms of language in use, we draw on existing literature as well as on data from our multi-year study of multilingual adolescents engaging in multimodal activities using the Meta Quest headset. The interactions between adolescents and researchers were designed to maximize key IVR affordances (embodied cognition, presence, agency, and contextualization), and examples from the IVR sessions suggest how the adolescents are developing multiliteracies. We conclude with ideas for future research that focus on empowering L2 learners to express themselves as they develop the necessary knowledge and skills to engage in multimodal literacies.
Aikawa, T. (2021, June 2–6). Virtual reality as a new modality for language learning [Conference presentation]. CALICO Annual Conference 2021: Global Realities, Online.
Araiza-Alba, P., Keane, T., & Kaufman, J. (2022). Are we ready for virtual reality in K–12 classrooms? Technology, Pedagogy and Education, 31(4), 471–491. https://doi.org/10.1080/1475939X.2022.2033307
Arya, D. J. (2022). Into the void of discourse. Linguistics & Education, 68, 100964. https://doi.org/10.1016/j.linged.2021.100964
Arya, D. J., Sultana, S., Levine, S., Katz, D., Galisky, J., & Karimi, H. (2022). Raising critical readers in the 21st century: A case of assessing fourth-grade reading abilities and practices. Literacy Research: Theory, Method, and Practice, 71(1). https://doi.org/10.1177/23813377221117100
Bailenson, J. (2018). Experience on demand: What virtual reality is, how it works, and what it can do (1st ed.). New York: W. W. Norton & Company, Inc.
Cheng, A., Yang, L., & Andersen, E. (2017). Teaching language and culture with a virtual reality game. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 541–549). New York: Association for Computing Machinery. https://doi.org/10.1145/3025453.3025857
Chun, D. M., Karimi, H., & Sañosa, D. J. (2022). Traveling by headset: Immersive VR for language learning. CALICO Journal, 39(2), 129–149. https://doi.org/10.1558/cj.21306
Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. https://doi.org/10.1007/BF02299088
Dagenais, D., Toohey, K., Bennett Fox, A., & Singh, A. (2017). Multilingual and multimodal composition at school: ScribJab in action. Language and Education, 31(3), 263–282. https://doi.org/10.1080/09500782.2016.1261893
Dalgarno, B., & Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? Learning affordances of 3-D virtual environments. British Journal of Educational Technology, 41(1), 10–32. https://doi.org/10.1111/j.1467-8535.2009.01038.x
Dede, C. J., Jacobson, J., & Richards, J. (2017). Introduction: Virtual, augmented, and mixed realities in education. In D. Liu, C. Dede, R. Huang, & J. Richards (Eds.), Virtual, augmented, and mixed realities in education (pp. 1–16). Singapore: Springer. https://doi.org/10.1007/978-981-10-5490-7_1
Di Natale, A. F., Repetto, C., Riva, G., & Villani, D. (2020). Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. British Journal of Educational Technology, 51(6), 2006–2033. https://doi.org/10.1111/bjet.13030
Haeyen, S., Jans, N., & Heijman, J. (2021). The use of VR Tilt Brush in art and psychomotor therapy: An innovative perspective. Arts in Psychotherapy, 76, 101855. https://doi.org/10.1016/j.aip.2021.101855
Hovious, A., Shinas, V. H., & Harper, I. (2021). The compelling nature of transmedia storytelling: Empowering twenty first-century readers and writers through multimodality. Technology, Knowledge and Learning, 26(1), 215–229. https://doi.org/10.1007/s10758-020-09437-7
Huang, H. M., Rauch, U., & Liaw, S. S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education, 55(3), 1171–1182. https://doi.org/10.1016/j.compedu.2010.05.014
Huang, X., Zou, D., Cheng, G., & Xie, H. (2021). A systematic review of AR and VR enhanced language learning. Sustainability, 13(9), 4639. https://doi.org/10.3390/su13094639
Hull, G. A., & Nelson, M. E. (2005). Locating the semiotic power of multimodality. Written Communication, 22(2), 224–261. https://doi.org/10.1177/0741088304274170
Johnson-Glenberg, M. C. (2018). Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers in Robotics and AI, 5, 81. https://doi.org/10.3389/frobt.2018.00081
Kern, R. (2000). Literacy and language teaching. Oxford: Oxford University Press.
Kern, R. (2015). Language, literacy, and technology. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139567701
Kern, R. (2021). Twenty-five years of digital literacies in CALL. Language Learning & Technology, 25(3), 132–150. http://hdl.handle.net/10125/73453
Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19. https://doi.org/10.1007/BF02299087
Lan, Y.-J. (2021). Language learning in virtual reality: Theoretical foundations and empirical practices. In Y.-J. Lan & S. Grant (Eds.), Contextual language learning. Singapore: Springer. https://doi.org/10.1007/978-981-16-3416-1_1
Luo, H., Li, G., Feng, Q., Yang, Y., & Zuo, M. (2021). Virtual reality in K-12 and higher education: A systematic review of the literature from 2000 to 2019. Journal of Computer Assisted Learning, 37(3), 887–901. https://doi.org/10.1111/jcal.12538
Ma, M., & Zheng, H. (2011). Virtual reality and serious games in healthcare. In S. Brahnam & L. C. Jain (Eds.), Advanced computational intelligence paradigms in healthcare 6 (pp. 169–192). Berlin: Springer. https://doi.org/10.1007/978-3-642-17824-5_9
Makransky, G., & Petersen, G. B. (2021). The cognitive affective model of immersive learning (CAMIL): A theoretical research-based model of learning in immersive virtual reality. Educational Psychology Review, 33(3), 937–958. https://doi.org/10.1007/s10648-020-09586-2
Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60, 225–236. https://doi.org/10.1016/j.learninstruc.2017.12.007
Maul, A. (2017). Rethinking traditional methods of survey validation. Measurement: Inter-disciplinary Research and Perspectives, 15(2), 51–69. https://doi.org/10.1080/15366367.2017.1348108
McClung, N. A. (2018). Learning to queer text: Epiphanies from a family critical literacy practice. Reading Teacher, 71(4), 401–410. https://doi.org/10.1002/trtr.1640
McLuhan, M. (1964). Understanding media: The extensions of man. New York: McGraw-Hill.
Mennecke, B. E., Triplett, J. L., Hassall, L. M., Conde, Z. J., & Heer, R. (2011). An examination of a theory of embodied social presence in virtual worlds. Decision Sciences, 42(2), 413–450. https://doi.org/10.1111/j.1540-5915.2011.00317.x
Mills, N., Courtney, M., Dede, C., Dressen, A., & Gant, R. (2020). Culture and vision in virtual reality narratives. Foreign Language Annals, 53, 733–760. https://doi.org/10.1111/flan.12494
New London Group (1996). A pedagogy of multiliteracies: Designing social factors. Harvard Educational Review, 66(1), 60–92. https://doi.org/10.17763/haer.66.1.17370n67v22j160u
Ou Yang, F.-C., Lo, F.-Y. R., Chen Hsieh, J., & Wu, W.-C. V. (2020). Facilitating communicative ability of EFL learners via high-immersion virtual reality. Educational Technology & Society, 23(1), 30–49.
Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778. https://doi.org/10.1016/j.compedu.2019.103778
Reinhardt, J., & Thorne, S. (2019). Digital literacies as emergent multifarious repertoires. In N. Arnold & L. Ducate (Eds.), Engaging language learners through CALL: From theory and research to informed practice (pp. 208–239). Sheffield: Equinox.
Sablan, J. R. (2019). Can you really measure that? Combining critical race theory and quantitative methods. American Educational Research Journal, 56(1), 178–203. https://doi.org/10.3102/0002831218798325
Selverian, M. E. M., & Lombard, M. (2010). Telepresence: A “real” component in a model to make human-computer interface factors meaningful in the virtual learning environment. Themes in Science and Technology Education, 2(1–2), 31–58.
Sherman, W., & Craig, A. (2003). Understanding virtual reality: Interface, application, and design. Presence, Teleoperators and Virtual Environments, 12, 441–442. https://doi.org/10.1162/105474603322391668
Slater, M., & Sanchez-Vives, M. V. (2014). Transcending the self in immersive virtual reality. Computer, 47(7), 24–30. https://doi.org/10.1109/MC.2014.198
Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 6(6), 603–616. https://doi.org/10.1162/pres.1918.104.22.1683
Street, B. (2003). What’s “new” in new literacy studies? Critical approaches to literacy in theory and practice. Current Issues in Comparative Education, 5(2), 77–91.
Swaffar, J. K., & Arens, K. (2005). Remapping the foreign language curriculum: An approach through multiple literacies. New York: Modern Language Association of America. https://doi.org/10.2307/4143872
Taguchi, N. (2021). Application of immersive virtual reality (VR) to pragmatics data collection methods: Insights from interviews. CALICO Journal, 38(2), 181–201. https://doi.org/10.1558/cj.41136
Tai, T.-Y., Chen, H. H.-J., & Todd, G. (2020). The impact of a virtual reality app on adolescent EFL learners’ vocabulary learning. Computer Assisted Language Learning, 35(4), 892–917. https://doi.org/10.1080/09588221.2020.1752735
Tan, L., Zammit, K., D’warte, J., & Gearside, A. (2020). Assessing multimodal literacies in practice: A critical review of its implementations in educational settings. Language and Education, 34(2), 97–114. https://doi.org/10.1080/09500782.2019.1708926
Tilhou, R., Taylor, V., & Crompton, H. (2020). 3D virtual reality in K-12 education: A thematic systematic review. In S. Yu, M. Ally, & A. Tsinakos (Eds.), Emerging technologies and pedagogies in the curriculum (pp. 169–184). Bridging human and machine: Future education with intelligence book series. Singapore: Springer. https://doi.org/10.1007/978-981-15-0618-5_10
Vázquez, C., Xia, L., Aikawa, T., & Maes, P. (2018). Words in motion: Kinesthetic language learning in virtual reality (pp. 272–276). IEEE 18th International Conference on Advanced Learning Technologies (ICALT). https://doi.org/10.1109/ICALT.2018.00069
Warner, C., & Dupuy, B. (2018). Moving toward multiliteracies in foreign language teaching: Past and present perspectives … and beyond. Foreign Language Annals 51, 116–128. https://doi.org/10.1111/flan.12316
Wu, B., Yu, X., & Gu, X. (2020). Effectiveness of immersive virtual reality using head-mounted displays on learning performance: A meta-analysis. British Journal of Educational Technology, 51(6), 1991–2005. https://doi.org/10.1111/bjet.13023
Yuan, C., Wang, L., & Eagle, J. (2019). Empowering English language learners through digital literacies: Research, complexities, and implications. Media and Communication, 7(2), 128–136. https://doi.org/10.17645/mac.v7i2.1912