Building a City in the Sky

Multiliteracies in Immersive Virtual Reality

Authors

  • Honeiah Karimi University of California, Santa Barbara
  • David Joshua Sañosa University of California, Santa Barbara
  • Kevin Hernandez Rios University of California, Santa Barbara
  • Phoebe Tran University of California, Santa Barbara
  • Dorothy M Chun University of California, Santa Barbara
  • Richert Wang University of California, Santa Barbara
  • Diana J Arya University of California, Santa Barbara

DOI:

https://doi.org/10.1558/cj.22838

Keywords:

affordances of IVR, embodiment, immersive virtual reality (IVR), multiliteracies, social VR

Abstract

The conceptualization of multiliteracies initiated by the New London Group (NLG, 1996) emphasized the situated nature of language use as a socially complex network of multimodal engagement. Inspired by this view of language and literacy, computer-assisted language learning and second language acquisition scholars have advocated for a broader scope of second language learning to include the development of multiliteracies (Reinhardt & Thorne, 2019; Warner & Dupuy, 2018). In this conceptual article, we explore the potential affordances of immersive virtual reality (IVR) for the development of multiliteracies. As we attempt to construct a working theory of IVR as a catalyst for understanding and creating multiple forms of language in use, we draw on existing literature as well as on data from our multi-year study of multilingual adolescents engaging in multimodal activities using the Meta Quest headset. The interactions between adolescents and researchers were designed to maximize key IVR affordances (embodied cognition, presence, agency, and contextualization), and examples from the IVR sessions suggest how the adolescents are developing multiliteracies. We conclude with ideas for future research that focus on empowering L2 learners to express themselves as they develop the necessary knowledge and skills to engage in multimodal literacies.

Author Biographies

Honeiah Karimi, University of California, Santa Barbara

Honeiah Karimi is a PhD student in Education with an emphasis on Quantitative Methods at University of California, Santa Barbara (UCSB). Her research interests include VR for language learning, data science, and educational measurement. She received a BA in Linguistics from UC Riverside and an MA in Linguistics from California State University, Fullerton.

David Joshua Sañosa, University of California, Santa Barbara

David Joshua Sañosa has been working with VR research/development since 2013. He is currently a PhD candidate studying at UCSB, focusing on sociocultural approaches to technology-mediated creative learning activity. He is a researcher and lead VR designer for the Community-Based Literacies (CBL) virtual reality project.

Kevin Hernandez Rios, University of California, Santa Barbara

Kevin Hernandez Rios has been part of the virtual reality project since 2020 as a research assistant and graduated from UCSB in 2021 with a BA in Sociology and a minor in History and Applied Psychology. He is currently a graduate student at the University of California, Davis. His research interests are in technology,
interconnectivity, transportation, and their functionalities together.

Phoebe Tran, University of California, Santa Barbara

Phoebe Tran has been a part of the virtual reality project as a research assistant at UCSB since 2019. In 2021, she graduated from UCSB with a BA in Sociology and Asian American Studies with a minor in Applied Psychology.

Dorothy M Chun, University of California, Santa Barbara

Dorothy M. Chun is Professor Emerita of Applied Linguistics at UCSB and Co-editor of Language Learning & Technology. Her research areas include L2 pronunciation and intonation, L2 vocabulary acquisition, CALL, and telecollaboration for intercultural learning. She is currently on a team developing VR games for learning.

Richert Wang, University of California, Santa Barbara

Richert Wang is Associate Teaching Professor in UCSB’s Computer Science Department and College of Creative Studies Computing program. His research focus is in computer science education and interdisciplinary applications of computer science.

Diana J Arya, University of California, Santa Barbara

Diana J. Arya is Associate Professor and Faculty Director in Education at UCSB. Their research interests converge on the intersection of multiliteracies and inter-disciplinary learning across K–20 contexts.

References

Aikawa, T. (2021, June 2–6). Virtual reality as a new modality for language learning [Conference presentation]. CALICO Annual Conference 2021: Global Realities, Online.

Araiza-Alba, P., Keane, T., & Kaufman, J. (2022). Are we ready for virtual reality in K–12 classrooms? Technology, Pedagogy and Education, 31(4), 471–491. https://doi.org/10.1080/1475939X.2022.2033307

Arya, D. J. (2022). Into the void of discourse. Linguistics & Education, 68, 100964. https://doi.org/10.1016/j.linged.2021.100964

Arya, D. J., Sultana, S., Levine, S., Katz, D., Galisky, J., & Karimi, H. (2022). Raising critical readers in the 21st century: A case of assessing fourth-grade reading abilities and practices. Literacy Research: Theory, Method, and Practice, 71(1). https://doi.org/10.1177/23813377221117100

Bailenson, J. (2018). Experience on demand: What virtual reality is, how it works, and what it can do (1st ed.). New York: W. W. Norton & Company, Inc.

Cheng, A., Yang, L., & Andersen, E. (2017). Teaching language and culture with a virtual reality game. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 541–549). New York: Association for Computing Machinery. https://doi.org/10.1145/3025453.3025857

Chun, D. M., Karimi, H., & Sañosa, D. J. (2022). Traveling by headset: Immersive VR for language learning. CALICO Journal, 39(2), 129–149. https://doi.org/10.1558/cj.21306

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. https://doi.org/10.1007/BF02299088

Dagenais, D., Toohey, K., Bennett Fox, A., & Singh, A. (2017). Multilingual and multimodal composition at school: ScribJab in action. Language and Education, 31(3), 263–282. https://doi.org/10.1080/09500782.2016.1261893

Dalgarno, B., & Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? Learning affordances of 3-D virtual environments. British Journal of Educational Technology, 41(1), 10–32. https://doi.org/10.1111/j.1467-8535.2009.01038.x

Dede, C. J., Jacobson, J., & Richards, J. (2017). Introduction: Virtual, augmented, and mixed realities in education. In D. Liu, C. Dede, R. Huang, & J. Richards (Eds.), Virtual, augmented, and mixed realities in education (pp. 1–16). Singapore: Springer. https://doi.org/10.1007/978-981-10-5490-7_1

Di Natale, A. F., Repetto, C., Riva, G., & Villani, D. (2020). Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. British Journal of Educational Technology, 51(6), 2006–2033. https://doi.org/10.1111/bjet.13030

Haeyen, S., Jans, N., & Heijman, J. (2021). The use of VR Tilt Brush in art and psychomotor therapy: An innovative perspective. Arts in Psychotherapy, 76, 101855. https://doi.org/10.1016/j.aip.2021.101855

Hovious, A., Shinas, V. H., & Harper, I. (2021). The compelling nature of transmedia storytelling: Empowering twenty first-century readers and writers through multimodality. Technology, Knowledge and Learning, 26(1), 215–229. https://doi.org/10.1007/s10758-020-09437-7

Huang, H. M., Rauch, U., & Liaw, S. S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education, 55(3), 1171–1182. https://doi.org/10.1016/j.compedu.2010.05.014

Huang, X., Zou, D., Cheng, G., & Xie, H. (2021). A systematic review of AR and VR enhanced language learning. Sustainability, 13(9), 4639. https://doi.org/10.3390/su13094639

Hull, G. A., & Nelson, M. E. (2005). Locating the semiotic power of multimodality. Written Communication, 22(2), 224–261. https://doi.org/10.1177/0741088304274170

Johnson-Glenberg, M. C. (2018). Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers in Robotics and AI, 5, 81. https://doi.org/10.3389/frobt.2018.00081

Kern, R. (2000). Literacy and language teaching. Oxford: Oxford University Press.

Kern, R. (2015). Language, literacy, and technology. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139567701

Kern, R. (2021). Twenty-five years of digital literacies in CALL. Language Learning & Technology, 25(3), 132–150. http://hdl.handle.net/10125/73453

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19. https://doi.org/10.1007/BF02299087

Lan, Y.-J. (2021). Language learning in virtual reality: Theoretical foundations and empirical practices. In Y.-J. Lan & S. Grant (Eds.), Contextual language learning. Singapore: Springer. https://doi.org/10.1007/978-981-16-3416-1_1

Luo, H., Li, G., Feng, Q., Yang, Y., & Zuo, M. (2021). Virtual reality in K-12 and higher education: A systematic review of the literature from 2000 to 2019. Journal of Computer Assisted Learning, 37(3), 887–901. https://doi.org/10.1111/jcal.12538

Ma, M., & Zheng, H. (2011). Virtual reality and serious games in healthcare. In S. Brahnam & L. C. Jain (Eds.), Advanced computational intelligence paradigms in healthcare 6 (pp. 169–192). Berlin: Springer. https://doi.org/10.1007/978-3-642-17824-5_9

Makransky, G., & Petersen, G. B. (2021). The cognitive affective model of immersive learning (CAMIL): A theoretical research-based model of learning in immersive virtual reality. Educational Psychology Review, 33(3), 937–958. https://doi.org/10.1007/s10648-020-09586-2

Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60, 225–236. https://doi.org/10.1016/j.learninstruc.2017.12.007

Maul, A. (2017). Rethinking traditional methods of survey validation. Measurement: Inter-disciplinary Research and Perspectives, 15(2), 51–69. https://doi.org/10.1080/15366367.2017.1348108

McClung, N. A. (2018). Learning to queer text: Epiphanies from a family critical literacy practice. Reading Teacher, 71(4), 401–410. https://doi.org/10.1002/trtr.1640

McLuhan, M. (1964). Understanding media: The extensions of man. New York: McGraw-Hill.

Mennecke, B. E., Triplett, J. L., Hassall, L. M., Conde, Z. J., & Heer, R. (2011). An examination of a theory of embodied social presence in virtual worlds. Decision Sciences, 42(2), 413–450. https://doi.org/10.1111/j.1540-5915.2011.00317.x

Mills, N., Courtney, M., Dede, C., Dressen, A., & Gant, R. (2020). Culture and vision in virtual reality narratives. Foreign Language Annals, 53, 733–760. https://doi.org/10.1111/flan.12494

New London Group (1996). A pedagogy of multiliteracies: Designing social factors. Harvard Educational Review, 66(1), 60–92. https://doi.org/10.17763/haer.66.1.17370n67v22j160u

Ou Yang, F.-C., Lo, F.-Y. R., Chen Hsieh, J., & Wu, W.-C. V. (2020). Facilitating communicative ability of EFL learners via high-immersion virtual reality. Educational Technology & Society, 23(1), 30–49.

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778. https://doi.org/10.1016/j.compedu.2019.103778

Reinhardt, J., & Thorne, S. (2019). Digital literacies as emergent multifarious repertoires. In N. Arnold & L. Ducate (Eds.), Engaging language learners through CALL: From theory and research to informed practice (pp. 208–239). Sheffield: Equinox.

Sablan, J. R. (2019). Can you really measure that? Combining critical race theory and quantitative methods. American Educational Research Journal, 56(1), 178–203. https://doi.org/10.3102/0002831218798325

Selverian, M. E. M., & Lombard, M. (2010). Telepresence: A “real” component in a model to make human-computer interface factors meaningful in the virtual learning environment. Themes in Science and Technology Education, 2(1–2), 31–58.

Sherman, W., & Craig, A. (2003). Understanding virtual reality: Interface, application, and design. Presence, Teleoperators and Virtual Environments, 12, 441–442. https://doi.org/10.1162/105474603322391668

Slater, M., & Sanchez-Vives, M. V. (2014). Transcending the self in immersive virtual reality. Computer, 47(7), 24–30. https://doi.org/10.1109/MC.2014.198

Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 6(6), 603–616. https://doi.org/10.1162/pres.1997.6.6.603

Street, B. (2003). What’s “new” in new literacy studies? Critical approaches to literacy in theory and practice. Current Issues in Comparative Education, 5(2), 77–91.

Swaffar, J. K., & Arens, K. (2005). Remapping the foreign language curriculum: An approach through multiple literacies. New York: Modern Language Association of America. https://doi.org/10.2307/4143872

Taguchi, N. (2021). Application of immersive virtual reality (VR) to pragmatics data collection methods: Insights from interviews. CALICO Journal, 38(2), 181–201. https://doi.org/10.1558/cj.41136

Tai, T.-Y., Chen, H. H.-J., & Todd, G. (2020). The impact of a virtual reality app on adolescent EFL learners’ vocabulary learning. Computer Assisted Language Learning, 35(4), 892–917. https://doi.org/10.1080/09588221.2020.1752735

Tan, L., Zammit, K., D’warte, J., & Gearside, A. (2020). Assessing multimodal literacies in practice: A critical review of its implementations in educational settings. Language and Education, 34(2), 97–114. https://doi.org/10.1080/09500782.2019.1708926

Tilhou, R., Taylor, V., & Crompton, H. (2020). 3D virtual reality in K-12 education: A thematic systematic review. In S. Yu, M. Ally, & A. Tsinakos (Eds.), Emerging technologies and pedagogies in the curriculum (pp. 169–184). Bridging human and machine: Future education with intelligence book series. Singapore: Springer. https://doi.org/10.1007/978-981-15-0618-5_10

Vázquez, C., Xia, L., Aikawa, T., & Maes, P. (2018). Words in motion: Kinesthetic language learning in virtual reality (pp. 272–276). IEEE 18th International Conference on Advanced Learning Technologies (ICALT). https://doi.org/10.1109/ICALT.2018.00069

Warner, C., & Dupuy, B. (2018). Moving toward multiliteracies in foreign language teaching: Past and present perspectives … and beyond. Foreign Language Annals 51, 116–128. https://doi.org/10.1111/flan.12316

Wu, B., Yu, X., & Gu, X. (2020). Effectiveness of immersive virtual reality using head-mounted displays on learning performance: A meta-analysis. British Journal of Educational Technology, 51(6), 1991–2005. https://doi.org/10.1111/bjet.13023

Yuan, C., Wang, L., & Eagle, J. (2019). Empowering English language learners through digital literacies: Research, complexities, and implications. Media and Communication, 7(2), 128–136. https://doi.org/10.17645/mac.v7i2.1912

Downloads

Published

2023-02-10

How to Cite

Karimi, H., Sañosa, D. J., Hernandez Rios, K., Tran, P., Chun, D. M., Wang, R., & Arya, D. J. (2023). Building a City in the Sky: Multiliteracies in Immersive Virtual Reality. CALICO Journal, 40(1), 24–44. https://doi.org/10.1558/cj.22838

Issue

Vol. 40 No. 1 (2023): XR: Crossing Reality to Enhance Language Learning

Section

Articles
Crossref
Scopus
Google Scholar
Europe PMC

Most read articles by the same author(s)