Empirical CALL evaluation
The Relationship Between Learning Process and Learning Outcome
DOI:
https://doi.org/10.1558/cj.v26i1.108-122Keywords:
Empirical CALL Evaluation, Learning Process, Learning OutcomeAbstract
CALL evaluation is important because it is the most efficient means to prove CALL effectiveness. While both learning process and learning outcome should be investigated in empirical evaluation, the precise relationship between the two needs to be examined closely. Only by doing so can we identify useful CALL design features that facilitate relevant user-computer interaction which can lead to an improved learning outcome. This study examined how certain user actions can affect or predict receptive/productive vocabulary retention in a computer-assisted vocabulary learning (CAVL) program. Results showed that only the user actions which contributed to identifiable levels of vocabulary processing--those at which (a) the initial meaning-form mappings of the vocabulary items are established and (b) the vocabulary items are practiced in different contexts--significantly accounted for vocabulary retention. In addition, user actions accounted for receptive vocabulary retention better than for productive vocabulary retention.
References
Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning & Technology, 5(1), 202-232. Retrieved June 2, 2008, from http://llt.msu.edu/vol5num1/alseghayer/default.html
Brace, N., Kemp, R., & Snelgar, R. (2006). SPSS for psychologists. Hampshire, NY: Palgrave Macmillan.
Chapelle, C. (2001). Computer applications in second language acquisition. Cambridge: Cambridge University Press.
Chapelle, C. (2003). English language learning and technology. Amsterdam, Philadelphia: John Benjamins Publishing Company.
Chapelle, C., Jamieson, J., & Park, Y. (1996). Second language classroom research traditions: How does CALL fit? In M. Pennington (Ed.), The power of CALL (pp. 33-53). Houston, TX: Athelstan.
Chapelle, C., & Mizuno, S. (1989). Student’s strategies with learner-controlled CALL. CALICO Journal, 7(2), 25-47. Retrieved June 2, 2008, from https://calico.org/p-5-Calico%20Journal.html
Chun, D. M., & Plass, J. J. (1996). Effects of multimedia annotations on vocabulary acquisition. The Modern Language Journal, 80(2), 183-198.
Colpaert, J. (2004). Design of online interactive language courseware: Conceptualisation, specification and prototyping. Research into the impact of linguistic-didactic functionality on software architecture. Unpublished doctoral dissertation, University of Antwerp, Belgium. Retrieved June 8, 2005, from http://www.didascalia.be/doc-design.pdf
De Ridder, I. (2002). Visible or invisible links: Does the highlighting of hyperlinks affect incidental vocabulary learning, text comprehension, and the reading process? Language Learning & Technology, 6(1), 123-146. Retrieved June 2, 2008, from http://llt.msu.edu/vol6num1/deridder/default.html
Desmarais, L., Duquette, L., Renié, D., & Laurier, M. (1998). Evaluating learning and interaction in a multimedia environment. Computers and the Humanities, 22(1), 1-23.
Garrett, N. (1991). Technology in the service of language learning: Trends and issues. The Modern Language Journal, 75(1), 74-101.
Goodfellow, R. (1994). A computer-based strategy for foreign-language vocabulary learning. Unpublished doctoral dissertation, Open University, Milton Keynes, UK.
Goodfellow, R. (1999). Evaluating performance, approach and outcome. In K. Cameron (Ed.), CALL: Media, design, and applications (pp. 109-140). Lisse, The Netherlands: Swets & Zeitlinger.
Grace, C. (1998). Personality type, tolerance of ambiguity, and vocabulary retention in CALL. CALICO Journal, 15(1-3), 19-44. Retrieved June 2, 2008, from https://calico.org/p-5-Calico%20Journal.html
Groot, P. J. M. (2000). Computer assisted second language vocabulary acquisition. Language Learning & Technology, 4(1), 60-81. Retrieved June 2, 2008, from http://llt.msu.edu/vol4num1/groot/default.html
Hegelheimer, V., & Tower, D. (2004). Using CALL in the classroom: Analyzing student interaction in an authentic classroom. System, 32(2), 185-205.
Hémard, D. (1997). Design principles and guidelines for authoring hypermedia language learning applications. System, 25(1), 9-27.
Hémard, D. (2004). Enhancing online CALL design: The case for evaluation. ReCALL, 16(2), 502-529.
Hwu, F. (2003). Learners’ behaviors in computer-based input activities elicited through tracking technologies. Computer Assisted Language Learning, 16(1), 5-29.
Hwu, F. (2005, May). The use of learner-behavior tracking technology in CALL studies: A review of research objectives, methods, and tracking data analysis techniques. Paper presented at the annual CALICO conference, East Lansing, MI.
Jamieson, J., & Chapelle, C. A. (1987). Working styles on computers as evidence of second language learning strategies. Language Learning, 37(3), 523-544.
Jamieson, J., Chapelle, C., & Preiss, S. (2005). CALL evaluation by developers, a teacher, and students. CALICO Journal, 23(1), 93-138. Retrieved June 2, 2008, from https://calico.org/p-5-Calico%20Journal.html
Jones, L. C., & Plass, J. J. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern Language Journal, 86(4), 546-561.
Laufer, B. (1998). The development of passive and active vocabulary in a second language: Same or different? Applied Linguistics, 19(2), 255-271.
Laufer, B., & Hill, M. (2000). What lexical information do L2 learners select in a CALL dictionary and how does it affect word retention? Language Learning & Technology, 3(2), 58-76. Retrieved June 2, 2008, from http://llt.msu.edu/vol3num2/laufer-hill/index.html
Levy, M. (1999). Design processes in CALL: Integrating theory, research and evaluation. In K. Cameron (Ed.), CALL: Media, design and applications (pp. 83-108). Lisse, The Netherlands: Swets & Zeitlinger.
Liou, H-C. (1997). Research of on-line help as learner strategies for multimedia CALL evaluation. CALICO Journal, 14(2-4), 81-96. Retrieved June 2, 2008, from https://calico.org/p-5-Calico%20Journal.html
Ma, Q. (2007a). Design and evaluation of a computer assisted vocabulary learning program for Chinese learners of English: A user-centred approach. Unpublished doctoral dissertation, Catholic University of Louvain, Belgium.
Ma, Q. (2007b). From monitoring user actions to controlling user actions: A new perspective on the user-centred approach to CALL. Computer Assisted Language Learning, 20(4), 297-321.
Ma, Q., & Kelly, P. (2006). Computer assisted vocabulary learning: Design and evaluation. Computer Assisted Language Learning, 19(1), 15–45.
Meara, P. (1990). A note on passive vocabulary. Second Language Research, 6(2), 150–154.
Nielsen, J. (2006). How to conduct a heuristic evaluation. Retrieved December 10, 2006, from http://www.useit.com/papers/heuristic/heuristic_evaluation.html
Smidt, E., & Hegelheimer, V. (2004). Effects of online academic lectures on ESL listening comprehension, incidental vocabulary acquisition, and strategy use. Computer Assisted Language Learning, 17(5), 517-556.
Tozcu, A., & Coady, J. (2004). Successful learning of frequent vocabulary through CALL also benefits reading comprehension and speed. Computer Assisted Language Learning, 17(5), 437-495.
Yeh, Y., & Wang, C. (2003). Effects of multimedia vocabulary annotation and learning styles on vocabulary learning. CALICO Journal, 21(1), 134-144. Retrieved June 2, 2008, from https://calico.org/p-5-Calico%20Journal.html
Vinther, J. (2005). Cognitive processes at work in CALL. Computer Assisted Language Learning, 18(4), 251-271.
