Quantitative considerations for improving replicability in CALL and applied linguistics

Authors

  • Luke Plonsky Northern Arizona University

DOI:

https://doi.org/10.1558/cj.v32i2.26857

Keywords:

computer-assisted language instruction, SLA, replication, quantitative research methods,

Abstract

There are a number of methodological practices commonly employed by CALL researchers that limit progress in the field. Some of these practices are particular to replication research, but most are more general and are found throughout the field. I describe in this paper two studies that are fabricated but that resemble much of what is found in published second language research. Each study corresponds to and contains a set of methodological issues. Following each study, I address the issues they illustrate, providing comments and suggestions for how the analyses could be modified to produce greater replicability and/or replicational value. I conclude with a summary of suggestions for quantitative reforms related to improving replication research and quantitative practices more generally in CALL and applied linguistics.

Author Biography

Luke Plonsky, Northern Arizona University

Luke Plonsky is Assistant Professor, Applied Linguistics at Northern Arizona University, Flagstaff, AZ, USA.

References

Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.

Gass, S., & Valmori, L. (in press). Replication in interaction and working memory research: Révész (2012) and Goo (2012). Language Teaching. http://dx.doi.org/10.1017/S0261444815000038

Godfroid, A., & Spino, L. (in press). Reconceptualizing reactivity research: Absence of evidence is not evidence of absence. Language Learning.

Jeon, E. H., & Yamashita, J. (2014). L2 reading Comprehension and its correlates: A meta-analysis. Language Learning, 64 (1), 160–212. http://dx.doi.org/10.1111/lang.12034

Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge.

Larson-Hall, J., & Plonsky, L. (in press). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. In J. M. Norris, S. Ross, & R. Schoonen (Eds), Improving and extending quantitative reasoning in second language research. Malden, MA: Wiley.

Norris, J. M. (in press). Statistical significance testing in second language research: Basic problems and suggestions for reform. In J. M. Norris, S. Ross, & R. Schoonen (Eds.), Improving and extending quantitative reasoning in second language research. Malden, MA: Wiley.

Norris, J. M., & Ortega, L. (2006). The value and practice of research synthesis for language learning and teaching. In J. M. Norris & L. Ortega (Eds), Synthesizing research on language learning and teaching, 3–50. Philadelphia, PA: John Benjamins.

Norris, J. M., Plonsky, L., Ross, S. J., & Schoonen, R. (in press). Guidelines for reporting quantitative methods and results in primary research. Language Learning.

Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85–110. http://dx.doi.org/10.1017/S0267190510000115

Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics, 116–132. New York: Cambridge University Press.

Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655–687. http://dx.doi.org/10.1017/S0272263113000399

Plonsky, L. (in press). Statistical power, p values, descriptive statistics, and effect sizes: A ‘back-to-basics’ approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research. New York: Routledge.

Plonsky, L., & Oswald, F. L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64 (4), 878–912. http://dx.doi.org/10.1111/lang.12079

Porte, G. (2013). Who needs replication research? CALICO Journal, 30, 10–15. http://dx.doi.org/10.11139/cj.30.1.10-15

Smith, B., & Schulze, M. (2013). Thirty years of the CALICO Journal – replicate, replicate, replicate. CALICO Journal, 30 (1), i–iv. http://dx.doi.org/10.11139/cj.30.1.i-iv

Downloads

Published

2015-04-23

How to Cite

Plonsky, L. (2015). Quantitative considerations for improving replicability in CALL and applied linguistics. CALICO Journal, 32(2), 232–244. https://doi.org/10.1558/cj.v32i2.26857

Issue

Section

Review article