Examining validity issues in classifying and exiting English learners through dynamic views of language proficiency
DOI:
https://doi.org/10.1558/jmtp.19826Keywords:
English learners, validity, English learner classificationAbstract
With the growing English learner population, this study identifies issues of fairness around cut score setting practices that impact English learner (EL) classifications and reclassifications in the US context. High-stakes decisions regarding the education of ELs have relied mostly on English language proficiency (ELP) assessments and the ELP-related classifications. The ELP-related classifications are central to ELs’ life cycles in US schooling. These classifications impact decisions as to whom to designate as EL, whom to administer content assessments to, and who benefits from instructional support programmes. This study examines the practices in the US pre-and post- ‘Every Child Succeeds Act’ (ESSA) of 2015 and takes a dynamic multilingualism perspective to discuss what validity evidence is missing in the current classification practices and what could be done to better align the classification practices with the dynamic languaging practices of multilingual learners.
References
Abedi, J. (2004). The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33.1, 4–14. https://doi.org/10.3102/0013189X033001004
Abedi, J. (Ed.). (2007). English Language Proficiency Assessment in the Nation: Current Status and Future Practice. University of California, Davis, School of Education.
Abedi, J. (2008a). Classification system for English language learners: Issues and recommendations. Educational Measurement: Issues and Practice, 27.3, 17–31. https://doi.org/10.1111/j.1745-3992.2008.00125.x
Abedi, J. (2008b). Measuring students’ level of English proficiency: Educational significance and assessment requirements. Educational Assessment, 13.2/3, 193–214. https://doi.org/10.1080/10627190802394404
American Educational Research Association, American Psychological Association and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. American Educational Research Association.
Bailey, A. L. and Huang, B. H. (2011). Do current English language development/proficiency standards reflect the English needed for success in school? Language Testing, 28.3, 343–365. https://doi.org/10.1177/0265532211404187
Bennett, R. E., Kane, M. and Bridgeman, B. (2011). Theory of Action and Validity Argument in the Context of Through-Course Summative Assessment. Educational Testing Service.
Buckendahl, C. W. and Davis-Becker, S. L. (2012). Setting passing standards for credentialing programmes. In G. J. Cisek (Ed.), Setting Performance Standards: Concepts, Methods, and Innovations (pp. 485–501). Routledge.
Callahan, R., Wilkinson, L., Muller, C. and Frisco, M. (2009). ESL placement and schools: Effects on immigrant achievement. Educational Policy, 23.2, 355–384. https://doi.org/10.1177/0895904807310034
Carlson, D. and Knowles, J. (2016). The effect of English language learner reclassification on student ACT scores, high school graduation, and postsecondary enrollment: Regression discontinuity evidence from Wisconsin. Journal of Policy Analysis and Management, 35.3, 559–586.
Chalhoub-Deville, M. (2016). Validity theory: Reform policies, accountability testing, and consequences. Language Testing, 33.4, 453–472. https://doi.org/10.1177/0265532215593312
Cimpian, J. R., Thompson, K. D. and Makowski, M. B. (2017). Evaluating English learner reclassification policy effects across districts. American Educational Research Journal, 54.1_suppl., 255S–278S. https://doi.org/10.3102/0002831216635796
Collier, V. P. and Thomas, W. P. (1989). How quickly can immigrants become proficient in school English. Journal of Educational Issues of Language Minority Students, 5.1, 26–38.
Conger, D. (2009). Testing, time limits, and English learners: Does age of entry affect how quickly students can learn English? Social Science Research, 38.2, 383–396. https://doi.org/10.1016/j.ssresearch.2008.08.002
Cook, V. (2012). Multicompetence. The Encyclopedia of Applied Linguistics, 1–6. https://doi.org/10.1002/9781405198431.wbeal0778
Creese, A. and Blackledge, A. (2015). Translanguaging and identity in educational settings. Annual Review of Applied Linguistics, 35, 20–35. https://doi.org/10.1017/S0267190514000233
Cummins, J. (1980). The cross-lingual dimensions of language proficiency: Implications for bilingual education and the optimal age issue. TESOL Quarterly, 14.2, 175–187. https://doi.org/10.2307/3586312
De Jong, E. J. (2004). After exit: Academic achievement patterns of former English language learners. Educational Policy Analysis Archives, 12(50). Retrieved 22 September 2005 from: http://epaa.asu.edu/epaa/v12n50/v12n50.pdf. https://doi.org/10.14507/epaa.v12n50.2004
Farnsworth, T. (2020). A review of validity evidence on K–12 English language proficiency assessment: Current state and future direction. Assessing English Language Proficiency in US K–12 Schools, 75–91. https://doi.org/10.4324/9780429491689-5
Faulkner-Bond, M. and Soland, J. (2020). Comparability when assessing English learner students. In A. I. Berman, E. H. Haertel and J. W. Pellegrino (Eds.), Comparability of Large-Scale Educational Assessments: Issues and Recommendations (pp. 149–175). National Academy of Education.
Florez, I. R. (2012). Examining the validity of the Arizona English language learners’ assessment cut scores. Language Policy, 11.1, 33–45. https://doi.org/10.1007/s10993-011-9225-4
García, O. (2009). Emergent bilinguals and TESOL: What’s in a name? Tesol Quarterly, 43.2, 322–326. https://doi.org/10.1002/j.1545-7249.2009.tb00172.x
García, O. and Wei, L. (2014). Language, bilingualism and education. In Translanguaging: Language, Bilingualism and Education (pp. 46–62). Palgrave Pivot. https://doi.org/10.1057/9781137385765_4
Genesee, F., Lindholm-Leary, K., Christian, D., Saunders, W. and Saunders, B. (2006). Educating English Language Learners: A Synthesis of Research Evidence. Cambridge University Press. https://doi.org/10.1017/CBO9780511499913
Grosjean, F. (1989). Neurolinguists, beware! The bilingual is not two monolinguals in one person. Brain and Language, 36.1, 3–15. https://doi.org/10.1016/0093-934X(89)90048-5
Hakuta, K., Butler, G. Y. and Witt, D. (2000). How long does it take English learners to attain proficiency? University of California, Linguistic Minority Research Institute, Policy Report 2000–1.
Hopkins, M., Thompson, K. D., Linquanti, R., Hakuta, K. and August, D. (2013). Fully accounting for English learner performance: A key issue in ESEA reauthorization. Educational Researcher, 42.2, 101–108. https://doi.org/10.3102/0013189X12471426
Kane, M. T. (2006). Content-related validity evidence. In S. M. Downing and T. M. Haladyna (Eds.), Handbook of Test Development (pp. 131–154). Erlbaum.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50.1, 1–73. https://doi.org/10.1111/jedm.12000
Kanno, Y. and Kangas, S. E. (2014). ‘I’m not going to be, like, for the AP’: English language learners’ limited access to advanced college-preparatory courses in high school. American Educational Research Journal, 51.5, 848–878. https://doi.org/10.3102/0002831214544716
Kenyon, D. M., MacGregor, D., Li, D. and Cook, H. G. (2011). Issues in vertical scaling of a K-12 English language proficiency test. Language Testing, 28, 383–400.
Kim, J. and Herman, J. L. (2009). A three-state study of English learner progress. Educational Assessment, 14.3–4, 212–231. https://doi.org/10.1080/10627190903422831
Kopriva, R. (2000). Ensuring accuracy in testing for English language learners. Council of Chief State School Officers.
Leung, C. and Valdés, G. (2019). Translanguaging and the transdisciplinary framework for language teaching and learning in a multilingual world. The Modern Language Journal, 103.2, 348–370. https://doi.org/10.1111/modl.12568
Linquanti, R. (2001). The redesignation dilemma: Challenges and choices in fostering meaningful accountability for English learners. University of California Linguistic Minority Research Institute, Santa Barbara, Policy Report 2001–1.
Linquanti, R. and Cook, G. (2013). Toward a ‘common definition of English learner’: Guidance for states and state assessment consortia in defining and addressing policy and technical issues and options. Council of Chief State School Officers.
Linquanti, R. and George, C. (2007) Establishing and utilizing an NCLB Title III accountability system: California’s approach and findings to date. In J. Abedi (Ed.), English Language Proficiency Assessment in the Nation: Current Status and Future Practice (pp. 105–118). University of California, Davis, School of Education.
Lyons, S., Dadey, N. and Garibay-Mulattieri, K. (2017). Considering English language proficiency within systems of accountability under the Every Student Succeeds Act. Center for Assessment.
MacSwan, J. and Rolstad, K. (2006). How language proficiency tests mislead us about ability: Implications for English language learner placement in special education. Teachers College Record, 108.11, 2304–2328. https://doi.org/10.1111/j.1467-9620.2006.00783.x
Mahoney, K. S. and MacSwan, J. (2005). Reexamining identification and reclassification of English language learners: A critical discussion of select state practices. Bilingual Research Journal, 29.1, 31–42. https://doi.org/10.1080/15235882.2005.10162822
Martiniello, M. (2008). Language and the performance of English language learners in math word problems. Harvard Educational Review, 78, 333–368. https://doi.org/10.17763/haer.78.2.70783570r1111t32
McLaughlin, M. W. and Shepard, L. A. (1995). Improving Education through Standards-Based Reform: A Report by the National Academy of Education Panel on Standards-Based Education Reform. National Academy of Education, Stanford University, CERAS Building, Room 108, Stanford, CA 94305-3084.
Morita-Mullaney, T. and Singh, M. (2021). Obscuring English learners from state accountability: The case of Indiana’s language blind policies. Educational Policy, 35.4, 621–645. https://doi.org/10.1177/0895904818823751
NCES. (2011). The condition of Education 2011 (2011–033). U.S. Department of Education. Retrieved from NCES https://nces.ed.gov/pubs2011/2011033.pdf
Parrish, T., Perez, M., Merickel, A. and Linquanti, R. (2006). Effects of the Implementation of Proposition 227 on the Education of English Learners, K-12: Findings from a Five-Year Evaluation. American Institutes for Research. https://doi.org/10.1037/e539652012-001
Ragan, A. and Lesaux, N. (2006). Federal, state, and district level English language learner programme entry and exit requirements: Effects on the education of language minority learners. Education Policy Analysis Archives, 14.20. Retrieved from: http://epaa.asu.edu/epaa/v14n20/
Rivera, C., Vincent, C., Hafner, A. and LaCelle-Peterson, M. (1997). Statewide assessment programme policies and practices for the inclusion of limited English proficient students. Clearinghouse on Assessment and Evaluation (ERIC Rep. No. EDO-TM-97-02).
Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33.3, 267–292. https://doi.org/10.3102/0162373711407912
Robinson-Cimpian, J. P. and Thompson, K. D. (2016). The effects of changing test-based policies for reclassifying English learners. Journal of Policy Analysis and Management, 35.2, 279–305. https://doi.org/10.1002/pam.21882
Shin, N. (2018). The effects of the initial English language learner classification on students’ later academic outcomes. Educational Evaluation and Policy Analysis, 40.2, 175–195.
Slama, R. B. (2012) A longitudinal analysis of academic English proficiency outcomes for adolescent English language learners in the United States. Journal of Educational Psychology, 104.2, 265. https://doi.org/10.1037/a0025861
Slama, R. B. (2014). Investigating whether and when English learners are reclassified into mainstream classrooms in the United States: A discrete-time survival analysis. American Educational Research Journal, 51.2, 220–252. https://doi.org/10.3102/0002831214528277
Stokes-Guinan, K. and Goldenberg, C. (2010). Use with caution: What CELDT results can and cannot tell us. The CATESOL Journal, 22.1, 189–202.
Tannenbaum, C., Boyle, A., Soga, K., Le Floch, K. C., Golden, L., Petroccia, M. and O’Day, J. (2012). National Evaluation of Title III Implementation: Report on State and Local Implementation. American Institutes for Research.
Tannenbaum, R. J. and Wylie, E. C. (2005). Mapping English language proficiency test scores onto the Common European Framework. ETS Research Report Series, 2005.2, i–46. https://doi.org/10.1002/j.2333-8504.2005.tb01995.x
Tannenbaum, R. J. and Katz, I. R. (2013). Standard-setting. In K. F. Geisinger, B. A. Bracken, J. F. Carlson, J.-I. C. Hansen, N. R. Kuncel, S. P. Reise and M. C. Rodriguez (Eds.), APA Handbook of Testing and Assessment in Psychology, Vol. 3: Testing and Assessment in School Psychology and Education (pp. 455–477). American Psychological Association.
Thompson, K. D. (2012). Are we there yet? Exploring English learners’ journey to reclassification and beyond. Unpublished doctoral dissertation, Stanford University.
Umansky, I. M. (2018). According to plan? Examining the intended and unintended treatment effects of EL classification in early elementary and the transition to middle school. Journal of Research on Educational Effectiveness, 11.4, 588–621. https://doi.org/10.1080/19345747.2018.1490470
Umansky, I. M. and Reardon, S. F. (2014). Reclassification patterns among Latino English learner students in bilingual, dual immersion, and English immersion classrooms. American Educational Research Journal, 51.5, 879–912. https://doi.org/10.3102/0002831214545110
Valdés, G., Poza, L. and Brooks, M. D. (2017). Language acquisition in bilingual education. In W. Wright, S. Boun, and O. García (Eds.), The Handbook of Bilingual and Multilingual Education (pp. 56–74). Wiley Blackwell. https://doi.org/10.1002/9781118533406.ch4
Wang, J., Niemi, D. and Wang, H. (2007). Impact of different performance assessment cut scores on student promotion. CSE (CRESST) Report 719. https://doi.org/10.1037/e643802011-001
Wolf, K. M., Farnsworth, T. and Herman, J. (2008). Validity issues in assessing English language learners’ language proficiency, Educational Assessment, 13.2, 80–107. https://doi.org/10.1080/10627190802394222
Wolf, M. K., Kao, J., Herman, J. L., Bachman, L. F., Bailey, A. L., Bachman, P. L. et al. (2008). Issues in Assessing English Language Learners: English Language Proficiency Measures and Accommodation Uses – Literature Review (CRESST Tech. Rep. No. 731). University of California Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Zehler, A. M., Fleischman, H. L. Hopstock, P. J., Pendzick, M. L. and Stephenson, T. G. (2003). Findings on Special Education LEP Students. Office of English Language Acquisition (OELA), US Department of Education (Contract No. ED-00-CO-0089).
