Adolescent test-taker characteristics
A qualitative validation study of a Chinese language reading examination in Singapore
DOI:
https://doi.org/10.1558/jmtp.19885Keywords:
Adolescent test-taker characteristics, qualitative research, test validation, literacy assessment, motivationAbstract
This qualitative validation study investigates the characteristics and needs of adolescent test-takers sitting the reading paper of the Singapore-Cambridge General Certificate of Education Ordinary-Level Chinese Language Examination (GCE 1160). Specifically, evidence gathered through semi-structured interview and document analysis was organised around the following three issues: adolescence and adolescent literacy; reading motivation, interests and authenticity; and new technologies and reading literacy. Findings from the analysis suggest that Singaporean adolescents are generally becoming less motivated to read extensively in the Chinese language, seemingly the direct result of digital technologies and heavy homework loads. It also appeared that motivation to read is connected to the relevance and appeal of texts available as well as the perceived value of reading in securing better education and job opportunities. Another pattern illustrated by the study was that the internet is the defining technology for literacy and learning for Singaporean adolescents. Whilst evidence suggested that the GCE 1160 reading examination is designed with adequate knowledge of adolescence and adolescent literacy, several threats to validity were identified, including the relatively low appeal of the passages to test-takers, and their relevance, authenticity and sensitivity to new technologies and reading literacy. Drawing on a socio-cognitive validation framework, the study analyses these threats to validity and outlines practical directions whereby policy-makers and test designers might develop the reading examination.
References
Alderson, J. C. (2000). Assessing Reading. Cambridge University Press. https://doi.org/10.1017/CBO9780511732935
Alderson, J. C. and Banerjee, J. (2002). Language testing and assessment (Part 2). Language Teaching, 35, 79–113. https://doi.org/10.1017/S0261444802001751
American Educational Research Association (AERA), American Psychological Association (APA) and National Council on Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing. AERA.
Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford University Press.
Bachman, L. F. and Palmer, A. S. (1996). Language Testing in Practice. Oxford University Press.
Beardsmore, H. B. (2003). Language shift and cultural implications in Singapore. In S. Gopinathan, A. Pakir, W. K. Ho and V. Saravanan (Eds.), Language, Society and Education in Singapore: Issues and Trends (pp. 85–98). Eastern Universities Press.
British Educational Research Association. (2018). Ethical Guidelines for Educational Research. Retrieved 1 April 2021 from: https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018-online
Bruner, J. S. (1966). Toward a Theory of Instruction. Norton.
Chall, J. S. (1996). Stages of Reading Development. Harcourt Brace.
Cheong, Y. Y. (2019). Singapore’s Chinese language education and assessment policy. Journal of Chinese Language Education, 17.2, 1–33.
Chin, C. K. (2011). Chinese Language Curriculum and Pedagogies of Singapore. Nanjing University Press.
Coiro, J., Knobel, M., Lankshear, C. and Leu, D. J. (Eds.). (2008). Handbook of Research on New Literacies. Lawrence Erlbaum.
Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer and H. I. Braun (Eds.), Test Validity (pp. 3–17). Lawrence Erlbaum.
Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience. Harper Perennial.
Geranpayeh, A. and Taylor, L. (2013). Examining Listening: Research and Practice in Assessing Second Language Listening. Cambridge University Press.
Gu, L. (2014). At the interface between language testing and second language acquisition: Language ability and context of learning. Language Testing, 31, 111–133. https://doi.org/10.1177/0265532212469177
Guo, X. (2011). Chinese language teaching in Singapore: Variations and objectives. Journal of Chinese Language Education, 9.1, 1–16.
Haertel, E. (2013). How is testing supposed to improve schooling? Measurement: Interdisciplinary Research and Perspectives, 11.1–2, 1–18. https://doi.org/10.1080/15366367.2013.783752
Heng, S. K. (9 April 2014). Opening address by Mr Heng Swee Keat, Minister for Education, at the International Conference of Teaching and Learning with Technology at the Suntec International Convention and Exhibition Centre. MOE Speeches. Retrieved 1 April 2021 from: https://www.aps.sg/files/in-the-news/opening-address-by-mr-heng-swee-keat-at-the-international-conference-of-teaching-and-learning-with-technology.pdf
Hesse-Biber, S. N. (2010). Mixed Methods Research: Merging Theory with Practice. Guilford Press.
International Reading Association. (2002). Integrating Literacy and Technology in the Curriculum: A Position Statement. International Reading Association.
Internet World Stats. (2020). Usage and Population Statistics. Retrieved 1 April 2021 from: http://www.internetworldstats.com/stats.htm
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38.4, 319–342. https://doi.org/10.1111/j.1745-3984.2001.tb01130.x
Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational Measurement (pp. 17–64). American Council on Education and Praeger.
Khalifa, H. and Weir, C. J. (2009). Examining Reading: Research and Practice in Assessing Second Language Reading. Cambridge University Press.
Koretz, D. (2008). Measuring Up: What Educational Testing Really Tells Us. Harvard University Press. https://doi.org/10.2307/j.ctv1503gxj
Lee, H. L. (22 October 2019). Speech in Chinese by PM Lee Hsien Loong at the 40th anniversary of the Speak Mandarin Campaign. Prime Minister Office. Retrieved 1 April 2021 from: https://www.pmo.gov.sg/Newsroom/PM-Lee-Hsien-Loong-at-40th-Anniversary-of-Speak-Mandarin-Campaign
Leu, D. J., McVerry, J. G., O’Byrne, W. I., Zawilinski, L., Castek, J. and Hartman, D. K. (2009). The new literacies of online reading comprehension and the irony of no child left behind: Students who require our assistance the most, actually receive it the least. In L. M. Morrow, R. Rueda and D. Lapp (Eds.), Handbook of Research on Literacy and Diversity (pp. 173–194). Guilford Press.
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational Measurement (pp. 13–103). Macmillan.
Messick, S. (1992). Validity of test interpretation and use. In M. C. Alkin (Ed.), Encyclopedia of Educational Research (pp. 1487–1495). Macmillan. https://doi.org/10.1002/j.2333-8504.1990.tb01343.x
Ministry of Education (MOE). (2020). MOE is refreshing its curriculum in three areas: Here’s how it will impact your child. Retrieved 1 April 2021 from: https://www.moe.gov.sg/microsites/cos2020/infographics/COS2020_refreshing-curriculum.pdf
Mother Tongue Languages Review Committee (MTLRC). (2011). The Mother Tongue Languages Review Committee Report: Nurturing Active Learners and Proficient Users. Singapore Ministry of Education.
National Library Board. (2017). 2016 National Reading Habits Study: Findings on Teenagers. Singapore National Library Board.
Newton, P. E. (2017). An Approach to Understanding Validation Arguments: OFQUAL Report. Retrieved 1 April 2021 from: https://www.gov.uk/government/publications/an-approach-to-understanding-validation-arguments
Newton, P. E. and Shaw, S. D. (2014). Validity in Educational and Psychological Assessment. Sage Publications. https://doi.org/10.4135/9781446288856
Oh, K. M. (7 April 2019). Less than 40% of students like to read in Chinese. Lianhe Zaobao. Retrieved 1 April 2021 from: https://www.zaobao.com.sg/znews/singapore/story20190407-946478
Patton, M. Q. (2015). Qualitative Research & Evaluation Methods: Integrating Theory and Practice. Sage Publications.
Piaget, J. (1970). Piaget’s theory. In P. Mussen (Ed.), Carmichael’s Manual of Child Psychology (pp. 703–732). Wiley.
Prime Minister’s Office, Singapore Department of Statistics, Ministry of Home Affairs, Immigration and Checkpoints Authority and Ministry of Manpower. (2019). Population in Brief 2019. Retrieved 1 April 2021 from: https://www.strategygroup.gov.sg/files/media-centre/publications/population-in-brief-2019.pdf
Scott, D. (2011). Assessment reforms: High stakes testing and knowing the contents of other minds. In R. Berry and B. Adamson (Eds.), Assessment Reform in Education: Policy and Practice (pp. 155–164). Springer. https://doi.org/10.1007/978-94-007-0729-0_11
Shaw, S. D. and Crisp, V. (2012). An approach to validation: Developing and applying an approach for the validation of general qualifications. Research Matters Special Issue, 3, 3–44.
Shaw, S. D. and Weir, C. J. (2007). Examining Writing: Research and Practice in Assessing Second Language Writing. Cambridge University Press.
Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405–450. https://doi.org/10.2307/1167347
Snow, C. E. and Sweet, A. (2003). Reading for comprehension. In A. Sweet and C. E. Snow (Eds.), Rethinking Reading Comprehension (pp. 1–11). Guilford Press.
Stobart, G. (2012). Foreword. Research Matters Special Issue, 3, 1.
Suri, H. (2011). Purposeful sampling in qualitative research synthesis. Qualitative Research Journal, 11.2, 63–75. https://doi.org/10.3316/QRJ1102063
Tan, C. L. (Ed.). (2011). From Practice to Practical: Teaching and Learning of Chinese as a Second Language. Nanjing University Press.
Tan, C. L. (2016). The present: An overview of teaching Chinese language in Singapore. In K. C. Soh (Ed.), Teaching Chinese Language in Singapore: Retrospect and Challenges (pp. 11–26). Springer. https://doi.org/10.1007/978-981-10-0123-9_2
Taylor, L. (Ed.). (2011). Examining Speaking: Research and Practice in Assessing Second Language Speaking. Cambridge University Press.
Vygotsky, L. S. (1987). The Collected Works of L.S. Vygotsky: Problems of General Psychology. Plenum.
Weir, C. J. (2005). Language Testing and Validation: An Evidence-Based Approach. Palgrave Macmillan.
World Health Organization. (2011). Maternal, New-Born, Child and Adolescent Health: Adolescent Development. Retrieved 1 April 2021 from: http://www.who.int/maternal_child_adolescent/topics/adolescence/dev/en/
