Linguistically based scales for assessment of young students’ writing
DOI:
https://doi.org/10.1558/wap.21504Keywords:
assessment, early school writing, comparative judgement, writing, linguistic resourcesAbstract
This study addresses the question of how different aspects of students’ writing achievement can be recognised and evaluated. We developed a linguistically based framework for criteria-based assessment, anchored in a functional view of language and language learning. The framework was used to determine what traits characterise texts at different Proficiency Groups based on comparative judgement and what traits characterise texts assessed differently. Altogether, 100 texts (written by students ages 6–9) representing four text genres were assessed and ranked using both comparative judgement (holistic assessment) and criteria-based analysis. The results indicate that texts generally are assessed as stronger (i.e., placed in a higher Proficiency Group) when comparative judgement is used than what the assessment of a specific language resource indicates. The results also indicate that assessment differences might be a result of different quality expectations for different genres. This points towards the need for genre- and subject-specific assessment criteria to scaffold students in their emergent disciplinary writing development.
References
Aull, L. (2015). Connecting writing and language in assessment: Examining style, tone, and argument in the U.S. Common Core standards and in exemplary student writing. Assessing Writing, 24, 59–73. https://doi.org/10.1016/j.asw.2015.03.002
Barkaoui, K. (2010). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly, 7(1), 54–74. https://doi.org/10.1080/15434300903464418
Beck, I. L., McKeown, M. G., & Worthy, J. (1995). Giving a text voice can improve students’ understanding. Reading Research Quarterly, 30(2), 220–238. https://doi.org/10.2307/748033
Beck, I. L., & McKeown, M. G. (2007). Increasing young low-income children’s oral vocabulary repertoires through rich and focused instruction. The Elementary School Journal, 107(3), 251–271. https://doi.org/10.1086/511706
Benton, T., & Gallacher, T. (2018). Is comparative judgement just a quick form of multiple marking? Research Matters: A Cambridge Assessment Publication, 26, 22–28.
Christie, F. (2012). Language education throughout the school years: A functional perspective. Wiley-Blackwell.
Cristie, F., & Derewianka, B. (2008). School discourse: Learning to write across the years of schooling. London.
Crossley, S. A., Roscoe, R., & McNamara, D. S. (2014). What is successful writing? An investigation into the multiple ways writers can write successful essays. Written Communication, 31(2), 184–214. https://doi.org/10.1177/0741088314526354
Fang, Z., & Coatoam, S. (2013). Disciplinary literacy: What you want to know about it. Journal of Adolescent & Adult Literacy, 56(8), 627–632. https://doi.org/10.1002/JAAL.190
Fang, Z., & Wang, Z. (2011). Beyond rubrics: Using functional language analysis to evaluate student writing. The Australian Journal of Language and Literacy, 34(2), 147–165. https://search.informit.org/doi/10.3316/ielapa.112929734219836
Folkeryd, J. W. (2006). Writing with an attitude: Appraisal and student texts in the school subject of Swedish. Acta Universitatis Upsaliensis : Uppsala University.
Folkeryd, J. W. (2014). Hjärtan, hjul och himlakroppar. Innehåll i elevers sakprosatexter på lågstadiet. In P. Andersson, P. Holmberg, A. Lyngfelt, A. Nordenstam & O. Widhe (Eds.), Mångfaldens möjligheter. Litteratur och språkdidaktik i Norden (pp. 115–131). Nationella nätverket för svenska med didaktisk inriktning.
Folkeryd, J. W. (2021). Teorier och metoder för textanalys – exemplet skrivande i skolans tidiga år. In T. Nygren (Ed.), Vetenskapsteori och forskningsmetoder i utbildningsvetenskap, Natur och kultur.
Fulcher, G. (2003). Testing second language speaking. Routledge.
af Geijerstam, Å. (2006). Att skriva i naturorienterande ämnen i skolan. Uppsala University.
Halliday, M. A. K. (1978). Language as social semiotic: The social interpretation of language and meaning. Edward Arnold.
Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3.). Arnold.
Heldsinger, S., & Humphry, S. (2010). Using the method of pairwise comparison to obtain reliable teacher assessments. The Australian Educational Researcher, 37(2), 1–19. https://doi.org/10.1007/BF03216919
Heller, R. (2010). In praise of amateurism: A friendly critique of Moje’s ‘call for change’ in secondary literacy. Journal of Adolescent & Adult Literacy, 54(4), 267–273. https://doi.org/10.1598/JAAL.54.4.4
Humphry, S. (2017). Academic literacies in the middle years: A framework for enhancing teacher knowledge and student achievement. Routledge.
Humphry, S., & Heldsinger, S. (2020). A two-stage method for obtaining reliable teacher assessments of writing. Frontiers in Education, 5, 6. https://doi.org/10.3389/feduc.2020.00006
Humphry, S., & McGrane, J. A. (2015). Equating a large-scale writing assessment using pairwise comparisons of performances. Australian Educational Researcher, 42(4), 443–460. Social Science Premium Collection. https://doi.org/10.1007/s13384-014-0168-6
Håland, A. (2016). Disciplinary literacy in elementary school: How a struggling student positions herself as a writer. The Reading Teacher, 70(4), 457–468. https://doi.org/10.1002/trtr.1541
Ivani?, R. (2004). Discourses of writing and learning to write. Language and Education, 18(3), 220–245. https://doi.org/10.1080/09500780408666877
Knoch, U. (2011). Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from? Assessing Writing, 16(2), 81–96. https://doi.org/10.1016/j.asw.2011.02.003
Lado, R. (1961) Language testing: The construction and use of foreign language tests: A teacher's book. McGraw Hill.
Langer, J. A. (1992). Reading, writing and genre development. In J. W. Irwin & M. A. Doyle (Eds.), Reading/writing connections: Learning from research, 32–54. International Reading Association.
Lesterhuis, M., Daal, T. van, Van Gasse, R., Coertjens, L., & Donche, V. (2018). When teachers compare argumentative texts: Decisions informed by multiple complex aspects of text quality. L1 Educational Studies in Language and Literature, 18, 1–22. https://doi.org/10.17239/L1ESLL-2018.18.01.02
Liberg, C. (2014). Att tala om innehåll och ämnesstämmor i elevtexter i tidiga skolår. In R. Hvistendahl & A. Roe (Eds.), Alle tiders norskdidaktiker: Festskrift til Fröydis Hertzberg på 70-årsdagen (pp. 141–156).
Liberg, C., Folkeryd, Jenny W., af Geijerstam, Å., & Nordlund, A. (2022). Teachers’ talk about young students’ writing of narrative and informational texts. Writing & Pedagogy, 13(1–3), 181–206. https://doi.org/10.1558/wap.21544
Liberg, C., Espmark, L., Wiksten, J., & Bütler, I. (1997). Upplevelsepresenterande, händelsetecknande, berättande och språkinlärning. Dept. of Linguistics, Uppsala University.
Macken-Horarik, M., Sandiford, C., Love, K., & Unsworth, L. (2015). New ways of working ‘with grammar in mind’ in school English: Insights from systemic functional grammatics. Linguistics and Education, 31, 145–158. https://doi.org/10.1016/j.linged.2015.07.004
Macken-Horarik, M., & Sandiford, C. (2016). Diagnosing development: A grammatics for tracking student progress in narrative composition. International Journal of Language Studies, 10(3), 61–94. https://acuresearchbank.acu.edu.au/item/878zq/diagnosing-development-a-grammatics-for-tracking-student-progress-in-narrative-composition
Madsen, H. (1983). Techniques in testing. Oxford University Press.
Martin, J. R. (2009). Genre and language learning: A social semiotic perspective. Linguistics and Education, 12. https://doi.org/10.1016/j.linged.2009.01.003
Martin, J., & White, P. (2005). The language of evaluation: Appraisal in English. Palgrave Macmillan. https://doi.org/10.1016/j.linged.2009.01.003
Moje, E. B. (2008). Foregrounding the disciplines in secondary literacy teaching and learning: A call for change. Journal of Adolescent & Adult Literacy, 52(2), 96–107. https://doi.org/10.1598/JAAL.52.2.1
Myers, M. (1980). A procedure for writing assessment and holistic scoring. ERIC Clearinghouse on Reading and Communication Skills, National Institute of Education: National Council of Teachers of English.
Olinghouse, N. G., & Leaird, J. T. (2009). The relationship between measures of vocabulary and narrative writing quality in second- and fourth-grade students. Reading and Writing: An Interdisciplinary Journal, 22(5), 545–565. https://doi.org/10.1007/s11145-008-9124-z
Pollitt, A. (2012). The method of adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice, 19(3), 281–300. https://doi.org/10.1080/0969594X.2012.665354
Ravid, D. (2006). Semantic development in textual contexts during the school years: Noun scale analyses. Journal of Child Language, 33(4), 791–821. https://doi.org/10.1017/S0305000906007586
Schleppegrell, M. J. (2004). The language of schooling: A functional linguistics perspective. Lawrence Erlbaum Associates, Publishers.
Shanahan, C., & Shanahan, T. (2014). Does disciplinary literacy have a place in elementary school? The Reading Teacher, 67(8), 636–639. https://doi.org/10.1002/trtr.1257
Skar, G. B., & Aasen, A. J. (2018). Å måle skriving som grunnleggende ferdighet. Acta Didactica Norden, 12(4). https://doi.org/10.5617/adno.6280
Skar, G. B., Jølle, L., & Aasen, A. J. (2020). Establishing rating scales to assess writing proficiency development in young learners. Acta Didactica Norden, 14(1), 30 sider. https://doi.org/10.5617/adno.7909
Swedish National Agency for Education. (2018). Curriculum for the Compulsory School, Preschool Class and School-age Educare 2011, revised 2018.
Tolchinsky, L., & Sandbank, A. (1994). Text production and text differentiation: Developmental changes and educational influences. Learning Environments and Psychological Development. Norwood, NJ: Ablex.
Wheadon, C., Barmby, P., Christodoulou, D., & Henderson, B. (2020). A comparative judgement approach to the large-scale assessment of primary writing in England. Assessment in Education: Principles, Policy & Practice, 27(1), 46–64. ERIC. https://doi.org/10.1080/0969594X.2019.1700212
