Extending Preselected Items Evaluation (PIE) with the ATA (American Translators Association) error categories for objective and targeted translation evaluation
DOI:
https://doi.org/10.1558/jalpp.21681Keywords:
ATA framework for standardized error marking, preselected items evaluation, professional translation, translation evaluation, translation quality assessment, translator trainingAbstract
This paper proposes to extend Preselected Items Evaluation (PIE), an item-based analytical method that seeks to provide greater objectivity in translation evaluation, with the error categories of the American Translators Association’s (ATA) Framework for Standardized Error Marking. The aim is to address a shortcoming of PIE, specifically its restriction to the identification of error numbers, disregarding error types. We describe the application of PIE and the ATA Framework for Standardized Error Marking and explain the advantages of combining PIE with the ATA error categories. We also outline the findings of a pilot study using this method and suggest proposals for further research.
References
Angelelli, Claudia (2009) Using a rubric to assess translation ability: Defining the construct. In Claudia V. Angelelli and Holly E. Jacobson (eds) Testing and Assessment in Translation and Interpreting Studies: A Call for Dialogue between Research and Practice, 13–47. Amsterdam: John Benjamins. https://doi.org/10.1075/ata.xiv.03ang
Angelelli, Claudia and Holly E. Jacobson (2009) Testing and Assessment in Translation and Interpreting Studies: A Call for Dialogue between Research and Practice. Amsterdam: John Benjamins. https://doi.org/10.1075/ata.xiv
Beeby, Allison, Mónica Fernández, Olivia Fox, Amparo Albir, Inna Kozlova, Anna Kuznik, Wilhelm Neunzig, Patricia Rodríguez et al. (2009) Results of the validation of the PACTE translation competence model: Acceptability and decision making. Across Languages and Cultures 10 (2): 207–230. https://doi.org/10.1556/Acr.10.2009.2.3
Colina, Sonia (2009) Further evidence for a functionalist approach to translation quality evaluation. Target: International Journal of Translation Studies 21 (2): 235–264. https://doi.org/10.1075/target.21.2.02col
Colman, Amy, Winibert Segers and Heidi Verplaetse (2021a) PIE & ATA: An innovative approach to translation evaluation. Conference session, Applied Linguistics and Professional Practice (ALAPP) 2021, Translational Data Analytics Institute, Ohio State University, 15–17 September. https://lirias.kuleuven.be/retrieve/626331
Colman, Amy, Winibert Segers and Heidi Verplaetse (2021b) The PIE method (Preselected Items Evaluation): An innovative approach to translation evaluation in translator training. Conference session, TRANSLATA IV, 4th International Conference on Translation and Interpreting Studies, Future Perspectives in Translation and Interpreting Studies, University of Innsbruck, Austria, 20–22 September. https://lirias.kuleuven.be/retrieve/628950
Colman, Amy, Winibert Segers and Heidi Verplaetse (2021c) Refining the PIE method (Preselected Items Evaluation) in translator training. Current Trends in Translation Teaching and Learning E 8: 236–276. https://doi.org/10.51287/cttle20218
Colman, Amy, Heidi Verplaetse and Winibert Segers (2022) Directionality in translator training: Contrastive evaluation of L1 and L2 translations using the PIE method (Preselected Items Evaluation) and the ATA Framework for Standardized Error Marking. Transletters: International Journal of Translation and Interpreting 6: 31–62.
Daems, Joke, Lieve Macken and Sonia Vandepitte (2014) On the origin of errors: A fine-grained analysis of MT and PE errors and their relationship. In Proceedings of the Ninth International Conference on Language Resources and Evaluation. Reykjavik, Iceland, 26–31 May 2014, 62–66. Luxembourg: European Language Resources Association (ELRA).
Doyle, Michael S. (2003) Translation pedagogy and assessment: Adopting ATA’s framework for Standard Error Marking. ATA Chronicle 2003 (November–December): 21–28.
Drugan, Joanna (2013) Quality in Professional Translation: Assessment and Improvement. London: Bloomsbury.
Ebel, Robert L. (1979) Essentials of Educational Measurement. Englewood Cliffs, NJ: Prentice Hall.
Eyckmans, June and Philippe Anckaert (2017) Item-based assessment of translation competence: Chimera of objectivity versus prospect of reliable measurement. Linguistica Antverpiensia (new series) 16: 40–56. https://doi.org/10.52034/lanstts.v16i0.436
Eyckmans, June, Philippe Anckaert and Winibert Segers (2009) The perks of norm-referenced translation evaluation. In Claudia Angelelli and Holly Jacobson (eds) Testing and Assessment in Translation and Interpreting Studies: A Call for Dialogue between Research and Practice, 73–93. Amsterdam: John Benjamins. https://doi.org/10.1075/ata.xiv.06eyc
Huertas Barros, Elsa and Juliet Vine (2018) Current trends on MA translation courses in the UK: Changing assessment practices on core translation modules. Interpreter and Translator Trainer 12 (1): 5–24. https://doi.org/10.1080/1750399X.2017.1400365
Koby, Geoffrey S. (2015) The ATA flowchart and framework as a differentiated error-marking scale in translation teaching. In Ying Cui and Wei Zhao (eds) Handbook of Research on Teaching Methods in Language Translation and Interpretation, 220–253. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-4666-6615-3.ch013
Koby, Geoffrey S. and Brian J. Baer (2005) From professional certification to the translator training classroom: Adapting the ATA error marking scale. Translation Watch Quarterly 1 (1): 33–45.
Koby, Geoffrey S., Paul Fields, Daryl R. Hague, Arle Lommel and Alan Melby (2014) Defining translation quality. Tradumàtica 12: 413–420. https://doi.org/10.5565/rev/tradumatica.76
Koby, Geoffrey S. and Isabel Lacruz (2017) The thorny problem of translation and interpreting quality. Linguistica Antverpiensia (new series) 16: 1–12. https://doi.org/10.52034/lanstts.v16i0.487
Kockaert, Hendrik and Winibert Segers (2014) Evaluation de la traduction: La méthode PIE (Preselected Items Evaluation). Turjuman 23 (2): 232–250.
Kockaert, Hendrik and Winibert Segers (2017) Evaluation of legal translations: PIE method (Preselected Items Evaluation). Journal of Specialised Translation 27: 148–163.
Kübler, Natalie, Alexandra Mestivier and Mojca Pecman (2018) Teaching specialised translation through corpus linguistics: Translation quality assessment and methodology evaluation and enhancement by experimental approach. Meta 63 (3): 807–825. https://doi.org/10.7202/1060174ar
Mariana, Valerie, Troy Cox and Alan Melby (2015) The multidimensional quality metrics (MQM) framework: A new framework for translation quality assessment. Journal of Specialised Translation 23: 137–161.
Phelan, Mary (2017) Analytical assessment of legal translation: A case study using the American Translators Association framework. Journal of Specialised Translation 27: 189–210.
Segers, Winibert (2007) IJkpuntenmethode. In Chris Van de Poel and Winibert Segers (eds) Vertalingen objectief evalueren: Matrices en ijkpunten, 21–25. Leuven: Acco.
Segers, Winibert and Hendrik Kockaert (2016) Can subjectivity be avoided in translation evaluation? In Marcel Thelen, Gys-Walt van Egdom, Dirk Verbeeck and Barbara Lewandowska-Tomaszczyk (eds) Translation and Meaning: New Series, Volume 1, 69–78. Lódz Studies in Language 41. Frankfurt am Main: Peter Lang.
Segers, Winibert, Hendrik Kockaert and Bert Wylin (2018) Vertaalevaluatie en subjectiviteit. Tijdschrift N/f 13: 41–51.
Segers, Winibert and Gys-Walt van Egdom (2018) De kwaliteit van vertalingen: Een terminologie van de vertaalevaluatie. Kalmthout, Belgium: Pelckmans Pro.
Tijtgat, Evelien and Winibert Segers (2019) Wat is een goede vertaling? Vertaalevaluatie: methodes en technieken. In Gert De Sutter and Isabelle Delaere (eds) In balans. Een inleiding tot vertaal- en tolkwetenschap, 307–328. Leuven: Acco.
van Egdom, Gys-Walt, Heidi Verplaetse, Iris Schrijver, Hendrik Kockaert, Winibert Segers, Jasper Pauwels, Henri Bloemen and Bert Wylin (2018) How to put the translation test to the test? On Preselected Items Evaluation and perturbation. In Elsa Huertas Barros, Sonia Vandepitte and Emilia Iglesias-Fernández (eds) Quality Assurance and Assessment Practices in Translation and Interpreting, 26–56. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-5225-3.ch002
Verplaetse, Heidi (2022) Translation quality in student specialized translation: The impact of corpus use. In Marie-Aude Lefer and Sylviane Granger (eds) Extending the Scope of Corpus-Based Translation Studies, 209–236. London: Bloomsbury Academic. https://doi.org/10.5040/9781350143289.0017
Verplaetse, Heidi and An Lambrechts (2022) Analytical translation quality assessment: Insights for translators’ training. In Said M. Shiyab (ed.) Research into Translation and Training in Arab Academic Institutions, 89–112. New York: Routledge. https://doi.org/10.4324/9781003034667-6
Waddington, Christopher (2004) Should student translations be assessed holistically or through error analysis? Lebende Sprachen 49 (1): 28–35. https://doi.org/10.1515/LES.2004.28
Published
Issue
Section
License
Equinox Publishing Ltd.
