Contrastive analyses of evaluation in text

Key issues in the design of an annotation system for attitude applicable to consumer reviews in English and Spanish


  • Maite Taboada Simon Fraser University
  • Marta Carretero Universidad Complutense de Madrid



discourse analysis, evaluative meaning, annotation systems, corpus analysis


This paper reports on part of the research on evaluative language currently carried out within the CONTRANOT project,[lxxviii] which aims at the creation and validation of contrastive functional descriptions through corpus analysis and annotation in English and Spanish. More concretely, we will present the coding scheme designed for Attitude, a subcategory of Appraisal as studied within Systemic-Functional Linguistics (Martin and White, 2005; White, 2003). The criteria for selection and annotation of spans of Attitude in the coding scheme are specified and illustrated with examples from the Simon Fraser University Review Corpus (Taboada, 2008), a corpus of consumer-generated reviews on hotels, books and movies, and a small-scale English-Spanish contrastive analysis of these reviews has been carried out. The scheme is to be used for the future annotation of evaluation in an English-Spanish corpus, CONTRASTES (Lavid, 2008; Lavid et al., 2007, 2010). Once annotated, the reviews will be part of this corpus.

Author Biographies

Maite Taboada, Simon Fraser University

Maite Taboada is Associate Professor of Linguistics at Simon Fraser University (Canada). She holds MA and PhD degrees from the Universidad Complutense de Madrid (Spain), and an MSc in Computational Linguistics from Carnegie Mellon University. Maite works in the areas of discourse analysis, systemic functional linguistics and computational linguistics.

Marta Carretero, Universidad Complutense de Madrid

Marta Carretero is Associate Professor in English Language and Linguistics at the Universidad Complutense, Madrid, where she currently lectures on the areas of pragmatics and functional linguistics. She has done extensive research on modality, including English-Spanish contrastive analysis, the influence of genre on modality and the relationship between modality and evidentiality, among other issues. Within this area she authors articles and reviews in international and Spanish periodical journals. She also has contributions in collective works, such as English modality in perspective: Genre analysis and contrastive studies (2004), Perspectives on evidentiality and modality (2004) and Studies on English modality: In honour of Frank Palmer (2009). She has directed two research projects on modality in English and Spanish. She is co-editor of the collective work A Pleasure of Life in Words: A Festschrift for Angela Downing (2006), and was managing editor of the academic journal Atlantis (2006-2008).


Bednarek, Monika (2009) Language patterns and ATTITUDE. Functions of Language 16 (2): 165–192.

Biber, Douglas (1988) Variation across Speech and Writing. Cambridge: Cambridge University Press.

Biber, Douglas (1995) Dimensions of Register Variation: A Cross-Linguistic Comparison. Cambridge: Cambridge University Press.

Biber, Douglas and Finegan, Edward (1989a) Drift and the evolution of English style: A history of three genres. Language 65 (3): 487–517.

Biber, Douglas and Finegan, Edward (1989b) Styles of stance in English: Lexical and grammatical marking of evidentiality and affect. Text 9.1: 93–124.

Bloom, K., Garg, N. and Argamon, S. (2007) Extracting appraisal expressions, Proceedings of HLT/NAACL 308–315. Rochester, NY.

Carretero, Marta (2002) The influence of genre and register on epistemic modality in spoken English: A preliminary study. Estudios Ingleses de la Universidad Complutense 10: 11–41.

Carretero, Marta (2004) The role of evidentiality and epistemic modality in three English spoken texts from legal proceedings. In J. Marín-Arrese (ed.) Perspectives on Evidentiality and Modality in English and Spanish, 25–62. Madrid: Editorial Complutense.

Carretero, Marta (2007) Subjectivity in English epistemic modality: A two-resource based approach. BELL New Series 5 (5): 97–111.

Carretero, Marta and Taboada, Maite (to appear) The annotation of Appraisal: How attitude and epistemic modality overlap in English and Spanish consumer reviews. In J. R. Zamorano (ed.) Thinking Modally: English and Contrastive Studies on Modality. Berne: Peter Lang.

Chafe, Wallace and Nichols, Johanna (1986) Evidentiality: The Linguistic Coding of Epistemology. Norwood, NJ: Ablex.

Coulson, Seana (2001) Semantic Leaps: Frame-shifting and Conceptual Blending in Meaning Construction. Cambridge: Cambridge University Press.

Esuli, Andrea and Sebastiani, Fabrizio (2006) Determining term subjectivity and term orientation for opinion mining, Proceedings of EACL-06, the 11th Conference of the European Chapter of the Association for Computational Linguistics. Trento, Italy.

Fauconnier, Gilles and Turner, Mark (2008) Rethinking metaphor. In R. Gibbs (ed.), Cambridge Handbook of Metaphor and Thought, 53–66. Cambridge: Cambridge University Press.

Goldberg, Andrew B. and Zhu, Xiaojin (2006) Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization, Proceedings of HLT-NAACL 2006 Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing, 45–52. New York.

Kennedy, Alistair and Inkpen, Diana (2006) Sentiment classification of movie and product reviews using contextual valence shifters. Computational Intelligence 22 (2): 110–125.

Lakoff, George and Johnson, Mark (1980) Metaphors We Live By. Chicago, IL: University of Chicago Press.

Lavid, Julia (2008) CONTRASTES: An online English-Spanish textual database for contrastive and translation learning.

Lavid, J., Arús, J. and Zamorano, J. R. (2007) Working with a Bilingual English-Spanish Database using SFL. Paper presented at the 34th International Systemic-Functional Congress, Odense, Denmark.

Lavid, J., Arús, J. and Zamorano, J. R. (2010) Towards an Annotated English-Spanish Corpus with SFL Textual Features. Paper presented at the 37th International Systemic-Functional Congress, Vancouver, Canada.

Martin, James R. (2000) Beyond exchange: Appraisal systems in English. In S. Hunston and G. Thompson (eds) Evaluation in Text: Authorial Distance and the Construction of Discourse, 142–175. Oxford: Oxford University Press.

Martin, James R. and White, Peter R. R. (2005) The Language of Evaluation. New York: Palgrave.

Pang, Bo and Lee, Lillian (2008) Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2 (1–2): 1–135.

Pang, B., Lee, L. and Vaithyanathan, S. (2002) Thumbs up? Sentiment classification using Machine Learning techniques, Proceedings of Conference on Empirical Methods in NLP, 79–86.

Stillar, Glenn (1998) Analyzing Everyday Texts: Discourse, Rhetoric and Social Perspectives. London: Sage.

Stubbs, Michael (1986) ‘A matter of prolonged field work’: Notes towards a modal grammar of English. Applied Linguistics, 7 (1): 1–25.

Taboada, Maite (2008) SFU Review Corpus [Corpus]. Vancouver: Simon Fraser University,

Taboada, Maite, Anthony, C., Brooke, J., Grieve, J. and Voll, K. (2008a) SO-CAL: Semantic Orientation CALculator. Vancouver: Simon Fraser University.

Taboada, Maite, Anthony, C. and Voll, K. (2006a) Creating semantic orientation dictionaries, Proceedings of 5th International Conference on Language Resources and Evaluation (LREC) 427–432. Genoa, Italy.

Taboada, M. Brooke, J., Tofiloski, M., Voll, K. and Stede, M. (2011) Lexicon-based methods for sentiment analysis. Computational Linguistics 37 (2): 267–307.

Taboada, M., Gillies, M. A., and McFetridge, Paul (2006b) Sentiment classification techniques for tracking literary reputation, Proceedings of LREC Workshop: Towards Computational Models of Literary Analysis 36–43. Genoa, Italy.

Taboada, Maite and Grieve, Jack (2004) Analyzing appraisal automatically. In Y. Qu, J. G. Shanahan and J. Wiebe (eds) Proceedings of AAAI Spring Symposium on Exploring Attitude and Affect in Text (AAAI Technical Report SS-04-07) 158–161. Stanford University, CA: AAAI Press.

Taboada, M., Voll, K. and Brooke, J. (2008b) Extracting Sentiment as a Function of Discourse Structure and Topicality (Technical Report No. 2008-20): Simon Fraser University.

Turney, Peter (2002) Thumbs up or thumbs down? Semantic orientation applied to un­supervised classification of reviews, Proceedings of 40th Meeting of the Association for Computational Linguistics 417–424. Philadelphia, PA.

Voll, Kimberly and Taboada, M. (2007) Not all words are created equal: Extracting semantic orientation as a function of adjective relevance, Proceedings of the 20th Australian Joint Conference on Artificial Intelligence 337–346. Gold Coast, Australia.

White, Peter (2002) Appraisal. In J.-O. Östman and J. Verschueren (eds) Handbook of Pragmatics 1–27. Amsterdam: John Benjamins.

White, Peter R. R. (2003) An Introductory Course in Appraisal Analysis. Retrieved March 16, 2009 from

Whitelaw, C., Garg, N. and Argamon, S. (2005) Using Appraisal groups for sentiment analysis, Proceedings of ACM SIGIR Conference on Information and Knowledge Management (CIKM 2005) 625–631. Bremen, Germany.

Whitelaw, Casey and Patrick, Jon (2004) Selecting systemic features for text classification, Proceedings Australasian Language Technology Workshop, 93–100. Sydney, Australia.



How to Cite

Taboada, M., & Carretero, M. (2012). Contrastive analyses of evaluation in text: Key issues in the design of an annotation system for attitude applicable to consumer reviews in English and Spanish. Linguistics and the Human Sciences, 6(1-3), 275–295.



Registers, Genres and Text Types

Most read articles by the same author(s)