A better assessment of personality in language-learning situations


  • Omar Karlin Toyo University
  • Sayaka Karlin Toyo Gakuen University




Personality, Rasch Analysis, Factor Analysis, Language-Learning Situations, Validity


Two hundred and sixty-two items, using a five-point Likert scale, were created and administered to 287 Japanese university students to measure five personality factors based on the extraversion, emotional stability, openness, agreeableness, and conscientiousness factors of the Big Five model of personality (McCrae and Costa, 1987). These items were then culled to 50 items by examining their suitability through factor analysis and Rasch analysis, resulting in two 50-item versions of the Questionnaire of English Environment Personality (QuEEP). Both versions of the QuEEP included 10 items for each of the five personality factors in the Big Five. Both versions of the QuEEP outperformed an established personality instrument not specifically intended for language-learning situations, the International Personality Item Pool Big Five Factor Markers test (IPIP BFFM), on five measures of validity, including content validity, the substantive aspect of construct validity, structural validity, external validity, and generalizability. As a result, it was concluded that the QuEEP, specifically the version derived from the Rasch analysis, was more effective than the IPIP BFFM at capturing personality situated in language-learning situations.

Author Biographies

Omar Karlin, Toyo University

Omar Karlin is currently a Lecturer of English at Toyo University, and completed his Doctorate in TESOL from Temple University in 2015. His research interests include test construction, teaching listening, and the intersection of language and personality.

Sayaka Karlin, Toyo Gakuen University

Sayaka Karlin is currently an Adjunct Professor of English at Toyo Gakuen University, and completed her Masters in TESOL from Temple University in 2018. Her research interests include vocabulary acquisition and the intersection of language and personality.


Apple, M. (2011). The Big Five personality traits and foreign language speaking confidence among Japanese EFL students. Unpublished doctoral dissertation. Temple University.(UMI No. 3457819).

Bachman, L. F., and Palmer, A. S. (1996). Language Testing in Practice. Oxford: Oxford University Press.

Benson, J. (1987). Detecting item bias in affective scales. Educational and Psychological Measurement, 47 (1), 55–67. https://doi.org/10.1177/0013164487471008

Bhavsar, K. S., Hess, C. E., and Surface, E. A. (2006, February). Investigating the construct validity of Goldberg’s 50-item personality scale. Paper presented at the annual Industrial-Organizational/Organizational Behavior (IOOB) Graduate Student Conference, Fairfax, VA.

Block, J., (1961). The Q-sort Method in Personality Assessment and Psychiatric Research. Springfield, IL: Thomas. https://doi.org/10.1037/13141-000

Bond, T. G., and Fox, C. M. (2007). Applying the Rasch model: Fundamental Measurement in the Human Sciences (2nd ed.). Mahwah, NJ: Erlbaum.

Budner, S. (1962). Intolerance of ambiguity as a personality variable. Journal of Personality, 39, 29–50. https://doi.org/10.1111/j.1467-6494.1962.tb02303.x

Busch, D. (1982). Introversion-extraversion and the EFL proficiency of Japanese students. Language Learning, 32 (1), 109–132. https://doi.org/10.1111/j.1467-1770.1982.tb00521.x

Carrell, P. L., Prince, M. S., andAstika, G. G. (1996). Personality types and language learning in an EFL context. Language Learning, 46 (1), 75–99. https://doi.org/10.1111/j.1467-1770.1996.tb00641.x

Chen, Y., Jiang, Y., and Mu, Z. (2015). A survey study: The correlation between introversion/extroversion and oral English learning outcome. Journal of Language Teaching and Research, 6 (3), 581–587. https://doi.org/10.17507/jltr.0603.14

Comrey, A. L., and Lee, H. B. (1992). A First Course in Factor Analysis (2nd ed.). Hillsdale, NJ: Erlbaum.

Costa, P. T. Jr., and McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PiR) and NEO Five-Factor Inventory (NEO-FFR) Professional Manual. Odessa, FL: Psychological Assessment Resources.

Costa, P. T. Jr., and McCrae, R. R. (1995). Domains and facets: Hierarchical personality assessment using the Revised NEO Personality Inventory. Journal of Personality Assessment, 64 (1), 21–50. https://doi.org/10.1207/s15327752jpa6401_2

Davis, M. (2010). Personality and its effect on relationships and teaching and learning styles. International Schools Journal, 29 (2), 22–29.

Dewaele, J. M., andFurnham, A. (1999). Extraversion: The unloved variable in applied linguistic research. Language Learning, 49 (3), 509–544. https://doi.org/10.1111/0023-8333.00098

Dörnyei, Z. (2005). The Psychology of the Language Learner: Individual Differences in Second Language Acquisition. Mahwah, NJ: Erlbaum.

Ehrman, M., and Oxford, R. (1995). Cognition plus: Correlates of language learning success. Modern Language Journal, 79, 67–89. Retrieved from http://www.jstor.org.libproxy.temple.edu/stable/329394. https://doi.org/10.1111/j.1540-4781.1995.tb05417.x

Eysenck, H. J., and Eysenck, S. B. (1963). Manual for the Eysenck Personality Inventory. San Diego, CA: Educational and Industrial Testing Service.

Furr, R. M., and Funder, D. C. (2007). Behavioral observation. In R. W. Robins, R. C. Fraley, and R. F. Krueger (Eds), Handbook of Research Methods in Personality Psychology, 273–291. New York: Guilford.

Gan, Z. (2011). An investigation of personality and L2 oral performance. Journal of Language Teaching and Research, 2 (6), 1259–1267. https://doi.org/10.4304/jltr.2.6.1259-1267

Goldberg, L. R. (1990). An alternative ‘description of personality’: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. https://doi.org/10.1037/0022-3514.59.6.1216

Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measur­ing the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, and F. Ostendorf (Eds), Personality Psychology in Europe, Vol. 7, 7–28. Tilburg, The Netherlands: Tilburg University Press.

Gotlib, I. H., and Meyer, J. P. (1986). Factor analysis of the multiple affect adjective check list: A separation of positive and negative affect. Journal of Personality and Social Psych­ology, 50 (6), 1161–1165. https://doi.org/10.1037/0022-3514.50.6.1161

Gough, H. G., and Heilbrun, A. B. (1983). The Adjective Check List Manual. Palo Alto, CA: Consulting Psychologists Press.

Gow, A. J., Whiteman, M. C., Pattie, A., and Deary, I. J. (2005). Goldberg’s ‘IPIP’ Big-Five Factor Markers: Internal consistency and concurrent validation in Scotland. Personality and Individual Differences, 39, 317–329. https://doi.org/10.1016/j.paid.2005.01.011

Heuchert, J. W. P., Parker, W. D., Stumpf, H., and Myburgh, C. P. H. (2000). The five-factor model in South African college students. American Behavioral Scientist, 44 (1), 112–125. https://doi.org/10.1177/00027640021956125

Hogan, R. (1986). Hogan Personality Inventory Manual. Minneapolis, MN: National Comp­uter Systems.

Jessee, S. A., O’Neill, P. N., and Dosch, R. O. (2006). Matching student personality types and learning preferences to teaching methodologies. Journal of Dental Education, 70, 644–651.

Knight, R. G., Chisholm, B. J., Marsh, N. V., and Godfrey, H. P. (1988). Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. Journal of Clinical Psychology, 44 (2), 203–206. https://doi.org/10.1002/1097-4679(198803)44:2<203::AID-JCLP2270440218>3.0.CO;2-5

Lanning, K. (1994). Dimensionality of observer ratings on the California adult Q-Set. Journal of Personality and Social Psychology, 67 (1), 151–160. https://doi.org/10.1037/0022-3514.67.1.151

Lee, C. H., Shim, J., and Yoon, A. (2005). The review about the development of Korean linguistic inquiry and word count. Korean Journal of Cognitive Science, 16, 93–121.

Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16, 878. Retrieved from http://www.rasch.org/rmt/rmt162f.htm. http://www.rasch.org/rmt/rmt162f.htm

Linacre, J. M. (2009). Winsteps (Version 3.68) [Computer Software]. Beaverton, OR: Winsteps.com.

McCrae, R. R.,and Costa, P. T. Jr. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81–90. https://doi.org/10.1037/0022-3514.52.1.81

Mehrabian, A. (1970). The development and validation of measures of affiliative tendency and sensitivity to rejection. Educational and Psychological Measurement, 30, 417–428. https://doi.org/10.1177/001316447003000226

Meyers, K. S., Gamst, G., and Guarino, A. J. (2006). Applied Multivariate Research: Design and Interpretation. Thousand Oaks, CA: Sage.

Naiman, N., Frolich, M., Stern, H. H., and Todesco, A. (1978). The Good Language Learner. Research in Education Series No. 7. Toronto, ON: The Ontario Institute for Studies in Education.

Paulhus, D. L., and Vazire, S. (2007). The self-report method. In R. W. Robins, R. C. Fraley, and R. F. Krueger (Eds), Handbook of Research Methods in Personality Psychology, 273–291. New York: Guilford.

Pilotte, W. J., and Gable, R. K. (1990).The impact of positive and negative item stems on the validity of a computer anxiety scale. Educational and Psychological Measurement, 50 (3), 603–610. https://doi.org/10.1177/0013164490503016

Singh, J. (2004). Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example. Journal of Business Research, 37, 184–208. https://doi.org/10.1016/S0148-2963(01)00302-2

Smith, R. M. (2000). Fit analysis in latent trait measurement models. Journal of Applied Measurement, 1 (2), 199–218.

Stevens, J. P. (2002). Applied Multivariate Statistics for the Social Sciences (4th ed.). Hillsdale, NJ: Erlbaum.

Saucier, G., Bel-Bahar, T., and Fernandez, C. (2007). What modifies the expression of personality tendencies? Defining basic domains of situation variables. Journal of Personality, 75, 479–504. https://doi.org/10.1111/j.1467-6494.2007.00446.x

Tabachnick, B. G., and Fidell, L. S. (2007). Using Multivariate Statistics (5th ed.). Boston, MA: Pearson.

Ten Berge, M. A., and Raad, B. D. (2002). The structure of situations from a personality perspective. European Journal of Personality, 16, 81–102. https://doi.org/10.1002/per.435

Verhoeven, L., and Vermeer, A. (2002). Communicative competence and personality dimensions in first and second language learners. Applied Psycholinguistics, 23, 361–374. https://doi.org/10.1017/S014271640200303X

Wiggins, J. S., Trapnell, P., and Phillips, N. (1988). Psychometric and geometric characteristics of the Revised Interpersonal Adjectives Scales (IAS-R), Multivariate Behavioral Research, 23, 517–530. https://doi.org/10.1207/s15327906mbr2304_8

Wolfe, E. W., and Smith, E. V. (2007). Instrument development tools and activities for measure validation using Rasch models: Part II – Validation activities. Journal of Applied Measurement, 8 (2), 202–234. Retrieved from http://www.jampress.org/backissue.htm

Wood, D., and Roberts, B. W. (2006). Cross-sectional and longitudinal tests of the personality and role identity structural model (PRISM). Journal of Personality, 74, 779–810. https://doi.org/10.1111/j.1467-6494.2006.00392.x

Zhang, W., Su, D., and Liu, M. (2013). Personality traits, motivation, and foreign language attainment. Journal of Language Teaching and Research, 4(1), 58–66. https://doi.org/10.4304/jltr.4.1.58-66

Zheng, L., Goldberg, L. R., Zheng, Y., Zhao, Y., Tang, Y., and Liu, L. (2008). Reliability and concurrent validation of the IPIP Big-Five Factor Markers in China: Consistencies in factor structure between Internet-obtained heterosexual and homosexual samples. Personality and Individual Differences, 45 (7), 649–654. https://doi.org/10.1016/j.paid.2008.07.009



How to Cite

Karlin, O., & Karlin, S. (2018). A better assessment of personality in language-learning situations. Journal of Research Design and Statistics in Linguistics and Communication Science, 4(1), 3–49. https://doi.org/10.1558/jrds.31833