RATER PERCEPTIONS IN TESTS OF ORAL EXPRESSION
Abstrakt
Abstract
Assessment of spoken performance is often viewed in terms of various different
categories relating to the language aspect of performance including provisions for vocabulary
use and linguistic resources, as well as accuracy of use, contrasted with fluency. Such
categories are often coupled with several other categories relating to other aspects of spoken
performance, and amongst those one almost always featured in rating schemes is that
of pronunciation. It is interesting to observe what happens when this category is omitted in
rating scales and how this affects the perceptions of oral examiners. In this investigation,
data from an Electronic Performance Support System (EPSS) obtained in training and
fine-tuning examiners of oral performance in tests of spoken performance in English from
different educational contexts is used to draw some intriguing research conclusions.
Key words: Electronic Performance Support System (EPSS)
Streszczenie
Systemy oceniania języka mówionego posługują się rozmaitymi skalami zawierającymi
różne kategorie odnoszące się do jakości języka zawierające podkategorie obejmujące
słownictwo i środki językowe, jak również poprawność językową zwykle na tle biegłości
i potoczystości w mówieniu. Te, z kolei, uzupełnione bywają rozlicznymi jeszcze
kategoriami, wśród których właściwie zawsze poczesne miejsce zajmuje wymowa. Warto
jednak przyjrzeć się sytuacji, gdzie tej kategorii brak w skalach oceniania i w jaki sposób
wpływa to na percepcje egzaminatorów języka mówionego. W tym badaniu dane pochodzące
z Elektroniczne Systemu Wspierania Oceniania (ang. EPSS) zgromadzone w trakcie
treningu i doszkalania egzaminatorów języka angielskiego z różnych kontekstów edukacyjnych
pozwalają na sformułowanie intrygujących wniosków badawczych
Słowa kluczowe: Elektroniczny system Wspierania Ocen (EPSS),
Bibliografia
REFERENCES
Bachman L.F., and Palmer, A. 2002. Language Testing in Practice. Oxford: University Press.
Barna M.L. 1994. Stumbling Blocks in Intercultural Communication. In Intercultural Communication,
eds. L.A. Samovar and R.E. Porter. Wadsworth.
Bennet M.J. 1993. Towards Ethnorelativism: A Developmental Model of Intercultural Sensitivity.
In Education for the Intercultural Experience. Yarmouth: Intercultural Press.
Bond T.G., and Fox, C.M. 2007. Applying the Rasch Model. Fundamental Measurement in the
Human Sciences. Toledo: University of Toledo Press.
Brown A., 2006. “An examination of the rating process in the revised IELTS Speaking Test”.
IELTS Research Reports Volume 6, IELTS Australia, Canberra and British Council, London.
Cavé C., Guaïtella I., and Santi S. 2002. “Eyebrow movements and voice variations in dialogue
situations”. In Proceedings of the 7th International Conference on Spoken Language Processinig,
eds. Hansen, J.H.L and Pellom, B.
Chambers L., Ingham K. 2011. “The BULATS Online SpeakingTest”. ESOL Research Notes,
vol 34.
De Velle S. 2008. “The revised IELTS Pronunciation scale”. ESOL Research Notes, vol 34.
Fulcher G. 2003. Testing Second Language Speaking. Harlow: Pearson Longman.
Guaïtella I., Santi S., Lagrue B., Cavé Ch. 2009. “Are Eyebrow Movements Linked to Voice
Variations and Turn-taking in Dialogue? An Experimental Investigation”. Language and
Speech 57.
Hall E., and Hall M. 1990. Understanding cultural differences: Germans, French and Americans.
Verlag: Intercultural Press.
Hall E. 1959. The silent language. Garden City: Doubleday.
Hall E. 1966. The hidden dimension. Garden Day: Doubleday Anchor Books.
Hawkey R. 2004. “A Modular Approach to Testing English Language Skills: The development
of the Certificates in English Language Skills, CELS, examinations”. Cambridge ESOL
Research Notes Volume 16.
Hildreth P.M. and Kimble Ch. 2004. Knowledge networks: innovation through communities of
practice. Idea Group Inc.
Hubbard C., Gilbert S. and Pidcock J. 2006. “Assessment processes in speaking tests: a pilot
verbal protocol study”. ESOL Research Notes, vol 24.
Hubbard Ch. 2011. “Cambridge ESOL Professional Support Network Extranet: Development
and impact”. ESOL Research Notes, vol 49.
Hymes D. 1964. Introduction: “Toward Ethnographies of Communication”. American Anthropologist
, 6.
Hymes D. 1972. On communicative competence. In Sociolinguistics, eds J. B. Pride and J.
Holmes. Harmondsworth: Penguin.
Krahmer E. and Swerts M. 2004. More about brows. In From brows to trust: Evaluating embodied
conversational agents, eds. Ruttkay, Z. and Pelachaud, C., Norwell: Kluwer Academic
Press.
Krakowian P. 2010. Modern Test Theory Explained. Warszawa: Scholar.
Krakowian P. 2011. Investigating Rater Performance in Tests of Oral Expression. Łódź: Wydawnictwo
Uniwersytetu Łódzkiego
Lustig M.W. and Koester J. 1993. Intercultural Competence. Interpersonal Communication
across Cultures. New York: Harper Collins College Publishers.
Lustig M.W. and Koester J. 2009. Intercultural Competence: Interpersonal Communication
Across Cultures. Boston: Allyn and Bacon.
Martinez L. 2009. “How Examiners of Different Severity Grade Candidates of Different Ability”.
Test Insights 2009. Measurement Research Associates, Inc.
Martinez L. 2010. “The Relationship between Examiner Severity and Consistency”. Test Insights
Measurement Research Associates, Inc.
Nakane I. 2007. Silence in Intercultural Communication: perceptions and performance. Amsterdam:
John Benjamins.
O’Sullivan B. 2008. Modelling Performance in Tests of Spoken Language. Frankfurt: Peter
Lang
Ockey G. J. 2009. “The effects of group members’ personalities on a test taker’s L2 group oral
discussion test scores”. Language Testing, 26.
Orr M. 2002. “The FCE Speaking test: using rater reports to help interpret test scores”. System,
vol 30, no 2.
Saint-Onge H. and Wallace D. 2003. Leveraging communities of practice for strategic advantage.
Oxford: Butterworth-Heinemann.
Scollon R. and Scollon S.W. 1995. Intercultural Communication. Oxford: Blackwell
Seed G. 2012. “Perceptions of authenticity in academic test tasks”. ESOL Research Notes, vol
Seiter J., Weger H., Jensen A. and Kinzer H. 2010. “The Role of Background Behavior in Televised
Debates: Does Displaying Nonverbal Agreement and/or Disagreement Benefit Either
Debater?” Journal of Social Psychology Vol. 150, No. 3.
Taylor L., and Falvey, P. 2007. “IELTS Collected Papers: Research in speaking and writing
assessment”. Cambridge ESOL Research Notes Volume 19.
Tynan R. 2015. “Creating ePortfolios to facilitate and evidence progress using learning technologies”.
Cambridge Exams Research Notes, vol 61
Weir C. and Milanovic M. 2003. “Continuity and Innovation: Revising the Cambridge Proficiency
in English Examination 1913-2002”. Cambridge ESOL Research Notes issue 15
Wenger E. 1998. Communities of Practice: Learning, Meaning, and Identity. Cambridge: University
Press.
Wenger E., McDermott R.A. and Snyder W. 2002, Cultivating communities of practice: a guide
to managing knowledge. Harvard: Business Press
Wilson M. 2005. Constructing Measures: An Item Response Model, Mahwah: Lawrence Erlbaum
Associates
Wright B.D. and Masters G. 1982. Rating Scale Analysis. San Diego: MESA Press
Wright B.D. and Stone M.H. 1979. Best test design. San Diego: MESA Press
