Hoppa till sidinnehåll
Bedömning

Investigating and Validating Spoken Interactional Competence: Rater Perspectives on a Swedish National Test of English

Publicerad:2018-12-04
Uppdaterad:2019-01-08

Linda Borger har undersökt bedömningen av de muntliga nationella delproven i engelska i gymnasieskolan utifrån olika validitetsaspekter.

Författare

Linda Borger

Handledare

Professor Gudrun Erickson, Göteborgs universitet Professor Monica Rosén, Göteborgs universitet

Opponent

Professor Barry O’Sullivan, British Council, London

Disputerat vid

Göteborgs universitet

Disputationsdag

2018-12-07

Titel (eng)

Investigating and Validating Spoken Interactional Competence: Rater Perspectives on a Swedish National Test of English

Institution

Institutionen för pedagogik och specialpedagogik

Investigating and Validating Spoken Interactional Competence: Rater Perspectives on a Swedish National Test of English

This thesis aims to explore different aspects of validity evidence from the raters’ perspective in relation to a paired speaking test, part of a high-stakes national test of English as a Foreign Language (EFL) in the Swedish upper secondary school. Three empirical studies were undertaken with the purpose of highlighting (1) the scoring process, (2) the construct underlying the test format, and (3) the setting and test administration. In Study I and II, 17 teachers of English from Sweden, using national performance standards, and 14 raters from Finland and Spain, using scales from the Common European Framework of Reference for Languages (CEFR), rated six audio-recorded paired performances, and provided written comments to explain their scores and account for salient features. Inter-rater agreement was analysed using descriptive, correlational and reliability statistics, while content analysis was used to explore raters’ written comments. In Study III, 267 upper secondary teachers of English participated in a nation-wide online survey and answered questions about their administration and scoring practices as well as their views of practicality. The responses were analysed using descriptive statistics and tests of association. Study I revealed that raters observed a wide range of students’ oral competence, which is in line with the purpose of the test. With regard to inter-rater agreement, the statistics indicated certain degrees of variability. However, in general inter-rater consistency was acceptable, albeit with clear room for improvement. A small-scale, tentative comparison between the national EFL standards and the reference levels in the CEFR was also made. In Study II, raters’ interpretation of the construct of interactional competence was explored. The results showed that raters attended to three main interactional resources: topic development moves, turn-taking management, and interactive listening strategies. As part of the decision-making process, raters also considered the impact of test-takers’ interactional roles and how students’ performances were interrelated, which caused some challenges for rating. Study III investigated teachers’ implementation practices and views of practicality. The results revealed variations in how the national speaking test was implemented at the local level, which has clear implications for standardisation but must be considered in relation to the decentralised school system that the national tests are embedded in. In light of this, critical aspects of the setting, administration and scoring procedures of the national EFL speaking tests were highlighted and discussed. In the integrated discussion, the different aspects of validity evidence resulting from the empirical data are analysed in relation to a socio-cognitive framework for validating language tests (O’Sullivan & Weir, 2011; Weir, 2005). It is hoped that the thesis contributes to the field of speaking assessment in two ways: firstly by showing how a theoretical framework can be used to support the validation process, and secondly by providing a concrete example of validation of a high-stakes test, highlighting positive features as well as challenges to be addressed.

Forskningsbevakningen presenteras i samarbete med

forskningsinstitutet Ifous

Läs mer
Stockholm

Modersmål

Välkommen till Skolportens fortbildning för dig som undervisar i modersmål! Konferensen ger både praktiska verktyg för din undervisning och fördjupning inom viktiga ämnen. Delta på plats i Stockholm den 5 februari eller på webbkonferensen, öppen 12 februari–5 mars.
Läs mer och boka
Åk F–9
5 feb 2025
Digital temaföreläsning

Flickor med autism och adhd i skolan

Skolportens digitala tema för dig som vill veta mer om flickor med autism och adhd i skolan! Titta när det passar dig mellan 7 april och 19 maj. Förutom forskningsbaserad information och praktiska verktyg får du med dig diskussionsfrågor inför samarbetet med kollegor.
Läs mer och boka
Åk F–Gy
7 apr – 19 maj 2025
Dela via: 

Relaterade artiklar

Relaterat innehåll

Senaste magasinen

Läs mer

Nyhetsbrev