 Center for Advanced Research on Language Acquisition (CARLA) | ||
 |
||
|
Introduction to the Speaking Assessment Rater TrainingAbout the CoSA The Contextualized Speaking Assessment (CoSA) is designed to determine if language learners are proficient at the Intermediate-Low (IL) level of the ACTFL Proficiency Guidelines for the purposes of articulation and fulfilling language study requirements. IL is the minimum target level for learners who have completed at least three years of language study at the secondary level or for post-secondary students who have completed 120 contact hours of language study (or an equivalent combination of secondary and post-secondary instruction). The CoSA are not intended to be used to identify proficiency levels above or below IL. The CoSA consists of five scored tasks, a warm-up and a wind-down. All scored tasks target functions, topics and discourse within situations appropriate to the Intermediate-Low level and identified by the ACTFL Guidelines. In responding to the five scored CoSA tasks, learners are asked to demonstrate that they can:
The CoSA takes about 20 minutes to administer. Each rated student sample is a minute in length. Criteria for the rubric used to rate the CoSA are derived from the ACTFL Proficiency Guidelines (1986) for Intermediate-Low speakers which had not yet been revised at the time the CoSA was developed and the Minnesota Articulation Project (MNAP) prepared evaluation criteria for the assessment.
Able to handle successfully a limited number of interactive, task-oriented, and social situations. Can ask and answer questions, initiate and respond to simple statements, and maintain face-to-face conversation, although in a highly restricted manner and with much linguistic inaccuracy. Within these limitations, can perform such tasks as introducing self, ordering a meal, asking directions, and making purchases. Vocabulary is adequate to express only the most elementary needs. Strong interference from native language may occur. Misunderstandings frequently arise, but with repetition, the Intermediate-Low speaker can generally be understood by sympathetic interlocutors.
Source: ACTFL Speaking Guidelines, Intermediate Low (1986) Members of the MNAP also felt strongly that for students to perform the functions of the Intermediate-Low level (describe self, order a meal, make purchases, etc.) in a manner comprehensible to a sympathetic interlocutor and to succeed in second year language courses at a post-secondary level, students must poses the ability to correctly use present tense of common verbs most of the time. CoSA rating criteria are in the form of a holistic rubric which asks raters to make judgments by forming an overall
In a holistic rubric, all aspects of a performance are equally considered during evaluation.
impression of a performance. The rubric is an attempt to simplify the rating process so that many students’ performances can be evaluated in a reasonable amount of time. Nonetheless, rating is difficult and time-consuming even when the decision to be made is reduced to whether or not a learner’s performance is at least at the Intermediate-Low level.
In holistic evaluation, raters make judgments by forming an overall impression of an entire performance and matching it to the best fit from among the descriptions for several criteria (e.g., range of vocabulary + grammatical accuracy + fluency) on a scale of generally four to six levels. Because the CoSA targets the Intermediate-Low level on the ACTFL Proficiency Scale, the holistic rubric for rating the CoSA consists of one level and five criteria. For more on holistic evaluation, please see: Virtual Assessment Center Learners adopt different approaches to taking a test of this type. One difficulty in rating their performance is that within a given level, a range of performance will be observed. Some learners proceed very carefully and only use structures they know how to say correctly. The quantity of speech they produce tends to be limited, and so does the quantity of error. Other learners respond very creatively to the tasks and exceed the amount of speech expected. They may also make many errors. The challenge for raters is to make judgments based on what they hear in the language samples. Ideally, more than one rater would evaluate the tests, and raters could discuss and compare test takers’ performances and their understanding of the criteria. Frequent consultations with other evaluators can help avoid “rater drift.” When raters drift from the criteria, they rely o their intuition or compare learners to each other. Start the Rater Training:Please choose your language - French, German, or Spanish
|
|