Evaluation

Process: Types of rubrics

Rubrics are generally categorized as generic or task-specific. As is so often the case in assessment, the line between the two categories may blur so that rating instruments appear more or less generic or task-specific. Indeed, many task-based rubrics are adaptations of generic scales. It is also possible to design hybrid rubrics that combine features of both types.

Generic rubrics can be applied to a number of different tasks. In language assessment, one frequently finds generic rubrics used with assessment tasks within a modality (generally writing and speaking) or mode (interpersonal and presentational). A truly generic rubric could be applied to any task within the same modality or mode.

The dimensions in a generic rubric for second-language assessment often emphasize features of language production, such as comprehensibility, accuracy, and vocabulary, without making reference to specific content or task details. Generic rubrics are often derived from models of language proficiency and/or second language acquisition.

Figure 1 shows a sample generic, analytic rubric for oral presentations (presentational mode) for Intermediate level learners adapted from the ACTFL Performance Guidelines for K-12 Learners (K-12 Guidelines). This example reflects a focus on features of second-language production, but additional dimensions might be included in order to measure such aspects of oral presentation as content coverage, organization, connection with audience, elocution, use of graphics, and so on.

The terminology in Figure 1 is accessible to language-teaching professionals, but it may not provide meaningful feedback to learners. Large-scale and external assessments for purposes such as certification, placement, articulation, and program evaluation often use generic scales that contain a high degree of professional language and require modification for classroom use.

A high school French teacher who commented on the rubric in Figure 1 indicated that, for classroom use, he prefers a rubric with short descriptors that he can take in at a glance and that serve primarily to refresh his memory of what performance is like at each step on the scale. Because he has only a few seconds to evaluate each student, and because he wants to spend as little class time as possible explaining the terms in the rubric to his students, he preferes simple vocabulary that neither he nor his students must ponder (J.-L. Roche, personal communication, October 22, 2002). Figure 2 presents an adaptation of the oral presentation generic rubric for classroom use.

It is certainly most efficient to design or identify rubrics that can be used for multiple purposes, but when weighing the use of generic versus task-based rubrics, efficiency is not the only important criterion. Tedick (2002) writes: "While some rubrics are created in such a way as to be generic in scope for use with any number of writing or speaking tasks, it is best to consider the task first and make sure that the rubric represents a good fit with the task and your instructional objectives. Just as a variety of task-types should be used in language classrooms, so should a variety of rubrics and checklists be used for assessing performance on those tasks. (p. 37)" For learners who are new to performance assessment and evaluation, Tedick recommends making students comfortable with the process by first using generic rubrics and gradually introducing task-specific rubrics.

Task-specific rubrics are used with particular tasks, and their criteria and descriptors reflect specific features of the elicited performance. Rubrics developed for a defined group of tasks within a modality or mode, such as writing narratives, performing role-plays, or exchanging e-mail messages may combine elements of language production with dimensions related to the content and language function(s) of the lesson/task. For example, if an assessment task requires learners to use a series of pictures to tell a story in the past about a visit to monuments in Paris, the scoring criteria would focus on language competencies related to narration in past tense along with one or more dimensions measuring content and cultural knowledge. A possible rubric for this task is shown in Figure 3.

Rubrics that combine features of generic and task-specific rubrics are very useful in classroom assessment because they provide feedback to learners on broad dimensions of language production along with their performance on the particular competencies and knowledge targeted by course content and aligned assessments. When adapting the rubrics for other tasks, teachers may keep the generic language production elements as they are and change one or two categories to focus on task expectations. For example, one might add level-appropriate, generic dimensions such as pronunciation or fluency to the task-specific categories of narration, use of past tense, and knowledge about monuments of Paris to the rubric in Figure 3.

Holistic, analytic, primary trait and multiple trait rubrics may be seen as different ways of selecting and organizing rating criteria. These rubric types come from different contexts, and although their particular uses and characteristics have converged in current practice, there are some general guidelines for choosing among them. In addition, each type has advantages and disadvantages.

In practice, you will probably find considerable variability in how rubric types are identified. Holistic and analytic scales may be identified as generic or task-specific, or they may include rating criteria of both types. Primary and multiple trait rubrics are essentially task-specific, but general language production categories may be added to multiple trait rubrics.