Types
of rubrics: Analytic scales
Analytic
scales are usually associated with generic rubrics and tend to focus
on broad dimensions of writing or speaking performance. These dimensions
may be the same as those found in a generic, holistic scale, but
they are presented in separate categories and rated individually.
Points may be assigned for performance on each of the dimensions
and a total score calculated.
Traditionally, analytic rubrics are associated with large-scale
assessment of general dimensions of language performance. However,
analytic rubrics certainly can be created or adapted for use in
classroom settings and with particular tasks (e.g., Taggart
et al., 1998). These rubrics often combine performance categories
from a generic rubric with categories directly related to a task,
such as demonstrating understanding of specific lesson content (Moskal,
2000). In practice, the names "analytic rubric" and
"multiple trait rubric" may be used interchangeably.
Performance dimensions commonly found in analytic rubrics include:
Speaking & Writing
- Content
- Vocabulary
- Accuracy/Grammar/Language Use
- Task fulfillment
- Appropriate use of language
- Creativity
- Sentence structure/Text type
- Comprehensibility
Writing
- Organization
- Style
- Mechanics
- Coherence and Cohesion
Speaking
- Fluency
- Pronunciation
- Intonation
Figure F presents an adaptation of a well-known analytic scale
for evaluating ESL
writing performance. Describing this rubric, Tedick
(2002) writes: "Note that the scale assigns different weights
to different features. This allows a teacher to give more emphasis
to content than to grammar or mechanics, for example. The option
to weigh characteristics on the scale represents an advantage to
analytic scoring." (p. 35).
Figure F2 shows an analytic scale for role plays and interviews
used with
students in first-year French courses at the University of Minnesota.
This rubric can be used with other languages. In this example, all
criteria are weighted equally.
There are more sample analytic rubrics in the Evaluation >
Examples section.
Advantages:
- They provide useful feedback to learners on areas of strength
and weakness.
- Their dimensions can be weighted to reflect relative importance.
- They can show learners that they have made progress over time
in some or all dimensions when the same rubric categories are
used repeatedly (Moskal,
2000).
Disadvantages:
- "The whole is greater than the sum of its parts."
Tedick
(2002) notes: "Separate scores for different aspects
of a student’s writing or speaking performance may be considered
artificial in that it does not give the teacher (or student) a
good assessment of the "whole" of a performance."
(p. 36).
- They take more time to create and use.
- There are more possibilities for raters to disagree. It is more
difficult to achieve intra- and inter-rater reliability on all
of the dimensions in an analytic rubric than on a single score
yielded by a holistic rubric.
- There is some evidence that raters tend to evaluate grammar-related
categories more harshly than they do other categories (McNamara,
1996), thereby overemphasizing the role of accuracy in providing
a profile of learners' proficiency.
|
