Evaluation:

Process: Creating rubrics

There are many rubric resources available to teachers—online and in published materials—so the first piece of advice we have to offer is: Find and adapt existing rubrics! It is rare to find a rubric that is exactly right for your situation and your students, but by using rubrics that have worked well for others as a starting point, you can save a great deal of time.

There are many rubric formats. In the grid format shown here, which is one of the possible ways to lay out a rubric, we illustrate a few common, frequently recommended, features of multiple trait rubrics:

• An even number (4 or 6) of levels of performance on the scale. When there is an odd number of levels, the middle level tends to become a catch-all category. With an even number of levels, raters have to make a more precise judgement about a performance when its quality is not at the top or bottom of the scale.
• High to low scale. In the graphic, the highest level of performance is described at the left. Students read first the description of an exemplary performance in each criterion. A few labels for a four-point scale include:

  4	3	2	1
  Exemplary	Excellent	Acceptable	Unacceptable
  Exceeds expectations	Meets expectations	Progressing	Not there yet
  Superior	Good	Fair	Needs work

• Limited number of dimensions or criteria. The criteria are those components that are most important to evaluate in the given task and instructional context. A rubric with too many dimensions may be unworkable in classroom assessment.
• Equal steps along the scale. The difference between 4 and 3 should be equivalent to the difference between 3 - 2 and 2 - 1. "Yes, and more", "Yes", "Yes, but", and "No" are ways for the rubric developer to think about how to describe performance at each scale point. Some common descriptive terms are listed in the chart below.

	4	3	2	1
Task requirements	All	Most	Some	Very few or none
Frequency	Always	Usually	Some of the time	Rarely or not at all
Accuracy	No errors	Few errors	Some errors	Frequent errors
Comprehensibility	Always comprehensible	Almost always comprehensible	Gist and main ideas are comprehensible	Isolated bits are comprehensible
Content coverage	Fully developed, fully supported	Adequately developed, adequately supported	Partially developed, partially supported	Minimally developed, minimally supported
Vocabulary Range Variety	Broad Highly varied; non-repetitive	Adequate Varied; occasionally repetitive	Limited Lacks variety; repetitive	Very limited Basic, memorized; highly repetitive

 

Considerations for Task-Based Rubric Development*

Getting Started: Determining the criteria that will be valued for a particular task.

1. Brainstorm all the possible elements or criteria that could be assessed in the performance task.
2. Determine which elements are "non-negotiable." Which criteria could be part of the task description as baseline requirements or provided as a checklist?
3. Prioritize the elements that are left:
   

   What are the content and language goals of the unit? What do you really want the students to emphasize in their performance? How important is the overall "look" of the project (interest, appeal, creativity, neatness)? Is culture represented in the rubric, if applicable? Are the standards you targeted represented in the rubric?

4. Determine 3 to 5 elements or criteria that will be incorporated in the rubric to define a quality performance. In trying to be thorough, an unwieldy rubric may be constructed, with so many elements being assessed that the rubric is time-consuming to fill out or oral performances will have to be taped (audio or video) in order to repeat them several times for the purposes of assessment.

Considering the Levels of the Rubric: Determining the number of levels and defining them.

1. How many levels of performance do you wish to include in the rubric? How should they be defined? For example, "does not meet expectations," "meets expectations," "exceeds expectations." Or you may choose to use simply a 3, 4, or 5-point scale, noting, however, that a 3-point scale does not account for the fluctuation that exists within the average range. Some suggest that a 4-point scale is ideal, and that more than 4 points makes a scale cumbersome and difficult to use.
2. Consider the elements or criteria you have chosen one at a time. Begin with the highest level of the scale to define top quality performance. This is the level that you want all students to achieve and it should be challenging. How would you describe a representation that exceeds expectations? meets expectations? does not meet expectations?
3. Are the levels you have created parallel? That is, are the criteria present in all levels?
4. Is there continuity in the difference between the criteria for exceeds vs. meets, and meets vs. does not meet expectations? The difference between a 2 and a 3 performance should not be more than the difference between a 3 and a 4 performance.
5. Do the levels reflect variants in quality and not a shift in importance of the criteria?
6. Is there an expectation of quality at the average (meets expectations) level of the scale?

Other issues to consider for rubric creation:

• Are the characteristics of each performance level described clearly? Will students be able to self-assess with the descriptors given? Will the descriptors give students enough information to know what they need to improve?
• Does the rubric adequately reflect the range of levels at which students may actually perform given tasks?
• Are the criteria at each level defined clearly enough to ensure that scoring is accurate, unbiased and consistent? Could several teachers use the rubric and score a student’s performance within the same range?
• Does the rubric attend to process as well as product?
• Are all criteria equally important, or does it make sense to weight an element more than the others?
• Are you attending carefully to the language used in the rubric? Use demonstrative verbs. Keep to observable behaviors. Avoid negatives ("begins without preparation" vs. "does not prepare"). Be specific. Instead of "many errors" you may want to specify "six or more errors". At the same time, be sure the rubric is generally qualitative in nature rather than quantitative.

Other issues to consider when using rubrics:

• Rubrics need to be piloted or field tested.
• Rubrics need to be discussed with students to create an understanding of expectations; you cannot write a paragraph defining each word in the rubric.
• Are the criteria at each level defined clearly enough to ensure that scoring is accurate, unbiased and consistent? Could several teachers use the rubric and score a student’s performance within the same range?
• There is a fine balance between modeling excellent work and creating a "template" that is replicated by the students ad nauseum to the detriment of creativity.
• "If a student can achieve a high score on all the criteria and still not perform well at the task, you have the wrong criteria" (Wiggins, cited in Clementi, 1999).
• Consider whether a rubric needs revision for a specific task. Do some of the criteria on the rubric go beyond this particular performance (that is, if you’ve created a rubric that is more "generic" and can be used for many tasks over time)?
• Make sure that the expectations in the rubric are directly aligned with the instruction of the lesson/unit. Students shouldn’t be expected to do what they haven’t been previously taught or shown.
• Some suggest that generic rubrics are more useful because creating rubrics is time-consuming and the more often they can be applied, the better. It is also more informative for students if the same rubric is used again and again, because they can see themselves making progress over time. On the other hand, generic rubrics are much less tied to the task and are not able to provide criteria for specific language use expectations or content knowledge.