Computer Adaptive Testing Project

The computer adaptive testing (CAT) research project at the Center for Advanced Research on Language Acquisition (CARLA) was funded by the U.S. Department of Education's International Studies and Research Program. The goal of the project was to develop a cost-effective, accurate, and flexible means to assess students' reading proficiency in French, German, and Spanish using a computer adaptive approach.

>> Frequently Asked Questions about computer adaptive testing...

To fulfill the goals of this grant the Second Language Assessment Team:

• constructed a reading proficiency model that guided item development. This model construction process involved consideration of how different language ability models, such as the ACTFL Proficiency Guidelines and Bachman's (1990) communicative language ability model, inform our test design.
   
• hosted an international seminar entitled "Issues in Computer Adaptive Testing and Reading Proficiency" under the direction of Dr. Micheline Chalhoub-Deville, on March 20 - 22, 1996. The seminar featured leading experts in the fields of computer adaptive testing, technology, second language reading proficiency, and second language assessment.
   
• assembled large proficiency-oriented item banks. For this activity, the CAT team invited teachers and language professionals to submit items and authentic materials.
   
• engaged in the pilot testing and calibration of items. Pilot testing was conducted in both traditional pencil-and-paper format and by computer. Items were administered to students to determine their quality and to obtain the necessary psychometric indices and qualitative information. Items were revised as needed to assure their quality and discriminative power. Items were incorporated into the Minnesota Language Proficiency Assessments.
   
• determined the CAT algorithm. This step involved estimating entrance and exit points and the various proficiency cut scores. Item response theory and a modified Angoff method were used to verify a priori hypothesized proficiency cut scores.
   
• assessed test takers' perceptions of the test via questionnaire. This information helped us to improve the quality of the test and enhance its user-friendliness.
   
• Published "An Operational Framework for Constructing a Computer Adaptive Test of L2 Reading Ability: Theoretical and Practical Issues -- Available as a PDF.