Using Performance Data

To help course directors and their faculty develop well-constructed questions and tests, AIMS generates a detailed performance report after each exam. The report provides data about the reliability and validity of the test, the performance of each item in the test, and the scores of each student. For more information about test validity and reliability, please refer to the USMLE Guide: Constructing Written Test Questions for the Basic and Clinical Sciences.

The AIMS Performance Report includes four components:

• student scores
• test statistics
• question statistics
• graphic distribution of scores

Student Scores

The first part of the AIMS Performance Report is a list of student scores in rank order, from the highest to the lowest score.

Test Statistics

The AIMS Performance Report also provides measurements of test reliability and validity, including the mean, standard deviation, lowest and highest scores, KR20 (Kuder & Richardson coefficient), and the standard error of measurement. These statistics are printed on the AIMS Performance Report, and can be viewed from within the AIMS system. Click on thumbnail below for an overview of the test statistics. For more information about student feedback, please refer to the section on Giving Feedback to Students.


Question Statistics

The third part of the AIMS Performance Report is the item analysis of each question in the exam. The report includes: the AIMS question ID number, correct answer, content category, number of students to try the question, number of students to correctly answer the question, the difficulty and discrimination indices, percentage of students in the upper and lower 27% of the class who correctly answer the question, and the distribution of answers across the options (see a sample report).

Graphical Distribution of Scores

The last part of the AIMS Performance Report is a graph depicting the distribution of students' scores, including the test number, academic year, course name, number of students, mean, standard deviation, and low/high scores (see a sample graph).

Marking Questions for Review

Course directors may decide to 'throw out' questions that they feel are poorly constructed or flawed. These questions are permanently 'marked' in the AIMS data bank so that they cannot be used again until they are modified.

updated December 2007