Using
Performance Data
Student
Scores
Test
Statistics
Questions
Statistics
Test
Graph
Marked
Questions
Using Performance Data to Improve Tests
To help course directors and their faculty develop well-constructed questions and tests, AIMS generates a detailed performance report after each exam. The report provides data about the reliability and validity of the test, the performance of each item in the test, and the scores of each student. For more information about test validity and reliability, please refer to the USMLE Guide: Constructing Written Test Questions for the Basic and Clinical Sciences.
The AIMS Performance Report includes four components:
- student scores
- test statistics
- question statistics
- graphic distribution of
scores
The first part of the AIMS Performance Report is a list of student scores in rank order, from the highest to the lowest score.
The AIMS Performance Report also provides measurements of test reliability and validity, including the mean, standard deviation, lowest and highest scores, KR20 (Kuder & Richardson coefficient), and the standard error of measurement. These statistics are printed on the AIMS Performance Report, and can be viewed from within the AIMS system. Click on thumbnail below for an overview of the test statistics. For more information about student feedback, please refer to the section on Giving Feedback to Students.
The third part of the AIMS
Performance Report is the item analysis of each question in the exam.
The report includes: the AIMS question ID number, correct answer, content
category, number of students to try the question, number of students to
correctly answer the question, the difficulty and discrimination indices,
percentage of students in the upper and lower 27% of the class who correctly
answer the question, and the distribution of answers across the options
(see a sample
report).
Graphical Distribution of Scores
The last part of the AIMS Performance Report is a graph depicting the distribution of students' scores, including the test number, academic year, course name, number of students, mean, standard deviation, and low/high scores (see a sample graph).
Course directors may decide to 'throw out' questions that they feel are poorly constructed or flawed. These questions are permanently 'marked' in the AIMS data bank so that they cannot be used again until they are modified.
Contact Information
Linda
J. Fisher, AIMS Coordinator (919) 843-9214
Office of Educational Development
UNC-CH School of Medicine
CB 7530 Rm. 322 MacNider Building
Chapel Hill, NC 27599-7530
main office: (919) 966-3641
OED web site: www/med.unc.edu/oed
updated June 2004
| School of Medicine | Office of Educational Development | Offices of Medical Education | UNC-CH |