Building a Question Bank

Users of the AIMS system are encouraged to create and use questions that are similar in structure to those used by the National Board of Medical Examiners for the USMLE exams. Question formats include single best answer, matching & extended matching, true/false, and essay. Each question is classified by subject category and may be linked to learning objectives, clinical scenarios, narratives, charts, graphs, or other images. Cumulative performance statistics are available for all questions that have been used in graded exams. Questions can be used in printed and computer-based tests, as well as self-assessments. There are currently over 10,000 questions in the AIMS system.

Main Elements of a Question

There are 3 main elements used in building the question or item data bank:

• category (the subject content of the question)
• question (e.g., correct answer, type, text & options, justification, performance statistics)
• narrative (e.g., clinical scenario, case, graph, chart, image)

Each of these elements is described below in more detail.

Creating and Editing Categories

Each question in the AIMS system is linked to one category that best describes the content of that question. Categories allow users to classify and group questions into general content areas which are consistent with the learning objectives of the course. The categories are critical to searching the data bank, building tests, reviewing performance data, and generating student feedback.

Below is a sample list of categories that could be used in a nephrology course. The letters of the category further identify the type of course and content area.

 
  
Creating and Editing Questions

The main AIMS Question Module has 4 components: General [Information], Question Text, Justification/Comments, and Statistics. Each of these elements are described below.

General Information

Click on the thumbnail below for an overview of the information stored about a question.

Special features of AIMS:

• each question in the data bank has an unique identifying number
• AIMS does not need a separate answer key- the correct answer is stored with the question
• each question may be linked to one or more learning objectives in the course
• the author of each question can be identified

Question Text

Questions and answers can be typed directly into AIMS or copied from other programs such as Microsoft Word™. Users can bold & underline text, use sub- and superscripts, and special characters such as Greek symbols. The cumulative number of students who choose each answer is also displayed as part of the question data (see image below).

Justification & Comments

Once course directors accumulate a data bank of questions in the AIMS system, they can 'release' a number of items for study purposes. Faculty can include a statement of justification (see image above) that will appear beside the answers in these self-assessment exams. For more information about creating and using self-assessments, please refer to the page on 'Designing Tests: Self Assessments'. General comments about each question can also be stored in AIMS.

Statistics

Each time a question is used in a graded exam the AIMS system calculates and stores cumulative performance statistics about that question (see image below). These statistics include: the difficulty & discrimination indices; the percentage of students in the upper & lower 27% of the class who try and correctly answer the question; the number of times the question has been used; the last time the question was used; and the number of students who have answered the question. For more information about question and test statistics see the page on Using Performance Data.


Creating and Editing Narratives

The AIMS 'narrative' can be a clinical scenario, patient case, chart, and/or image that provides background information for one or more questions. The image below depicts a sample narrative used in a cardiovascular course. The main elements of a narrative include: an unique identifying number, author, objective, subject category, course, text of the narrative, and a linked image (if appropriate). From this screen users can also see AIMS questions that are linked to this narrative.

Using Images

The AIMS system can accommodate three image formats-- jpg, bmp, and gif. The image should be of sufficient size and resolution to be easily displayed in both printed and web formats. The images are linked to various questions through the AIMS narrative (see image above).

Searching for Questions

AIMS has a search engine that can be used to find questions in the data bank. A search can be done by question number, author, category, or a word or phrase in the question text. Users can edit, delete or view existing questions from the search screen (see image below).


updated December 2007