ACE Home | Contact Us | Center for Aging and Health
School of Medicine  |  UNC Home
Mission       Goals       Meet the Team       CQI       Assessment Tools       Educational Resources
Unit Orientation and Expectations       Admitting      Clinical Practice

Continuous Quality Improvement: An Educational Tool for the Training of Medical Professionals

Review by Debra L. Bynum, MD
Division of Geriatric Medicine
University of North Carolina at Chapel Hill


With support from the Donald W. Reynolds Foundation


Revised July 2006

CQI and Medical Education: Background

Traditionally, medical education has revolved around a curriculum based upon content and examinations. Key curriculum components thus included the learning methods, educational strategies, and “aims and objectives.” The current focus has shifted to “outcome based education,” emphasizing the product rather than the process. The essential question becomes, “what kind of doctor is being produced?” “Competency” replaces “learning objective” – not just “what will the student learn”, but what will the doctor be able to do? What skills and capabilities will the doctor possess? [1, Harden 1999].

The ACGME at its 1999 meeting endorsed the following six competencies for residents in the areas of

1. Patient care

2. Medical knowledge

3. Practice-based learning and improvement

4. Interpersonal and communication skills

5. Professionalism

6. Systems-based practice

Two of these core competencies, practice- based learning and improvement (PBL) and systems based practice (SBP), have received special attention as areas that focus on a physician’s ability to utilize the processes of Continuous Quality Improvement (CQI) to improve care. Without the ability to reflect and analyze one’s own practice and to work within a larger system in order to improve the quality of care delivered at all levels, even the most knowledgeable and professional doctor will be of limited value.

ACGME : Competency in Practice-Based Learning and Improvement

“Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices. Residents are expected to:

 -analyze practice experience and perform practice-based improvement activities using a systemic methodology

 -locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems

 -obtain and use information about their own population of patients and the larger population from which their patients are drawn

 -apply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness

 -use information technology to manage information, access on-line medical information; and support their own education

 -facilitate the learning of students and other health care professionals”

ACGME: Competency in Systems-Based Practice

 “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value. Residents are expected to:

 -understand how their patient care and other professional practices affect other health care professionals, the health care organization, and the larger society and how these elements of the system affect their own practice

 -know how types of medical practice and delivery systems differ from one another, including methods of controlling health care costs and allocating resources

 -practice cost-effective health care and resource allocation that does not compromise quality of care

 -advocate for quality patient care and assist patients in dealing with system complexities

 -know how to partner with health care managers and health care providers to assess, coordinate, and improve health care and know how these activities can affect system performance”

 [2, ACGME Outcome Project 1999].

The Push for Quality Improvement

In short, the issue of Quality Improvement is a current reality in medical practice and education. Complicating this though are multiple forces, multiple agendas, and multiple perspectives. Patients, third-party payers, purchasers of health plans, administrators, and evaluators of health care systems are all actively interested in the “quality” of care provided. But multiple questions remain, including what is quality, who defines quality, how is quality measured, how is good quality compensated, is bad quality penalized, how is quality reported?

It is important to understand several of the major forces behind incorporating Quality Improvement into medical care and education. Below are some of the major players:

 1. Institute of Medicine (IOM): a current main goal is to improve patient safety after releasing in 2001 a report, “Crossing the Quality Chasm.” Well known and often cited report, “To Err is Human – Building a Safer Health System,” states that over 90,000 deaths each year are due to preventable medical errors.

2. Center for Medicare and Medicaid Services (CMS): CMS has had an active role with the Department of Health and Human Services (HHS) to develop “quality indicators” with the goal of improving the quality of health care by requiring accountability and public disclosure. The other key is the goal of CMS to tie reimbursement to performance based upon these specific quality-of-care indicators.

3. Department of Health and Human Services (HHS): Quality Initiative program started in 2001.

a. Nursing Home Quality Initiative (NHQI) 2002
b. Home Health Quality Initiative (HHQI) 2003
c. Hospital Quality Initiative (HQI) 2003

  • 22 measures of “quality”
  • Core areas of assessment: MI, CHF, pneumonia, prevention of surgical infection


d. Physician-focused Quality Initiative 2004

  • Doctor’s Office Quality Project
  • ESRD QI program

e. Physician Voluntary Reporting Program 2005

4. National Committee for Quality Assurance (NCQA): An independent, non-profit organization that essentially serves as the “watchdog” of managed care. The organization provides information about quality of managed care plans, and gives “accreditation” to those agencies that meet their standards. Their reviews of various Preferred Provider Organizations (PPOs) and HMOs provides data that is often used by employers and patients to make decisions about health plans. They manage a large data set, known as the Health Plan Employer and Data Set (HEDIS), that is being used to follow performance measures. The NCQA has been in support of the CMS efforts to tie reimbursement and quality.

5. ACGME: Incorporation of PBLI and SBP into core competencies (see above).

6. American Board of Internal Medicine (ABIM): The ABIM has now incorporated a Practice Performance Requirement and Practice Improvement Module (PIM) into the Maintenance of Certification Program (10-year recertification). “The short term goal for implementing this requirement in ABIM’s maintenance of Certification program is to introduce certified physicians to the competencies of Practice-Based Learning and Improvement and Systems Based Practice… The long term goal for the requirement is that physicians will be competent in improvement science and will have the information systems needed to meet the quality reporting requirements of patients, purchasers and payers.” Each Practice Assessment Pathway includes a Practice Improvement Module to be completed online, surveys of peers/patients/staff that must be completed (25 required), and a self-directed quality measurement and improvement project that usually consists of chart and patient reviews.

Teaching “CQI”

Several studies have demonstrated positive methods of teaching CQI techniques and processes to resident physicians. The best programs involve residents in the projects – just teaching about CQI does not work, residents need to have hands on experience with how to actually perform CQI [3, 4,5,6,7,8,9,10,11,12,13,14,15, 16]. Involving residents in active CQI projects is essential; without resident involvement, the likelihood of having significant impact with any hospital wide policy change is low [17, 14]

What is CQI and How Do We “Teach” it? A Primer on CQI Processes and Measurements:

Continuous Quality Improvement, as originally outlined and described by W. Edwards Deming and Joseph Juran, is basically a collection of principles and methods aimed at letting individuals improve the systems in which they work with a scientific approach of testing changes to demonstrate improvement [5].

Key Concepts of CQI:

  • Success is defined as meeting the needs of those served (the patient, the hospital, the family, the physicians, the staff).
  • Quality is defined by “how well we meet those needs.”
  • Individuals are usually not the problem; most physicians and staff are not incompetent or uncaring. Usually quality is impeded by the system, especially when the individual has little control over the system.
  • “Unintended or unnecessary variation in processes can lead to unwanted variation in outcomes.”
  • The process of CQI is based upon scientific theory – experimenting with a change in the system and then testing the outcomes to demonstrate any improvement.
    • Plan-Do-Study-Act (PDSA) cycles [5, 18].

Basic Method of CQI Process:

  1. Identify the Problem
    1. Significant problem with observable impact on quality or care or patient safety
    2. Importance of resident input to identify problems in the system
    3. Target the problem: make it specific as possible!
  2. Analyze the Problem
    1. Consider baseline data or audits or outcomes of interest.
  3. Suggest a Solution
    1. Be specific in intervention.
  4. Implement the Solution
  5. Evaluate Change
    1. Repeat data collection, chart audits.
    2. Measure outcome changes or improvements.
    3. If no change, consider what has been learned in process and go back to step 1 [3].

CQI: Terms and Definitions
1. Patient Safety
: the absence of the potential for, or occurrence of, health care associated injury to patients. Created by avoiding medical errors as well as taking action to prevent errors from causing injury.
2. Error: mistakes made in the process of care that result in, or have the potential to result in, harm to patients. Mistakes include the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.


Error of Commission: result of an action that is taken.
Error of Omission: result of an action that is not taken.


3. Incident: unexpected or unanticipated events or circumstances not consistent with the routine care of a particular patient, which could have, or did lead to, an unintended or unnecessary harm to a person, or a complaint, loss or damage.

4. Near Miss: as an occurrence of an error that did not result in harm.

5. Adverse Event: an injury resulting from a medical intervention.

6. Preventable Adverse Event: harm that could be avoided through reasonable planning or proper execution of an action.

(Taken from Pronovost, “Defining and measuring patient safety,” 19, and AHRQ and IOM definitions of terms).

Measuring Quality and Safety:

Key Points:
1. Quality and Safety may be hard to separate;
at what point does “quality” cross over to safety? (example, use of beta blockers and ASA as quality indicators of post MI care could be considered to impact patient safety given the strong evidence for mortality and morbidity reduction in this setting…)
2. Quality is a complex and multifaceted entity with multiple attributes which may require multiple means of measurement.
3. Assessing Quality and Safety require an assessment of a “gold standard”; this is usually based upon evidence based guidelines. This emphasizes the need to transfer knowledge gained (evidence) from clinical studies into practice.
4. Full measurement and assessment will require determination of Numerator and a Denominator!

    1. Numerator: rate of event/problem/outcome of choice
    2. Denominator: “at risk” (time/events/cases/patients) [19]

How can we measure patient safety?

  • Measures based upon single event(s)
    • Examples:
      • Peer review
      • Mortality and Morbidity reviews
      • Investigation of liability claims
      • Incident reports
      • Investigation of complaints
    • Drawback:
      • Numerator only, no denominator; therefore unable to estimate rate.
      • Since there is no denominator and due to selection bias in cases that are identified, incident type reports cannot serve as measures/rates of performance in patient safety
    • Benefit:
      • helps to clarify and identify the problem.
      • Essentially serves as “screening tool.”
    • Review of “near misses” as incident reports especially useful for identifying system flaws.
      • Incident reports that focus on system and organization much more useful and productive than standard incident reporting system that focuses on punitive action against individual people.
      • Standard “mortality and morbidity” incident type may not be as effective at identifying key targets for improvement as the “near misses.”
  • Measures of rates: requires denominator to calculate number of events/problems/outcome per number “at risk”

[Adapted from Pronovost 2005, 19]

How to Use the Methods and Principles of CQI for the Education of Resident Physicians: Meeting the Core Competencies of Practice-Based Learning and Improvement and Systems-Based Practice


1. Get the resident’s buy in
. Whenever possible, have the residents arrive at a specific topic or identify a problem that needs addressing or evaluating.


2. Identify a problem based upon incident reports, mortality/morbidity reviews, complaints, claims, etc. (the “screening” process as discussed above).


3. Specify and focus the problem. There should be a very clear and simple question. Instead of the general issue of delirium, the more specific problem of the inappropriate use of atypical antipsychotics in elderly patients in the inpatient setting is more focused and manageable. True quality improvement relies on multiple small steps of change.


4. Specify the outcome change desired. What will be measured?


5. Review the evidence based upon a literature review. What are the ideal standards of care relating to the specific question or problem? What actions have been taken by other facilities or organizations? What resources are available through well designated Quality Indicators through Medicare/Medicaid, NCQA, IOM.


6. Identify the denominator if at all possible. What is the “at risk” group? For instance, if the outcome is looking at patients who develop PE after hip surgery, the denominator would be all patients admitted to undergo the procedure.


7. Identify key players in your facility that will interested in the results; identify key housestaff, other physicians, and staff who will play a part in the process; identify others such as hospital and clinic administrators, the office of CQI, and other parties that would be interested in the problem and documented improvement.


8. Create a “PDSA” cycle as above. “Plan, Do, Study, Act”

    1. Plan: determine process changes and collect baseline data.
    2. Do: implement process improvement.
    3. Study: evaluate effectiveness of intervention, gather follow up data.
    4. Act: implement a modified or refined intervention based upon follow up data.


9. Analyze the problem
. Create baseline data.

    1. Baseline chart audits
    2. Baseline data from hospital or clinic administration or other data sets

10. Suggest a Solution. Create a focused plan of intervention.

11. Implement the Solution. Advertise the change, get active participation from all involved.


12. Evaluate the Change. May need professional assistance for data analysis depending on level of CQI project undertaken. If no change is observed, re-evaluate process and review important learning points and repeat cycle of study.

CQI: How Effective Is It in Improving Care?

CQI has a strong foundation in the worlds of business and aviation safety, but is it effective at improving the quality of medical care? Despite the push for CQI in practice and education, it might be surprising to many that there is actually very little data to support that CQI interventions have a significant, lasting, consistent impact on patient care. What is the data?

Multiple studies and reviews have demonstrated that it is difficult to change physician behavior. Passive approaches with an emphasis upon provider feedback do not work to change physician behavior. A multi-faceted approach with provider feedback, reminders, compensation plans and guidelines are required to demonstrate any significant change in behavior, and the long lasting effect after these systems are gone is not known [20].

In addition, many changes in quality of care do not rely upon the physician alone but require the system to be effective. In a study by Halm in 2004, a multicenter intervention including treatment guidelines, critical pathways, education sessions, pocket reminders, standardized orders and patient education materials had essentially no significant impact on the overall care of patients with pneumonia. Despite an increased use of guideline recommendations, outcomes such as time to receive antibiotics rely upon much more than education [21, 22].

Even in large projects (that report success), such as the Cooperative Cardiovascular Project supported by Medicare/CMS, the sustained improvements in care are modest [23]. In a 2004 review of Quality Improvement Programs and their impact after 2 years, there was essentially little to no consistent association between QI projects and quality of health plans [24, 25].

In summary, the effectiveness is not clear. There are multiple reasons why CQI initiatives in medicine are complicated and multiple unanswered questions:

  1. What is quality?
  2. Who defines quality?
  3. Who measures quality
  4. How does the provider fit into QI programs?
  5. How is good quality compensated?
  6. How is bad quality penalized?
  7. How is quality reported to the public?
  8. How is quality sustained over time without endless and continuous resources?

So, why bother with CQI? Several main reasons include:

  • As physicians, we are and will increasingly be faced with the forces of managed care, CMS, and other payers in addition to patients who are pushing for accountability and improved quality.
  • Although the data to date are not there to prove that CQI improves patient outcomes, the idea that improving quality of care is important and up to those who are actively involved in care is essential.
  • The processes of QI will be better and more effective if we as physicians are actively involved rather than passive recipients of mandated change.
  • It is a mandatory component to education at a resident level, CME level, and board recertification level.
  • CQI is based upon Evidence-Based Medicine, essentially attempting to translate evidence from ongoing studies into strategies for providing optimal care to patients.
  • CQI in the form of Practice-Based Learning and Improvement is essentially a form of Case-Based Education – we learn most profoundly when faced with real cases and questions.

CQI: Beyond Meeting Core Competencies.

We have shifted our focus in medical education from the process of teaching to the outcome of the education. This emphasis on “Outcome Based Education” has led to the development of “competencies” – what should a competent physician be able to do? With multiple forces including the public, third-party payers, Medicare/Medicaid, hospital administration, the ABIM/ABFM, and mandate statements from the IOM, IHI, and NCQA emphasizing the need to improve the quality and safety of medical care delivered, core competencies for medical training now include practice- and system-based improvements. Much focus is now being made on how to “teach” CQI and how to incorporate these competencies and methods of CQI into the curriculum of resident education.

Upon further review, using the methods of quality improvement may have much more to offer than meeting core requirements. It may even have another outcome equal in improving the quality of patient care. It may serve as an educational tool in and of itself – utilizing the methods of quality improvement to teach core topics of internal medicine. Review of our individual practice and errors in addition to review of system-based problems can serve as a very powerful and lasting educational tool. The processes of quality improvement work well as a structure and foundation upon which to build upon, layering bricks of knowledge and core topics, creating a framework for the other core competencies to be taught. In a recent survey, physicians viewed as “high performing” in real practice reported several important keys to their ongoing education – namely a practice-based learning approach based upon questions that arose from specific cases and problems [26].

Future Directions: Will using the methods of CQI and structuring educational programs to meet the needs of instruction in Practice-Based Learning and Improvement and Systems-Based Practice serve as a useful educational tool? Will residents benefit with improved knowledge skills if taught core topics in such a format compared to the traditional method of didactic lectures? Will residents be more competent in other proficiencies such as professionalism and communication if they learn to work with an interdisciplinary team in identifying system based problems and means for improvement? Will overall patient care be improved by physicians who learn within a framework of identifying and creating opportunities for change and improvement? Ziegelstein and Fiebach summarized this process of education well by describing it as teaching the importance of “the mirror” and “the village” in learning how to care for patients – looking into the mirror to improve individual practice, and looking at the village to improve our systems of care [27]. Will the “mirror” and the “village” serve as new frameworks for medical education, a truly outcomes-based educational directive aimed at creating physicians who are prepared to optimally provide quality care?

Resources:

  1. Institute for Healthcare Improvement: IHI
  2. Institute of Medicine : IOM
  3. Agency for Healthcare Research and Quality (AHRQ)
    1. www.qualityindicators.ahrq.gov
  4. ACGME: Core Competencies
  5. ABIM: Accreditation and Recertification Processes
  6. National Committee for Quality Assurance
  7. Centers for Medicare and Medicaid Services/CMS

References:

Harden RM, Crosby JR, Davis MH. AMEE Guide: Outcome-based education: an introduction to outcome based education. Medical Teacher.1999; 21(1): 7-14.

ACGME Outcome Project: General Competencies. Feb 1999.

Djuricich AM. A continuous quality improvement curriculum for residents: addressing core competency, improving systems. Acad Med. 2004; 79(suppl 10) S65-7.

Ogrinc G, Headrick LA, Mutha S, Coleman MT, O’Donnell J, Miles PV. A framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003; 78(7): 748-56.

Headrick LA, Richardson A, Priebe GP. Continuous Improvement Learning for Residents. Pediatrics. 1998; 101(4): 768-774.

Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Int Med. 2004; 19 (5 Pt 2): 496-500.

Coleman MT, Nasraty S, Ostapchuk M, Wheeler S, Looney S, Rhodes S. Introducing practice-based learning and improvement ACGME core competencies into a family medicine residency curriculum. Jt Comm J Qual Saf. 2003; 29(5): 238-47.

Schillinger D, Wheeler M, Fernandez A. The populations and quality improvement seminar for medical residents. Acad Med. 2000; 75: 562-563.

Holmboe E, Scranton R, Sumption K, Hawkins R. Effect of medical record audit and feedback on residents’ compliance with preventive health care guidelines.

Dyne PL, Strauss RW, Rinnert S. Systems-based practice: the sixth core competency. Acad Emerg Med. 2002; 9(11): 1270-7.

Mohr JJ, Randolph GD, Laughon MM, Schaff E. Integrating improvement competencies into residency education: a pilot project from a pediatric continuity clinic. Ambulatory Pediatrics. 2003; 3(3): 131-136.

Frey K, Edwards F, Altman K, Spahr N, Gorman RS. The collaborative care curriculum: an educational model addressing key ACGME core competencies in primary care residency training. Med Education. 2003; 37(9): 786.

Reich LM, Rand AD. Comprehensive Educational Performance Improvement (CEPI): an innovative, competency-based assessment tool. Mount Sinai J of Med. 2005; 72(5): 300-306.

Weingart SN, Tess A, Driver J, Aronson MD, Sands K. Innovations in education and clinical practice: creating a quality improvement elective for medical house officers. JGIM. 2004; 19(8): 861.

Swanson TK, Eilers GM. Physician and staff acceptance of continuous quality improvement. Fam Med. 1994; 26(9): 583-6.

Headrick LA, Knapp M, Neuhauser D. Working from upstream to improve health care: the IHI Interdisciplinary Professional Education Collaborative. Joint Comm J Qual Improvement. 1996; 22: 149-164.

Ashton CM. Invisible doctors: making a case for involving medical residents in hospital quality improvement programs.

Deming WE. The New Economics. Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study; 1993.

Pronovost PJ, Thompson DA, Holzmueller CG, Lubomski LH, Morlock LL. Defining and measuring patient safety. Critical Care Clinics. 2005; 21(1).

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E. Changing Provider Behavior: an overview of systematic reviews of interventions. Med Care. 2001; 39(8 suppl 2): 112-45.

Halm EA. Limited Impact of a Multicenter Intervention to improve the quality and efficiency of pneumonia care. Chest. 2004; 126(1).

Marciniak TA, Mosedale L, Ellerbeck EF. Quality improvement at the national level. lessons from the Cooperative Cardiovascular Project. Eval Health Prof. 1998; 21: 525-36.

Peterson ED. Should we link payment to quality? Am Heart Journal. 2004. 148(5).

Borenstein J, Badamgarav E, Henning J, Anacleto DJ, Weingarten SR. Association between quality improvement activities performed by managed care organizations and quality of care. Am J Med. 2004; 117: 297-304.

Jha AK, Schneider EC. From Motives to Results: Improving the effectiveness of quality improvement. Am J Med. 2004; 117(5).

Ziegelstain RC, Fiebach NH. The mirror and the village: a new method for teaching practice-based learning and improvement and systems-based practice. Acad Med. 2004; 79(1): 83-8.

Scherer LA. Videotape review leads to rapid and sustained learning. Am J Surg 2003; 85(6): 516-20.

Cook D. Quality Improvement: Best evidence in clinical practice and clinical evidence in best practice. Crit Care Med 2006; 34(1).

Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional dogma. Lancet 2005; 363: 1147-1154.

Chassin Mr, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA 1998; 280: 1000-5.

This page last updated 9/12/2007.

This webpage is the property of the Acute Care of the Elderly (ACE) inpatient unit at UNC Hospitals, Chapel Hill, North Carolina, USA. All images and design are copyrighted to ACE. Please do not use images from the webpage for any reason unless you obtain written permission from the director of the unit, Debra Bynum, MD, to do so.
2008.
About the photos on this webpage: The photos are by Sarah Madry and are copyrighted to her. They consist of image-edited photos of architectural elements of UNC Hospitals. Please do not use, copy, distribute or otherwise manipulate or take these photos by any method.
Thank you.