• Perspectives on Safety
  • Published March 2008

Creation of a Medical Procedure Service to Improve Patient Safety

Perspective

Introduction and Rationale

Prior to the introduction of the Medical Procedure Service (MPS) in 2002, medical procedures at our institution (Beth Israel Deaconess Medical Center) were performed and taught using the time-honored apprenticeship model of "see one, do one, teach one." Residents were expected to independently perform and teach most medical procedures (central venous line placement, thoracentesis, paracentesis, and lumbar punctures). Procedural instruction was limited and poorly standardized; furthermore, trainee performance was infrequently supervised or assessed. In addition, the most qualified faculty were often unavailable to help teach, perform, supervise, or evaluate these procedures.

In 2001, the Vice Chair for Quality in the Department (Dr. Mark Aronson) gathered a small group of faculty to discuss how we performed medical procedures and the educational process for house staff. Although there had not been a sentinel event or specific concerns regarding procedural complications, this group recognized that the time had come to develop a new model of procedural instruction for our trainees. In addition, they believed that it was not in the best interest of our trainees or patients to expect an inexperienced second-year resident to teach and supervise an intern to perform these procedures on complex medical patients. These concerns resulted in the development of a comprehensive and novel Medical Procedure Service (MPS). The aims of this program were to: (i) create a new and standardized approach to teaching, performing, and evaluating inpatient medical procedures; (ii) better determine when a trainee is competent by assessing both clinical knowledge and psychomotor skill; and (iii) ultimately, improve patient safety.

Configuration

The MPS consists of select faculty skilled in performing inpatient procedures. These attendings focus on all components of the procedure, including obtaining consent, utilization of proper sterile technique, knowledge of relevant anatomic landmarks, teaching the psychomotor skills required for the specific procedure, and identifying and managing complications. Central venous lines (CVL) and thoracenteses are performed under the direct supervision of interventional pulmonology attendings, and operators use a portable ultrasound device whenever indicated. Lumbar punctures and paracenteses are performed under the guidance of designated, experienced hospitalist physicians who do monthly rotations on the procedure service while also serving as the medical consult attendings. Table 1 displays the volume and characteristics of procedures performed by house staff in a typical year.

When an intern on a non–intensive care unit (ICU)-based rotation needs to perform a medical procedure, they contact the MPS resident through a unique pager number. The MPS resident discusses the indications, risks, and benefits with the primary team and patient; assesses the patient and then pages the appropriate procedure attending; and schedules a mutually convenient time and location for the procedure. Interventional pulmonary and select hospitalist faculty provide supervision and teaching to the procedure resident, who covers procedures from 7 AM to 7 PM Monday through Saturday; on Sundays the service is covered by an ICU resident. Overnight, the intensivist on call in the hospital supervises urgent medical procedures, which are performed by the most experienced resident involved in the patient's care. Thus, with the exception of emergent procedures, all medical procedures are supervised by experienced faculty. Direct faculty supervision helps offset the costs of the program, as attendings now generate clinical revenue for procedures that were previously unbilled. Recognizing the importance of this educational and quality improvement endeavor and the time commitment of the faculty involved (and the fact that professional fee billings do not cover these costs), the Department of Medicine directs additional teaching funds to the pulmonary and hospitalist medicine budgets.

In our initial model, the intern caring for the patient was the primary operator of the procedure. This arrangement promoted continuity of care and allowed the trainee to understand the full scope of the rationale, benefits, and risks of procedures. However, as a result, MPS faculty had to assess the level of knowledge and skill and provide education to a different resident with every procedure, which was time-intensive. Additionally, weeks to months would elapse before a resident could perform another similar procedure, hindering his or her ability to refine and reapply new skills. Thus, after the first year, we modified the model by creating a dedicated procedure rotation covered by a second-year resident. This intensive learning experience provides the resident multiple, condensed opportunities for deliberate practice and allows the assigned faculty member to tailor instruction individually and provide longitudinal evaluation. This also frees the busy ward teams to continue in other patient-related activities rather than rushing to perform procedures. In addition, procedures that were previously directed to interventional radiology are now conducted by the residents' colleagues on the MPS.

Because the MPS has removed interns from doing many ward procedures on their patients, we did worry about how to ensure that interns would achieve at least a minimal level of procedural competency before their postgraduate year 2 (PGY2) procedure service rotation. To meet this goal, we expect interns to perform these procedures during their ICU rotations. This is done under the direct supervision of the ICU attending; all ICU attendings provide the same guidance, supervision, and feedback as the procedure service faculty. In addition, to guarantee a minimal proficiency at CVL insertion (the procedure most likely to be required on an emergent basis), on the first afternoon of their first ICU rotation, all interns participate in an extensive CVL course that utilizes simulation technology.

The MPS has evolved over time to better address the needs of our trainees. For instance, we now provide opportunities for the residents rotating on the service to participate in elective cardioversions and right heart catheterization. Previously, as a minimal marker of procedural competence, the American Board of Internal Medicine (ABIM) required each resident to perform a minimum number of designated procedures prior to completion of residency. In 2007, recognizing that types and numbers of procedures required by an internist will vary depending on one's chosen field of expertise, the ABIM modified its requirements and no longer requires all graduates to competently perform all procedures. In light of these recent changes, the procedure service is now an elective rotation, whereas it was previously required for every second-year resident. This change allows residents who are interested in procedure-oriented fields or who are interested in becoming competent at medical procedures to have adequate time to master these skills and allows the program to better assess their level of competence at these procedures. Although the rotation is optional, the procedure service remains continuously staffed by residents, as more residents are interested in the rotation than are slots available. To allow all residents to participate, some rotations are scheduled for less than a full 3-week block.

Education and Evaluation

A Web-based multimedia curriculum complements individualized education and feedback provided by the faculty. These materials include a review of the procedure, annotated by interactive diagrams, links to videos, and bibliographic references, and self-assessment quizzes. The Web site also aids in documentation; it provides standardized checklists to be used prior to the procedure, prefilled consent forms, and procedure note templates. Figure 1 shows a sample portion of a self-assessment quiz.

After every procedure, the trainee completes an online procedure log that records aspects of the procedure (e.g., complications and needle passes) and assesses procedural comfort. An alert is then automatically sent to the supervising attending physician, who completes an online evaluation. This form documents the trainee's cognitive and psychomotor skills, procedural complications, and other data such as number of needle passes and time to completion of the procedure.

The new ABIM regulations regarding procedural training (1) leave the determination of competence to the individual residency programs. However, there are no clear standards by which to determine competence in these common medical procedures. In addition to enhancing education and patient safety, the direct supervision and assessment provided by our MPS allows us to better define trainee competence. Previously, the determination of competence relied exclusively on one simply completing a predetermined number of procedures. Table 2 shows a sample of our new requirements for interns to be deemed "competent" at a CVL.

Research

Measuring the effectiveness of the MPS program presents an opportunity for educational innovations to intersect with clinical research. Data collected from self-assessment instruments, resident procedure logs, faculty evaluations, and electronic medical records provide the foundation for research evaluation. Figure 2 shows how these sources of data span the levels of Kirkpatrick's framework for evaluation (2), from basic trainee metrics to impact on the patient. We believe that the combination of these measures forms one approach to procedural competence.

During our first year, when use of the MPS was elective, we utilized these data to demonstrate that complication rates of the MPS compare favorably to national benchmarks (3), that residents who use the procedure service are twice as likely to be comfortable performing procedures (4), and that residents preferentially choose the MPS for more medically complex patients (GCH, unpublished data). Now that the MPS is required for performing procedures, we are taking advantage of the increased sample size to attain two ultimate research goals for the MPS: to determine whether the MPS has achieved a statistically significant decrease in complications rates and to characterize procedural competence.

In our work in procedural competence assessment, we have launched two related initiatives. We developed a CVL performance checklist validated by experts. These experts also helped determine a minimum passing score, which we will corroborate with other performance outcomes. Second, recognizing the power of simulation as a "safety net" for procedural training, we launched a randomized study whereby interns in the intervention group are taught CVL placement on simulators prior to performing them on patients in the ICU rotation. Using the validated checklist, they are assessed on simulators before and after the intervention. Thus, we have addressed all levels of Miller's pyramid of competence assessment (5), as depicted in Table 3.

Conclusions

As with any new program or intervention that affects an entire department, the success of this program depends on the collaboration and cooperation of many. The willingness to dedicate the resources and faculty support to an untested endeavor was a direct reflection of the institutional spirit and dedication to providing the best possible care for our patients and education for our students and residents. At our institution, the MPS has been implemented only in the Department of Medicine; however, it has garnered interest from and collaboration with other clinical departments within our hospital, as well as from other medical centers. It has widespread support from hospital administration and program directors, and as such, our program serves as an exemplar of how the educational and clinical missions of the teaching hospital share common goals, by protecting our patients as they undergo potentially morbid procedures.

C. Christopher Smith, MD
Associate Director, Internal Medicine Residency Program, Beth Israel Deaconess Medical Center
Associate Chief, Herrman L. Blumgart Medical Firm, Beth Israel Deaconess Medical Center
Co-Director, Rabkin Fellowship in Medical Education at Harvard Medical School and Beth Israel Deaconess Medical Center
Assistant Professor of Medicine, Harvard Medical School

Grace C. Huang, MD
Director of Assessment, Carl J. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center
Assistant Professor of Medicine, Harvard Medical School

Acknowledgment: The authors owe special thanks to those who showed an uncommon foresight and dedication to make the Medical Procedure Service at Beth Israel Deaconess Medical Center successful, including Drs. Moellering, Weinberger, Zeidel, Strewler, Weiss, Reynolds, Feller-Kopman, Gordon, Ernst, Aronson, Clardy, and Weingart.

References

Back to Top

1. Procedures Required for Internal Medicine. American Board of Internal Medicine. http://www.abim.org/certification/policies/imss/im.aspx#procedures. Accessed January 24, 2008.

2. Kirkpatrick, DL. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler, 1994. ISBN: 1881052494.

3. Smith CC, Gordon CE, Feller-Kopman D, et al. Creation of an innovative inpatient medical procedure service and a method to evaluate house staff competency. J Gen Intern Med. 2004;19:510-513. [go to PubMed]

4. Huang GC, Smith CC, Gordon CE, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119:71.e17-71.e24. [go to PubMed]

5. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(suppl 9):S63-S67. [go to PubMed]

Tables

Back to Top

Table 1. Procedures Performed by House Staff in 2006
(Go to table citation in commentary)

  N (%)
Total Procedures 1174 (1)
Central venous line 467 (40)
Lumbar puncture 232 (20)
Paracentesis 258 (22)
Thoracentesis 217 (18)
Complications  
None 1102 (94)
Bleeding 34 (3)
Pneumothorax 9 (1)
Dry tap 29 (2)
Urgency  
Elective 636 (54)
Urgent 431 (37)
Emergent 107 (9)
Timing  
Day 769 (66)
Night 248 (21)
Overnight 157 (13)
Location  
Intensive care unit 494 (42)
Ward 482 (41)
Emergency department 157 (13)
Other 41 (3)

Table 2: Beth Israel Deaconess Internal Medicine Residency 2007–2008 Procedural Competency Guidelines for Interns Performing Central Venous Line Placements (CVL)
(Go to table citation in commentary)

Procedure

Minimum Requirement

Competence

CVL placement

Completion of simulation training course (prior to performing on patients)

Completion of simulation training course (prior to performing on patients)

Achieve minimum passing score on CVL simulation checklist and cognitive quiz

Achieve mastery score on CVL simulation checklist

2 supervised procedures with an average overall score of 3* or greater

Minimum of 7 documented and supervised procedures

 

Average overall score on last 2 procedures: 4.5* or greater

 

No areas of deficiency identified on last 2 procedures

 

Achieve mastery score on CVL simulation checklist

*Rating Scale for Assessment

1

Deficient: unable to complete procedure without assistance.

2

 

3

Adequate skill: knew essential skills to complete procedure.

4

 

5

Proficient: demonstrated mastery of procedure skills.

Table 3: Components of the MPS Address All Levels of Miller's Pyramid
(Go to table citation in commentary)

Behavioral assessment

Does

Procedures on real patients

Shows

Performance on simulators

Cognitive assessment

Knows how

Faculty evaluation

Knows

Self-assessment quizzes

Figures

Back to Top

Figure 1. Sample Portion of the Self-Assessment Quiz for Central Venous Line Placement.
(Go to figure citation in commentary)
Self-assessment quiz text reads: Which of the following is a relative contraindication to placement of a subclavian or internal jugular central venous line? A. Cellulitis over anticipated insertion site. B. Injury or previous surgery to superior vena cava (e.g., superior vena cava syndrome). C. Inability to tolerate pneumothorax. D. Assisted ventilation with high end expiratory pressure. E. All the above. (E is highlighted in red.) In addition to the above-mentioned relative contraindications to placement of a central venous line, morbid obesity is also considered a relative contraindication. Absolute contraindications include inadequate operator experience or supervision, uncooperative patient, or a significant bleeding disorder or anticoagulation therapy.

Figure 2. How Different Sources of Data Address Kirkpatrick's Levels of Evaluation.
(Go to figure citation in commentary)
Table showing how sources of data address Kirkpatrick's levels of evaluation. Kirkpatrick's level

 
Back to Top