Unexpected Collateral Effects of Simulation-Based Medical Education

Barsuk, Jeffrey H. MD, MS; Cohen, Elaine R.; Feinglass, Joe PhD; McGaghie, William C. PhD; Wayne, Diane B. MD

Purpose: Internal medicine residents who complete simulation-based education (SBE) in central venous catheter (CVC) insertion acquire improved skills that yield better patient care outcomes. The collateral effects of SBE on the skills of residents who have not yet experienced SBE are unknown.

Method: In this retrospective, observational study, the authors used a checklist to test the internal jugular and subclavian CVC insertion skills of 102 Northwestern University second- and third-year internal medicine residents before they received simulation training. The authors compared, across consecutive academic years (2007-2008, 2008-2009, 2009-2010), mean pretraining scores and the percent of trainees who met or surpassed a minimum passing score (MPS).

Results: Mean internal jugular pretest scores improved from 46.7% (standard deviation = 20.8%) in 2007 to 55.7% (+/-22.5%) in 2008 and 70.8% (+/-22.4%) in 2009 (P < .001). Mean subclavian pretest scores changed from 48.3% (+/-25.5%) in 2007 to 45.6% (+/-31.0%) in 2008 and 63.6% (+/-27.3%) in 2009 (P = .04). The percentage of residents who met or surpassed the MPS before training for internal jugular insertion was 7% in 2007, 16% in 2008, and 38% in 2009 (P = .004); for subclavian insertion, the percentage was 11% in 2007, 19% in 2008, and 38% in 2009 (P = .028).

Conclusions: SBE for senior residents had an effect on junior trainees, as evidenced by pretraining CVC insertion skill improvement across three consecutive years. SBE for a targeted group of residents has implications for skill acquisition among other trainees.

Use of simulation-based education to improve resident learning and patient care in the medical intensive care unit: A randomized trial

PMID:

 

22033049

CJ Schroedl, TC Corbridge, ER Cohen, SS Fakhran… - Journal of Critical Care, 2011

Abstract

Purpose

The purpose of this study is to determine the effect of simulation-based education on the knowledge and skills of internal medicine residents in the medical intensive care unit (MICU).

Methods and Materials

From January 2009 to January 2010, 60 first-year residents at a tertiary care teaching hospital were randomized by month of rotation to an intervention group (simulator-trained, n = 26) and a control group (traditionally trained, n = 34). Simulator-trained residents completed 4 hours of simulation-based education before their medical intensive care unit (MICU) rotation. Topics included circulatory shock, respiratory failure, and mechanical ventilation. After their rotation, residents completed a standardized bedside skills assessment using a 14-item checklist regarding respiratory mechanics, ventilator settings, and circulatory parameters. Performance of simulator-trained and traditionally trained residents was compared using a 2-tailed independent-samples t test.

Results

Simulator-trained residents scored significantly higher on the bedside skills assessment compared with traditionally trained residents (82.5% ± 10.6% vs 74.8% ± 14.1%,P = .027). Simulator-trained residents were highly satisfied with the simulation curriculum.

Conclusions

Simulation-based education significantly improved resident knowledge and skill in the MICU. Knowledge acquired in the simulated environment was transferred to improved bedside skills caring for MICU patients. Simulation-based education is a valuable adjunct to standard clinical training for residents in the MICU.

Keywords: Simulation training; Medical education; Medical intensive care unit; Clinical competence; Quality of health care

Medical Education Featuring Mastery Learning With Deliberate Practice Can Lead to Better Health for Individuals and Populations

 Pubmed ID: 22030671

McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. 

Medical education can lead to better health for individuals and populations when it has effective, evidence-based features and is delivered under the right conditions. Effective, evidence-based features include mastery learning (ML), deliberate practice (DP), and rigorous outcome measurement (ML and DP are both defined below). The right conditions include a committed and skillful faculty, curriculum integration and institutional endorsement, and health care system acceptance. Translation of medical education outcomes to measurable downstream effects on improved patient care practices and better health for individuals and populations is demonstrated by educational and health services research programs that are thematic, sustained, and cumulative.

ML is an especially stringent form of competency-based education where learners acquire essential knowledge and skill measured rigorously against fixed achievement standards without regard to the time needed to reach the outcome. Mastery indicates a much higher level of performance than competence alone, and evidence shows that ML leads to longer skill maintenance without significant decay. Educational outcomes are uniform in ML with little or no variation, whereas educational time varies among trainees.¹ In medical education, ML has been used chiefly for acquisition and maintenance of clinical procedural skills such as advanced cardiac life support (ACLS), thoracentesis, and central venous catheter (CVC) insertion. ML can also be used to acquire and refine cognitive and affective educational outcomes. The ability to engage a family in a difficult conversation about end-of-life issues is a clinical skill amenable to ML just like performance of a lumbar puncture. Work is now under way to evaluate these and other clinical mastery outcomes.

DP embodies strong and consistent educational interventions grounded in information processing and behavioral theories of skill acquisition and maintenance.² DP has at least nine elements: (1) highly motivated learners with good concentration, (2) well-defined learning objectives that address knowledge or skills that matter clinically, at an (3) appropriate level of difficulty for the medical learners, with (4) focused, repetitive practice of the knowledge or skills, that leads to (5) rigorous measurements that yield reliable data, which provide (6) informative feedback from educational sources (e.g., teachers, simulators) that promotes frequent (7) monitoring, error correction, and more DP that enables (8) performance evaluation toward reaching a mastery standard, and allows (9) advancement toward the next clinical task or unit. The goal of DP is constant skill improvement. Research shows that DP is a much more powerful predictor of professional accomplishment than experience or academic aptitude.

Medical education and evaluation research programs that incorporate ML and DP principles, and evaluate outcomes with measurement and methodological rigor, are beginning to show translational results in patient care practices and patient outcomes.³ Many of these educational programs use health care simulation technology as a curriculum driver. Examples of improved patient care include reduced complications and higher success rates at CVC insertion, improvement in laparoscopic surgical skill, better adherence to guidelines during ACLS team responses, and increased competence in several types of endoscopy. Better health for individuals and populations linked directly to medical education programs has been demonstrated through reduced rates of catheter-related bloodstream infections; reduced birth complications due to shoulder dystocia (brachial plexus injury), low Apgar scores, and infant brain injury from neonatal hypoxic–ischemic encephalopathy; and lower postsurgical complications among cataract surgery patients.³ Advancements in medical education, evaluated rigorously, can produce better patient health as judged statistically and clinically.

Powerful and effective medical education programs do not exist in a vacuum. They include not only such curriculum features as ML, DP, and reliable outcome measurement but also faculty and administrative commitment, curriculum support expressed as financial and human capital, and a health care system whose culture embraces professional competence evaluation in service of patient care quality and patient safety at all levels. Medical education programs are being recognized as complex service interventions that are affected by the context in which they are delivered. This context is highly variable but has a powerful role in determining the ultimate success of the program. A new, interdisciplinary academic field called implementation science, and the scholarly journal that bears its name, holds promise to teach the medical education community how to develop, launch, and sustain educational programs that improve health for individuals and populations.

Medical school and residency curricula must change to adopt a competency-based approach featuring structured learning experiences tied to assessments that yield reliable data. Research shows convincingly that ML and DP linked to competence assessment can improve health outcomes. Expansion of this model is needed to better prepare trainees for independent and group practice and to ensure competent medical care for patients and society.

Is It the Athlete Or the Equipment? An analysis of the top swim performances from 1990-2010.

O'Connor LM, Vozenilek JA.
J Strength Cond Res. 2011 Sep 29. [Epub ahead of print]
PMID: 21964430

Abstract

Forty-three world record swims were recorded at the 2009 Fédération Internationale de Natation (FINA) World Championship meet in Rome. Of the 20 FINA recognized long-course (50m pool) swimming events, men set new world records in 15 of those events while women did the same in 17 events. Each of the men's world records and 14 of the 17 women's records still stand. These performances were unprecedented; never before had this many world records been broken in such a short period of time. There was much speculation that full-body, polyurethane, technical swimsuits were the reason for the conspicuous improvement in world records. Further analysis led FINA to institute new rules on January 1, 2010, that limited the types of technical swimsuits that could be worn by athletes. No long-course world record has been broken since then. We sought to understand this phenomenon by analyzing publically available race data and exploring other possible causes including: improvements in other sports; improvements in training science; changes in rules and regulations; gender differences; anaerobic vs. aerobic events; unique talent; and membership data.

PMID:

21964430

[PubMed - as supplied by publisher]

Simulation and quality improvement in anesthesiology.

Anesthesiol Clin. 2011 Mar;29(1):13-28. Epub 2010 Dec 16.

Abstract

Simulation, a strategy for improving the quality and safety of patient care, is used for the training of technical and nontechnical skills and for training in teamwork and communication. This article reviews simulation-based research, with a focus on anesthesiology, at 3 different levels of outcome: (1) as measured in the simulation laboratory, (2) as measured in clinical performance, and (3) as measured in patient outcomes. It concludes with a discussion of some current uses of simulation, which include the identification of latent failures and the role of simulation in continuing professional practice assessment for anesthesiologists.

Copyright © 2011 Elsevier Inc. All rights reserved.

PMID:

 

21295750

 

[PubMed - indexed for MEDLINE]

Simulation-based team training in healthcare.

Simulation-based team training in healthcare.

Posted:

Simul Healthc. 2011 Aug;6 Suppl:S14-9
Authors: Eppich W, Howard V, Vozenilek J, Curran I
: Simulation-based team training (SBTT) in healthcare is gaining acceptance. Guidelines for appropriate use of SBTT exist, but the evidence base remains limited. Insights from other academic disciplines with sophisticated models of team working may point to opportunities to build on current frameworks applied to team training in healthcare. The purpose of this consensus statement is threefold: (1) to highlight current best practices in designing SBTT in healthcare and to identify gaps in current implementation; (2) to explore validated concepts and principles from relevant academic disciplines and industries; and (3) to identify potential high-yield areas for future research and development.
PMID: 21817858 [PubMed - in process]

Evaluating the Impact of Simulation on Translational Patient Outcomes.

Pubmed ID:21705966

Simul Healthc. 2011 Jun 23. [Epub ahead of print]

McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D.

Source

From the Center for Education in Medicine (W.C.M.), Northwestern University Feinberg School of Medicine, Chicago, IL; Southmead Hospital (T.J.D.), Bristol, UK; College of Medicine, Mayo Clinic (W.F.D.), Rochester, MN; Kaiser Permanente Program Offices (C.M.L.), Oakland, CA; and Department of Surgery (D.S.), Carolinas Healthcare System, Charlotte, NC.

Abstract

INTRODUCTION:

A long and rich research legacy shows that under the right conditions, simulation-based medical education (SBME) is a powerful intervention to increase medical learner competence. SBME translational science demonstrates that results achieved in the educational laboratory (T1) transfer to improved downstream patient care practices (T2) and improved patient and public health (T3).

METHOD:

This is a qualitative synthesis of SBME translational science research (TSR) that employs a critical review approach to literature aggregation.

RESULTS:

Evidence from SBME and health services research programs that are thematic, sustained, and cumulative shows that measured outcomes can be achieved at T1, T2, and T3 levels. There is also evidence that SBME TSR can yield a favorable return on financial investment and contributes to long-term retention of acquired clinical skills. The review identifies best practices in SBME TSR, presents challenges and critical gaps in the field, and sets forth a TSR agenda for SBME.

CONCLUSIONS:

Rigorous SBME TSR can contribute to better patient care and improved patient safety. Consensus conference outcomes and recommendations should be presented and used judiciously.

PMID:

21705966

~

A Retrospective Review of TATRC Funding for Medical Modeling and Simulation Technologies

Pugh, Carla M. MD, PhD; Bevan, Matthew G. PhD; Duve, Rebecca J. MS; White, Heather L. BA; Magee, J. Harvey BA; Wiehagen, Gene B. BS

Pub med ID 21546864

Introduction: In February 2000, the U.S. Army's Telemedicine and Advanced Technology Research Center (TATRC) and the U.S. Army's Simulation, Training, and Instrumentation Command cohosted an Integrated Research Team conference in Maryland. The goal of the conference was to enable end users, researchers, materiel developers, and other government agencies to present their conceptions of how modeling and simulation could and should be developed to meet military medical needs. During the past 9 years, TATRC has funded more than 175 projects relating to simulation.

Methods: This study was a retrospective review of TATRC's Modeling and Simulation Training projects (N = 175).
Results: Our results show that most (>75%) of the funded projects in this study involved industry. More than 85% of the projects that involved industry focused on technology development. Industry development projects seemed to meet their deliverables in a timely fashion. However, academia projects using industry-developed technologies and prototypes were delayed largely because the technologies did not meet their needs.

Discussion: There seems to be a measurable gap between industry's definition of a completed product technology and academia's ability to implement and use the technology in interactive learning environments. Our findings support the need for a standardized strategic design process that involves a strong industry-academia collaboration and early end-user testing to better facilitate the development of sound requirements that guide technology development.

________

Using Second Life Virtual Simulation Environment for Mock Oral Emergency Medicine Examination.

Schwaab J, Kman N, Nagel R, Bahner D, Martin DR, Khandelwal S, Vozenilek J,
Danforth DR, Nelson R.


From the Department of Emergency Medicine (JS, NK, DB, DRM, SK, RNe), the Center
for Education and Scholarship (RNa), and the Department of Obstetrics and
Gynecology (DRD), The Ohio State University, Columbus, OH; and the Department of
Emergency Medicine, Feinberg School of Medicine, Northwestern University (JV),
Chicago, IL.

ACADEMIC EMERGENCY MEDICINE 2011; 18:1-4 © 2011 by the Society for Academic
Emergency Medicine

ABSTRACT: Objectives:  Oral examination is a method used to
evaluate emergency medicine (EM) residents and is a requirement for board
certification of emergency physicians. Second Life (SL) is a virtual
three-dimensional (3-D) immersive learning environment that has been used for
medical education. In this study we explore the use of SL virtual simulation
technology to administer mock oral examinations to EM residents.

Methods:  This was a prospective observational study of EM residents who had previously
completed mock oral examinations, participating in a similar mock oral
examination case scenario conducted via SL. EM residents in this training program
completed mock oral examinations in a traditional format, conducted face to face
with a faculty examiner. All current residents were invited to participate in a
similar case scenario conducted via SL for this study. The examinee managed the
case while acting as the physician avatar and communicated via headset and
microphone from a remote computer with a faculty examiner who acted as the
patient avatar. Participants were surveyed regarding their experience with the
traditional and virtual formats using a Likert scale.

Results:  Twenty-seven EM
residents participated in the virtual oral examination. None of the examinees had
used SL previously. SL proved easy for examinees to log into (92.6%) and navigate
(96.3%). All felt comfortable communicating with the examiner via remote
computer. Most examinees thought the SL encounter was realistic (92.6%), and many
found it more realistic than the traditional format (70.3%). All examinees felt
that the virtual examination was fair, objective, and conducted efficiently. A
majority preferred to take oral examinations via SL over the traditional format
and expressed interest in using SL for other educational experiences (66.6 and
92.6%, respectively).

Conclusions:  Application of SL virtual simulation
technology is a potential alternative to traditional mock oral examinations for
EM residents.


PMID: 21521404 [PubMed - as supplied by publisher]

Does Simulation-Based Medical Education With Deliberate Practice Yield Better Results Than Traditional Clinical Education? A Meta-Analytic Comparative

PubMed ID: 21512370
Acad Med. 2011 Apr 20. [Epub ahead of print]

McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB.

Abstract
PURPOSE: This article presents a comparison of the effectiveness of traditional clinical education toward skill acquisition goals versus simulation-based medical education (SBME) with deliberate practice (DP).


METHOD: This is a quantitative meta-analysis that spans 20 years, 1990 to 2010. A search strategy involving three literature databases, 12 search terms, and four inclusion criteria was used. Four authors independently retrieved and reviewed articles. Main outcome measures were extracted to calculate effect sizes.


RESULTS: Of 3,742 articles identified, 14 met inclusion criteria. The overall effect size for the 14 studies evaluating the comparative effectiveness of SBME compared with traditional clinical medical education was 0.71 (95% confidence interval, 0.65-0.76; P < .001). CONCLUSIONS: Although the number of reports analyzed in this meta-analysis is small, these results show that SBME with DP is superior to traditional clinical medical education in achieving specific clinical skill acquisition goals. SBME is a complex educational intervention that should be introduced thoughtfully and evaluated rigorously at training sites. Further research on incorporating SBME with DP into medical education is needed to amplify its power, utility, and cost-effectiveness.

PMID: 21512370 [PubMed - as supplied by publisher]

Dr. McGaghie is Jacob R. Suker, MD, Professor of Medical Education, professor of preventive medicine, and director of evaluation, Northwestern University Clinical and Translational Sciences Institute, Northwestern University Feinberg School of Medicine, Chicago, Illinois. Dr. Issenberg is Michael S. Gordon, MD professor of medicine and assistant director, Gordon Center for Research in Medical Education, University of Miami Miller School of Medicine, Miami, Florida. Ms. Cohen is research assistant, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois. Dr. Barsuk is assistant professor of medicine, Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois. Dr. Wayne is associate professor of medicine and director, Internal Medicine Residency Training Program, Northwestern University Feinberg School of Medicine, Chicago, Illinois.

Structured interdisciplinary rounds in a medical teaching unit: improving patient safety.

PubMed ID: 21482844

Arch Intern Med. 2011 Apr 11;171(7):678-84.

O'Leary KJ, Buck R, Fligiel HM, Haviley C, Slade ME, Landler MP, Kulkarni N, Hinami K, Lee J, Cohen SE, Williams MV, Wayne DB.

Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, 211 E Ontario St, Seventh Floor, Chicago, IL 60611. keoleary@nmh.org

Abstract

BACKGROUND: Effective collaboration and teamwork is essential to providing safe hospital care. The objective of this study was to assess the effect of an intervention designed to improve interdisciplinary collaboration and lower the rate of adverse events (AEs).

METHODS: The study was a controlled trial of an intervention, Structured Inter-Disciplinary Rounds, implemented in 1 of 2 similar medical teaching units in a tertiary care academic hospital. The intervention combined a structured format for communication with a forum for regular interdisciplinary meetings. We conducted a retrospective medical record review evaluating 370 randomly selected patients admitted to the intervention and control units (n = 185 each) in the 24 weeks after and 185 admitted to the intervention unit in the 24 weeks before the implementation of Structured Inter-Disciplinary Rounds (N = 555). Medical records were screened for AEs. Two hospitalists confirmed the presence of AEs and assessed their preventability and severity in a masked fashion. We used multivariable Poisson regression models to compare the adjusted incidence of AEs in the intervention unit to that in concurrent and historic control units.

RESULTS: The rate of AEs was 3.9 per 100 patient-days for the intervention unit compared with 7.2 and 7.7 per 100 patient-days, respectively, for the concurrent and historic control units (adjusted rate ratio, 0.54; P = .005; and 0.51; P = .001). The rate of preventable AEs was 0.9 per 100 patient-days for the intervention unit compared with 2.8 and 2.1 per 100 patient-days for the concurrent and historic control units (adjusted rate ratio, 0.27; P = .002; and 0.37; P = .02). The low number of AEs rated as serious or life-threatening precluded statistical analysis for differences in rates of events classified as serious or serious and preventable.

CONCLUSION: Structured Inter-Disciplinary Rounds significantly reduced the adjusted rate of AEs in a medical teaching unit.

PMID: 21482844 [PubMed - in process]

Toward a simulation and assessment method for the practice of camera-guided rigid bronchoscopy.

1. Stud Health Technol Inform. 2011;163:535-41.

http://www.ncbi.nlm.nih.gov/pubmed/21335852

Salud LH, Peniche AR, Salud JC, de Hoyos AL, Pugh CM.

Northwestern University, Department of Surgery, Feinberg School of Medicine.

We have developed a way to measure performance during a camera-guided rigid
bronchoscopy using manikin-based simulation. In an effort to measure contact
pressures within the airway during a rigid bronchoscopy, we instrumented pressure
sensors in a commercially available bronchoscopy task trainer. Participants were
divided into two groups based on self-reported levels of expertise: novice (none
to minimal experience in rigid bronchoscopy) and experts (moderate to extensive
experience). There was no significant difference between experts and novices in
the time taken to complete the rigid bronchoscopy. However, novices touched a
greater number of areas than experts, showing that novices induce a higher number
of unnecessary soft-tissue contact compared to experts. Moreover, our results
show that experts exert significantly less soft tissue pressure compared to
novices.


PMID: 21335852 [PubMed - in process]

Expanding the use of simulators as assessment tools: the new pop quiz.

Kaye AR, Salud LH, Domont ZB, Blossfield Iannitelli K, Pugh CM.

Northwestern University Feinberg School of Medicine, Department of Surgery.
Stud Health Technol Inform. 2011;163:271-3
PMID: 21335802


Abstract
This study introduces a novel way to implement simulation in medical education. We investigated the feasibility of integrating a newly developed breast examination simulator into a breast exam technique lecture while also collecting detailed data on medical students' breast exam skills. Results indicate that it is feasible to integrate simulation technology into the classroom environment and collect detailed performance data that can be analyzed and used for skills assessment.

PMID: 21335802

Comparison of Checklist and Anchored Global Rating Instruments for Performance Rating of Simulated Pediatric Emergencies

Adler, Mark D. MD; Vozenilek, John A. MD; Trainor, Jennifer L. MD; Eppich, Walter J. MD, MEd; Wang, Ernest E. MD; Beaumont, Jennifer L. MS; Aitchison, Pamela R. RN; Pribaz, Paul J. MS; Erickson, Timothy MD; Edison, Marcia PhD; McGaghie, William C. PhD

Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare:
February 2011 - Volume 6 - Issue 1 - pp 18-24
doi: 10.1097/SIH.0b013e318201aa90

Purpose: To compare the psychometric performance of two rating instruments used to assess trainee performance in three clinical scenarios.

Methods: This study was part of a two-phase, randomized trial with a wait-list control condition assessing the effectiveness of a pediatric emergency medicine curriculum targeting general emergency medicine residents. Residents received 6 hours of instruction either before or after the first assessment. Separate pairs of raters completed either a dichotomous checklist for each of three cases or the Global Performance Assessment Tool (GPAT), an anchored multidimensional scale. A fully crossed person × rater × case generalizability study was conducted. The effect of training year on performance is assessed using multivariate analysis of variance.

Results: The person and person × case components accounted for most of the score variance for both instruments. Using either instrument, scores demonstrated a small but significant increase as training level increased when analyzed using a multivariate analysis of variance. The inter-rater reliability coefficient was >0.9 for both instruments.

Conclusions: We demonstrate that our checklist and anchored global rating instrument performed in a psychometrically similar fashion with high reliability. As long as proper attention is given to instrument design and testing and rater training, checklists and anchored assessment scales can produce reproducible data for a given population of subjects. The validity of the data arising for either instrument type must be assessed rigorously and with a focus, when practicable, on patient care outcomes.

Use of a continuing medical education course to improve fellows' knowledge and skills in esophageal disorders

PubMed ID: 21309911

Dis Esophagus. 2011 Feb 10. doi: 10.1111/j.1442-2050.2010.01161.x. [Epub ahead of print]

Kim HC, Pandolfino JE, Komanduri S, Hirano I, Cohen ER, Wayne DB.

Departments of Gastroenterology andMedicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA.

Abstract

Advanced esophageal endoscopic procedures such as stricture dilation, hemostasis tools, and stent placement as well as high-resolution manometry (HRM) interpretation are necessary skills for gastroenterology fellows to obtain during their training. Becoming proficient in these skills may be challenging in light of higher complication rates compared with diagnostic procedures and infrequent opportunities to practice these skills. Our aim was to determine if intensive training during a continuing medical education (CME) course boosts the knowledge and skills of gastroenterology fellows in esophageal diagnostic test interpretation and performance of therapeutic procedures. This was a pretest-posttest design without a control group of a simulation-based, educational intervention in esophageal stricture balloon dilation and HRM interpretation. The participants were 24 gastroenterology fellows from 21 accredited US training programs. This was an intensive CME course held in Las Vegas, Nevada from August 7 to August 9, 2009. The research procedure had two phases. First, the subjects were measured at baseline (pretest) for their knowledge and procedural skill. Second, the fellows received 6 hours of education sessions featuring didactic content, instruction in HRM indications and interpretation, and deliberate practice using an esophageal stricture dilation model. After the intervention, all of the fellows were retested (posttest). A 17-item checklist was developed for the esophageal balloon dilation procedure using relevant sources, expert opinion, and rigorous step-by-step procedures. Nineteen representative HRM swallow studies were obtained from Northwestern's motility lab and formed the pretest and posttest in HRM interpretation. Mean scores on the dilation checklist improved 81% from 39.4% (standard deviation [SD]= 33.4%) at pretest to 71.3% (SD = 29.5%) after simulation training (P < sd =" 16.4%)" sd =" 15.8%),">

New directions in simulation-based surgical education and training: Validation and transfer of surgical skills, use of nonsurgeons as faculty...

New directions in simulation-based surgical education and training: Validation and transfer of surgical skills, use of nonsurgeons as faculty, use of simulation to screen and select surgery residents, and long-term follow-up of learners

Daniel J. Scott MD, FACSa, Carla M. Pugh MD, PhD, FACSb, E. Matthew Ritter MD, FACSc, Lenworth M. Jacobs MD, MPH, FACSd, Carlos A. Pellegrini MD, FACS, FRCSI (Hon)e and Ajit K. Sachdeva MD, FRCSC, FACSf, ,
a University of Texas Southwestern Medical Center, Dallas, TX
b Northwestern University Feinberg School of Medicine, Chicago, IL
c Uniformed Services University, Bethesda, MD
d University of Connecticut School of Medicine, Hartford, CT
e University of Washington, Seattle, WA
f American College of Surgeons, Chicago, IL
Accepted 16 November 2010. Available online 5 February 2011.

The Consortium of American College of Surgeons-Accredited Education Institutes was created to explore new opportunities in simulation-based surgical education and training beyond the scope of individual accredited institutes. During the Third Annual Meeting of the Consortium of American College of Surgeons-Accredited Education Institutes Consortium, 4 work groups addressed the validation and transfer of surgical skills, the use of nonsurgeons as faculty, the use of simulation to screen and select surgery residents, and long-term follow-up of learners. The key elements from the deliberations and conclusions are summarized in this manuscript.
Article Outline

Validation and transfer of surgical skills
Use of nonsurgeons as faculty for simulation-based surgical education
Use of simulation to screen and select surgery residents
Long-term follow-up of learners after participation in simulation-based surgical education and training
Conclusions
References

Implementation science: Addressing complexity in medical education.

PMID: 21275541

Med Teach. 2011;33(2):97-8.

McGaghie WC.

A report on the piloting of a novel computer-based medical case simulation for teaching and formative assessment of diagnostic laboratory testing

Clarence D. Kreiter, Thomas Haugen, Timothy Leaven,Christopher Goerdt, Nancy Rosenthal, William C. McGaghie and Fred Dee

Department of Pathology, University of Iowa Carver College of Medicine, Iowa City, IA, USA;

Northwestern University Feinberg School of Medicine, Chicago, IL, USA

Citation: Medical Education Online 2011, 16: 5646 - DOI: 10.3402/meo.v16i0.5646

Objectives: Insufficient attention has been given to how information from computer-based clinical case simulations is presented, collected, and scored. Research is needed on how best to design such simulations to acquire valid performance assessment data that can act as useful feedback for educational applications. This report describes a study of a new simulation format with design features aimed at improving both its formative assessment feedback and educational function.

Methods: Case simulation software (LabCAPS) was developed to target a highly focused and well-defined measurement goal with a response format that allowed objective scoring. Data from an eight-case computerbased performance assessment administered in a pilot study to 13 second-year medical students was analyzed using classical test theory and generalizability analysis. In addition, a similar analysis was conducted on an administration in a less controlled setting, but to a much large sample (n 143), within a clinical course that utilized two random case subsets from a library of 18 cases.

Results: Classical test theory case-level item analysis of the pilot assessment yielded an average case discrimination of 0.37, and all eight cases were positively discriminating (range 0.11 0.56). Classical test theory coefficient alpha and the decision study showed the eight-case performance assessment to have an observed reliability of s G 0.70. The decision study further demonstrated that a G 0.80 could be attained with approximately 3 h and 15 min of testing. The less-controlled educational application within a large medical class produced a somewhat lower reliability for eight cases (G 0.53). Students gave high ratings to the logic of the simulation interface, its educational value, and to the fidelity of the tasks.

Conclusions: LabCAPS software shows the potential to provide formative assessment of medical students’ skill at diagnostic test ordering and to provide valid feedback to learners. The perceived fidelity of the performance tasks and the statistical reliability findings support the validity of using the automated scores for formative assessment and learning. LabCAPS cases appear well designed for use as a scored assignment, for stimulating discussions in small group educational settings, for self-assessment, and for independent learning. Extension of the more highly controlled pilot assessment study with a larger sample will be needed to confirm its reliability in other assessment applications.

Keywords: computer-based simulation; clinical skills assessment; formative assessment; laboratory medicine; performance
assessment

Faculty evaluation of simulation-based modules for assessment of intraoperative decision making.

Pugh CM, Darosa DA, Santacaterina S, Clark RE.

Surgery. 2011 Jan 7. [Epub ahead of print]

Northwestern University, Chicago, IL.

PMID: 21216420 [PubMed - as supplied by publisher]

Abstract

BACKGROUND: Previous studies using simulation-based curricula have focused largely on technical skills. We developed a set of simulation-based modules that focus on intraoperative decision making. The objective of this study was to conduct a faculty evaluation of: (1) the usefulness of 4 newly developed, simulation-based modules; (2) the curricular need to train and assess intraoperative decision making skills of the residents; and (3) potential for resident benefit.

METHODS: Simulation-based modules were developed using a cognitive task analysis (CTA) framework. The CTA framework involved faculty interviews focusing on 4 operative tasks that span a range of complexity: (1) creation of small and large bowel stoma, (2) laparoscopic ventral hernia repair, (3) pancreaticojejunostomy, and (4) lymph node biopsy during a mediastinoscopy. An experienced psychologist conducted task-specific, one-on-one interviews with fellowship-trained specialists who perform these operations in their practice. Two faculty were interviewed for each procedure. The interviews lasted a minimum of 1 hour and focused on critical decisions, error prevention, error recognition, and error rescue strategies. The coded interview summaries were used as development guides for the simulation-based learning modules. Each module included locally developed physical models for the simulated operative tasks combined with oral and paper-based questions. The physical models were fabricated in such a way that simulated operative tasks could be performed using standard surgical instruments. To assess the newly developed simulation-based modules, 8 volunteer faculty (50% overlap with the interview pool) participated in a simulation-based exercise during a one-on-one session and then completed an 8-item survey cast on a 5-point Likert agreement scale (1 = strongly disagree, 5 = strongly agree). One of the items was worded negatively to ensure internal consistency. An independent observer recorded faculty session times and assessed faculty engagement in the task (1 = not engaged, 5 = extremely engaged).

RESULTS: On average, faculty spent 60 minutes completing each simulation-based exercise. Over 80% of this time was spent performing the operative tasks as they would during a real-life procedure. Mean engagement rating was 4.9 (maximum 5.0, SD = 0.3). Survey results show strong agreement on the importance of training and assessing intraoperative decision making, and that residents would likely benefit from the simulation-based modules.

CONCLUSION: We developed 4 high-fidelity simulation-based modules to assess intraoperative decision making. Faculty agree strongly on the importance and need for additional modules.