Man Versus Machine
Are carotid stent simulators ready for education and certification?
Medical competency often requires the mastery of new skill sets. With the advent of carotid artery stenting has come the need for cardiologists, vascular surgeons, interventional radiologists, neurosurgeons, and even some neurologists to master the technical skills required to perform this procedure. A number of surgical and medical specialty societies have recently developed consensus guidelines suggesting carotid stent volumes necessary to achieve proficiency. Specifically, the American College of Cardiology (ACC), American College of Physicians (ACP), Society for Cardiac Angiography and Intervention (SCAI), Society of Vascular Medicine and Biology (SVMB), and the Society of Vascular Surgery (SVS) recommend that a minimum of 30 diagnostic cerebrovascular angiograms (15 as a supervised primary operator), and a minimum of 25 supervised carotid interventions (at least half as primary operator), must be performed before a trainee can be considered competent.
Some societies (eg, ACC) have recommended that training programs include on-line (Web-based), graded didactic training, on-site proctoring and metric-based simulator training (ACC 2004) in addition to meeting these case volume requirements. Others have suggested that, given the number of operators necessary to enable widespread access to protected carotid stenting and the limited number of eligible patients requiring these procedures, endovascular simulation might be a suitable substitute for a proportion of suggested operator experience. Simulator training, whether considered above and beyond, or instead of, case volume, has been proposed as a component of the certification process as well.
Five essential elements of medical simulation training include curriculum, training tools, performance evaluation, data collection, and debriefing.1 Since the introduction of the first fully interactive patient simulator in the 1960s, the health care industry has rapidly embraced the use of medical simulators as a component of medical training.2 Today, there are more than 40 virtual reality, graphical, and mannequin- and screen-based simulators available for initial and ongoing training of health care professionals. Simulators are now routinely utilized for training in anesthesiology, ophthalmology, laparoscopy, endoscopy, and endovascular therapy.
Application of Medical Simulators as a Teaching Tool
Simulation offers a number of potential advantages over traditional medical training.3 Simulators provide an environment that is safe for patients and trainees alike, allow easy access to a wide variety of clinical scenarios such as rare but important adverse clinical events, offer the ability to actively develop training opportunities rather than passively waiting for them to arise, enable immediate feedback, permit repetition of procedures until proficiency is achieved, create an opportunity to train a team of individuals who would typically work together on a common task, create a more flexible and less costly training environment, as well as foster critical thinking, active learning, and confidence building.
The 1999 Institute of Medicine report, “To Err Is Human,” underlined the high prevalence of medical errors and their impact on patient safety in our country.4 Procedure-related complications comprise an important subset of medical errors, the latter of which impact 7.5 million people annually and result in more than 320,000 deaths and nearly $9 billion in health care costs.5 To this end, it is incumbent upon the health care industry to develop and incorporate processes that facilitate the safe and effective performance of medical (endovascular and other) procedures.
Medical simulators provide an environment in which a physician can repeatedly practice the core components of a procedure without impacting patient safety. The underpinning of simulator-based training is the provision of highly accurate and realistic scenarios in which the physician is required to perform a procedure using realistic tools while maintaining the patient’s overall health. Without compromising patient safety, physicians are able to review, repeat, and reassess their ability to perform a procedure and locate areas in which they require further training and enhancement prior to actual patient contact. Medical simulators combined with traditional training methods provide a comprehensive and fully exhaustive learning opportunity.6
Medical simulation training programs have resulted in improved performance, shorter response time, and less deviation from practice standards than nonsimulator training.7 Furthermore, simulators increase trainee confidence and competence, and they improve patient safety.8 Finally, simulator training yields cost and process efficiencies when these devices are integrated into training programs. For example, studies have demonstrated greater procedural efficiency (less procedure time, less fluoroscopy, and better visualization of tissues by x-ray) among simulator-trained individuals attempting to implant ventricular and atrial device leads.9
Endovascular fellows initially trained on simulators outperform traditionally trained fellows in a number of areas, including more independently completed procedures, shorter procedure times, and fewer interoperative errors. There are few studies that have specifically examined the role of carotid stent simulator training on outcomes. Hsu et al found that previous endovascular experience was associated with better performance on a carotid stent simulator. They also observed that proctored simulator training sessions shortened procedure times in both novice and experienced operators, but much more so in the former group.10 Similarly, Dayal et al found that a simulator-based carotid stent training program was associated with shorter procedure and fluoroscopy times among novice but not experienced users.11 Accordingly, the SCAI, SVMB, and SVS, representing the majority of physicians who routinely perform carotid stent procedures, publicly support the use of virtual reality simulation for carotid stent training.12
THE USE OF SIMULATORS FOR CERTIFICATION
Six core components are commonly used to determine medical competence: patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and systems-based practice.13 Each of these is typically evaluated during residency and fellowship training programs; however, the physician’s technical skills, which are at the core of this competency, are often not formally evaluated.14 It is often difficult to replicate or be present during a procedure to evaluate a physician’s skills. Additionally, such observations and evaluations are subjective by nature and require significant justification. In such instances, the use of medical simulators to evaluate physician technical skills for certification purposes may be highly appropriate and useful. Some entities have already begun to use high-fidelity medical simulators as part of procedural certification. At Brown Medical School, the housestaff’s ability to meet advanced cardiac life-saving criteria has been assessed using a simulator.15 The Joint Commission on Allied Health Personnel in Ophthalmology (JCAHPO) uses computer-based simulation for the Certified Ophthalmic Technician skills evaluation, which is conducted annually at more than 250 test centers in the US. This computer-based simulation has fully replaced the hands-on skill evaluation that was conducted in previous years.16 Nevertheless, most professional societies, medical associations, and licensing boards have been opposed to using simulators as a primary component of the certification process. This hesitancy has occurred partly because it has been difficult to tie simulator outcomes to real-life procedural outcomes. In short, there is a paucity of data to support the validity, reliability, and reproducibility of simulator training or its translation into clinical practice.3
During the past 5 years, a number of simulators have added performance metrics that evaluate user abilities in key areas. Medical simulators that offer evidence-based performance metrics are a useful tool in determining a physician’s understanding and incorporation of best practices, appropriate use of tools and devices, management of patient complications, effective navigation through the patient anatomy, and overall competence in performing a defined procedure without the need to use actual patients. Objective evaluation metrics based on industry standards combined with trainee observation while on the simulator could be used to determine the physician’s competence for certification purposes. In contrast, the mere presence as a second operator during a procedure (a common situation in training programs and one supported by the previously noted carotid stent consensus guidelines) does not afford the trainee the type of hands-on clinical experience needed to perform such intensive procedures safely and effectively.
Simulators are revolutionizing the practice of endovascular medicine. The procedural efficiencies they create are well-documented. Their impact on the ultimate safety and effectiveness of endovascular procedures is less well-established. There is a great need to relate simulator competence to real-life clinical outcome in well-designed studies. Given the infrequent occurrence of periprocedural adverse events associated with carotid stenting, such studies will require the use of surrogate outcomes (eg, embolization as detected by transcranial Doppler or diffusion-weighted MRI) if they are to be adequately powered. Ahead of such studies, there remains ample evidence to justify the use of simulators as an integral part of carotid stent training programs. Nevertheless, until data emerge that relate simulator-based training to clinical outcomes, it will be difficult to replace man with machine in the certification process.
Herbert D. Aronow, MD, MPH, is a coronary and peripheral interventionist at Michigan Heart, PC, in the Michigan Heart and Vascular Institute at St. Joseph Mercy Hospital in Ann Arbor, Michigan. He has disclosed that he is a paid consultant for Cordis and Medtronic. Dr. Aronow may be reached at firstname.lastname@example.org.
1. Schaefer JJ III. Integrated medical simulation, simulators in critical care and beyond. Dunn WW, ed. Current Opinion in Critical Care, Des Moines, IL, 2004; 72-75.
2. Gaba DM. The future vision of simulation in health care. Qual Saf Health Care. 2004;(suppl 1):i2-10.
3. Grenvik A, Schaefer JJ III. Medical simulation training coming of age. Crit Care Med. 2004;32:2549-2550.
4. Institute of Medicine: To Err is Human. Kohn LT, Corrigan JM, Donaldson MS, eds. Washington, DC: National Academy Press; 1999.
5. Null G. Death by medicine, part I. E-Healthy News. November 26, 2003. Available at http://www.mercola.com/2003/nov/26.
6. Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA.1999;282:861-866.
7. Hammond J. Simulation in critical care and trauma education and training. Curr Opin Crit Care. 2004;10:325-329.
8. Doyle DJ. Simulation in medical education: focus on Anesthesiology. Med Educ Online. 2002, 7:16. Available at http://www.med-ed-online.org.
9. Wong AK. Full scale computer simulators in anesthesia training and evaluation. Can J Anaesth. 2004; 51:455-464.
10. Hsu JH, Younan D, Pandalai S, et al. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40:1118-1125.
11. Dayal R, Faries PL, Lin SC, et al. Computer simulation as a component of catheter-based training. J Vasc Surg. 2004;40:1112-1117.
12. Gallagher AG, Cates CU. Virtual reality training for the operating room and cardiac catheterization laboratory. Lancet. 2004;364:1538-1540.
13. Lipner R. Developing metrics and testing strategies. American College of Cardiology Conference, March 9, 2005; Orlando, Florida.
14. Fried MP, Satava R, Weghorst S, et al. Identifying and reducing errors with surgical simulation. Qual Saf Health Care. 2004;(suppl 1):i19-26.
15. Kobayashi, L. High fidelity medical simulation enhances housestaff satisfaction with ACLS certification and recertification training. Academic Emergency Medicine 2003. 2003; 10:488.
16. Joint Commission on Allied Health Personnel in Ophthalmology. COT Skill Evaluation Content Areas and Test Information. JCAHPO 2005. Available at http://www.jcahpo.org/skilleval.htm.
Abrahamson S, Denson J, Wolf R. Effectiveness of a simulator in training anesthesiology resident. J Med Educ 1969; 44: 515-519.
Alinier G, Gordon R, Hunt B. Effectiveness of the use of simulation in nurse education: study design and initial results. International Meeting on Medical Simulation. San Diego, CA, 2003.
Aloisio G, Barone L, Bergamasco M, et al. Computer-based simulator for catheter insertion training. Stud Health Technol Inform. 2004;98:4-6.
Avrahami R, Marin M, Haddad M, et al. [A new endovascular simulator]. Harefuah. 1998;135:304-305.
Aziz I, Lee J, Lee JT, et al. Accuracy of three-dimensional simulation in the sizing of aortic endoluminal devices. Ann Vasc Surg. 2003;17:129-136.
Barach P, Satish U, Streufert S. Healthcare assessment and performance: using simulation. Simulation Gaming. 2001;32:147-155.
Barnes SZ, Morr DR, Oggero E, et al. The realization of a haptic (force feedback) interface device for the purpose of angioplasty surgery simulation. Biomed Sci Instrum. 1997;33:19-24.
Berry E, Marsden A, Dalgarno KW, et al. Flexible tubular replicas of abdominal aortic aneurysms. Proc Inst Mech Eng [H]. 2002;216:211-214.
Beyea SC. Human patient simulation: a teaching strategy. AORN J. 2004;80:738, 741-742.
Boulet JR, Murray D, Kras J, et al. Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology. 2003;99:1270-1280.
Buck GH. Development of simulators in medical education. Gesnerus. 1991;48 Pt 1:7-28.
Champion HR, Gallagher AG. Surgical simulation—a ‘good idea whose time has come.’ Br J Surg. 2003;90:767-768.
Chong CK, Brennan J, How TV, et al. A prototype simulator for endovascular repair of abdominal aortic aneurysms. Eur J Vasc Endovasc Surg. 1997;13:330-333.
Chong CK, How TV, Black R, et al. Development of a simulator for endovascular repair of abdominal aortic aneurysms. Ann Biomed Eng. 1998;26:798-802.
Chui CK, Li Z, Anderson JH, et al. Training and pretreatment planning of interventional neuroradiology procedures—initial clinical validation. Stud Health Technol Inform. 2002;85:96-102.
Cooper JB, Taqueti VR. A brief history of the development of mannequin simulators for clinical education and training. Qual Saf Health Care. 2004;13(suppl 1):i11-18.
Cotin S, Dawson SL, Meglan D, et al. ICTS, an interventional cardiology training system. Stud Health Technol Inform. 2000;70:59-65.
Curry L, Gass D. Effects of training in cardiopulmonary resuscitation on competence and patient outcome. Can Med Assoc J. 1987;137:491-496.
Davies JM, Helmreich RL: Virtual reality in medical training. Can Med Assoc J. 1997;157:1352-1353.
Davies JM, Helmreich RL: Simulation: it’s a start. Can J Anaesth. 1996;43:425-429.
Forrest FC, Taylor M. High level simulators in medical education. Hosp Med. 1998;59:653-655.
Forrest FC, Taylor MA, Postlethwaite K, et al. Use of a high-fidelity simulator to develop testing of the technical performance of novice anaesthetists. Br J Anaesth. 2002;88:338-344.
Fujino T, Kobayashi M, Calderhead RG. The computer and simulation surgery. Keio J Med. 2001;50(suppl 2):39-85.
Good ML. Patient simulation for training basic and advanced clinical skills. Med Educ. 2003;37(suppl)1:14-21.
Gordon MS, Ewy GA, Felner JM, et al. A cardiology patient simulator for continuing education of family physicians. J Fam Pract.1981;13:353-356.
Jacomides L, Ogan K, Cadeddu JA, et al. Use of a virtual reality simulator for ureteroscopy training. J Urol. 2004;171:320-323;discussion 323.
Kapur PA, Steadman RH. Patient simulator competency testing: ready for takeoff? Anesth Analg. 1998;86:1157-1159.
Kneebone R. Simulation in surgical training: educational issues and practical implications. Med Educ. 2003;37:267-277.
Kneebone RL, Scott W, Darzi A, et al. Simulation and clinical practice: strengthening the relationship. Med Educ. 2004;38:1095-1102.
Kras J, Murray D, Woodhouse J, et al. The validity of a simulation-based anesthesia acute care skills evaluation. International Meeting on Medical Simulation. San Diego, CA, 2003.
Lane JL, Slavin S, Ziv A. Simulation in medical education: a review. Simulation Gaming. 2001;32:297-314.
McMahon GT, Monaghan C, Falchuk K, et al. A simulator-based curriculum to promote comparative and reflective analysis in an internal medicine clerkship. Acad Med. 2005;80:84-89.
Morgan PJ, Cleave-Hogg D. A worldwide survey of the use of simulation in anesthesia. Can J Anaesth. 2002;49:659-662.
Morgan PJ, Cleave-Hogg D. Comparison between medical students’ experience, confidence and competence. Med Educ. 2002;36:534-539.
Olympio MA. Simulation saves lives. ASA Newsletter. 2001;15-19.
Quek FK, Kirbas C, Charbel FT. AIM: attentionally based interaction model for the interpretation of vascular angiography. IEEE Trans Inf Technol Biomed. 1999;3:151-157.
Riley RH, Grauze AM, Chinnery C, et al. Three years of “CASMS”: the world’s busiest medical simulation centre. Med J Aust. 2003;179:626-630.
Sajid AW, Safford RE. Multimedia computer-assisted instruction in cardiology. Arch Intern Med. 1995;155:197-203.
Solares G, Gonzalez AM, Escalante C, et al. Simulators in medicine teaching. Novelty or necessity? Med Clin (Barc). 2004;122:798-799.
Takashina T, Shimizu M, Katayama H. A new cardiology patient simulator. Cardiology. 1997;88:408-413.
Takayesu J, Gordon J, Farrell S, et al. Learning emergency medicine and critical care medicine: what does high-fidelity patient simulation teach? (Abstract #319). Acad Emerg Med. 2002;9:476-477.
Wan J, Steele B, Spicer SA, et al. A one-dimensional finite element method for simulation-based medical planning for cardiovascular disease. Comput Methods Biomech Biomed Engin. 2002;5:195-206.
Weller J, Wilson L, Robinson B. Survey of change in practice following simulation-based training in crisis management. Anaesthesia. 2003;58:471-473.
Weller JM. Simulation in undergraduate medical education: bridging the gap between theory and practice. Med Educ. 2004;38:32-38.
Whittaker DR, Dwyer J, Fillinger MF. Prediction of altered endograft path during endovascular abdominal aortic aneurysm repair with the Gore Excluder. J Vasc Surg. 2005;41:575-583.
Wright MC, Taekman JM, Endsley MR. Objective measures of situation awareness in a simulated medical environment. Qual Saf Health Care. 2004;13(suppl)1:i65-71.
Yambe T, Yoshizawa M, Tabayashi K, et al. Virtual percutaneous transluminal coronary angioplasty system for an educational support system. Artif Organs. 1998;22:710-713.