Training With Endovascular Simulators in Graduate Medical Education Programs
There is a growing role for this technology, although specific applications and cost effectiveness are challenges to be overcome.
New educational technologies, such as high-fidelity endovascular procedure simulators, are being used to help endovascular specialists acquire peripheral vascular interventional procedure skills.1-3 Opportunities to use endovascular procedure simulators have become common at regional and national meetings such as VIVA, TCT, and specialty society meetings. The FDA requirements for training new users on approved carotid artery stents and distal embolic protection devices include training on a virtual reality (VR) simulator. This level of acceptance has clearly contributed to the growing use of endovascular simulators.
NEW DIRECTIONS IN GRADUATE MEDICAL EDUCATION
The past decade has seen substantial changes in how graduate medical education (GME) is provided. Many of these changes are driven by the organization that accredits North American residency and fellowship programs, the Accreditation Council for Graduate Medical Education (ACGME).
The ACGME expects each training program to provide the educational experiences necessary for a resident to gain specific knowledge, skills, behaviors, and attitudes requisite of a physician in the specialty.4 In 1999, the ACGME endorsed a set of core competencies for residents in the areas of:
- patient care
- medical knowledge
- practice-based learning and improvement
- interpersonal and communication skills
- systems-based practice
Patient safety concerns also led the ACGME to implement a mandatory 80-hour-per-week cap on resident work hours, which became effective July 1, 2003.4 This move was, in part, to mitigate the negative effects of fatigue on physician performance. There was a broader implication, however. This move reflected the increasing focus on residency training as an educational rather than service activity. In implementing this change, programs were required to consider ways to provide a thorough educational experience with fewer available contact hours.5,6
The use of simulation technology has been cited as one way to meet the challenge of more efficiently providing clinically focused education.6 Simple devices teach hand-eye coordination, and more sophisticated VR trainers, such as the current generation of endovascular simulators, teach complex tasks and sequencing. Residents can acquire and practice basic skills before applying them with patients. The learning environment of a skills laboratory also offers a less stressful and more controlled situation. The resident can take extra time on a simulator or repeat procedures as needed to master the procedural tasks being taught.
In the past, the ACGME evaluated GME programs on their potential to educate, based on compliance with specific accreditation requirements. The modern focus, however, is on outcomes, with critical evaluation of the educational processes and accomplishments of training programs. This focus is the basis for the ACGME's Outcome Project, a long-term initiative to emphasize educational outcome assessment.7 The Outcome Project started in 2001, and is now in its integration phase. Residency programs are expected to provide objective resident performance data, use external measures of performance, and make data-driven changes in their curricula and programs when opportunities for improvement are recognized.
Implementation of competency-based education programs are a component of the ACGME Outcome Project. A competency-based program focuses on learner performance in reaching specific goals and objectives. With this approach, task performance becomes the focus, not the number of repetitions. Using the example of iliac artery stenting, the learner is evaluated on the ability to perform the procedure appropriately, not simply on the number of cases logged. Elements of this evaluation may be simple or complex, but the ACGME now expects to have objective or quantifiable performance information. This is not just to assess the quality of the residents being trained, but each program is required to use resident performance and outcome assessment data to evaluate the program's effectiveness.
The ACGME has required GME programs to document and demonstrate that there are appropriate learning opportunities in each core competency domain, that they use multiple assessment methods, and that aggregate data are used to improve the educational program.
In the core competency domains of patient care and medical knowledge, patient care simulations or procedural training devices (such as endovascular procedure simulators) can provide standardized experiences that can augment clinical and didactic instruction. In surgical disciplines, it has been shown that individuals trained initially on a simulator are subsequently more efficient in the operating room, and they make fewer technical missteps than peers trained with traditional, patient-centered models.6,8-11 Further, simulators have the potential to be used as assessment tools. Again, using the example of iliac artery stenting, elements of a resident's procedural skill and knowledge can be assessed by use of fluoroscopy time, volume of contrast used, accuracy of stent positioning and deployment, or other metrics that can be recorded by the simulator.
The move toward competency-based training is appropriate, as the current case experience requirements of the specialty boards are quite low. For a vascular surgeon's training, the ACGME Residency Review Committee (RRC) for Vascular Surgery requires the resident to complete 200 major vascular reconstructions, not specifying the number of endovascular versus open cases.12 The Society of Vascular Surgery guidelines for credentialing suggest 100 diagnostic arteriogram and 50 interventions as a minimum experience for a surgeon seeking hospital privileges.13 The RRC for Interventional Cardiology requires fellows to perform at least 250 coronary interventions, but despite the requirement that the fellow has knowledge of peripheral arterial disease management of peripheral arterial complications, there is no case requirement for any noncardiac interventions.14 The RRC for Vascular and Interventional Radiology requires that fellows document participation in at least 500 cases, with the stipulation that these cover the range of the specialty. No specific guidance is provided regarding the expected number of any particular category or type of case (eg, arterial intervention or iliac stenting).
TYPES OF ENDOVASCULAR PROCEDURES MODELED ON CURRENT SIMULATORS
Several companies produce endovascular procedure simulators and related educational content (Table 1). These devices can present a variety of clinical scenarios and procedural simulations, including cardiac and peripheral cases. Available cardiac procedural simulations include right and left heart catheterization, angioplasty, and stenting. Modules are under development for modeling structural and valvular heart disease interventions. Specific to peripheral vascular applications, there are modules for angiography, angioplasty, and stenting of iliac, renal, femoral, and carotid arteries. Simulations of endovascular repair of abdominal aortic aneurysms are also being developed.
APPLICATIONS OF SIMULATOR TRAINING IN A TEACHING CENTER
Simulation-based endovascular procedure training has been most widely used for training practicing physicians in advanced procedures, such as carotid artery stenting and for device-specific training (such as the use of an embolic protection device). Simulation-based training appears to be especially well suited for use in the setting of the academic medical center, though, as is ideal for providing an introduction to endovascular procedures for residents and fellows who have yet to develop sophisticated procedural skills.
Recent experience from UC Davis offers one perspective on how endovascular simulation training can be used in the setting of an academic medical center. UC Davis has established an educational laboratory that in 2003 incorporated a fixed installation of an endovascular procedure simulator (SimSuite, Medical Simulation Corp., Denver, CO). The SimSuite, other VR procedure simulators, and robotic surgery training devices are operated in a facility designated the "Center for Virtual Care." This facility has incorporated a variety of simulation training tools to create a virtual hospital environment for practicing medical procedures (Figures 1-3). The center can mimic a single intensive care unit or replicate the flow of patients through the medical centerÑfrom the field to transport through emergency treatment and surgery to the intensive care unit. The SimSuite, however, is most commonly used as a stand-alone educational laboratory.
Some components of the training program include self-directed learning (ie, online or computer-based instruction), but all of the procedural training on the simulator is done with direct interaction of an instructor and a learner in a one-to-one or one-to-two setting. Faculty physicians mentor trainees in some of the training events, but most of the training is provided by a full-time, dedicated educational specialist who has both clinical knowledge and technical expertise with the simulator.
Most participants in the training program at UC Davis Medical Center are physicians in the university's GME programs, including cardiology fellows and residents from the departments of surgery, internal medicine, anesthesiology, and emergency medicine. Nonphysicians are frequently involved in simulator training as well, often in the context of training for team-based care. Approximately 40% of simulation training events at UC Davis are for nurses or technologists.
AVAILABILITY AND COSTS OF SIMULATION TRAINING
Simulator training for teaching procedural skills and the use of VR and other training aids is becoming more common in surgical specialties. Korndorffer et al, from the Tulane Center for Minimally Invasive Surgery, surveyed 253 general surgery program directors to determine the perceived value, prevalence, equipment, types of training, supervision, and costs of the labs.15 One hundred sixty-two (64%) responded, with 88% of the responders who consider skills labs effective in improving operating room performance; 55% already had implemented skills labs. Of 89 programs with skills labs, 99% used video-trainer equipment, and 46% used virtual reality trainer equipment. Skills lab training was mandatory in 55% and was supervised in 73% of the programs. The mean development cost was $133,000. The range of reported costs was broad ($300 to $1,000,000), as there is significant variability in the equipment and training practices in existing labs. It was suggested that strategies are needed for more widespread implementation of skills labs, and that standards should be developed to facilitate uniform adoption of validated curricula to optimize training efficiency and educational benefit. Though general surgery program directors consider skills labs important, 45% of programs still have yet to provide this training to their residents. Resource constraints are a likely reason.
The availability of endovascular simulator training for residents in cardiology, radiology, and surgery programs is not well documented. The costs associated with operating an endovascular simulator may vary, depending on the equipment used, the level of utilization, number and types of support personnel, and other factors. In a 2005 article, Carl Patow, MD, MPH, cited the following costs: "HealthPartners provided start-up funds to purchase $350,000 in simulation technology, with additional funding of $150,000 in 2003 and $100,000 in 2004, while Metropolitan State offered space and curriculum development services. Planned fundraising aims to add $800,000 to $1 million over the next 3 years to purchase additional simulation technology and for a permanent director, instructional design staff, and technology support staff."16 The article does not provide details regarding which types of simulation technology were purchased or in what quantity.
The investment range for a "simulation program" is based on the needs of the particular center. The budget will vary based on the complexity of needs, breadth of contingencies that it serves, and level of service support. UC Davis invests $300,000 to $400,000 annually on the simulation tools and service support to meet its endovascular educational objectives.
The cost-effectiveness of simulation training has not been confirmed, but even moderate increases in the efficiency of care in a teaching hospital may provide economic benefit. Health system costs related to the use of the operating room for resident teaching, for example, have been estimated to be approximately $50,000 per surgery resident (due to increased operative times and decreased efficiency that occur when operating with a trainee).17 Helping the learner become more efficient in the performance of clinical procedures decreases costs. Avoidance of complications would be expected to save even more. Unfortunately, cost-savings gleaned from improved training processes may not be easily converted into financial support for the training programs.
UC DAVIS VASCULAR SURGERY FELLOWS TRAINING PROGRAM
One way to diminish the cost of each program is to use simulation training centers as regional resources, allowing several institutions' GME programs to share the use of a single facility. For the past 3 years, a regional program for new vascular surgery fellows has been taught at UC Davis. Educational grants from Boston Scientific Corporation (Natick, MA) have allowed a single simulator site to provide training to individuals from 11 programs over a five-state area in the western US. Fellows participate in a series of 2-day endovascular training programs that utilize the SimSuite simulator, didactic instruction, computer-based training, and tabletop procedure demonstrations. The course evaluations by the participants suggest that fellows want to have training with endovascular simulation as a part of their curriculum, either within their own institution (if available), or through participation in regional training programs.
Data have also been collected to evaluate the impact of simulation-based training on the endovascular novices' skills. Performance on a standardized iliac intervention case was used to assess skills and knowledge at the beginning of the training program and was repeated at completion of the training. Expressed as percent change from each individual's initial performance value (mean change from baseline), total procedure time was shortened 54%, volume of contrast use decreased 44%, and fluoroscopy time was reduced by 48%. Selection of angioplasty balloon catheters and stents was improved; the average number of catheters used and stents deployed decreased. Both objective metrics and subjective faculty assessments also suggested a substantial improvement in each fellow's performance. Details of this experience are pending publication in the Journal of Vascular Surgery.
Our experience to date has demonstrated the usefulness and a high level of resident acceptance of endovascular procedure simulation when used as an integral part of a structured endovascular skills curriculum. Plans are now being discussed for converting this western regional program into a national program utilizing regional training centers, potentially offering this type of experience to the majority of incoming vascular surgery trainees.
MEDICAL SPECIALTIES AND SIMULATION
Several specialty societies and boards are implementing or evaluating simulation as an assessment tool.
The independent American Board of Vascular Medicine (ABVM) is the first organization to incorporate the use of an endovascular VR simulator in a formal testing setting. The ABVM offers credentialing examinations in vascular and endovascular medicine. The endovascular examination includes a written (computer-delivered) knowledge test. In addition, candidates are tested on procedural skills on a simulator.18
The American Board of Internal Medicine (ABIM) has completed a 10-center pilot study to evaluate the use of an endovascular simulator as an assessment tool for certification or recertification examinations in interventional cardiology.19 There were three groups of participants, 120 subjects in total, that completed test scenarios: (1) cardiology fellows early in their second year of training; (2) interventional cardiology fellows or interventional cardiologists at the start of their career; or (3) board-certified interventional cardiologists with 3 or more years of practice experience. Performance on the simulator could discriminate between differences in expertise and the magnitude of the differences varied by specific case scenario. Analysis of the data is ongoing, and additional follow-up studies are in development.
The American College of Surgeons (ACS) is also interested in educational technologies, and it is actively supporting the development of regional surgical training centers that utilize bench models, simulations, simulators, virtual reality simulation, and related technologies. Through the Division of Education, the ACS has instituted a program for the accreditation of regional centers, with the goal of creating "a network of accredited Education Institutes across the country to assist surgeons, residents, medical students, and other members of the surgical team in achieving the requisite knowledge and skills in order to provide optimal care to surgery patients."20 By promoting the development of Educational Institutes and research to develop and validate assessment tools, the ACS is working toward a longer-term goal of better assessment of technical competence as part of maintenance of certification.
Currently available endovascular procedure simulators are useful educational tools for training endovascular specialists, but they are only one component of a broad curriculum. Cognitive and skills training on computer-based learning modules and simulated cases can make a resident or fellow better prepared for "live" clinical cases, and studies from other surgical disciplines suggest that training in a skills laboratory reduces errors in actual practice.
Practical assessment of procedural competency is a goal that has been only partially met with traditional assessment methods. Feldman et al found that performance on a simulator was better at identifying residents with below-average technical skills than evaluation methods that included only written tests or traditional subjective evaluations of residents at the end of their rotations.21 Data from multiple assessment instruments, however, should be considered for assessment of technical competency, as no single method provides a complete picture of physician competency.
While preliminary data suggest that performance on simulators can distinguish novice, learner, and expert operators, uses of endovascular simulators as formal testing tools are limited. It will be useful to consider the experience of the ABVM after a sufficient number of candidates for the endovascular board examination have been tested. Other boards, however, await additional validation studies before implementing simulation-based testing.
The use of objective metrics from a simulator, however, does not need to be a "stand-alone" testing method. Many specialty boards, including the Vascular Surgery Board of the American Board of Surgery, include an oral examination as part of the testing process. Oral examiners present clinical scenarios, and then the candidate describes a set of actions to be taken. The experts evaluate the candidate on knowledge (including procedural knowledge), decision-making, and ability to recognize and respond to complications. The evaluation is subjective, although scoring guidelines are used. Incorporating simulated cases into a testing scenario with an expert examiner might be an intermediate step, something less than relying completely on a computer-generated score from the testing device. This approach would give the expert evaluator a role in determining how well performance on the simulated case reflected the candidate's knowledge and procedural skills.
In addition to gaining board certification, new GME program graduates need to be able to gain hospital privileges. Privileges to perform endovascular procedures are conferred by individual institutions, typically through medical staff committees that have been assigned to ensure that staff credentials and practice privileges are granted only to providers with the training and experience to ensure safe and effective care. Most hospitals require proctoring or supervised practice prior to awarding full privileges, and there is usually a requirement for periodic review and renewal.
At present, endovascular cases performed on a simulator have not been accepted as an adequate substitute for actual procedural experience, but this could be considered in the future, once the validity of such an approach is demonstrated. Training with simulators, however, already has direct relevance to the credentialing process, as simulation training programs teach essential cognitive elements, and simulations help with learning and practicing angiographic and interventional skills. Simulation may also be better for training physicians in the recognition and management of serious intraprocedural complications, as these events may be rare enough to have never been encountered during the performance of actual cases during training.
Even though endovascular simulator training is new to GME programs, there is a growing recognition of the value to this emerging technology. Validating specific applications and making the process cost effective are the challenges that will need to be met to make this technology a fully integral part of GME.
David L. Dawson, MD, is Professor, UC Davis Vascular Center, Division of Vascular and Endovascular Surgery in Sacramento, California. He has disclosed that he holds no financial interest in any product or manufacturer mentioned herein. Dr. Dawson may be reached at (916) 734-8122; email@example.com.