At Issue: Competency to perform surgery
Question: Can someone's competency to perform particular orthopedic procedures be tested? If so, how?
Click Here to Manage Email Alerts
Competency, proficiency can be accurately assessed
The answer is yes. Although the term competence does not have a universally accepted definition, many consider it to represent the acquisition of the bare minimum skill set that is acceptable. Proficiency, on the other hand, typically indicates a level of mastery of a particular skill that is well above competency. Competency assessments are generally “described” whereas proficiency is rigidly defined by the attainment of an unambiguously defined benchmark. The groundbreaking level 1, randomized, blinded, prospective study by Richard L. Angelo, MD, PhD, clearly demonstrates that beyond competency, proficiency can be accurately and reliably assessed for the individual learner.
The Global Rating Scales or Likert-type grades are subjectively based assessments of surgical competence and, as such, are less reliable than objective evaluations. The Copernicus Initiative headed by Angelo and his colleagues in conjunction with the Arthroscopy Association of North America focused on finding a superior method of surgical training with the ability to objectively evaluate surgical performance. Proficiency based progression (PBP) surgical training offers the clear and superior alternative to our current “apprenticeship” teaching model that we relied upon in the past. PBP training is best described as the acquisition of a necessary skill set to a predefined proficiency benchmark before progressing on to learning the next set of skills within the curriculum, eg, the trainee must acquire the ability to tie an arthroscopic knot to proficiency before he or she learns how to insert a suture anchor to proficiency. Proximate feedback (addressing trainee mistakes by the instructor at the time they occur) is critical as it enables the trainee to eliminate errors in technique. The deliberate practice of specific skills allows the student to repeatedly practice and master challenging techniques. Summative feedback offers both an overall impression of the trainee’s progress as well as a determination as to whether the proficiency benchmark was attained for the compete procedure.
Index arthroscopic Bankart surgery
Angelo and colleagues determined that an arthroscopic Bankart procedure was the most suitable index operation for an investigation into the merits of PBP training. Metric characterization required a task deconstruction of the Bankart procedure in which unequivocally clear steps (45), errors (77) and sentinel errors were identified, and a metric sheet was developed with which grading of the procedure could be accomplished. A Delphi panel of experienced arthroscopic shoulder surgeons was convened and the metrics were further refined. The metrics were stress-tested and shown to demonstrate construct validity, namely the ability to distinguish between the performance of novice and advanced arthroscopic surgeons. A team of raters, who would grade the final surgical videos of the completed Bankart procedure, went through extensive training and ultimately demonstrated an inter-rater reliability that exceeded 90%. The benchmark for proficiency was then determined based on the mean performance of the experienced surgeons who completed the index procedure.
Copernicus research culminated in a randomized, prospective, blinded study in which PGY 4 and 5 residents were randomized to traditional apprenticeship-style learning cohort or PBP training. The findings were compelling. Nearly 70% of the PBP group achieved proficiency in performing the arthroscopic Bankart procedure compared to only 29% of the traditional group.
PBP curriculum is foundational
That PBP is superior to the apprenticeship model is clearly evident. Furthermore, a PBP curriculum should form the foundation for not only those who are in training programs, but also for those who are active and lifelong learners seeking to improve their skills and knowledge. In addition to the Bankart procedure, AANA has created validated metrics for an arthroscopic rotator cuff repair and is actively developing the metrics for a hip labral repair and an ACL reconstruction. If objective skills credentialing is a future requirement, Angelo and his colleagues have developed the pathway to such a goal.
Competency, or better yet, proficiency can be objectively measured. As with most substantive issues, a greater demand for time and energy is a necessary component to success with PBP training efforts. In time, as validated metrics are created for more orthopedic procedures, PBP training will be the gold standard to which we all aspire.
- References:
- Angelo RL, et al. Arthroscopic rotator cuff repair metrics: Establishing face, content, and construct validity. Arthroscopy. (accepted for publication 2019).
- Angelo RL. Arthroscopy. 2015;doi:10.1016/j.arthro.2015.05.013.
- Angelo RL, et al. Arthroscopy. 2015;doi:10.1016/j.arthro.2015.04.092.
- Angelo RL, et al. Arthroscopy. 2015;doi:10.1016/j.arthro.2015.07.001.
- Pedowitz RA, et al. Arthroscopy. 2015:doi:10.1016/j.arthro.2015.06.021.
- For more information:
- Richard R.K. Ryu, MD, can be reached at The Ryu Hurvitz Orthopedic Clinic, 2936 De La Vina St., Santa Barbara, CA 93105; email: rknrmd@gmail.com.
Disclosure: Ryu reports no relevant financial disclosures.
Practice, integrity should determine surgeon’s competency
This headline of an article by Sandra G. Boodman appeared in the April 25, 2016 Washington Post: “Do you need complex surgery? Some doctors may not have much practice.” Boodman wrote, “The largely unfettered ability of surgeons with minimal experience to perform high-risk procedures, particularly in hospitals caring for significant numbers of patients, has been the subject of a contentious, long-running battle known as a volume outcome debate.”
Volume is one of the critical factors in a surgeon’s competency. Some officials fear this emphasis on volume penalizes low-volume surgeons in smaller hospitals who can have good outcomes. These low-volume surgeons have increased mortality, longer lengths of stay, higher complication rates, longer operative times, higher readmission rates and higher revision rates. Similarly, low-volume hospitals are associated with a longer length of stay, less likely discharge home, higher transfusion rates and higher revision rates, according to an article by Jon J. P. Warner, MD, and colleague. “Although volume most certainly plays a role in determining outcomes and cost, the more important principle is to measure outcomes and treatment results may improve with such measurement,” the authors wrote.
Currently, it is impractical to restrict surgeons or low-volume centers from performing complex surgical procedures. The solution is to improve care by the low-volume surgeon. Despite the surgeon’s opinion that volume does not matter, payers recognize this volume factor and eventually patients will recognize it. Some large organizations are partnering with medical insurance companies to direct patients with specific problems to specific hospitals and surgeons, even paying airfare and hotel reservations for the patient and family members.
Observe complex procedures
Reverse total shoulder arthroplasty (TSA) and superior capsular reconstruction are both new, complex shoulder operations. How do surgeons with busy practices learn new procedures like these? The traditional mentoring approach for surgeon-training consists of these simple steps: I do it while you watch; I do it while you help me; you do it and I help you; and you do it and I watch.
The mentee usually develops competence which, repeated over time, leads to confidence and eventually to independence. However, there are underlying basic skills that should allow the surgeon to become more confident in advanced versions of these procedures. One example is moving from an anatomical shoulder arthroplasty to a reverse TSA. Another example is a surgeon advancing from arthroscopic repair of small rotator cuff tears to repair of larger tears that involves tendon releases. However, a move to these procedures requires advanced skills in arthroplasty selection or suture management. These complex advanced procedures should not be performed without practice and instruction on a model or cadaver, as well as the observation of an accomplished expert surgeon. The surgeon learning the new technique may discover he or she does not have the skills to perform this procedure.
Surgeon age and competence
Is surgeon competency related to age? In a 2016 article, Krista L. Kaups, MD, MSc, pointed out that competency, not age, determines the surgeon’s ability to practice. It is true that our vision and hearing begin to decline at age 50, cognitive function has a steep decline after age 65 and dementia occurs in 5% to 10% of individuals aged older than 65 years. Twenty percent of physicians are not competent after the age of 65 years. Nevertheless, many older surgeons maintain their “accumulated knowledge” and the extent of these changes vary significantly from one individual to another.
It is thought that Will Durant once said, “We are what we repeatedly do. Excellence, then, is not an act, but a habit.” Surgeons strive for excellence. Others have said what someone does is not who they are. Surgeons should not let the ego and pride of their reputation interfere with demanding and analytical critical preoperative thinking. The principle, “First, do no harm,” which is attributed to Hippocrates, should guide surgeons in balancing competing priorities and surgical practice. Often factors that are not directly related to the patient enter into a surgeon’s intraoperative decision-making. We must be aware that these sources of pressure, such as income or community status, may interfere with critical self-reflection. The surgeon may simply be unprepared to take on a new, complex surgical procedure. We, as surgeons, may be the best judge of our own capabilities and know when to refer a patient with a complex problem to a more experienced colleague.
It is up to the surgeon’s integrity, hopefully unaffected by ego, to determine what newer procedures he or she is able to successfully perform. After 48 years as an active orthopedic surgeon, I believe my best decisions concerning my patients were often not to proceed with a surgical procedure. Many patients improved with a cortisone shot, simple home physical therapy and, most importantly, time.
- References:
- Kaups KL. AMA J Ethics. 2016;doi:10.1001/journalofethics.2016.18.10.pfor1-1610.
- Leung A, et al. Acad Med. 2012;doi:10.1097/ACM.0b013e3182677587.
- Warner JJP, et al. Arthroscopy. 2017;doi:10.1016/j.arthro.2017.03.023.
- For more information:
- James C. Esch, MD, can be reached at Orthopaedic Specialists of North County, 1770 Troy Lane, Oceanside, CA 92054; email: jesch@shoulder.com.
Disclosure: Esch reports no relevant financial disclosures.
Skills, procedures must evolve and remain safe
Some define competency as the ability to successfully or efficiently do something. It is the minimal expectation when the “something” is surgery on humans. Although many surgeons trained in a system based on the see one, do one, teach one principle, this is not the best way to teach or learn surgical skills. A 2019 Dr. Death Podcast highlighted the need for better assessment of surgeon competency. It discussed Christopher Duntsch, MD, a neurosurgeon who was unable to complete many of his spine surgeries, which resulted in 33 of 38 patients who were maimed or died. In Duntsch’s case, hospitals and state and national medical boards did not to act on the multiple opportunities they had to stop an incompetent surgeon from operating. In defense of these institutions, there is presently no validated method to objectively measure or rate a surgeon’s abilities. Few articles on surgeon competency discuss how to address this among practicing physicians.
Orthopedic resident training
Current research is underway to identify objective, step-based learning programs to educate, as well as evaluate surgeons in training. The Accreditation Council of Graduate Medical Education requires that residency programs teach and assess six core competencies: medical knowledge, patient care, interpersonal communications skills, professionalism, systems-based practice and practice-based learning and improvement. Validated educational tools, which include basic simulations and more advanced virtual reality, augmented reality and haptic feedback simulators, have been introduced to provide both consistent education and evaluation of residents’ competency. These educational platforms transcend general orthopedics and include many orthopedic subspecialties including arthroscopy, trauma and upper extremity.
One example is PBP training from the AANA Copernicus Initiative, which teaches skills the trainee must master before progressing to more advanced techniques. Presently, there is no mandate or method to assess a surgeon’s technical skills post-training. Surgeons choose either a written or oral exam for board recertification. Surgical skills are not assessed. Post-procedure patients properly evaluated with validated outcome measurement tools may indirectly suggest the surgeon is competent. However, this may not be the best method to assess technical skills or decision-making. Presently, work is being done on simulation-based assessments similar to those used for airline pilots. Further research will be necessary to understand how or if these simulators will provide equivalent value for surgeons.
The natural evolution of disease treatment requires physicians to learn new procedures. Again, there is no standardized method to educate surgeons or assess their skills before they perform these procedures. Ultimately, the onus remains on surgeons to accurately and honestly assess their skills before embarking on a new procedure.
For orthopedic surgeons to remain current, their surgical skills and procedures must evolve. However, this must be done responsibly so that our patients do not become our experiments.
- References:
- Accreditation Council of Graduate Medical Education. Available at www.acgme.org. Accessed Oct. 15, 2019.
- Bayona S, et al. J Bone Joint Surg Am. 2014;doi:10.2106/JBJS.M.00304.
- Bernard JA, et al. J Surg Educ. 2016;doi:10.1016/j.jsurg.2016.04.023.
- Coughlin RP, et al. J Bone Joint Surg Am. 2015;doi:10.2106/JBJS.N.01140.
- Koehler RJ, et al. Arthroscopy. 2015;doi:10.1016/j.arthro.2015.06.011.
- Lopez G, et al. J Bone Joint Surg Am. 2015;doi:10.2106/JBJS.N.01269.
- Pedersen P, et al. Acta Orthop. 2014;doi:10.3109/17453674.2014.917502.
- Purnam MD, et al. J Surg Educ. 2015;doi:10.1016/j.jsurg.2014.11.001.
- Van Heest A, et al. J Hand Surg Am. 2012;doi:10.1016/j.jhsa.2011.10.050.
- Van Heest AE, et al. J Bone Joint Surg Am. 2019;doi:10.2106/JBJS.17.01512.
- Vankipuram M, et al. J Biomed Inform. 2010;doi:10.1016/j.jbi.2010.05.016.
- For more information:
- Shariff K. Bishai, DO, MS, FAOAO, can be reached at Associated Orthopedists of Detroit, 24715 Little Mack Ave., Suite 100, St. Clair Shores, MI 48080; email: skbishai@yahoo.com.
Disclosure: Bishai reports no relevant financial disclosures.