This article is more than 5 years old. Information may no longer be current.
Orthopaedic trainee assessments should include a range of evaluations
Click Here to Manage Email Alerts
Editor’s note: In this article from the FORTE organization, one member discusses how orthopaedic trainee assessments can be improved in light of the European Working Time Regulations. Look for more articles from FORTE members in future issues of Orthopaedics Today Europe.
Following the introduction of the European Working Time Regulations, The Royal College of Surgeons of England estimates 400,000 hours of surgical time have been lost, and, with it, significant opportunity for training. A survey in 2010 revealed 80% of consultants and 66% of trainees felt patient care and training had suffered as a direct result of the European Working Time Regulations (EWTR).
To address concerns surrounding quality assurance in training and practice and to ensure transparency and accountability, the Intercollegiate Surgical Curriculum Programme (ISCP) was introduced for trainees in the United Kingdom. ISCP is now a mainstay of trauma and orthopaedic surgical training. As part of a comprehensive curriculum — at the center of which lies patient safety — it also provides a platform for assessment and feedback.
Methods of assessment fall into two broad categories: formative and summative. The latter provides a quantifiable record of progress, at the end of a course or period of study, to determine the level of learning that has taken place. A grade or mark can be attributed against a set standard, with examples, including the European Board of Orthopaedics and Trauma and Fellowship of the Royal Colleges of Surgeons examinations. Summative assessment is undeniably necessary to ensure residents at the end of their training can demonstrate suitable knowledge, skills and attitudes in the specialty, to progress to safe independent practice. How can they achieve such competence?
Assess everything from skills to professionalism
While summative assessment can be considered assessment of learning, formative assessment is assessment for learning, intended to support development. Workplace-based assessments (WBAs) are the tools with which evidence of learning events can be captured within the working environment, and constructive feedback given. In the wider field of surgery, much emphasis is placed on acquiring knowledge and operative skills, although most would agree these are not the only domains that make a good surgeon, let alone a safe one.
Research among surgeons has identified clinical judgment/decision-making as the most important factor in determining patient outcomes, but attitudes and professional behavior are also considered fundamental. Assessments should, therefore, encompass these core domains of knowledge, skills, judgment and professionalism in order to give a holistic view of the development of a trainee.
Within ISCP, there are a number of WBAs that can be used to look at one or more of these areas: procedure-based assessments and direct observation of procedural skills assess operative skills; clinical evaluation exercises assess examination skills and professionalism, and case-based discussions (CBDs) assess knowledge, judgment and professionalism. Nonoperative technical skills for surgeons further deconstructs judgment and professionalism to examine the human factors side of surgical practice through assessment of the four domains of situation awareness, decision-making, communication and teamwork, and leadership, according to a 2011 study by Crossley and colleagues. CBDs, in particular, can be used to demonstrate a trainee’s ability to bridge the gap between “lower level” knowledge and comprehension of a topic, and higher order, reflective thinking that involves deconstructing a clinical situation into components for evaluation that will improve practice in the future.
Consider value of feedback
Reflection is crucial for trainees, but the most valued aspect of formative assessment is feedback. In a study, Hattie went so far as to state, “The most powerful single moderator that enhances achievement is feedback.” High quality, timely feedback from trainers is crucial, and the evidence strongly supports taking an educational approach where this is available frequently, based on studies by van de Ridder, Eraut, Van de Wiel and Watling. Such an approach can be supported through the use of multi-source feedback, which is a method that collates feedback from a number of colleagues, including physicians, nurses and other health care professionals, and allows the trainee to compare this with his or her own self-assessment of progress.
Assessment of day-to-day surgical practice is a crucial way of developing and demonstrating the skills and behaviors that we, as trainees, will need to provide the best possible care for patients. There are a number of valid and reliable WBAs available, made feasible in the workplace by online platforms and smartphone apps. Whichever tool is used, the onus is on each of us, with our limited working time, to reflect on practice and maximize the value from each learning event we encounter. However, our trainers, according to the study by Memon, can have the greatest influence on our development, through high quality feedback, to yield the greatest possible benefit.
- References:
- Crossley J, et al. Br J Surg. 2011;doi:10.1002/bjs.7478.
- Eraut M. Informal learning in the workplace: evidence on the real value of work-based learning (WBL). Development and Learning in Organizations. 2011;doi:10.1108/14777281111159375.
- Hattie J. Influences on student learning. University of Auckland, New Zealand: Inaugural professorial lecture; 2011.
- ISCP Evaluation Report. ISCP Management Group. 2012. Available at: www.iscp.ac.uk.
- Memon MA, et al. Acad Med. 2010; doi:10.1097/ACM.0b013e3181d74bad.
- The Royal College of Surgeons of England. Surgery and the European Working Time Directive: Background Briefing; 2010.
- Van de Ridder JMM, et al. Med Educ. 2015; doi: 10.1111/medu.12744.
- Van de Wiel M, et al. Adv Health Sci Educ Theory Pract. 2011; doi:10.1007/s10459-010-9246-3.
- Van der Vleuten C. Adv Health Sci Educ Theory Pract. 1996; doi:10.1007/BF00596229.
- For more information:
- Danny Ryan, MA, MB BChir (Cantab), PGCME, FRSA, MRCSEng, MFSTEd, MacadMEd, can be reached at 6 The Hendre, Overton Park Road, Cheltenham, Gloucestershire, United Kingdom GL50 3BW; email: danny.ryan@cantab.net.
Disclosure: Ryan reports no relevant financial disclosures.