BJS-2025-0288
cse058
https://doi.org/10.58974/bjss/azbc024
Continuing surgical education
General
Digests
Resident selection in surgery
Kristine Hagelsteen, MD PhD
Consultant Pediatric Surgeon Director of Postgraduate Medical Education Skåne University Hospital Department of Clinical Sciences Lund University Sweden
Chris Mathieu
Associate Professor of Organisational and Labour Sociology Department of Sociology Lund University Sweden
11072023
There is a global interest in and growing knowledge about how best to select residents in surgery. In 2010, Paice et al claimed that selection was “the missing link in patient safety work”1. The starting point for collegial discussions on the matter and opinions about selection are things we know and have experience of; and we know that there is a spectrum running from excellent to unsuitable surgeons. Some continue to ask, are surgeons “born or made”, i.e., is it innate talent, or their training that matters? Is competence-based education better than the apprenticeship model?2,3. Further, it is well-established that all facets of competence matter for patient outcomes, not only technical skill4-6. The detrimental effect a less competent or even dysfunctional colleague can have on patients and the workplace environment is also widely recognised7. Attrition from surgical residency programs has been reported to be up to 20%, reflecting bad investments of resources8. Another challenge is that surgery accounts for most adverse events in health care, and that up to 15 % of patients in elective surgery suffer a treatment-related injury9,10. We all want the next generation to be better than us. Selection is thus a foundational factor in raising the standard level of treatment and care, reducing adverse events, improving workplace collaboration and heightening the reputation of the profession.
Some characteristics have been found important for surgeons to be successful, such as having a strong academic background and commitment to lifelong learning; possessing sufficient manual dexterity and fine motor skills to reach proficiency; displaying emotional stability; extroversion and conscientiousness; good communication skills and ability to work in teams; critical thinking ability; situational awareness; robust decision-making and problem-solving skills11-14. A more recent addition to the list is “technical orientation”, i.e. willingness and ability to work in a technology-based environment15.
How can successful selection be defined and measured?
A common surrogate measure for defining success in residency selection is the percentage of residents that complete the training program, but efforts to include more detailed outcome assessments exists8,16 . To simplify, the goal of selection can be thought of as a procedure to a) identify those with potential to become competent surgeons within a reasonable time, with robustness to face the profession’s increasing complexity; and b) identify those who will potentially become a “struggling resident or doctor”, or exhibit “early warning signs” of tendencies towards various forms of unsuitability17,18. Though these seem like mutually exclusive groups, some candidates may exhibit characteristics, accomplishments and qualifications apparently putting them into, and even at the top of the first population, while also having tendencies that may be harder to detect or actively hidden, that place them in the latter. This is why selection processes need to focus not just on excellence, but also unsuitability simultaneously.
Most candidates will fall in the first group, but arguably, it might be more important to identify those in the latter due to long-term detrimental effects on patients and work environment if recruited17,18. Behavior to screen for and keep under observation that are worrisome in a potential surgical resident include indecisiveness, timidity, lack of self-awareness, dishonesty, and insufficient receptivity for feedback. Further, it has been shown that deficient communication skills, empathy, mental health, stress coping, professionalism, interpersonal skills, commitment to documentation, and cognitive abilities such as problem solving and clinical reasoning will make a surgical resident less likely to succeed17-19.
Selection is a first step towards future job performance, but it is impossible to eliminate uncertainty completely20,21. Several selection tests and assessment tools have been developed, including ratings of multiple, specifically-defined competencies, skills, behaviors, personality factors, and personal characteristics. These tests and assessments can help in the selection but are never the complete answer15,22,23.
Selection of surgical residents in practice
Most research on surgical selection has been performed in, and thus reflects the practice of, Anglo-Saxon countries24. The variation of selection procedures between countries is striking25. Selection can be centralized or regionalized and standardized with a strong emphasis on medical school performance or exam scores, without taking personal characteristics into account. In other settings, the selection process may be decentralized down to the level of the local hospital, with emphasis on job interviews and work trial periods. Noteworthy is that according to the Best Evidence Medical Education (BEME) guide no. 45, test scores and grades are pointed out as insufficient in selecting applicants for a complex and multidimensional role like residency training26. Likewise, traditional CVs and letters of recommendation inconsistently predict performance and should not be the only basis for a selection decision23,26. A period of work trial is valid in seeing actual performance and non-technical skills, but has the inherent risk of personal bias influencing assessments and permanent employment decisions20.
Currently, the most valid and agreed-upon selection tool is a structured interview performed by more than one assessor27,28. Behavior-based questions are superior to descriptive ones as they are less biased and better reflect non-cognitive skills and inform about past behavior and reasoning (see example)23,28,29. Structured interviews furthermore afford the opportunity to evaluate and compare applicants’ ability to communicate effectively as well as probing for non-technical skills, warning signs, attitudes, and motivations17,28,29.
Another valid selection method is Situational Judgement Testing (SJT), which is a written test with hypothetical behavior-based questions that can be used to assess candidates’ ability to choose the most appropriate approach to scenarios encountered in the workplace and justify chosen courses of action30. This test is considered to reflect job performance and problem-solving abilities, and is capable of discriminating well between average and low performers30. Conducting a SJT is part of the selection process in United Kingdom, among other countries, and content is developed locally and revised annually (see example)31. A challenge with STJs is that they need to be locally developed to be relevant, demanding expertise and financial resources which may limit its feasibility in some, especially decentralized, contexts.
Text box SJT and behavioral questions
Despite research and the myriad of selection processes practiced worldwide, we still don’t know exactly what the spectrum of a suitable candidate accommodates, and how to measure or foresee it predictively into an unknown future. To study resident performance outcomes requires longitudinal studies with qualitative and quantitative empirical analysis of surgical resident trajectories to understand how the initial endowments and dispositions of a resident are developed or redressed in the given training program. In an ongoing longitudinal study aiming to improve the selection process in the authors’ institution, we track residents from start to finish. At baseline, we have measured such things as visuospatial ability, personality, multisource feedback and technical aptitude. We correlate this data with interviews, collegial and supervisor assessments and final tests when residents five years later are considered ready for specialist certification. We have so far found that higher scores on a visuospatial ability test correspond to better performance in a 2D laparoscopy simulator, but not in the 3D virtual reality simulator32. The importance of this finding will thus be revealed when we compare the performance results at the end of the study, having followed the cohort of 50 residency candidates towards certification.
Conclusion
It is a challenge to select suitable individuals to surgical residency since personal development is complex, includes factors of uncertainty and is not a linear process easy to predict based on single characteristics. Furthermore, residents are exposed to different learning environments. Structured interviews with more than one interviewer, using behavioral questions currently seem to be the most valid tool as well as situational judgement testing, which reflects the importance of non-technical skills in surgery. The authors believe more and continuous multidisciplinary, longitudinal research, with mixed methods and a global perspective is needed to improve our understanding of how to improve selection, especially since the field of surgery is constantly evolving.
References
1
Paice AG, Aggarwal R, Darzi A. Safety in surgery: is selection the missing link? World journal of surgery 2010;34:1993-2000.
2
Louridas M, Szasz P, Fecso AB, et al. Practice does not always make perfect: need for selection curricula in modern surgical training. Surgical endoscopy 2017;31:3718-27.
3
Rodriguez-Paz JM, Kennedy M, Salas E, et al. Beyond "see one, do one, teach one": toward a different training paradigm. Postgraduate medical journal 2009;85:244-9.
4
Suliburk JW, Buck QM, Pirko CJ, et al. Analysis of Human Performance Deficiencies Associated With Surgical Adverse Events. JAMA Netw Open 2019;2:e198067.
5
Shubeck SP, Kanters AE, Dimick JB. Surgeon leadership style and risk-adjusted patient outcomes. Surgical endoscopy 2019;33:471-4.
6
Fecso AB, Szasz P, Kerezov G, Grantcharov TP. The Effect of Technical Performance on Patient Outcomes in Surgery: A Systematic Review. Annals of surgery 2017;265:492-501.
7
Roberts NK, Williams RG. The hidden costs of failing to fail residents. J Grad Med Educ 2011;3:127-9.
8
Khoushhal Z, Hussain MA, Greco E, et al. Prevalence and Causes of Attrition Among Surgical Residents: A Systematic Review and Meta-analysis. JAMA Surgery 2017;152:265-72.
9
Nilsson L, Risberg MB, Montgomery A, Sjodahl R, Schildmeijer K, Rutberg H. Preventable Adverse Events in Surgical Care in Sweden: A Nationwide Review of Patient Notes. Medicine (Baltimore) 2016;95:e3047.
10
Anderson O, Davis R, Hanna GB, Vincent CA. Surgical adverse events: a systematic review. American journal of surgery 2013;206:253-62.
11
Royal Australasian College of Surgeons R. Surgical Competence and Performance. A guide to aid the assessment and development of surgeons. 2020 February 2020.
12
Arora S, Sevdalis N, Suliman I, Athanasiou T, Kneebone R, Darzi A. What makes a competent surgeon?: Experts' and trainees' perceptions of the roles of a surgeon. The American Journal of Surgery 2009;198:726-32.
13
Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World journal of surgery 2008;32:548-56.
14
Vaughan LA, Quick JA. Evidence-Based Selection of Surgical Residents. The Surgical clinics of North America 2021;101:667-77.
15
Gazit N, Ben-Gal G, Eliashar R. Using Job Analysis for Identifying the Desired Competencies of 21st-Century Surgeons for Improving Trainees Selection. Journal of surgical education 2022.
16
Torbeck L, Canal DF, Choi J. Is Our Residency Program Successful? Structuring an Outcomes Assessment System as a Component of Program Evaluation. Journal of surgical education 2014;71:73-8.
17
Hagelsteen K, Johansson BM, Bergenfelz A, Mathieu C. Identification of Warning Signs During Selection of Surgical Trainees. Journal of surgical education 2019;76:684-93.
18
Taira T, Santen SA, Roberts NK. Defining the "Problem Resident" and the Implications of the Unfixable Problem: The Rationale for a "Front-door" Solution. West J Emerg Med 2019;20:43-9.
19
Guerrasio J, Garrity MJ, Aagaard EM. Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006-2012. Academic medicine : journal of the Association of American Medical Colleges 2014;89:352-8.
20
Schmidt F HJ. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin 1988;Vol 124(2):pp. 262-74.
21
Highhouse S. Stubborn Reliance on Intuition and Subjectivity in Employee Selection. Ind Organ Psychol-Us 2008;1:333-42.
22
Louridas M, Szasz P, de Montbrun S, Harris KA, Grantcharov TP. Can We Predict Technical Aptitude?: A Systematic Review. Annals of surgery 2016;263:673-91.
23
Schaverien MV. Selection for Surgical Training: An Evidence-Based Review. Journal of surgical education 2016;73:721-9.
24
Patterson F, Roberts C, Hanson MD, et al. 2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions. Medical Teacher 2018;40:1091-101.
25
Hagelsteen K, Pedersen H, Bergenfelz A, Mathieu C. Different approaches to selection of surgical trainees in the European Union. BMC medical education 2021;21:363.
26
Roberts C, Khanna P, Rigby L, et al. Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45. Med Teach 2018;40:3-19.
27
Patterson F, Rowett E, Hale R, et al. The predictive validity of a situational judgement test and multiple-mini interview for entry into postgraduate training in Australia. BMC medical education 2016;16:87.
28
Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the Interview in Resident Candidate Selection: A Review of the Literature. J Grad Med Educ 2015;7:539-48.
29
Strand EA, Moore E, Laube DW. Can a structured, behavior-based interview predict future resident success? American journal of obstetrics and gynecology 2011;204:446 e1- e13.
30
Patterson F, Zibarras L, Ashworth V. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Med Teach 2016;38:3-17.
31
Situational Judgement Test (SJT). NHS England: Health Education England, 2023. (Accessed 2023-04-24, at https://medical.hee.nhs.uk/medical-training-recruitment/medical-specialty-training/public-health/public-health-assessment-centre/situational-judgement-paper.)
32
Pedersen H, Stahl D, Ekelund M, et al. Visuospatial ability is associated to 2D laparoscopic simulator performance amongst surgical residents. Surg Open Sci 2023;11:56-61.
.png)
.jpg){kind=logo}
