Resident selection in surgery
Kristine Hagelsteen, MD PhD
Consultant Pediatric Surgeon
Director of Postgraduate Medical Education
Skåne University Hospital
Department of Clinical Sciences
Associate Professor of Organisational and Labour Sociology
Department of Sociology
11 July 2023
There is a global interest in and growing knowledge about how best to select residents in surgery. In 2010, Paice et al claimed that selection was “the missing link in patient safety work” 1. The starting point for collegial discussions on the matter and opinions about selection are things we know and have experience of; and we know that there is a spectrum running from excellent to unsuitable surgeons. Some continue to ask, are surgeons “born or made”, i.e., is it innate talent, or their training that matters? Is competence-based education better than the apprenticeship model? 2,3. Further, it is well-established that all facets of competence matter for patient outcomes, not only technical skill 4-6. The detrimental effect a less competent or even dysfunctional colleague can have on patients and the workplace environment is also widely recognised 7. Attrition from surgical residency programs has been reported to be up to 20%, reflecting bad investments of resources 8. Another challenge is that surgery accounts for most adverse events in health care, and that up to 15 % of patients in elective surgery suffer a treatment-related injury 9,10. We all want the next generation to be better than us. Selection is thus a foundational factor in raising the standard level of treatment and care, reducing adverse events, improving workplace collaboration and heightening the reputation of the profession.
Some characteristics have been found important for surgeons to be successful, such as having a strong academic background and commitment to lifelong learning; possessing sufficient manual dexterity and fine motor skills to reach proficiency; displaying emotional stability; extroversion and conscientiousness; good communication skills and ability to work in teams; critical thinking ability; situational awareness; robust decision-making and problem-solving skills 11-14. A more recent addition to the list is “technical orientation”, i.e. willingness and ability to work in a technology-based environment 15.
How can successful selection be defined and measured?
A common surrogate measure for defining success in residency selection is the percentage of residents that complete the training program, but efforts to include more detailed outcome assessments exists 8,16 . To simplify, the goal of selection can be thought of as a procedure to a) identify those with potential to become competent surgeons within a reasonable time, with robustness to face the profession’s increasing complexity; and b) identify those who will potentially become a “struggling resident or doctor”, or exhibit “early warning signs” of tendencies towards various forms of unsuitability 17,18. Though these seem like mutually exclusive groups, some candidates may exhibit characteristics, accomplishments and qualifications apparently putting them into, and even at the top of the first population, while also having tendencies that may be harder to detect or actively hidden, that place them in the latter. This is why selection processes need to focus not just on excellence, but also unsuitability simultaneously.
Most candidates will fall in the first group, but arguably, it might be more important to identify those in the latter due to long-term detrimental effects on patients and work environment if recruited 17,18. Behavior to screen for and keep under observation that are worrisome in a potential surgical resident include indecisiveness, timidity, lack of self-awareness, dishonesty, and insufficient receptivity for feedback. Further, it has been shown that deficient communication skills, empathy, mental health, stress coping, professionalism, interpersonal skills, commitment to documentation, and cognitive abilities such as problem solving and clinical reasoning will make a surgical resident less likely to succeed 17-19.
Selection is a first step towards future job performance, but it is impossible to eliminate uncertainty completely 20,21. Several selection tests and assessment tools have been developed, including ratings of multiple, specifically-defined competencies, skills, behaviors, personality factors, and personal characteristics. These tests and assessments can help in the selection but are never the complete answer 15,22,23.
Selection of surgical residents in practice
Most research on surgical selection has been performed in, and thus reflects the practice of, Anglo-Saxon countries 24. The variation of selection procedures between countries is striking 25. Selection can be centralized or regionalized and standardized with a strong emphasis on medical school performance or exam scores, without taking personal characteristics into account. In other settings, the selection process may be decentralized down to the level of the local hospital, with emphasis on job interviews and work trial periods. Noteworthy is that according to the Best Evidence Medical Education (BEME) guide no. 45, test scores and grades are pointed out as insufficient in selecting applicants for a complex and multidimensional role like residency training 26. Likewise, traditional CVs and letters of recommendation inconsistently predict performance and should not be the only basis for a selection decision 23,26. A period of work trial is valid in seeing actual performance and non-technical skills, but has the inherent risk of personal bias influencing assessments and permanent employment decisions 20.
Currently, the most valid and agreed-upon selection tool is a structured interview performed by more than one assessor 27,28. Behavior-based questions are superior to descriptive ones as they are less biased and better reflect non-cognitive skills and inform about past behavior and reasoning (see example) 23,28,29. Structured interviews furthermore afford the opportunity to evaluate and compare applicants’ ability to communicate effectively as well as probing for non-technical skills, warning signs, attitudes, and motivations 17,28,29.
Another valid selection method is Situational Judgement Testing (SJT), which is a written test with hypothetical behavior-based questions that can be used to assess candidates’ ability to choose the most appropriate approach to scenarios encountered in the workplace and justify chosen courses of action 30. This test is considered to reflect job performance and problem-solving abilities, and is capable of discriminating well between average and low performers 30. Conducting a SJT is part of the selection process in United Kingdom, among other countries, and content is developed locally and revised annually (see example) 31. A challenge with STJs is that they need to be locally developed to be relevant, demanding expertise and financial resources which may limit its feasibility in some, especially decentralized, contexts.
Despite research and the myriad of selection processes practiced worldwide, we still don’t know exactly what the spectrum of a suitable candidate accommodates, and how to measure or foresee it predictively into an unknown future. To study resident performance outcomes requires longitudinal studies with qualitative and quantitative empirical analysis of surgical resident trajectories to understand how the initial endowments and dispositions of a resident are developed or redressed in the given training program. In an ongoing longitudinal study aiming to improve the selection process in the authors’ institution, we track residents from start to finish. At baseline, we have measured such things as visuospatial ability, personality, multisource feedback and technical aptitude. We correlate this data with interviews, collegial and supervisor assessments and final tests when residents five years later are considered ready for specialist certification. We have so far found that higher scores on a visuospatial ability test correspond to better performance in a 2D laparoscopy simulator, but not in the 3D virtual reality simulator 32. The importance of this finding will thus be revealed when we compare the performance results at the end of the study, having followed the cohort of 50 residency candidates towards certification.
It is a challenge to select suitable individuals to surgical residency since personal development is complex, includes factors of uncertainty and is not a linear process easy to predict based on single characteristics. Furthermore, residents are exposed to different learning environments. Structured interviews with more than one interviewer, using behavioral questions currently seem to be the most valid tool as well as situational judgement testing, which reflects the importance of non-technical skills in surgery. The authors believe more and continuous multidisciplinary, longitudinal research, with mixed methods and a global perspective is needed to improve our understanding of how to improve selection, especially since the field of surgery is constantly evolving.
Part of the charitable activity of the Society, BJS Academy is an online educational resource for current and future surgeons.
The Academy is comprised of five distinct sections: Continuing surgical education, Young BJS, Cutting edge, Scientific surgery and Surgical news. Although the majority of this is open access, additional content is available to BJS subscribers and strategic partners.