Author: Scott Sypek
Sypek, Scott, 2019 Selection into Vocational Medical Training, Flinders University, College of Medicine and Public Health
Terms of Use: This electronic version is (or will be) made publicly available by Flinders University in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. You may use this material for uses permitted under the Copyright Act 1968. If you are the owner of any included third party copyright material and/or you believe that any material has been made available without permission of the copyright owner please contact copyright@flinders.edu.au with the details.
Selection into vocational medical training is high stakes and competitive. A process is needed to select successful trainees from a large pool of applicants. Through a literature review and mixed methods analysis of a local case study, this study sought to identify the factors that make an effective selection process for vocational medical training.
Ultimately an effective process will select trainees that are successful in training and in becoming competent specialists. However, in this context, there are few meaningful measures of trainee performance, most trainees will eventually complete training and there is a low attrition rate which makes predictive validity studies difficult. Instead, other indices are used as proxies for the effectiveness and quality of selection tools and processes. These include reliability, various types of validity, acceptability and feasibility. No one tool is able to perform well across all these areas. In fact, beyond the type of tool, there are several factors that determine a tool’s utility in selection. These include the constructs measured by the tool and how these relate to the purpose of selection, the content, the format, the scoring system applied, and the number and training of assessors. Typically studies in this area are case reports of selection processes that focus on optimising the psychometric properties of selection tools. A gap in the literature is that there is no accepted standard theoretical or conceptual framework to guide selection process design and implementation.
The methods used to combine tool data and make selection decisions contribute to determining whether a selection process is effective. Many case reports in the literature and the local case study use a reductionist approach to decision-making. Information collected from the different selection tools is converted to a numerical score, which is summed to develop a rank list of applicants. This approach allows applicants to compensate for poor performance in one tool with better performance in another and therefore the weighting applied to each selection tool score has a significant influence on selection outcome. The quantification of qualitative data collected means that in this reductionist algorithm, valuable information about each applicant is lost when making final selection decisions. Constructing a selection process that considers all of these factors is complex and frameworks for designing and implementing selection processes are needed.
Assessment in medical education has faced many of the same challenges seen in selection. A Programmatic Assessment framework has been proposed to aid assessment practices. This framework involves the multi-method systematic collection of data about a learner, the careful selection of tools mapped to curriculum outcomes, and procedures for collating information collected about learners. The local case study is viewed through the lens of Programmatic Assessment. Utility of Programmatic Assessment principles to design a selection process provides a means to map domains to selection tools, combine information gathered on each applicant and facilitate decision making processes.
Keywords: Medical Education, Programmatic Assessment, Selection, Training, Medicine
Subject: Medicine thesis
Thesis type: Masters
Completed: 2019
School: College of Medicine and Public Health
Supervisor: Julie Ash