Author: Kathryn Brotchie
Brotchie, Kathryn, 2015 Developing a tool for OSCE writers and reviewers to aid the identification of station-level errors., Flinders University, School of Medicine
Terms of Use: This electronic version is (or will be) made publicly available by Flinders University in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. You may use this material for uses permitted under the Copyright Act 1968. If you are the owner of any included third party copyright material and/or you believe that any material has been made available without permission of the copyright owner please contact copyright@flinders.edu.au with the details.
The core outcome of medical education programs is competent medical practitioners. Assessment of competence throughout all the stages of a doctor’s career is necessary to ensure patient safety and effective practice. A popular approach to the assessment of clinical skills is the Objective Structured Clinical Examination (OSCE), originating in Scotland in 1975 and now globally accepted as a reliable and valid method. Despite its widespread adoption, concerns exist about the cost of conducting OSCEs, the impact on students of the high stakes examination process, authenticity issues, use of checklists and implementation errors including poorly written station items. The aim of this research is on improving the quality of the assessment of competence using the OSCE format, by aiding station developers and reviewers to identify station-level errors. The guiding research question is:
What aspects of the OSCE item writing process are prone to errors that undermine the quality of this assessment format and how can these be overcome?
This thesis provides an insight into the concept of errors in OSCE stations used for the assessment of clinical competence of medical practitioners. The use of flawed stations undermines the candidate’s opportunity to perform to the best of his or her ability, and ultimately reduces confidence in the results of that assessment. Importantly, a flawed assessment may prevent a competent doctor from becoming licenced to practice, reducing the available medical workforce and patient access to healthcare. Equally, a flawed assessment may allow incompetent doctors to practice unsupervised on patients or promote a medical student to the clinical environment when they are not yet ready to learn in that setting, compounding the learning deficit.
This project has met the aim by creating a tool to aid OSCE writers and reviewers to identify and correct errors affecting the validity, reliability, feasibility and educational impact of the assessment of clinical competence. Using a design-based research approach, a tool to aid in the quality improvement of OSCE station writing has been developed. Using a three phase iterative process, this thesis outlines the steps and decisions through the development of the OWSAT – the OSCE Writers Assessment Tool. This process has included considerable iteration, peer review at national and international conferences, and application of the tool against a database of OSCE stations.
Whilst not yet validated, the OWSAT contains questions highlighting aspects of OSCE writing that require reflection by the OSCE station writer or reviewer in search of improved station performance and overall assessment quality. It is anticipated that OWSAT will assist OSCE station writers and reviewers to identify errors at the writing or reviewing stage of the assessment process, and consequently enhance the systems that use OSCE assessment approach to determine competence in the medical profession.
Keywords: OSCE, errors, clinical skills assessment,
Subject: Medicine thesis
Thesis type: Professional Doctorate
Completed: 2015
School: School of Medicine
Supervisor: Associate Professor Linda Sweet