Author: Nyoli Valentine
Valentine, Nyoli, 2024 What makes assessment fair?, Flinders University, College of Medicine and Public Health
Terms of Use: This electronic version is (or will be) made publicly available by Flinders University in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. You may use this material for uses permitted under the Copyright Act 1968. If you are the owner of any included third party copyright material and/or you believe that any material has been made available without permission of the copyright owner please contact copyright@flinders.edu.au with the details.
Introduction
Traditionally, ‘objectivity’ has been seen as the only approach to fairness. Equating fairness to objectivity may be intuitive, however this can force a quantification of quality which can lead to a neglect of unquantifiable qualities, can hinder learning through the reduction of rich information to a numerical score, and in reality is very challenging to achieve. It also reduces a complex, multi-dimensional and contextual construct to a single linear, non-representative rule with limited fitness for purpose. An ontological shift, looking beyond objectivity, is needed to better understand what makes assessment fair.
Methods
I took a social constructivist stance, assuming fairness as a reality is socially constructed by multiple stakeholders, and that individuals and social groups share interpretations and understandings of fairness. Collecting data from multiple perspectives provided a richer and more nuanced understanding.
A hermeneutic literature review was undertaken for a scholarly knowledge synthesis of the definitions, factors and key questions associated with fairness in assessment. Two studies then explored how supervisors, learners and assessment leads conceptualise fair judgement. The first study used semi-structed interviews with vignettes, and the second engaged online focus groups with assessment leads from Australian and New Zealand medical schools. Initial analysis of study two revealed fair judgement is best studied as a complex adaptive system and so data analysis proceeded using a complexity lens. In a third study, online focus groups with academic leaders from the Netherlands explored how external systems’ forces on the complex adaptive system can impair fairness from emerging.
Results
In line with complexity theoretical notions, the same four elements of fairness (transparency, fitness for purpose, accountability and credibility) occurred throughout the data. These elements interacted with each other at all levels in the assessment program and behaved like a fractal. Within a complex adaptive system, a system’s behaviour relies less on the mere presence of the individual components but more on the dynamic strength and nature of the interactions between them. In line with this, people seek to create fairness through managing the interplay between fitness for purpose, credibility, transparency and accountability when interacting with others rather than using them as a tick box list.
Assessors used different strategies to influence the interactions between fairness components, including utilising narratives, aggregating evidence from multiple sources, procedural strategies, enabling a learning culture allowing for learner agency, articulating reasonable expectations of learners and ensuring a sound theoretic basis of assessment design.
Discussion
Considering fairness as a complex adaptive system changes our views about how we both approach and seek to improve assessment as well as a perspective to navigate the tensions of unpredictable real-world clinical and learning situations. In line with a complexity perspective, fairness can only emerge through the interaction of its components. This requires agile assessment, with assessors and learners who are able to adapt to different contexts using a variety of different strategies. Refraining from using strict regulations to supporting interactions and allowing learning by action is more likely to support the emergence of fairness. Viewing fairness through a lens of complexity rather than as a linear, causal model will enable better understanding of what is fair assessment and lead to more purposeful, meaningful changes which are more aligned with 21st century assessment.
Keywords: Assessment, fair, health professions education, medical education
Subject: Health Education thesis
Thesis type: Doctor of Philosophy
Completed: 2024
School: College of Medicine and Public Health
Supervisor: Lambert Schuwirth