Author: Wajid Hussain
Hussain, Wajid, 2021 Implementing digital integrated quality management systems for achieving quality and accreditation of engineering programs, Flinders University, College of Education, Psychology and Social Work
Terms of Use: This electronic version is (or will be) made publicly available by Flinders University in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. You may use this material for uses permitted under the Copyright Act 1968. If you are the owner of any included third party copyright material and/or you believe that any material has been made available without permission of the copyright owner please contact copyright@flinders.edu.au with the details.
Accreditation being the prime driver of outcomes assessment in most engineering campuses across the US and internationally, means that several engineering programs have adopted a macro approach to applying outcomes to student learning. Manual systems have been employed to complete a very cumbersome task of collating the assessment of student outcomes information and utilize the data for Continuous Quality Improvement (CQI), a crucial requirement for accreditation. Any typical undergraduate engineering program involves learning several hundred skills by the time of graduation. Authentic outcome-based CQI systems should assess all required skills for all students. Manual CQI systems have forced many programs worldwide to adopt short cut approaches and opt for simplistic learning models, selective sampling of students, courses, skills, and primitive methods of their evaluation to avoid massive expenditure of resources and the daunting task of manual management of data collection and reporting activities. Unfortunately, the focus of assessment and evaluation has shifted from achieving authentic outcome-based standards and student centered CQI to models that target minimal accreditation standards. Several practical issues related with manual CQI systems have been mentioned in literature and presented systematically in this research to emphasize their effect on improvement of student learning and propose practical solutions. Engineering programs are now faced with the challenge on how to implement the OBE paradigm, philosophy and principles using manual quality processes. The question is whether digital technology can be used to embed and collate assessment outcomes to help programs implement ‘authentic’ OBE that has continuous feedback on progress and therefore achieves realistic CQI.
In this thesis, I present a comprehensive methodology based on authentic OBE frameworks for implementing digital Integrated Quality Management Systems (IQMS) for the Islamic University civil, electrical and mechanical engineering programs. Six comprehensive PDCA quality cycles were implemented to ensure Total Quality Management of the education process. The Faculty Course Assessment Report and Performance Vector Table methodology were employed to implement embedded assessment technology. Specific performance indicators and topic specific hybrid rubrics were used to tightly align assessment with actual student learning activity. A customized web-based software EvalTools® that integrates Learning Management (LMS), Outcomes Assessment (OAS) and Continuous Improvement Management (CIMS) Systems streamlined the data collection and reporting activity for accreditation requirements. An Advising Module enabled effective developmental advising based on outcomes data collected for every individual student. A Remote Evaluator Module enabled virtual accreditation audits facilitating social-distancing during the COVID19 pandemic. A comprehensive eight phase meta-framework was employed to conduct Mixed Methods Theory Based Impact Evaluations of the IQMS providing detailed guidelines to both accreditors and engineering programs for conducting credible remote evaluations. Program evaluation using this novel meta-framework would help consider a range of aspects such as context, construct, causal links, processes, technology, data collection and outcomes results of CQI activity required for credible remote audits of automated digital quality systems. The IQMS was seamlessly operated for a full six-year period with more than a million documents systematically reported as objective evidence on a cloud based environment and followed with a successful ABET accreditation.
Keywords: digital quality systems, ABET, accreditation, outcomes, OBE, assessment, program evaluation, remote audits
Subject: Education thesis
Thesis type: Doctor of Philosophy
Completed: 2021
School: College of Education, Psychology and Social Work
Supervisor: Prof. Lindsey Conner