Beyond Selection: The Use of Situational Judgement Tests in the Teaching and Assessment of Professionalism.

Abstract

Professionalism is a critical attribute of medical graduates. Its measurement is challenging. The authors sought to assess final-year medical students' knowledge of appropriate professional behavior across a broad range of workplace situations. Situational judgement tests (SJTs) are used widely in applicant selection to assess judgement or decision making in work-related settings as well as attributes such as empathy, integrity, and resilience. In 2014, the authors developed three 40-item SJTs with scenarios relevant to interns (first-year junior doctors) and delivered the tests to final-year medical students to assess aspects of professionalism. As preparation, students discussed SJT-style scenarios; after the tests they completed an evaluation. The authors applied the Angoff method for the standard-setting process, delivered electronic individualized feedback reports to students post test, and provided remediation for students failing to meet the cut score. Evaluation revealed that the tests positively affected students' learning and that students accepted them as an assessment tool. Validity and reliability were acceptable. Implementation costs were initially high but will be recouped over time. Recent improvements include changes to pass requirements, question revision based on reliability testing, and provision of detailed item-level feedback. Work is currently under way to expand the item bank and to introduce tests earlier in the course. Future research will explore correlation of SJT performance with other measures of professionalism and focus on the impact of SJTs on professionalism and interns' ability to deal with challenging workplace situations.