Development and validation of a multisource feedback tool to assess medical trainee competence in patient safety

Abstract

Advances in medical sciences have led to the development of sophisticated clinical procedures and treatment modalities. Such advances are often associated with a potential for adverse events. Hence, the work environment of today’s health care provider has become significantly complex and demanding. The shift in the landscape of our health care system has called for a parallel shift in how residents are trained and assessed in the area of patient safety. Assessment is important in ensuring that future physicians are competent in the provision of safe, quality patient care. To date, there has been minimal research on the assessment of residents’ patient safety competencies. Multisource Feedback (MSF) (or 360° assessment), involves rating performances within the workplace through the use of multiple raters, with whom the ratee works, and through self-assessment. The overall purpose of this study was to develop a valid MSF tool for assessing residents’ patient safety competencies. The proposed project involved a five-phase mixed method research (MMR) study consisting of a scoping literature review, environmental scan, expert and stakeholder consultations, competency mapping and development, Delphi surveys, and a pilot test. Patient safety assessment skills were rated by experts using a Delphi survey. The tool was then piloted with health professionals and residents to determine the importance, clarity, utility, and feasibility of its use in postgraduate medical education (PGME). A 75% consensus rating threshold was applied to the Delphi and pilot data, which resulted in a 53-item supervisor, resident, and co-worker survey and a 26-item MSF Patient Safety iAssessment Tool360° (PSAT360°). The PSAT360° is regarded by experts, health professionals, and residents as a valuable and feasible tool that has broad applicability across PGME programs. This research resulted in an MSF tool that has established content validity and is grounded in principles of MSF, assessment of clinical performance, competency-based training, and required organizational practices (ROPs).