Guide to Evaluation Forms for Resident Clinical Performance Assessment
Susan J. Lieff M.D., M.Ed., FRCPC
Evaluation forms can be used for either summative or formative purposes. The goal of formative assessments is to give the trainee specific feedback that facilitates reflection on performance, identification of areas in need of development and decision-making on strategies to improve. They are used strictly to facilitate learning and are not tied to passing or failing. On the other hand, summative evaluations are used for documenting successful completion or passing such as completion of a residency core training requirement, graduation, or certification. The forms provided here can be used for either formative or summative purposes. It is important to separate the feedback/teaching function from the assessment/decision-making function. Encourage immediate feedback for formative purposes. Residents should never be asked to participate in any summative (formal evaluation) process without ample notification.
Members of the American Association for Geriatric Psychiatry Resident Evaluation Task Force were asked to create resident performance assessment form templates that incorporate the Accreditation Council for Graduate Medical Education (ACGME) core competency constructs and categories. It became apparent to task force members, that the creation of a single, ideal form, applicable to the clinical supervisory experience, and useful for both formative and summative purposes, that also meets the requirements of ACGME was unrealistic. Different competency constructs require different methods. Additionally, the ACGME-required midterm and final year summative evaluations are not consistent with providing the immediate and detailed formative feedback necessary to facilitate learning in the clinical supervisory context. Hence, a number of global assessment forms were created that can be used as a starting point for this new evaluative process. It is anticipated that as programs become increasingly familiar with the core competencies, ACGME will require more objective methods than a global assessment alone, as it is the least reliable and valid method of assessment. Nonetheless, currently, there is demand for such forms. To enhance the validity and reliability of the information gathered with these forms a number of augmentation strategies are offered. 360-degree evaluations and patient surveys have also been included to cover some of the newer competency categories such as interpersonal and communication skills and professionalism.
At the start
At the outset of the rotation, trainees need to be informed of the criteria, methods and frequency of evaluations. They also need to know who will be doing evaluations for both formative and summative purposes. They should be provided with copies of all forms, and the minimum required to achieve competence or pass the rotation should be clearly articulated. When Likeart scales are used it is important to define the cutoff for achievement of competence. For example, is a 3 average for their level of training or is a 5 required to pass?
Evaluation Templates Included:
- The Geriatric Fellow Competency Assessment Form
- Interview Evaluation Form
- Evaluation of Geriatric Psychiatry Fellow Performance Form
- 360-degree Evaluation Form
- Patient and Family Memberís Evaluation of Geriatric Psychiatry Fellow
The Geriatric Fellow Competency Assessment Form
The Geriatric Psychiatry Fellow Competency Assessment Form uses a type of global assessment with a bottom up approach, looking at the clinical and teaching setting and integrating the core competencies into that framework. It does not comprehensively cover all aspects of the core competencies but certainly most. It should be supplemented by documenting "mini-evaluations" that have taken place over the course of a rotation. "Mini-evaluations" such as chart stimulated recalls, oral exams, record reviews, etc. provide part of the "evidence" for how a traineeís scores were determined. These structured approaches to the evaluation of clinical performance can be documented at the time of performance and measured relative to a standard.
Interview Evaluation Form
The Geriatric Psychiatry Fellow Competency Assessment Form assessment does not provide the detailed feedback necessary for the evaluation of interviewing skills. Since interviewing skills often require their own assessment template a sample of an interviewing skills evaluation is provided for consideration and adaptation.
Evaluation of Geriatric Psychiatry Fellow Performance Form
The Evaluation of Geriatric Psychiatry Fellow Performance Form, another type of global assessment, is a top down approach that has core competencies spelled out with anchor points. This evaluation should also be supplemented with multiple data points such as observed interviews, record reviews, chart stimulated recall, etc. Faculty may prefer to use one scale for the quarterly and another for the end of year or, alternatively, one for formative and the other for summative assessments.
To provide reliability and validity, the 360-degree evaluations need to be completed by a minimum of 7 to 10 allied health professionals, peers, or colleagues. An even greater number of evaluations must be completed if patients and faculty members are the reviewers. It is recommended that a structured process that does not directly involve the trainee should be used for distribution and collection of these surveys.
Patient and Family Memberís Evaluation of Geriatric Psychiatry Fellow
The form included here can be distributed to both patients and family members and used in conjunction with the 360-degree evaluation.
Enhance Reliability and Validity
- Use multiple observers and employ multiple observations in different clinical situations to enhance reproducibility. Clinical competence is highly case and situation specific and is therefore difficult to generalize.
- Implement a strategy for systematic sampling of cases, situations, diagnoses and tasks and supplement this with observation of performance in standardized clinical cases to provide control over the case mix.
- Minimize delays between observation and documentation, avoiding the typical midterm and endpoint evaluations by encouraging raters to make performance notes at the time of observation using notebooks, instant event report cards, oral exams, chart stimulated recall, etc.
The templates included here can serve as a starting point to assist program directors and faculty in the development of their own competency evaluation forms for trainees. As the forms are used, consider two questions for future revisions. Are the items as a group truly representative of and sufficient to capture the constructs within the categories of core competencies? Do these forms accurately and reproducibly measure resident performance within the core domains?
- ACGME Toolbox of Assessment Methods. ACGME Outcome project. http://www.acgme.org/outcome/assess/toolbox.aspacgme
- Williams RG. Klamen DA. McGaghie WC. Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings. Teaching & Learning in Medicine 2003. 15(4):270-92
- Yudkowsky R. Lieff SJ. Master Educator Workshop I: Assessment. Association for Academic Psychiatry Annual Meeting. 0ct 3, 2003. Philadelphia, Pennsylvania