AAAAI Modules

Resources Relevant to the Implementation
of Core ACGME-Required Competencies

Module II
Practice-based learning and improvement
Assessment


Assessment of fellow by faculty
Rating scale American Board of Internal Medicine (ABIM) [adapted]		In its Portfolio for Internal Medicine Residency Programs: Evaluating your residents: New competencies, new forms, the ABIM suggests a number of questions that may be used to assess the competency of practice-based learning and improvement. The ABIM suggests that programs determine the appropriateness of question use. An adaptation of the ABIM questions was chosen for the AAAAI toolbox for a number of reasons. First, the questions focus solely on aspects of practice-based learning and improvement. This targeted assessment is needed both to assure compliance with the ACGME mandate and also to provide useful, targeted formative feedback to fellows. Nine questions, representing four components of practice-based learning and improvement, were selected for use. Although the ABIM recommends use of a 9-point scale ranging from unsatisfactory to superior, there is considerable evidence that programs have difficulty using this scale in a standard way. The adapted version of the items suggested for use by A/I training programs uses a 5-point frequency scale with descriptors for each point. The ABIM has granted permission for use of its core competencies by A/I training programs. The adapted ABIM form is an example of a modification of assessment via global rating. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of global rating for assessment.
		TOP

Assessment of fellow by learners
Rating scale Clinical Teaching Effectiveness Instrument		The Clinical Teaching Effectiveness Instrument was developed at the Cleveland Clinic. This tool focuses on assessing specific dimensions of effective teaching identified from the literature and from qualitative data obtained from interviews with stakeholders. Development and validity studies of the instrument are documented in: Copeland, H.L., & Hewson, M. G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2): 161-66. The Clinical Teaching Effectiveness Instrument was chosen for the AAAAI Toolbox for a number of reasons. The tool is based on theoretical constructs that have been identified as being important to clinical teaching. Moreover, the utility of these constructs has been validated by groups of stakeholders. Additionally, the tool was designed to be a useful way of providing clinical teachers with feedback. Since a goal of feedback is improvement, use of the tool itself becomes a means of demonstrating competency in practice-based learning and improvement. The psychometric qualities of the tool have been rigorously tested. Scores obtained have been found to be reliable and the instrument has demonstrated content and criterion-related validity. The literature on assessment indicates that, to be generalizable, a number of observations need to be made. During the development and testing of the Clinical Teaching Effectiveness Instrument, an average of eight assessments were completed for each clinical teacher. The Clinical Teaching Effectiveness Instrument is an example of assessment via rating scale. The ACGME, in its draft Toolbox of Assessment Methods , provides some information about the properties and uses of rating scales for assessment.
		TOP

Multi-purpose assessment of practice-based learning & improvement
Portfolio assessment		Portfolios provide a way to collect evaluative information about several aspects of practice-based learning and improvement. Aspects that can be assessed in this manner include analysis of one’s own practice, participation in improvement projects, and the triad of using information technology to identify evidence-based resources, applying research and statistical methods to appraise the evidence, and using evidence, to name a few. Portfolios are collections of materials prepared by the learner that show evidence of learning or achievement relative to a plan. While their use in graduate medical education is relatively new, portfolios have the potential to be useful in many aspects of competency assurance. Important components of effective portfolio use include creation of a learning plan with specific goals and objectives, collection of materials showing evidence of achievement, reflection by the learner, and assessment of the materials by the learner and by faculty. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of portfolios for assessment.
		TOP

		Return to Table of Contents
		Return to Assessment Overview
		Return to Practice-based learning & improvement Overview
		Move forward to Practice-based learning & improvement Resources

Home