An adaptation of the ABIM questions was chosen for the AAAAI toolbox for a number of reasons. First, the questions focus clearly on aspects of systems-based practice. This targeted assessment is needed both to assure compliance with the ACGME mandate and also to provide useful, targeted feedback to fellows. Seven questions, representing three components of practice-based learning and improvement, were selected for use.

Although the ABIM recommends use of a 9-point scale ranging from unsatisfactory to superior, there is considerable evidence that programs have difficulty using this scale in a standard way. The adapted version of the items suggested for use by A/I training programs uses a 5-point frequency scale with descriptors for each point.

The ABIM has granted permission for use of its core competencies by A/I training programs.

It should be noted that while these questions provide a summary assessment, the information needed to answer the questions must be obtained from a variety of other types of observation and assessment. For example, to assess the degree to which a fellow begins discharge planning at admission, one would rely on evaluation of case presentations and chart reviews. Another example would be to assess the degree to which a fellow utilizes hospital resources effectively by relying on information obtained through chart-stimulated recall.

The adapted ABIM form is an example of a modification of assessment via global rating. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of global rating for assessment.

Portfolios provide a way to collect evaluative information about several aspects of systems-based practice. Aspects that can be assessed in this manner knowledge of delivery systems and patient advocacy, for example.

Portfolios are collections of materials prepared by the learner that show evidence of learning or achievement relative to a plan. While their use in graduate medical education is relatively new, portfolios have the potential to be useful in many aspects of competency assurance. Important components of effective portfolio use include creation of a learning plan with specific goals and objectives, collection of materials showing evidence of achievement, reflection by the learner, and assessment of the materials by the learner and by faculty.

The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of portfolios for assessment.