AAAAI Modules

Resources Relevant to the Implementation
of Core ACGME-Required Competencies

Module I
Interpersonal & communication skills
Assessment


Assessment of fellow by faculty
Checklist: SEGUE Framework		The SEGUE Framework was developed by Gregory Makoul at the Program in Communication and Medicine at the Northwestern University Medical School. Designed to be completed by either a faculty observer or a standardized patient, the SEGUE Framework focuses on specific components of communication and assesses whether or not they were present during an encounter. Development and beginning validity studies of the instrument are documented in: Makoul, G. (2001). The SEGUE Framework for teaching and assessing communication skills. Patient Education and Counseling, 45(1), 23-34. * This site requires free registration. It is extremely slow to load. You may choose to request a print copy of the article from your library.* The SEGUE Framework was chosen for the AAAAI Toolbox for a number of reasons. First, it focuses on both important elements of interpersonal and communication skill, the content and the process of the medical encounter. While this assessment tool was originally developed for use at the undergraduate medical education level, the components of communication assessed by the tool are applicable at all levels of training. For example, although the level of content expected of a fellow is different from that expected of a medical student, the inclusion of this component in the medical interview is expected of both. The elements included in the Framework were selected based on review of two literatures – education and research – and on work in two contexts. The items each describe a single task and, as a result, are clear and unambiguous. The Framework is coded dichotomously, thus simplifying the scoring and making the attainment of a high degree of inter-rater reliability more likely. Finally, information gained from use of the Framework can provide residents with feedback targeted very specifically at content and process skills. Dr. Makoul has granted permission for use of the SEGUE Framework by A/I training programs. While standardized patient-based assessment is useful, it is not very practical for A/I training programs, which are typically small in size. Observation by faculty of an actual patient encounter, especially if conducted on an unannounced basis, is likely to provide a more realistic snapshot of resident behavior than scheduled time with a standardized patient. The literature on assessment indicates that, to be generalizable, a relatively large number of observations need to be made. At this point, since the purposes of assessing interpersonal and communication skills in A/I training programs are to demonstrate compliance with the ACGME mandate and to provide formative feedback to fellows, occasional observations conducted over the course of the fellowship should suffice. The SEGUE Framework is an example of assessment via checklist. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of checklists for assessment.
		TOP

Assessment of fellow by patients
Patient survey: American Board of Internal Medicine (ABIM) [adapted]		The ABIM‘s Continuous Professional Development Program uses patient assessment as one measure of physician skill. Designed to be completed by patients who have been contacted by mail some time after the visit, the ten-item instrument focuses on the physician-patient interaction that occurred during the medical encounter. An adaptation of the ABIM instrument was chosen for the AAAAI toolbox for a number of reasons. First, the questions focus solely on aspects of the physician-patient interaction. Instruments that look at the concept of satisfaction more globally often include questions about other aspects of the medical visit, such as access to appointments, wait time, and convenience of parking. While this information is useful, a more targeted assessment of interpersonal and communication skills is needed both to assure compliance with the ACGME mandate and also to provide useful, targeted formative feedback to fellows. Ten questions are enough to provide information about important specific components of the medical interaction, but are few enough not to be onerous for the patient to complete. Two adaptations to the instrument are suggested. First, the items should be shortened so that each reflects a single behavior. Research has shown that when multiple behaviors are assessed in one question, neither those doing the assessment nor those being assessed can be sure which components are being measured. The University of Pennsylvania Health System, in conjunction with Independence Blue Cross, created truncated single items from the ABIM instrument for reporting purposes. This adapted version of the items is suggested for use by A/I training programs. The second recommendation concerns how the instrument is administered. Since response to mailed surveys is low and may tend to reflect the evaluation of patients who are either very happy or very unhappy with their care, we suggest that the questions be handed out to patients by clinic staff at the end of the visit and completed on the spot. Having all of a fellow’s patients complete the forms during some period of time—one week every six months, for example— is likely to provide more complete and useful information. Care should be taken to ensure that the time period used is representative. The ABIM has granted permission for use of its core competencies by A/I training programs. The adapted ABIM form is an example of assessment via patient survey. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of patient surveys for assessment.
		TOP

Multi-purpose assessment of interpersonal & communication skills
Portfolio assessment		Portfolios provide a way to collect evaluative information about several aspects of interpersonal and communication skills. Aspects that can be assessed in this manner include communication with other health care professionals, issues of cultural competence, use of educational tools, and shared decision making, to name a few. Portfolios are collections of materials prepared by the learner that show evidence of learning or achievement relative to a plan. While their use in graduate medical education is relatively new, portfolios have the potential to be useful in many aspects of competency assurance. Important components of effective portfolio use include creation of a learning plan with specific goals and objectives, collection of materials showing evidence of achievement, reflection by the learner, and assessment of the materials by the learner and by faculty. The ACGME, in its draft Toolbox of Assessment Methods, provides some information about the properties and uses of portfolios for assessment.
		TOP

		Return to Table of Contents
		Return to Assessment Overview
		Return to Interpersonal & communication skills Overview
		Move forward to Interpersonal & communication skills Resources

Home