OBJECTIVE—There is a dearth of information on the extent to which diabetic patients receive care congruent with the chronic care model (CCM) and evidence-based behavioral counseling. This study evaluates a new instrument to fill this gap.

RESEARCH DESIGN AND METHODS—A heterogeneous sample of 363 type 2 diabetic patients completed the original Patient Assessment of Chronic Illness Care (PACIC), along with additional items that allowed it to be scored according to the “5As” (ask, advise, agree, assist, and arrange) model of behavioral counseling. We evaluated relationships between survey scores and patient characteristics, quality of diabetes care, and self-management.

RESULTS—Findings replicated those of the initial PACIC validation study but with a much larger sample of diabetic patients and more Latinos. Areas of CCM activities reported least often were goal setting/intervention tailoring and follow-up/coordination. The 5As scoring revealed that patients were least likely to receive assistance with problem solving and arrangement of follow-up support. Few demographic or medical characteristics were related to PACIC or 5As scores, but survey scores were significantly related to quality of diabetes care received and level of physical activity.

CONCLUSIONS—The PACIC and the new 5As scoring method appear useful for diabetic patients. Its use is encouraged in future research and quality improvement studies.

The chronic care model (CCM) (1,2) is receiving widespread acceptance as a framework for developing and implementing evidence-based activities to improve care for chronic illnesses (3,4). The CCM appears applicable for a variety of chronic illnesses (5), including diabetes (6,7), and potentially for preventive services (8). However, there are few instruments to assess the level of CCM-congruent activities that patients receive. To inform quality improvement programs, compare different health care settings, and evaluate intervention studies, it is necessary to have practical assessment tools to evaluate the delivery of CCM activities (9).

The primary assessment procedure that has been used to date is the Assessment of Chronic Illness Care (10). This scale is completed by health care team members and appears particularly useful for helping teams identify gaps and generate innovations. It is less practical for widespread application, however, and subject to clinician overreporting, as are many clinician report instruments. Since unobtrusive observation is not feasible for large-scale application, asking patients to report the CCM-related activities that they have received seems like a valuable method of providing CCM implementation data.

Recently, Glasgow et al. (11) reported preliminary data on the Patient Assessment of Chronic Illness Care (PACIC), a 20-item survey of the extent to which patients report having received CCM-based services that they could reasonably be expected to observe. That report suggests that the PACIC has reasonable psychometric characteristics and is appropriate for a variety of chronic conditions. This original study was conducted at an integrated health maintenance organization and included patients with a variety of different illnesses, 41 of whom (16%) had diabetes. This study did not have many diabetic patients, Latino respondents, or any patients from mixed-payer medical offices, however, and the present study addresses these issues.

A similar situation exists concerning the “5As” (ask, advise, agree, assist, and arrange) model of behavior change. This framework is increasingly adopted, is evidence based (12,13), appears appropriate to guide quality improvement efforts, and applies to diabetes self-management (14,15). The 5As is a patient-centered model of behavioral counseling that is congruent with the CCM and has been frequently used to enhance self-management support and linkages to community resources, two key CCM components (5,8,14,15). There are few practical assessment tools to evaluate the extent to which the 5As are delivered and that do not rely on clinician reports. The present study addresses this issue by adding six additional items to the original PACIC instrument, which when combined with existing PACIC items, permits scoring of five-item subscales on delivery of each of the 5As, as well as an overall 5As score.

The purpose of this report is to evaluate the appropriateness of the PACIC, and the revised 5As scoring method, for a larger sample of diabetic patients, for Latino patients, and among patients receiving their primary care from a wide range of providers. Specific questions addressed include the following:

  1. How do the results of the PACIC in this more diverse diabetes sample compare to the original PACIC study?

  2. Does the new 5As scoring method provide useful data?

  3. How do PACIC and 5As scores relate to 1) patient characteristics, 2) quality of diabetes-specific care received, and 3) self-management behaviors of healthy eating and physical activity?

The Diabetes Priority study was conducted with patients of 52 physicians in 30 primary care practices located throughout Colorado (16). The target population for the present report consisted of 641 type 2 diabetic patients who completed the study, were still receiving primary care from the same provider, and had known addresses at study conclusion.

The PACIC-5As version (Fig. 1) contained 26 items (the original 20 and 6 additional items to produce subscales reflecting each of the 5As of behavioral counseling recommended by the U.S. Preventative Services Taskforce) (12). Respondents answered each item with a response from 1 = almost never to 5 = almost always. There are five PACIC subscales related to different aspects of providing collaborative care congruent with the CCM (1,4): patient activation, delivery system/practice design, goal setting/tailoring, problem solving/contextual, and follow-up/coordination. The original PACIC and its subscales have been found to be internally consistent (α for overall scale = 0.93), to conform to a hypothesized factor structure, to be moderately reliable over time (r = 0.58 over 3 months), and to correlate as predicted with other measures of primary care quality (11).

As shown in the Fig. 1, the survey can also be scored to provide five items on each of the 5As subscales of assess, advise, agree, assist, and arrange (12,13). A modified five-item version of the Behavioral Risk Factor Surveillance System (17) physical activity questions was used to assess frequency and duration of moderate and vigorous exercise. The 17-item Block Dietary Data Systems “Fat Screener” (18) estimated participants’ intake of dietary fat. This screener has been shown to correlate well with the gold standard 100-item Block Food Frequency Questionnaire with respect to dietary intake of total fat, saturated fat, and percent of calories from fat (ρ = 0.60, P < 0.0001).

Quality of diabetes care was assessed at the final contact for the parent study by two composite scales consisting of items from the National Committee for Quality Assurance/American Diabetes Association Provider Recognition Program (19). These included a laboratory assessments composite score calculated from items that asked participants to recall when they had last received each of five recommended assessments from their physician (e.g., cholesterol, blood pressure, foot exam, dilated eye exam). The Provider Recognition Program assessment also included four items that involved receipt of self-management counseling for patients on diabetes-related lifestyle aspects of care (e.g., setting a self-management goal, nutrition education or therapy, self-monitoring of blood glucose). These items were summarized to provide a self-management/behavior change composite score.

Demographics collected earlier included sex, education, race/ethnicity, employment status, and yearly income. Medical characteristics collected at conclusion of the parent study included number of chronic conditions, HbA1c (A1C), and total and HDL cholesterol (Table 1).

Survey procedures

Between 12 and 24 months elapsed between participants’ last study visit and the survey. Packets included a summary of earlier study outcomes, the surveys, a cover letter, and a postage-paid “opt-out” postcard to return if the recipient did not want to complete the surveys. Participants completed informed consent as part of the parent study, and all procedures were approved by relevant institutional review boards.

Participant responses were tracked via a Microsoft Access database. Queries generated a list of nonrespondents who did not return the survey or the opt-out postcard (13.9%). Follow-up calls were made to 364 (56.9%) participants who received the surveys but did not return them within 2 weeks. The calls were scripted to remind participants to mail back the survey and offered the option to complete the survey over the phone. Three call attempts were made.

Of 364 participants, 46% (n = 167) were reached directly and 42% (n = 153) received messages. Eleven percent (n = 41) were not reachable via telephone, and 1% (n = 4) were deceased. Of 167 participants that were reached directly, 44% (n = 73) returned their surveys, 16% (n = 26) refused, and 41% (n = 68) did not respond. In summary, a total of 363 patients (63% of those contacted and/or presumed to have received a survey) completed a survey.

Respondent characteristics

Of nine patient characteristics analyzed, the only significant difference between respondents and nonrespondents was on sex (47% of respondents were women compared with 55% of nonrespondents, P = 0.042). There were no significant differences between respondents and nonrespondents on age, education, ethnicity, or income. Additionally, respondents did not differ from nonrespondents in medical characteristics, including number of comorbidities, A1C, and cholesterol levels (Table 1). Respondents appeared representative of type 2 diabetic patients: they were older (average age 64 years), and >60% had two or more chronic illnesses in addition to diabetes. Just over 10% were Latino, and 42% had an annual family income of <$30,000.

PACIC scale

The average overall score on the original PACIC items was 3.2 of a possible 5, just above the center point of the scale. Inspection of distributional characteristics revealed that there was adequate variability (Table 2) on the overall scale and all of the subscales (only 3–9% had subscale scores <1.5 [4% on the summary score] and 7–22% had subscale scores >4.5 [9% in the summary scale]). Within-group repeated-measures ANOVA results revealed significant differences among scale means. Bonferoni post hoc follow-up tests revealed that patient activation (M = 3.6) and delivery system/practice design (M = 3.5) did not differ significantly from one another but were rated as occurring significantly more often than the other subscales (Table 2). The problem solving/contextual scale (M = 3.4) was intermediate and significantly different from all other subscales, and the goal setting/tailoring and follow-up coordination scales were rated as occurring significantly less often than the other activities. The intraclass correlation of patient PACIC scores within physician was 0.023. To ensure that this moderate nesting did not inflate our results, we also conducted ANOVAs including physician as a random factor. All results were very similar, with resulting P values changing by ≤0.01. Between groups, repeated-measures ANOVA results showed that there were no significant differences associated with sex, ethnicity, income, or number of comorbid conditions (Table 2). The internal consistency α for the overall PACIC score was 0.96.

5As scoring

For the 5A scales, within-group repeated-measures results revealed that there were significant differences among the subscales. The mean for the overall 5As summary score was 3.2 of a possible 5, and there was adequate variability and distribution on all of the scales (only 4–16% had subscale scores <1.5 [4% on the summary scale] and only 8–15% had subscale scores >4.5 [7% on the summary scale]). Bonferoni post hoc tests showed that the arrange mean (M = 2.7) was significantly lower than all other 5As scale means (Table 3). The assist scale (M = 3.1) produced the next lowest mean score and was significantly different from all other scales. The intraclass correlation of the 5A scores within physician was 0.029. Therefore, we conducted ANOVAs including physician as a random factor. Again, conclusions were unchanged, and the resulting P values were almost identical to those of the analysis without physician. Between groups, repeated-measures ANOVAs revealed no significant effects due to sex, ethnicity, income, or number of comorbid conditions (Table 3). Internal consistency analyses indicated that the α for the 5As summary score was 0.97.

Relationship to diabetes care and self-management

The PACIC and the 5As summary scores were both significantly related to quality of diabetes care received, using both the composite laboratory assessment (r = 0.23 for both, P < 0.001) and the composite behavior change/self-management counseling scales (r = 0.25 for CCM scoring and 0.24 for 5As scoring, both P < 0.001).

Preliminary analyses revealed that female sex was significantly correlated with both physical activity (ρ = −0.19, P = 0.001) and fat consumption (ρ = −0.21, P < 0.001). Annual income <$30,000 per year was significantly correlated with fat consumption (ρ = 0.15, P = 0.007). To adjust for these associations, correlations were run both with and without sex and income partialled out. Both the PACIC and the 5As summary scales were significantly correlated with physical activity, in both unadjusted and adjusted analyses (r = 0.17 for both PACIC and 5As summary scores, P < 0.005), but were not significantly correlated with fat consumption.

This study provides a replication of results on a promising new instrument and reports on a new way of scoring the PACIC that should be of interest to the diabetes management research and clinical communities. There is a clear need for practical measures to assess implementation of both the CCM and the 5As approach to diabetes self-management; the PACIC appears to fill this need. The Institute of Medicine has concluded that patient-centered collaborative care is essential to improving chronic illness care (4), and valid measures of such care are sorely needed. Both the original PACIC and the new 5As version appear appropriate for type 2 diabetes.

Within-scale analyses produced important results concerning how often different CCM and 5As activities are conducted. Consistent with the original PACIC validation study, goal setting and follow-up support activities were conducted significantly less often than other actions, and this highlights important areas for future research. Collaborative goal setting is a core aspect of the CCM, and follow-up contact is critical across all types of diabetes management. The 5As analysis indicated that problem-solving assistance and especially arranging follow-up support were conducted less often than the other As. These results are consistent with research using different survey instruments (20) and indicate an important area for quality improvement and diabetes self-management research. Identifying practical and cost-effective interventions for problem-solving/action planning and follow-up support are important challenges (21), since these last two As are key aspects of effective self-management support (14,22,23).

Both scoring methods were generally unrelated to patient characteristics but significantly and moderately related to both measures of quality of diabetes care received and level of physical activity. These findings provide conceptual support for the PACIC, as it showed relationships with constructs that it was hypothesized to be related to and was not related to factors with which it should not be related (24).

Our data suggest conceptual and interpretive cautions and limitations to both scoring methods. Although the overall summary scores appear useful, the subscales are so highly intercorrelated that it may not make sense to interpret them separately. The overall correlation between the PACIC and 5As summary scores is not surprising, since 19 of the same items are used in both composites and since patient-centered self-management support is central to the CCM.

Study limitations include the modest survey return rate, that this was a cross-sectional analysis, and the length of time between assessment of self-management and quality of care assessments and the PACIC survey. These concerns are offset somewhat by the apparent representativeness of participants and the fact that any changes in patient characteristics since their assessment should have reduced the magnitude of relationships found. Strengths include the replication and extension of the initial PACIC validation study with diabetic patients, inclusion of a large and heterogeneous sample of type 2 diabetic patients from many clinical settings, analyses involving Latino patients, and the new 5As items and scoring methods. Future research is indicated to evaluate the PACIC against objective measures of clinician behaviors (e.g., video or audiotapes), determine the sensitivity of the PACIC to intervention effects, evaluate the scale with Spanish speaking and racial minorities, and determine whether the PACIC can be shortened without loss of predictive power.

This study was funded by grant no. HS10123 from the Agency for Health Care Research and Quality.

Appreciation is expressed to Barbara McCray for assistance with participant mailings and follow-up phone calls. We would like to acknowledge the cooperation we received from Copic Insurance Company in the initial recruiting of physician offices.

1.
Wagner EH: Chronic disease management: what will it take to improve care for chronic illness?
Eff Clin Pract
1
:
1
–4,
1998
2.
Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J: Improving chronic illness care: translating evidence into action.
Health Aff
20
:
64
–78,
2001
3.
Bodenheimer T, Wagner EH, Grumbach K: Improving parimary care for patients with chronic illness: the chronic care model, Part 2.
JAMA
288
:
1909
–1914,
2002
4.
Institute of Medicine, Committee on Quality of Health Care in America:
Crossing the Quality Chasm: A New Health System for the 21st Century
. Washington, DC, National Academy Press, 2001
5.
Wagner EH, Glasgow RE, Davis C, Bonomi AE, Provost L, McCulloch D, Carver P, Sixta C: Quality improvement in chronic illness care: a collaborative approach.
Jt Comm J Qua Improv
27
:
63
–80,
2001
6.
Chin MH, Cook S, Drum ML, Jin L, Guillen M, Humikowski CA, Koppert J, Harrison JF, Lippold S, Schaefer CT, the Midwest Cluster Health Disparities Collaborative: Improving diabetes care in midwest community health centers with the Health Disparities Collaborative.
Diabetes Care
27
:
2
–8,
2004
7.
Siminerio LM, Piatt G, Zgibor JC: Implementing the chronic care model for improvements in diabetes care and education in a rural primary care practice.
Diabetes Educ
31
:
225
–234,
2005
8.
Glasgow RE, Orleans CT, Wagner EH, Curry SJ, Solberg LI: Does the chronic care model serve also as a template for improving prevention?
Milbank Q
79
:
579
–612,
2001
9.
McGlynn EA: An evidence-based national quality measurement and reporting system.
Med Care
41:I 8–I15, 2003
10.
Bonomi AE, Glasgow RE, Wagner EH, Von Korff M: Assessment of chronic illness care (ACIC): a practical tool for quality improvement.
HSR
37
:
791
–820,
2001
11.
Glasgow RE, Wagner E, Schaefer J, Mahoney L, Reid R, Greene S: Development and validation of the Patient Assessment of Chronic Illness Care (PACIC).
Med Care
43
:
436
–444,
2005
12.
Whitlock EP, Orleans CT, Pender N, Allan J: Evaluating primary care behavioral counseling interventions: an evidence-based approach.
Am J Prev Med
22
:
267
–284,
2002
13.
Goldstein MG, Whitlock EP, DePue J, the Planning Committee of the Addressing Multiple Behavioral Risk Factors in Primary Care Project: Multiple behavioral risk factor interventions in primary care: summary of research evidence (Review).
Am J Prev Med
27(Suppl. 2)
:
61
–79,
2004
14.
Glasgow RE, Funnell MM, Bonomi AE, Davis C, Beckham V, Wagner EH: Self-management aspects of the improving chronic illness care breakthrough series: implementation with diabetes and heart failure teams.
Ann Behav Med
24
:
80
–87,
2002
15.
Glasgow RE, Davis CL, Funnell MM, Beck A: Implementing practical interventions to support chronic illness self-management in health care settings: lessons learned and recommendations.
Jt Comm J Qual Safety
29
:
563
–574,
2003
16.
Glasgow RE, Nutting PA, King DK, Nelson CC, Cutter G, Gaglio B, Kulchak Rahm A, Whitesides H: Randomized effectiveness trial of a computer-assisted intevention to improve diabetes care.
Diabetes Care
28
:
33
–39,
2005
17.
Brownson RC, Jones DA, Pratt M, Blanton C, Heath GW: Measuring physical activity with the Behavioral Risk Factor Surveillance System.
Med Sci Sports Exerc
32
:
1913
–1918,
2000
18.
Block G, Gillespie C, Rosenbaum EH, Jenson C: A rapid food screener to assess fat and fruit and vegetable intake.
Am J Prev Med
18
:
284
–288,
2000
19.
NCQA, American Diabetes Association: Diabetes physician recognition program [article online],
2003
. Available from www.ncqa.org/dprp/dprpmain.htm. Accessed 30 April 2005
20.
Glasgow RE, Eakin EG, Fisher EB, Bacak SJ, Brownson RC: Physician advice and support for physical activity: results from a national survey.
Am J Prev Med
21
:
189
–196,
2001
21.
Piette J: Enhancing support via interactive technologies.
Curr
DiabRep
2
:
160
–165,
2002
22.
Center for the Advancement of Health:
Essential Elements of Self-Management Interventions
. Washington, DC, U.S. Govt. Printing Office, 2002
23.
Lorig KR, Holman HR: Self-management education: history, definition, outcomes, and mechanisms.
Ann Behav Med
26
:
1
–7,
2003
24.
Shadish WR, Cook TD, Campbell DT:
Experimental and Quasi-Experimental Design for Generalized Causal Inference
. Boston, MA, Houghton Mifflin, 2002

A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.