| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
a University of California, Los Angeles, School of Medicine, Borun Center for Gerontological Research
b Jewish Home for the Aging, Reseda, CA
c Sepulveda Veterans Administration Medical Center, Sepulveda, CA
Correspondence: Sandra F. Simmons, PhD, Jewish Home for the Aging and the UCLA Borun Center for Gerontological Research, 7150 Tampa Avenue, Reseda, CA 91335. E-mail: ssimmons{at}ucla.edu.
Decision Editor: Eleanor S. McConnell, RN, PhD
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Outcomes of care • Satisfaction • Assessment • Incontinence
Federal nursing home (NH) survey procedures require that a sample of NH residents be interviewed about the quality of their daily care; however, an objective definition of the term "interviewable" is not provided for use (
Department of Health and Human Services 1995
). The lack of an objective definition has resulted in the exclusion of unknown numbers of residents from interviews during the survey process to evaluate NH care quality.
We have previously described criteria based on Minimum Data Set (MDS;
Simmons et al. 1997) assessment scales that identified residents capable of accurately reporting the occurrence of daily care activities, such as toileting and walking assistance, with 88% sensitivity (i.e., inclusion of appropriate residents) and 70% specificity (i.e., exclusion of inappropriate residents). The inappropriate exclusion of residents with cognitive impairment who could accurately describe care accounted for the reduction in specificity of the MDS-based selection criteria (
Simmons et al. 1997). Two other studies have also shown that a significant portion of verbally communicative, cognitively impaired NH residents are capable of reliably reporting their satisfaction with daily NH care and preferences for care; moreover, one of these studies also showed that NH staff were poor at identifying which residents were appropriate for interview (
Maris, Soberman, Murray, and Norton 1996;
Sansone, Schmitt, Nichols, Phillips, and Belisle 1998).
MDS-based selection criteria can potentially be used to identify residents who should be included in interviews to evaluate the quality of daily NH care. This research applies the MDS-based resident selection criteria from our previous study (
Simmons et al. 1997) to a subpopulation of the NH (incontinent residents), and compares those criteria to other performance-based cognitive status screening instruments to determine if the sensitivity and specificity of the selection criteria can be improved. The primary advantage of MDS-based resident selection criteria is that NH staff are required to complete the MDS for every resident.
The purpose of this study was to determine the best method of identifying incontinent NH residents capable of providing accurate interview information about daily NH care practices. Incontinent residents were targeted for inclusion in this study because they represent the majority of a typical community NH population (
Health Care Financing Administration 1997;
Strahan 1997). Furthermore, incontinence in NH residents is known to be associated with greater cognitive and physical limitations (
Mohide 1986;
Ouslander, Kane, and Abrass 1982), which, in turn, are associated with poorer care quality (
Coyne, Reichman, and Berbig 1993;
Lacks and Pillemer 1995). It is useful, therefore, to develop resident selection criteria that can be applied to a large and particularly vulnerable subgroup of the NH population.
Four questions were addressed in this study: (a) What percentage of incontinent NH residents are capable of accurate self-report with respect to the occurrence of daily care practices? (b) What chart (i.e., MDS) information can be used as criteria to identify incontinent residents capable of accurately describing daily care; and what is the sensitivity and specificity of the selection criteria? (c) Does the sensitivity and specificity of the MDS-based selection criteria improve when performance-based cognitive status measures are added to the criteria? and (d) What cutoff values can be established for the criteria to select incontinent residents appropriate for interview?
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
), is consistent with current NH care practice guidelines. Briefly, incontinence care (i.e., pad changes and toileting assistance) and mobility assistance (i.e., walking or wheelchair propulsion) were rendered by research staff every 2 hours, 4 times per day, 5 days per week, between 8 a.m. and 4 p.m. Research staff documented the number of times each resident required a pad change, accepted toileting assistance or walking assistance, and engaged in wheelchair mobility in addition to the extent of staff assistance provided to maximize independence. A research assistant who was unaware of the frequency of each care activity occurrence (i.e., whether or not a resident accepted or refused a care activity) interviewed residents using the "accuracy validation interview" at the end of an 8-hour period following research staff incontinence and mobility care provision. The "accuracy validation interview" is a brief interview with 20 structured questions regarding the occurrence of (e.g., "Did someone help you to walk today?"), frequency (e.g., How many times did someone help you walk today?"), and the extent of staff assistance provided (e.g., "Do you need someone with you/to help you when you walk?") for incontinence and mobility care practices. The interview was administered by research staff to each participant independently in a private room to ensure confidentiality. Interviewers were trained and supervised in the administration of the standardized interview protocol, including prompts for each question. Interviewers were aware of hearing and/or visual limitations; however, they were not aware of cognitive status. Nonverbal responses (i.e., nod or shake of the head to indicate yes or no, respectively; indicating frequency with fingers) were acceptable. No cognitive screening criteria were used to select residents for interview.
Measures
Background Variables
Age, gender, ethnicity, length of NH stay in days, and the presence or absence of a chart diagnosis of depression or dementia were recorded from each participant's medical record. Three established MDS scale scores were derived from individual MDS items from each participant's most recent MDS assessment, version 2.0: Activities of Daily Living (MDS-ADL), Social Initiative–Involvement (MDS-SII), and the Depression Rating Scale (MDS-DRS). The reliability, validity, and item content of each of these scales are described elsewhere (
Burrows, Morris, Simon, Hirdes, and Phillips 2000;
Hartmaier, Sloane, Guess, and Koch 1994;
Hartmaier et al. 1995;
Hawes et al. 1995;
Mor et al. 1995;
Morris et al. 1994).
Cognitive Status
The Cognitive Performance Scale (CPS) was calculated from the most recent MDS assessment. The CPS is based on five MDS items (see Table 1 ), and the total score ranges from 0 (cognitively intact) to 6 (severely impaired). In addition, all 10 MDS items related to cognitive functioning were used to create two scale scores, Direct and Indirect, and one subscale score, Recall. The Direct and Indirect scales were recently recommended by
Hartmaier and colleagues 1995 as a more useful way to denote the cognitive functioning of NH residents than the CPS. The Direct scale consists of seven items related to memory, orientation, and decision-making skills, which yield a composite score with a range of 0 (i.e., no memory problems, oriented, independent decision making) to 9 (i.e., severely impaired) for each participant (see Table 1 ). The Recall subscale comprises the following four orientation items that are also part of the Direct scale: (a) current season, (b) location of own room, (c) staff names and/or faces, and (d) awareness of being in an NH, with a score range of 0 to 4 (i.e., able to recall all four items). Thus, in order to sum all items for the Direct scale total score, the Recall subscale score must be reverse coded (i.e., 4 = unable to recall any of the four items). The Indirect scale consists of 10 items related to comprehension, communication, problem behaviors, self-feeding ability, and continence status, which results in a composite score with a range of 0 (i.e., no impairment) to 20 (i.e., impaired on all items) for each participant (see Table 1 ).
|
;
Folstein, Folstein, and McHugh 1975;
Molloy, Alemayehu, and Roberts 1991). We expected that both the MMSE and TSI instruments would provide better indicators of residents' cognitive functioning than the MDS cognitive items because of the difference in assessment approaches (i.e., performance-based versus NH staff categorical ratings). In addition, the TSI is designed to discriminate specifically among lower cognitive functioning residents (i.e., MMSE total score <11).
Definition of Accurate Self-Report
Each participant's ability to accurately describe their received care was assessed in several ways. First, we computed a chance-corrected measure of agreement (i.e., Kappa) between each participant's self-report and research staff documentation of care activity occurrence. Participants with a Kappa value 
0.70 were classified as capable of "accurate" self-report. The proportion of participants who would be considered accurate based on lower Kappa values (i.e., 0.60) was also examined. Second, the percentage of interview questions answered accurately was calculated for each participant by type of question: (1) all 20 interview questions, (2) only the 6 questions related to frequency of care activity occurrence, and (3) the remaining 14 questions related to occurrence and whether or not staff assistance was provided for each activity. Residents were classified as accurate if their percent agreement was 80%. The three different percent agreement values were compared for the group of participants with paired-samples t tests, and the proportion of participants who would be considered accurate based on the three different percent agreement values were compared with chi-square analyses.
Data Analyses
To determine if there were differences between those who provided complete "accuracy interview" data and those who did not, demographic, MDS, and performance-based cognitive status variables were compared with independent-samples t tests for the continuous variables and chi-square analyses for the categorical variables.
Predicting Accurate Residents: Validation of a Screening Instrument
In order to identify measures that predicted accuracy status for participants, an exploratory correlational analysis was first conducted to identify independent variables (i.e., demographic characteristics, MDS-derived scale scores, and performance-based cognitive assessment scores) significantly correlated with either measure of resident accuracy (i.e., Kappa or percent agreement). The independent variables that proved to be significantly correlated with resident accuracy, but not significantly intercorrelated at a level 0.80, were entered into a series of logistic regression analyses to predict membership in the accurate group based on each of the definitions of accuracy described earlier. For the MDS-derived scale that was most highly associated with residents' ability to describe care accurately, the sensitivity and specificity values were then calculated for various cutpoints that could be used to discriminate accurate from inaccurate residents.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
;
Ouslander et al. 1982).
|
2 (1, N = 186) = 35.3, p < .001). A total of 65 (35%) participants were considered accurate based on all 20 interview questions. A total of 59 (32%) of the 186 participants were classified as "accurate" according to the Kappa value 0.70 criterion. These 59 participants represent 27% of those 220 residents initially targeted for interview. When the Kappa criterion was lowered to 0.60, 73 participants (39%) were deemed "accurate."
In our previous work, the MDS-derived ADL, CPS, and SII scales significantly predicted accuracy status defined as 80% or more correct on occurrence and assistance questions on an "accuracy interview" (
Simmons et al. 1997). In this study, the following variables were significantly correlated with each participant's percent agreement: ADL, CPS, Direct, Indirect, Recall, and MMSE scores (see Table 2 ). These same variables were also significantly correlated with Kappa values, with the addition of the presence or absence of a depression diagnosis (see Table 2 ). Only two intercorrelations between MDS-based cognitive functioning scales exceeded a value of 0.80: CPS-Direct, r = .882; and Recall-Direct, r = .942. The remainder of the variables listed in Table 2 (i.e., demographic characteristics, mood indicators) were not significantly related to accuracy according to any criterion.
When the MDS-based selection criteria from the previous study (i.e., CPS, ADL, and SII scales) were applied to the sample of incontinent residents in this study using the 80% agreement percentage based only on the occurrence and assistance questions as the accuracy criterion, 63% were correctly identified as appropriate for interview (i.e., sensitivity) and 79% were correctly identified as inappropriate for interview (specificity), with an overall correct classification rate of 72%. The CPS scale was the primary significant predictor in the equation (b = -.70, SE = .16, R = -.27, p < .001) with the ADL scale approaching significance (b = -.53, SE = .29, R = -.07, p = .06) and the SII scale not significant (b = .04, SE = .13, R = .00, p = .77).
In order to compare the predictive value of the MDS-derived scales to the predictive value of the MMSE, we computed a series of logistic regression models. The collinearity between three potential predictors (i.e., Direct-CPS, Direct-Recall) prevented these variables from being entered into the same model. The following MDS-derived independent variables were entered into a predictive model: ADL, Direct, and Indirect scales (Table 3 , Model 1); and ADL, CPS, Indirect, and Recall scales (Table 3 , Model 2). A third model was analyzed in which the MMSE total score was entered as the independent variable (Table 3 , Model 3). The results of the three logistic regression models were comparable regardless of the criterion used to determine accuracy status (i.e., Kappa value 0.70 or 0.60, percent agreement based on all 20 interview questions versus 14 occurrence and assistance questions versus 6 frequency questions). The results displayed in Table 3 are based on the percent agreement criterion for only questions related to staff occurrence and assistance, as this criterion resulted in the largest number of participants being deemed capable of accurate self-report with respect to their daily care. These results show negligible differences among the three models with respect to predicting residents' ability to accurately describe their received care.
|
1, 2, or 3. Three logistic regression analyses were then conducted in order to determine the sensitivity (i.e., correct classification of accurate) and specificity (i.e., correct classification of inaccurate) that would result from each cutoff value. The sensitivity and specificity of a Recall subscale cutoff value 1 was 82% and 54%, respectively, with an overall correct classification rate of 67%. A total of 113 (64%) of the participants had a Recall subscale score 1. If a Recall subscale cutoff value 2 was used, the sensitivity decreased to 64% and the specificity increased to 75%. The overall correct classification rate remained approximately the same, 70%. A total of 78 (44%) of the participants scored 2 on the Recall subscale. Finally, a cutoff value 3 (n = 56, 32% of participants) resulted in 57% sensitivity and 92% specificity (i.e., exclusion of the majority of inaccurate residents but at the expense of excluding many who were accurate as well), with an overall correct classification rate of 75%. |
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
), which documented that many NH residents are capable of accurately describing their received care and that MDS-derived cognitive status variables are excellent predictors of their accuracy status. Surprisingly, the use of a standardized cognitive performance test (i.e., MMSE) did not improve upon the predictive value of the MDS-based cognitive performance scales. Based on the results of this study, the most time-efficient and simple approach to identify incontinent NH residents capable of accurately describing the care that they receive would be to calculate the MDS Recall subscale score and include all residents in the interview who score 2 or higher on this scale. This calculation could be completed quickly if one has access to the MDS information, which is available for all NH residents. The use of the highly efficient MDS Recall scale to identify incontinent NH residents capable of accurate self-report is more defensible than the untested approaches to screening that have been described in other studies (
Aller and Coeling 1995;
Lavizzo-Mourey, Zinn, and Taylor 1992;
Pearson, Hocking, Mott, and Riggs 1992). Most typically, the approach used in other studies to identify NH residents capable of interview is to have NH staff use subjective criteria to identify residents or to apply MMSE cutoff scores (
Aller and Coeling 1995;
Berlowitz, Du, Kazis, and Lewis 1995;
Lavizzo-Mourey et al. 1992). The use of such criteria results in a much larger proportion of residents being excluded from interview as compared to the use of the MDS-based Recall scale score of 2 or higher in this study. It is important to note, however, that the intent of this study was to identify incontinent NH residents who could accurately describe the daily care that they received as opposed to identifying residents who could simply respond to interview questions. The care practices used in this study to verify accuracy were frequent daily occurrences in the context of the larger clinical trial (e.g., toileting and walking assistance) and concrete, unambiguous events. It is, therefore, not surprising that a measure of residents' ability to recall other everyday events (e.g., staff names/faces, awareness of being in an NH) should predict accuracy. We cannot say, based on these results, whether this screening method would perform similarly well when trying to identify NH residents who can reliably respond to questions about their mood, experience of pain, satisfaction, and preferences related to daily care. On the one hand, the ability to understand these types of questions and report subjective, emotional experiences may require abstract reasoning, which might be lost when recall is preserved in an individual with cognitive impairment. On the other hand, these experiences are subjective and immediate. There is, thus, no requirement that recall be intact in order to reliably report such personal experiences, emotions, and preferences. The primary value of identifying accurate residents, as we did in this study, lies in the quality assurance arena. As there is reason to believe that chart documentation of many care activities may be inaccurate (e.g., how often a nursing assistant records that toileting or walking assistance was rendered to an individual resident), it is necessary to assess NH care quality with multiple approaches. In addition to chart review, these alternative approaches involve either directly observing daily care routines or interviewing residents. We have described a screening methodology that will efficiently permit interview quality assurance strategies to be implemented in the NH setting.
|
Acknowledgments |
|---|
Received for publication April 18, 2000. Accepted for publication March 26, 2001.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  A. N. Rahman and J. F. Schnelle The Nursing Home Culture-Change Movement: Recent Past, Present, and Future Directions for Research Gerontologist, April 1, 2008; 48(2): 142 - 148. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. E. Fisher, L. D. Burgio, B. E. Thorn, and J. M. Hardin Obtaining self-report data from cognitively impaired elders: methodological issues and clinical implications for nursing home pain assessment. Gerontologist, February 1, 2006; 46(1): 81 - 88. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. F. Schnelle, D. Osterweil, and S. F. Simmons Improving the Quality of Nursing Home Care and Medical-Record Accuracy With Direct Observational Technologies Gerontologist, October 1, 2005; 45(5): 576 - 582. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. H. Zarit, P. C. Griffiths, and S. Berg Pain Perceptions of the Oldest Old: A Longitudinal Study Gerontologist, August 1, 2004; 44(4): 459 - 468. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. F. Simmons, M. P. Cadogan, G. R. Cabrera, N. R. Al-Samarrai, J. S. Jorge, L. Levy-Storms, D. Osterweil, and J. F. Schnelle The Minimum Data Set Depression Quality Indicator: Does It Reflect Differences in Care Processes? Gerontologist, August 1, 2004; 44(4): 554 - 564. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. F. Schnelle, B. M. Bates-Jensen, L. Levy-Storms, V. Grbic, J. Yoshii, M. Cadogan, and S. F. Simmons The Minimum Data Set Prevalence of Restraint Quality Indicator: Does It Reflect Differences in Care? Gerontologist, April 1, 2004; 44(2): 245 - 255. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. P. Cadogan, J. F. Schnelle, N. Yamamoto-Mitani, G. Cabrera, and S. F. Simmons A Minimum Data Set Prevalence of Pain Quality Indicator: Is It Accurate and Does It Reflect Differences in Care Processes? J. Gerontol. A Biol. Sci. Med. Sci., March 1, 2004; 59(3): M281 - M285. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. F. Schnelle Improving Nursing Home Quality Assessment: Capturing the Voice of Cognitively Impaired Elders J. Gerontol. A Biol. Sci. Med. Sci., March 1, 2003; 58(3): M238 - 239. [Full Text] [PDF]
|
|
|
|
|
|
S. F. Simmons, S. Babineau, E. Garcia, and J. F. Schnelle Quality Assessment in Nursing Homes by Systematic Direct Observation: Feeding Assistance J. Gerontol. A Biol. Sci. Med. Sci., October 1, 2002; 57(10): M665 - 671. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
