| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
Correspondence: Address correspondence to Robert L. Kane, MD, University of Minnesota School of Public Health, D351 Mayo (MMC 197), 420 Delaware Street SE, Minneapolis, MN 55455. E-mail: kanex001{at}umn.edu
 |
Abstract |
|---|
 TOP Abstract BackgroundMethodsResultsDiscussionReferences |
|---|
Key Words: Quality of life • Nursing homes • Resident reports
The first step is, of course, to specify and test self-report measures of quality-of-life outcomes experienced by individual residents. That work was the first part of the CMS contract and is described elsewhere (Kane, 2001). However, even after reliable and valid measures of any given resident's quality of life are developed, many questions arise about whether such individual information can be summarized to characterize the quality of life of the nursing facility. For some accountability purposes, it is necessary to combine the individual data in a way that characterizes the nursing home as a whole. Moreover, such aggregation will facilitate studying the role of nursing facility characteristics in influencing the quality of its residents' lives. This article describes our method of aggregating individual data to the facility level and the extent to which such facility-level data could serve to differentiate among a sample of nursing homes. It is designed to test the feasibility of using this quality-of-life data to distinguish among nursing facilities.
|
Background |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
Some controversy exists, as yet uninformed by much data, about the extent to which a nursing home can actually influence the social and psychological domains of quality of life. For example, outcomes such as meaningful activity, relationships, and the like may be heavily determined by social factors outside facility control, such as family structure and availability, quality of family relationships, and resident's interests, education, and even personality. It is also reasonable to hypothesize that quality-of-life levels in a nursing home will be related to various health and disability characteristics such as health status, prognosis, functional abilities, sensory abilities, pain, persistent serious depressive illness, and cognitive abilities. These factors can affect how a resident appraises his or her quality of life. Nursing homes, however, can likely influence some of these potential mediators of quality of life directly, especially pain, functional status, and depression (particularly depressive affect that is not part of a long-standing psychiatric diagnosis). Although it is difficult for nursing homes to overcome the effects of extreme sensory impairment, high disease burden, irreversible cognitive impairment, and poor prognoses, we suggest that nursing homes can take effective steps to improve or maintain quality of life for those who may be at highest risk of poor quality of life, including those with dementia, those who are facing imminent death, and those who have limited external social support systems.
Indicators of quality of a health care organization may be expressed at several levels. One way to generate indicators is to identify facility-level structural and process factors that relate to outcomes of interest, such as nursing staff levels or infection control. Another approach is to aggregate individual-level data on outcomes of interest (e.g., proportion of bedsores or proportion of urinary tract infections) to describe the facility. When such outcomes are aggregated, adjustments are necessary for aspects of the case mix that are out of the control of the facility. With the use of the second approach, nursing home data generated through the Minimum Data Set (MDS) have been under study for more than a decade and various health-oriented outcome indicators have been developed by use of different case-mix adjustment strategies (Zimmerman et al., 1995). For quality-assurance purposes, and for presenting meaningful comparative data to the public, it is necessary to avoid describing nursing homes as better in quality if a substantial amount of the differences can be attributed to the characteristics of the residents who are admitted. In contrast, it is also imperative not to overadjust and, therefore, fail to hold nursing homes accountable for resident characteristics that they may be able to change (e.g., bed-bound or wheelchair-bound residents, or residents who are depressed).
A central task in creating facility-level measures of quality of life is to adjust appropriately for differences in case-mix across facilities, although this need is not always appreciated (Davis, 1991). The choice of case-mix adjusters is important; they should reflect elements that might influence quality of life, but they should not include items that are under the control of the nursing home, lest important differences in quality are adjusted away. Although a growing body of information is available on case-mix adjustment for quality of care in nursing homes (Braun, 1991; Mukamel, 1997; Phillips et al., 1996; Porell & Caro, 1998), little work has been done on quality of life.
The MDS assessment is performed by various staff members (nurses, social workers, therapists, and activities personnel) based on their knowledge of the resident during the time period of the assessment. Although considerable detail is provided about how to define each MDS element, ultimately the MDS is composed of staff observations and reflections, not resident reports; although the MDS manuals emphasize the value of resident input, neither specific instructions nor structured mechanism to obtain such input is provided. Quality of life as a phenomenon is poorly addressed by MDS 2.0. Among the 24 MDS-based Quality Indicators created by the Center for Health Services Research and Analysis at the University of Wisconsin, 2 were labeled as tapping quality of life: prevalence of daily physical restraints and prevalence of little or no activity (Zimmerman et al., 1995). However, quality of life is a more complex phenomenon than can be readily captured by these two statistics; moreover, it is a subjective phenomenon that requires direct statements from residents.
In this study, we envisaged quality of life as a multidimensional concept, and we developed measures for a spectrum of quality-of-life domains that the literature and focus groups had identified as important: comfort, autonomy, privacy, dignity, meaningful activity, relationships, food enjoyment, security, functional competence, and spiritual well-being (Kane, 2001; Kane et al., 2003). Conceptually, and on the basis of early analyses, we had reason to expect that the quality-of-life score on some domains would be worse for residents with poor cognitive abilities, though again this might be related to the kind of care and environments and general milieu ordinarily offered to those with worse cognitive abilities. In contrast, for some other domains that require more cognitive processing, such as privacy and dignity, we made no specific hypotheses about the influence of declining cognitive abilities on the quality-of-life scores. Therefore, an adjustment for cognitive status was necessary. Moreover, cognitive status is not a characteristic that a nursing home can easily influence through providing worse care.
To characterize a nursing home on any outcome, one must sample a sufficient number of residents to yield stable estimates (that is, the estimate should not be subject to the idiosyncratic outcomes of one or two residents). Given that data on quality of life would have to be collected directly from residents, a sampling approach seemed practical. Another goal we had for this study, therefore, was to determine how many resident appraisals were sufficient to yield stable estimates of the quality-of-life domains.
|
Methods |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
At this stage in measure development, our procedures demanded a great deal from facilities, and their participation in the study was voluntary. If a facility declined to participate, we randomly made a replacement within the size and rural–urban designation. Because our goal in this paper is to examine the patterns of performance across the facilities in the sample and not to attempt any generalization to the larger universe of nursing facilities, we propose no effort at weighting to compensate for the selection procedures. Instead, we examine the effects of various facility characteristics on the facilities' performance.
Consistent with our belief that each resident is the best judge of his or her quality of life, we collected the data on quality of life directly from residents. The resident sampling plan within each facility yielded 50 residents from each of up to five units per facility. It was based on two factors: the percentage of private rooms and residents' cognition. In the first step we sought to ensure that we had at least 10 residents in each facility in private rooms if possible. These residents were randomly selected first and assigned to cognitive categories. Then we drew the rest of the sample to achieve a 50–50 split in those with high and low cognitive scores until the facility sample included 50 residents. (Comatose residents and residents under the age of 65 were not included.) Interviews were attempted with all residents who were sampled, regardless of cognitive level. We abstracted cognitive functioning information from the MDS data on all residents in the facility, using a variant of the Cognitive Performance Scale (CPS; Morris et al., 1994). Specifically, we eliminated from the CPS the item on being comatose or in a vegetative state (because those residents were excluded) and the item on feeding. Our scale used a short-term memory problem (yes–no), a long-term memory problem (yes–no), and a 0–3 rating of decision-making capability to form a 6-point scale (0–5), where 5 represents severe impairment. We formed two strata and evenly sampled from residents with scores of 0–2 and 3–5. Among the latter (i.e., those who were more impaired), 52.8% completed enough of the questionnaire to enable construction of at least one quality-of-life domain scale, and for 33.5% we could construct 9 or more of the 10 domains measured. Among the higher functioning residents (with scores from 0 to 2), we could construct at least one quality-of-life measure for 90.8% of residents and 9 or more domains for 78.7% of the residents.
Data
Quality-of-life data came from in-person interviews conducted in the participating facilities by teams of university data collectors who received a week of training and testing to establish their interrater reliability. They were supervised in the field by periodic visits, but no formal interrater reliability testing was done after their initial certification. As described elsewhere, we created individual scale scores for each of the 10 domains, which were created by using a combination of factor analysis (to create domain scales) and cluster analysis (to shorten the length of the scales; Kane et al., 2001; Kane et al., 2003). Although the primary response mode used a Likert scale, the questions allowed for dichotomous responses from residents who could not use the Likert responses. We used this dual response option to encourage participation of cognitively impaired respondents. We combined the two response modes by using a z-score adjustment described elsewhere. These scales were subjected to the usual psychometric tests for reliability and validity, the details of which are provided elsewhere (Kane et al., 2003). The original data collected included a large number of questions designed to create scales for the postulated 11 domains. A first analytic step in creating the scales for these domains was to identify the questions that best captured each of them. For this analysis, we used the short versions of the scales, which consisted of three to six items per domain. We tested the internal reliability of these scales by using Cronbach's alpha coefficients, which essentially measure the degree to which each item in a putative scale correlates with the total scale score. Cronbach's alpha coefficients for the scales and the number of items for each scale are shown in Table 1. Ideally, these scores should exceed.80. All but two alphas were above.60 and four were above.70. Confirmatory factor analyses showed that the presumed factor structure was supported for the 10 domains reported here (Kane et al., 2003).
|
Analysis
We implemented the comparison of quality-of-life domain scores among facilities on two levels: raw scores and scores adjusted for differences in patient characteristics. First, we calculated quality-of-life domain scores by using all available resident-level responses and aggregated them by facilities to obtain nonadjusted facility-level average quality-of-life domain scores. We also used resident-level data to develop a case-mix-adjusted model that regressed individual quality-of-life domain scores on selected resident characteristics obtained from MDS data. These included ADL, cognition, age, gender, and length of stay. We dichotomized length of stay into fewer than 3 months and 3 months or more.
To enable the comparison of facilities while adjusting for patient characteristics, we calculated differences between the observed individual quality-of-life domain scores and the expected ones calculated based on the case-mix-adjusted model already described. We averaged the standardized resident-level residuals by facility to obtain facility-level adjusted scores. We transformed the distribution of the facility-level adjusted scores for all 40 facilities by using z scores. As a result, we expressed the differences among facilities in units of the standard deviation of average scores for all 40 facilities. By construction (assuming close to a normal distribution), we could expect that approximately half of all facilities would have positive and another half would have negative standardized scores in the range from –3 to +3.
We used mixed-effect linear models with main effects to fit the data. We used 10 components of the quality-of-life instrument as dependent variables. Independent variables included various combinations of Facility and Interviewer factors, and covariates. The allocation of interviewers by facilities was not planned in advance, and post hoc cross-tabulation revealed a very unbalanced design with many empty cells. This imbalance dictated the use of a type of sum of squares that can accommodate the design with empty cells. Covariates included in the models served as risk adjustors and included length of stay in the nursing facility and MDS-based cognition score (six-level ordinal variable), the ADL score, age, education (five-level ordinal variable), and binary variables representing race (White, other), marital status, gender, and the presence of living children. We constructed these variables on the basis of the measured values to obtain reasonably unskewed distributions. We implemented calculations by using the Univariate General Linear Model and Variance Components procedures in SPSS 11.0.
|
Results |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
By design, half the sample was rural. There was a considerable range in staffing. The proportion of homes that were proprietary was much lower than the national average, partly because Minnesota and New York have high numbers of nonprofits, but also because most of the refusals occurred in for-profit facilities, whereas all the facilities oversampled for private rooms were nonprofit. There was considerable variation in the proportion of cognitively impaired residents. This distribution was generally similar across four of the states (averaging about 55%), but California homes had a much higher proportion (67%) than the rest of the sample.
The mean facility scores for each domain are shown in Table 2. We created an average score across domains by dividing the additive scale by the number of questions for the particular domain. Each quality-of-life domain could be scored between 4 and 1, with higher scores reflecting higher quality of life. Facility-level scores varied from 2.70 (meaningful activity) to 3.67 (dignity). The extent of between-facility variation in the scores is reflected in the standard deviations, which varies over 100% (from 0.101 for dignity to 0.212 for spiritual well-being).
|
|
|
|
|
An important question is the sample size needed to produce significantly different facility scores. To estimate the effect of different sample sizes, we used a method described by Dupont and Plummer (1990). This method, which produces results that are in close agreement with those of Pearson and Hartley (1970), uses the relationship of the between-group difference and the within-group difference. Because these data could be used to describe facility performance, we must take special care to ensure that false positives (Type II errors) are avoided. Hence, we should use a conservative alpha value. Table 7 shows the estimated sample sizes needed to detect differences between two facilities that exceed 1 SD within groups, but the choice of a 1-SD threshold difference is arbitrary. In the first case, the risk of declaring a difference simply as a result of chance variation in sampling (alpha) is less than 5%, and the chance of declaring no difference when there really might be one (beta or 1-power) is 20%. In the second case, the alpha level is set at 1% and the beta is set at 10%. The sample size varies among the domains. For the first case it runs from 7 (spiritual well-being) to 12 (dignity). For the second, more stringent case, it runs from 13 to 22. Thus, a sample size of 22 per nursing home would permit a reasonable comparison across facilities. This number represents the number of nursing home residents who responded to an adequate number of the questions about quality of life. Presumably, a somewhat larger sample would have to be approached to net this amount. If smaller units within a facility were to be compared, then the necessary sample size would increase considerably.
|
|
Discussion |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
At this point we have used a separate quality-of-life score for each of the 10 domains. A further step would be to create a weighted score to establish a single quality-of-life score for each facility. This would render comparisons simpler, but might, in fact, obscure differences of interest, especially when the nursing home profile has both positive and negative deviations, a situation that occurred 30% of the time in our data (See Table 3). Moreover, from the perspective of the quality-assurance requirements, nursing homes are accountable for outcomes in specific areas such as dignity, rather than the general construct Quality of Life.
One concern with using domain averages is that if these average scores are near the upper or lower ends of the possible range, a ceiling or floor effect could result, and we would not be able to detect any positive or negative deviations. However, the results in Table 2 show that this potential threat did not occur. None of the average facility-level domain scores approached either end of the possible range.
A next step will be to establish the validity of these results by comparing the facility performance in quality of life to other markers such as deficiencies in survey results and the quality indicators derived from the MDS. In addition, it is worth exploring whether quality of life is influenced by a variety of administrative policies, such as staffing and program emphasis. For this we will be using a different sample of facilities that is more likely to include facilities with bad regulatory histories, which will not have the bias that comes from using a volunteer sample.
The analysis of the effect of adjusting for residents' characteristics suggests two things: (a) The proportion of total variance explained by the facility is modest, so resident factors undoubtedly explain a substantial amount of the variance; however, (b) adjusting for the characteristics does not importantly influence the amount of variance explained by the facility. In other words, there is a small but important facility effect, which is separate from the effect of the residents.
Some limitations of this study should be acknowledged. We have used z-scoring techniques to simplify the data presentation, but these transformations could create apparently large differences when the actual differences in scores are small. Care must be taken in interpreting the relative performance of any facility. However, if attention is focused on patterns across domains, the risk of overinterpretation is minimized. The sample used for this study was not intended to be nationally representative. The emphasis here was on developing a method for aggregating resident-level quality-of-life domain scores to the facility level. The actual findings are perhaps less noteworthy than is the approach used, because this sample of facilities was artificially created to emphasize single rooms and rural status. The general lack of statistically significant relationships between quality-of-life domain scores and nursing home characteristics may be attributed in part to the limited sample size. The sample itself is atypical of the general distribution of nursing homes in the United States. The lower proportion of proprietary homes may reflect the effect of this sampling. Further work is needed to see how well this approach can discriminate among a larger, more representative sample of facilities.
While analyzing the possibility of various quality-of-life domains to discriminate facilities, we found that difference in patient characteristics between facilities (the most commonly used reason for adjustment) changed the resolution of the comparison very little. A much greater source of variation was attributable to the measurement process (i.e., allocation of interviewers to facilities). Because the study had not been planned to examine this effect, we had to rely on a post hoc analysis of this observational study with a very unbalanced design and a sample size that was insufficient to test this effect. Nevertheless, we demonstrated the importance of the measurement process by comparing the relative variances associated with two random factors (we found Interviewer and Facility factors to have comparable variance). If confirmed, this finding would affect the process of comparing quality-of-life domains among facilities. Such studies should be designed to prevent confounding the Facility and Interviewer factors to allow their effects to be separated.
The interviewer effect is troublesome. We spent a week training the interviewers in uniform approaches to interviewing and establishing that they met the performance criteria as observed by their trainers. The logistics of the study made it much easier to concentrate a few interviewers in each facility, thereby increasing the effects of inter-interviewer differences, but also making it harder to distinguish among interviewer, facility, and state effects. Because different groups of interviewers worked in different states, we cannot say for certain how much is an interviewer and how much is a state effect. In an operational model it will be important to use the same interviewers across many facilities to at least convert potential bias into error. We were pleased in later work in this contract to see that it was possible to train both nursing home staff and surveyors in how to conduct resident interviews. The levels of interrater reliability (based on a different sample of residents) for domain scores between those trainees and our own interviewer staff were of the order of 0.61 to 0.75.
Even though the amount of overall variance explained by facilities is small when other factors are considered, it is important. Whereas resident characteristics are critical in predicting quality of life, some portion of the variance remains under the control of facilities. They thus can be held accountable for this important aspect of nursing home life, if appropriate case-mix adjustments are made in the analysis of the results. The specific pattern of strengths and weakness across the various domains may be especially informative for quality-improvement efforts.
This project suggests that it is possible to collect data on nursing home residents' quality of life and to use that information to discriminate among facilities. Such data can expand the dimensions of quality and can play an important role in helping people choose among nursing facilities. The feasibility demonstrated by this work has led to active discussion about incorporating quality-of-life items into MDS 3.0 and surveyor procedures.
|
Footnotes |
|---|
1 Division of Health Services Research and Policy, University of Minnesota School of Public Health, Minneapolis.
2 Department of Health Services Administration, Center for Bioethics and Health Law, University of Pittsburgh, PA.
3 Robert C. Byrd Health Sciences Center, West Virginia University Institute for Health Policy Research, Morgantown.
4 Department of Psychology, St. Cloud State University, MN.
Decision Editor: Linda S. Noelker, PhD
Received for publication April 4, 2003. Accepted for publication October 21, 2003.
|
References |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  E. A. Miller and V. Mor Balancing Regulatory Controls and Incentives: Toward Smarter and More Transparent Oversight in Long-Term Care Journal of Health Politics Policy and Law, January 1, 2008; 33(2): 249 - 279. [Abstract] [PDF]
|
|
|
|
 |
|
 R. L. Kane, R. A. Kane, B. Bershadsky, H. Degenholtz, K. Kling, A. Totten, and K. Jung Proxy Sources for Information on Nursing Home Residents' Quality of Life J. Gerontol. B. Psychol. Sci. Soc. Sci., November 1, 2005; 60(6): S318 - S325. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
