Home
HOME ARCHIVE SEARCH TABLE OF CONTENTS

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Services
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
PubMed
Right arrow PubMed Citation
The Gerontologist 43:483-492 (2003)
© 2003 The Gerontological Society of America

Providing Outcomes Information to Nursing Homes: Can It Improve Quality of Care?

Nicholas G. Castle, PhD1

Correspondence: Address correspondence to Nicholas G. Castle, PhD, RAND, 201 North Craig Street, Suite 102, Pittsburgh, PA 15213-1516. E-mail: CASTLE{at}RAND.org


    Abstract
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Purpose: This study examined whether providing outcomes information to 120 nursing homes facilitated improvements in quality over a 12-month period, as compared with 1,171 facilities not receiving this information. The outcomes information provided consisted of a report mailed to administrators that examined six measures of care quality. These were the rates of physical restraint use, urethral catheterization, contractures, pressure ulcers, psychotropic medication use, and certification survey quality of care deficiencies. Design and Methods: Data used in this investigation came from the 1998 and 1999 On-line Survey, Certification and Recording (OSCAR) system. With the use of generalized least squares regression and each of the six quality indicators as dependent variables, risk-adjustment models were developed by using aggregate resident variables as independent variables. These risk-adjustment models were used to compare the outcome measures for the intervention facilities with the same outcome measures in other facilities in the same states (Kansas, Maine, Mississippi, New York, Texas, and South Dakota). The difference between 1998 predicted scores less actual scores was calculated, and the difference between 1999 predicted scores less actual scores for each facility was calculated. Subtracting these 1998 difference scores from the 1999 difference scores gives some indication of the change in outcomes controlling for resident mix. Results: Physical restraint use and psychotropic medication use were significantly lower after 12 months in the intervention facilities, suggesting that quality had improved. Implications: This study may provide evidence that some of the outcomes initiatives currently being pursued in the long-term care arena will positively affect quality of care.

Key Words: Quality • Outcomes • OSCAR


Generally, outcomes information has some relation to the model popularized by Donabedian (1980), who stated that quality could be measured in terms of structures, processes, and outcomes. He hypothesized that good (bad) structure may promote good (bad) process and good (bad) process in turn may promote good (bad) outcomes. Donabedian regarded clinical outcomes as the most important quality measures. He envisioned outcomes as a change in patients' status that could be attributed to antecedent care. Similarly, Guadagnoli and McNeil (1994, p. 14) defined outcomes research as "linking the type of care received by a variety of patients with a particular condition to positive and negative outcomes in order to identify what works best for which patients."

Outcomes information is ubiquitous in today's health care environment. For example, managed care plans are required to compile outcomes data, which in turn are used to create a report card—the Health Plan Employer Data Information Set (HEDIS). Outcomes are also used in nursing homes. The new performance initiative, ORYX (not an acronym), from The Joint Commission on Accreditation of Healthcare Organizations (JCAHO), uses outcome scores as part of the accreditation process of nursing homes. The Centers for Medicare and Medicaid Services (CMS; formerly HCFA) has also incorporated outcomes in its Health Care Quality Improvement Program (HCQIP). This program focuses on improving the outcomes of care for Medicare and Medicaid beneficiaries, including those in nursing homes (Gagel, 1995).

These measurement and reporting activities may be important because outcomes information has the potential to improve quality of care (Mukamel & Mushlin, 2001; Shortell et al., 1995). However, we have little information on whether the use of outcomes in nursing homes actually does improve quality of care, and by how much. Nevertheless, we have an urgent need to facilitate and understand outcomes in this setting. Nursing homes are often criticized for their poor quality of care (Institute of Medicine [IOM], 1986). Many of these criticisms are recent and serious (e.g., General Accounting Office [GAO], 1998, 1999a, 1999b, 1999c; IOM, 2001), to the degree that 25% of nursing homes were identified as having quality problems that can either harm residents or place them at risk of death (GAO, 1999a). In this context, studying whether outcomes information can facilitate improvements in quality is important.

In this study, I examine whether providing outcomes information to 120 nursing homes facilitated improvements in quality over a 12-month period. The outcomes information provided to these facilities consisted of six well-accepted indicators of care quality in nursing homes. These were the rates of physical restraint use, urethral catheterization, contractures, pressure ulcers, psychotropic medication use, and certification survey quality of care deficiencies (Day & Klein, 1987; Graber & Sloane, 1995; Gustafson, Sainfort, Van Konigsveld, & Zimmerman, 1990; Holmes, 1996; Mukamel, 1997; Phillips, Hawes, & Fries, 1993; Phillips, Hawes, Mor, Fries, Morris, & Nennstiel, 1996; Spector & Takada, 1991).

Several of the outcomes I examine were addressed as quality issues in the 1987 Nursing Home Reform Amendments (NHRA) of the Omnibus Budget Reconciliation Act (OBRA). As recommended by the IOM report on Improving the Quality of Care in Nursing Homes, one of the goals of these regulatory provisions was to make nursing home care consistent with expert recommendations of care quality (IOM, 1986). A more recent IOM report (IOM, 2001) examining quality of care in nursing homes recommended using outcomes information for quality assurance, including the six I examine. The Medicare–Medicaid annual nursing home certification survey also emphasizes reducing the incidence of the resident outcomes I examine (Holmes, 1996). Thus, the outcomes information I studied has considerable significance to nursing homes.

I propose that professional norms can create pressure for changes in quality. This is the conceptual framework used to explain how the provision of outcomes information could facilitate improvements in nursing home quality. Many organizations that promote and monitor quality stress the public accountability of quality measures (Mukamel & Mushlin, 2001). For example, the National Committee on Quality Assurance (NCQA) states that its mission "is to provide information that enables purchasers and consumers of managed health care to distinguish among plans based on quality, thereby allowing them to make more informed health care purchasing decisions" (www.ncqa.org). Moreover, the NCQA web-page symbol is a bridge between government, health care organizations, employers, and consumers, further exploiting the public accountability dimension of its mission. Presumably, in these cases, market forces provide pressure for changes in quality (Chassin, Hannan, & DeBuono, 1996; Mukamel & Mushlin, 2001). In my investigation, outcome reports were not publicly available. In this case, professional norms probably created pressure for changes in quality, rather than market forces. Recent studies by Frayne and Geringer (2000), Angelelli, Gifford, Shah, and Mor (2001), and Castle and Fogel (2002) support the notion that professional norms can influence outcomes.


    Methods
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Data
Data used in this investigation came from the 1998 and 1999 OSCAR system. The OSCAR data are collected by state licensure and certification agencies as part of the Medicare–Medicaid certification process, and include approximately 17,000 facilities. I primarily used OSCAR data from 1,291 facilities found in six states (Kansas, Maine, Mississippi, New York, Texas, and South Dakota). These states were chosen because they participated in the CMS's Multi-State Case-Mix Demonstration Project, and thus have been of interest to researchers. However, OSCAR data from any state could have been used in this investigation. Only six states were used because I had limited resources for the intervention and I needed to limit the sample.

The OSCAR data contain aggregated resident data and facility data. Resident data elements relevant to this study include the outcomes I examined: the number of residents who are restrained, catheterized, have contractures, have pressure ulcers, or are given psychotropic medications. Facility data relevant to this study include the sixth outcome I examined, the number of certification survey health deficiencies.

The OSCAR data are widely used. First, they are often used by researchers as a secondary source of nursing home characteristics (Castle, 2000; Graber & Sloane, 1996; Harrington & Carrillo, 1999; Harrington, Zimmerman, Karon, Robinson, & Beutel, 2000). Second, the data are publicly available on the CMS web site and many individual state web sites. Third, the data are frequently used by the GAO (GAO, 1998, 1999a, 1999b, 1999c). Thus, the data have considerable face validity, and many items are also considered accurate and reliable (Harrington et al., 2000).

In a survey of 400 nursing homes, I determined that OSCAR data items were highly correlated with directly measured facility characteristics (such as bed size, ownership, and profit status). All items had levels of agreement exceeding 80%, an often-used minimum reliability standard (Porell, Caro, Silva, & Monane, 1998). This prior work suggests that the facility-level OSCAR items have adequate reliability. Some data limitations are evident with the aggregate resident OSCAR data, however.

Most data elements pertaining to resident characteristics are obtained by direct observation by the surveyors, including all of the elements used in this investigation. However, these data are limited because 24-hr observation is not possible. The information the surveyors report is pertinent only for the time they make rounds in the facility, which usually occurs during the day shift. The outcomes I use may be biased because other shifts may not follow day shift practices. One further limitation of the data is that resident care practices may change when an inspection is anticipated. No prior notice is given when surveyors inspect facilities, but inspections are conducted every 9 to 15 months.

Sample Selection and Intervention
The OSCAR data were used to produce individual outcomes reports for 120 nursing homes; 120 facilities were used simply because I had only enough resources to include this number of facilities in the intervention group. Facilities were stratified by state, and then a random sample of approximately 10% of facilities was chosen from each state's pool of eligible facilities. Eligible facilities included 74 nursing homes from Kansas, 23 from Maine, 81 from Mississippi, 465 from New York, 632 from Texas, and 16 nursing homes from South Dakota.

I excluded hospital-based facilities and facilities that were part of a retirement center because they tend to be unrepresentative of other nursing homes in terms of both staff and clients (Singh & Schwab, 2000). Facilities with fewer than 100 beds were also excluded, because with so few residents many of the outcome measures would be either zero or of low prevalence. Facilities excluded from my study represented 47% of nursing homes in the total sample. This may have some impact on the representativeness of my results, but clearly because of these data concerns I would be unable to determine whether outcomes information could improve the quality of care in hospital-based facilities or facilities that were part of a retirement center and smaller facilities, without substantially increasing the sample size of the study. Spector and Mukamel (1998) describe some of the problems in evaluating outcomes in small nursing homes.

I produced individual outcomes reports and mailed them to the administrator of each of these 120 nursing homes during the second quarter of 1998. This mailing also included a letter describing the source of the data and how they were used in the report. Because the OSCAR is a secondary data source, no informed consent was used, although administrators were assured that no individual facilities would be identified in any publications. In addition, my telephone number was given to administrators for any questions or concerns they might have had.

The OSCAR information is lagged by approximately 3–12 months. Early 1998 OSCAR data were used to produce the outcomes report, and early and late 1999 OSCAR data were used for the post-intervention analyses. Early and late 1999 data were used so that the data represented the outcomes approximately 12 months post-intervention.

These outcome reports were approximately 30 pages in length and consisted of several parts (a sample report is available at www.rand.org/health/tools). First, background information on the definition of each measure (restraint use, catheterization, contractures, pressure ulcers, psychotropic medications, and certification survey quality of care deficiencies) and important legislative efforts to reduce the prevalence of these outcomes in nursing homes was provided. This section primarily used materials found in long-term care association journals familiar to many administrators. The NHRA was the legislative effort described in detail. Second, facility information regarding rates of restraint use, catheterization, contractures, pressure ulcers, psychotropic medications, and certification survey quality of care deficiencies was provided. This included tables and figures showing comparisons of the levels of these indicators: (a) for each individual facility compared with the averages of nursing homes in the same zip code, and (b) for each individual facility compared with the averages of nursing homes in the same state. Finally, tables and figures were included showing risk-adjusted comparisons of the levels of these indicators: (a) for each individual facility compared with the averages of nursing homes in the same zip code, and (b) for each individual facility compared with the averages of nursing homes in the same state. This also included a simple description of the method used for risk adjustment. All tables and figures were in color and formatted to fill a full page. In this way they could be more effectively used by the facility (e.g., for posting in visitor or staff lounge areas). In unsolicited calls from administrators, they cited the reports used in total quality management (TQM), board meetings, and loan applications.

The outcomes reports were designed specifically for this project, using principles inherent to TQM. That is, the information provided was useful in changing the care process, was easy for the facility to collect itself, and was simple. All tables used numbers given in percents, and all figures were either bar charts or pie charts. The outcomes reports were pilot tested with four administrators. This involved sending the suggested report to the administrators and following up with face-to-face interviews, typically lasting between 30 min and 1 hr. The feedback resulted in some changes to the text and the suggestion that all figures and tables be formatted to fill a full page. After these revisions were made, the outcomes report was further tested with four different administrators. This resulted in only minor wording revisions to the text.

Quality Measures
Quality indicators are quantitative measures that reflect, at least in part, the quality of care rendered by a health care provider. Incidence and prevalence rates of conditions or treatments are one type of indicator—counts of survey deficiencies are another. The quality indicators available on the OSCAR system are mainly prevalence based, and they are not risk adjusted for residents' characteristics. For that reason they must be interpreted conservatively for any particular facility, or, as in this analysis, risk adjusted. This controls for confounding effects that might be attributed to differences in resident populations.

The IOM lists numerous well-accepted quality indicators that can be used in nursing homes (IOM, 1986). Many of these could have been included in my outcomes reports. My selection of quality indicators was based on their availability in the OSCAR data, previous use by researchers, and that they occurred with relatively high prevalence. The indicators I use are the rates of physical restraint use, urethral catheterization, contractures, pressure ulcers, psychotropic medication use, and certification survey quality of care deficiencies. In all cases the surveyors provide the values for each indicator. In this analysis, I use the count of certification survey quality of care deficiencies. For the other quality indicators, the number of residents with each measure is divided by the total census to give a proportional value. The following sections describe these indicators in greater detail.

Physical Restraints
Use of vests, wrist restraints, ankle restraints, or geri-chairs are included as physical restraints. The documented prevalence rates of physical restraint use vary widely. Recent studies have reported rates of 0% to 59% of residents restrained (Ejaz, Folmar, Kaufmann, Rose, & Goldman, 1994). They are an important quality indicator because they are associated with an increased risk of morbidity and mortality in nursing home residents (Phillips et al., 1993). Lower levels of physical restraint use are generally regarded as beneficial.

Catheters
Rates of indwelling catheter use in nursing homes range from 1.5% to 21% (Warren, Steinberg, Hebel, & Tenney, 1989). Spector and Takada (1991) found that residents in facilities that had moderate to high use of urethral catheterization had twice the probability of functional decline, as did residents in low-use facilities. Assuming some degree of causal connection, high catheterization rates imply lower quality of resident care (Ouslander & Kane, 1984).

Contractures
Contractures are an abnormal shortening and stiffening of muscle tissue that can decrease the range of motion at a joint. This can produce a change in gait and decrease in walking velocity—which are major risk factors for falls—and may also limit mobility in daily life. Contractures are frequently used as proxy measures of care quality as they are effectively postponed and corrected by exercise programs, massage, and physical therapy (Granger, Seltzer, & Fishbein, 1987).

Pressure Ulcers
Pressure ulcers affect both the comfort and the medical outcomes of nursing home residents with impaired mobility. Even though guidelines for the prevention and treatment of pressure ulcers are well established, their prevalence in nursing homes varies widely. Nursing homes with the lowest prevalence of pressure ulcers have rates as low as 3% (Allman, 1989) whereas those with the highest prevalence have rates as high as 21% (Brandeis, Ooi, Hossain, Morris, & Lipsitz, 1994).

Psychotropic Medications
Psychoactive drugs are defined as medications "that affect psychic function, behavior, or experience" (Harrington, Tompkins, Curtis, & Grant, 1992, p. 823). They are generally classified as one of four types of medication: antianxiety, sedative or hypnotic, antipsychotic, or antidepressant. The general concern with these psychoactive drugs is that the rates of use may be excessive or clinically unjustified. In addition, there is also a concern that antidepressants may in some cases be underutilized in nursing homes (Harrington et al., 1992). Because of this more complex relationship with antidepressants, I focus only on the antianxiety, sedative or hypnotic, and antipsychotic psychoactive drugs.

Survey Violations
Finally, nursing home code violations (deficiencies) are departures from federal nursing home standards, as identified by state or federal nursing home inspectors. These are related to many nursing home processes of care, from care quality to fire safety (Harrington & Carrillo, 1999; Harrington et al., 2000). Only quality-related deficiencies are used in this analysis. These include 19 deficiencies ranging from a "facility must provide appropriate treatment and services to maintain or improve resident's abilities in the activities of daily living" to "residents have the right to be free from unnecessary drugs" (GAO, 1999a, p. 13). Nursing home code violations are frequently used as proxy measures of care quality (Mukamel, 1997). Four recent government reports, for example, use code violations as quality measures (GAO, 1998, 1999a, 1999b, 1999c).

Statistical Methods
The data were first used to calculate outcome measures for each facility. With the use of generalized least squares regression and each of the six quality indicators as dependent variables, predicted facility scores were obtained by using aggregate resident variables as independent variables. This approach creates different risk-adjustment models for each outcome.

For each regression model, the resident risk factors used as dependent variables were chosen so as to maximize the adjusted coefficient of multiple determination (adjusted R2). So as not to overspecify the models, the risk factors were developed on data pooled for both years from all states (N = 16,718), but they excluded the five states of interest in this investigation, facilities with fewer than 100 beds, and hospital-based facilities and facilities that were part of a retirement center. Resident risk factors were entered into the models in a stepwise fashion. In all cases the adjusted R2 estimated on the test data were similar (within 5%) to the evaluation data, indicating that when they are applied to the evaluation data, the models retain their predictive power.

No well-accepted risk-adjustment model(s) for assessing resident outcomes in nursing homes exists. Therefore, in preliminary analyses (not shown), I examined other risk-adjustment models. These models included the use of logistic regression analyses and generalized estimating equations. The results from these other risk-adjustment models were all highly similar to those presented here. I also examined the use of state covariates in the models to determine risk adjusters. Four region covariates were included in the certification survey quality of care deficiencies analyses (Midwest, Northeast, South, and West), but the results from these other risk-adjustment models were all highly similar to those presented here.

I examined the correlations between the variables to identify whether the data had any problems of collinearity (not reported). Most of the correlations between risk-adjustment variables were small. The highest correlations were found between the activities of daily living (ADLs) variables, although, based on a threshold of.8, these variables showed no problems of collinearity (Kennedy, 1992). I also ran the analyses by using ADL summary scores (e.g., Cohen & Dubay, 1990; Harrington et al., 2000) rather than with individual ADL variables. The results reported here are similar with the use of both methods. Values for regression tolerance statistics (not reported) showed no problems of multicollinearity (SAS, 1990).

The variables used for risk adjustment are described in the following section and in Table 1. Each of the regression models included slightly different variables, but in all cases they incorporated variables used by others (e.g., Mukamel, 1997). The adjusted R2 values ranged from.12 to.45 (see Appendix 1). These values are consistent with those of other nursing home outcome studies (e.g., Mukamel, 1997; Harrington et al., 2000).


View this table:
[in this window]
[in a new window]
 
Table 1. Operational Definition of Variables.

 

View this table:
[in this window]
[in a new window]
 
Appendix 1. Risk Adjustment Models Used in Analyses: Stepwise Multivariate Regression Models for Physical Restraints, Urethral Catheterization, Pressure Ulcers, Psychoactive Drugs, and Quality of Care Deficiencies.

 
Some state variation in the use of nursing home inspections is known to occur (Day & Klein, 1987). The CMS is trying to remedy this (Harrington et al., 2000), but clearly this could be an analytic concern when multistate data are used. In this analysis I took into consideration this potential geopolitical limitation by calculating quality scores within each facility's state for both intervention and nonintervention nursing homes. This is also prudent because the definition of some of the aggregate resident variables (e.g., pressure ulcer rates) may be dependent on state nursing home policies. This is especially so for the health-related deficiencies. The number of code violations varies from state to state (Day & Klein, 1987). This "state-dependent" approach has been used by others investigating outcomes in nursing homes (Porell & Caro, 1998).

For each dependent variable, the difference between the predicted score from the regression analysis minus the actual observed score for the facility was used as the quality score for purposes of our analyses. In all cases, positive scores indicate a higher level of quality than expected, and negative scores a lower level of quality than expected. For example, a score of 6% for restraint use suggests relatively better care, whereas a value of -6% suggests lower quality for restraint use.

I used this analytic approach for 1998 and 1999 OSCAR data for all facilities receiving outcomes information (n = 120). With the same methods and data, a predicted minus actual observed score was also calculated for other facilities in the state that did not receive outcomes information. This method has been used by others with both nursing home and hospital data, and it is generally referred to as a difference in differences method (Iezzoni, 1994). The average predicted score minus the actual observed score was calculated for each outcome, for both the intervention and nonintervention groups, and t tests were used to compare the significance of the difference in values between the groups (Moore & McCabe, 1993).

Operational Definitions of Variables
The quality indicators used in the analyses are described in detail in Table 1. This table also shows the additional independent variables used to calculate each of the six quality indices. This includes the ADLs of transfer, dressing, eating, toilet use, bathing, and feeding. The numbers of residents dependent in each ADL is used to calculate a proportion. Similarly, the numbers of residents who were bladder incontinent, bowel incontinent, bedfast, or chairfast, and who experienced mental retardation, dementia, psychiatric diagnoses, or depression are used to calculate proportions.


    Results
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Table 2 presents descriptive statistics for the outcome variables used in the analysis, separated by 1998 and 1999 values, and for intervention facilities and other facilities in the five states. The values reported are not unusual and are similar to values reported by other studies using these factors (e.g., Mukamel, 1997; Harrington et al., 2000). With the use of t tests, the 1998 values for the intervention facilities and other facilities in the five states were compared. With the exception of survey quality of care health deficiencies, these were nonsignificant (p <.05). This would be expected because the intervention facilities were randomly chosen from eligible facilities in the five states. The significant difference (p =.047) in survey quality of care health deficiencies cannot be explained other than by random chance.


View this table:
[in this window]
[in a new window]
 
Table 2. Non-Risk-Adjusted Values for Quality Indicators Aggregated From Intervention and Nonintervention Facilities.

 
Table 2 also shows the comparison of the 1999 non-risk-adjusted values for the intervention facilities and other facilities in the five states by using t tests. Physical restraint use, psychotropic medication use, and quality of care deficiency citations were significantly lower in the intervention facilities. However, raw rates of these variables that are not risk adjusted are generally a function of both quality and resident mix (Mukamel, 1997). Thus, the results presented in Table 2 may be confounded with differences in resident mix.

In multivariate analyses, raw rate variables were used, but resident risk factors were also included as controls in the estimating equations. These estimating equations were used for the 1998 and 1999 data. The difference between 1998 predicted scores less actual scores and the 1999 predicted scores less actual scores gives some indication of the change in outcomes while controlling for resident mix. Positive scores indicate improvements in the outcomes, and negative scores indicate worsening outcomes. Table 3 shows these values and compares the results for the intervention facilities and other facilities in the five states by using t tests. This shows that physical restraint use and psychotropic medication use were significantly different. Both had positive values, indicating that physical restraint use was 5.7% lower in the intervention facilities and psychotropic medication use was 5.0% lower in the intervention facilities. Urethral catheterization, contractures, pressure ulcers, and quality of care health deficiencies all decreased in the intervention facilities. However, these improvements in quality were not significantly different (at p <.001) from those in the nonintervention facilities.


View this table:
[in this window]
[in a new window]
 
Table 3. Predicted Minus Actual Values for Quality Indicators Aggregated From Intervention and Nonintervention Facilities.

 

    Discussion
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
The results of this study provide some indication that providing outcomes information to nursing homes may be beneficial. Physical restraint use and psychotropic drug use are areas of care quality that showed significant improvement in nursing homes 1 year after they were provided with outcomes information in these areas. Information for urethral catheterization, contractures, pressure ulcers, and quality of care health deficiencies was also provided; these factors did all positively change over the course of this study, but they were not statistically significant.

Physical restraints are visible measures of quality. During a visit to a facility, the use of vests, belts, mittens, and wrist and ankle restraints can be readily identified. Consumer groups such as the National Citizens Coalition for Nursing Home Reform, professional groups such as the American Nurses Association, and accrediting bodies such as JCAHO have also been particularly successful in sensitizing the public and caregivers about the indiscriminate use of restraints in nursing homes (Kane, Williams, Williams, & Kane, 1993). This may account for the significant results for physical restraint use.

Likewise, a variety of studies have shown that the use of psychotropic drugs varies significantly between nursing homes (Kane et al., 1993). Some of this variation was shown to be due to factors unrelated to resident status and to inappropriate use (Beers, Avorn, Soumerai, Daniel, Sherman, & Salem, 1988; Ray, Federspiel, & Schaffner, 1980). For example, Beers and associates (1988) found that 61% of residents who took antidepressants did not have a recorded diagnosis of depression. One reason for these discrepancies was that, in nursing homes, psychotropic drugs were sometimes used to control disruptive behavior and nocturnal restlessness (Garrard et al., 1991; Ray et al., 1980). Because they were used in this way, these drugs became known as "chemical restraints." As with physical restraints, they are often the subject of concern. Therefore, I was not surprised with the observed decline.

Clearly, these are our speculations as to why we observed these results. I do not have any specific information regarding how these changes came about. I do know, however, that the outcomes reports were used by facilities. In a follow-up survey of administrators, I determined that 86% of facilities used the outcomes report, and 73% found the reports to be extremely useful. Many administrators reported giving the reports to personnel involved with quality improvement; some displayed the results in public areas or areas accessible only to staff. Other uses of the reports were for board meetings, residence councils, and loan applications.

I do propose one further speculation to explain some of our findings. That is, when using quality initiatives (QIs), now common in nursing homes, one area for improvement is often chosen. Our follow-up survey also found this to be the case. Background information and figures such as local and national benchmarks are often used in QIs. This information was included in the outcomes reports. It also may be that physical restraint use and psychotropic drug use are areas of care quality easily amenable or important for QIs; urethral catheterization, contractures, pressure ulcers, and quality of care deficiency citations may be the focus of future initiatives. Residents are often admitted to nursing homes catheterized, with contractures, or with pressure ulcers, making QIs complex in these areas. This of course also highlights one limitation of this study; that is, we examine a rather short and arbitrary period of time. This study also has other weaknesses.

Flood, Shortell, and Scott (1994) have shown that a narrow focus on single quality measures are misleading, and may lead to erroneous, or incomplete, conclusions. By including six such measures, my approach might better capture the overall effect of providing outcomes information. However, other outcome measures are prevalent in nursing homes, including resident satisfaction and other survey deficiencies. These other outcome measures should also be examined. For example, although previous research has used quality of care deficiencies (GAO, 1999a), this measure consists of an unweighted count of only 19 of a total of 185 separate deficiencies.

My reliance on OSCAR data is also a weakness of this study. The data are collected by surveyors and may predispose our analyses to ecological fallacy. Ascertainment bias from surveyors for both the quality measures and risk-adjustment variables may occur, leading to either overadjusted or underadjusted values. For sure, my quality measures are not as valid as those based on individual resident level data. A more refined approach suggested for future analyses would be to use individual assessments of residents available in data such as the Minimum Data Set.

Most of my outcome measures are proportional dependent variables. A substantial body of research using similar proportional dependent variables in the hospital setting has used generalized least squares regression for risk adjustment. Thus, my approach has some precedent. However, other studies in the hospital arena have shown that outcome results can be dependent on the method of risk adjustment (Iezzoni, 1994). Few similar studies exist using nursing home data (c.f., Mukamel & Brower, 1998), but it is likely that my outcome results are somewhat dependent on the method of risk adjustment used (Mukamel & Brower, 1998). It should be noted, however, that a gold standard to risk-adjust nursing home outcomes does not exist.

There is considerable debate over the type of risk adjustment that should be used in long-term care, and the factors that should be included in these models (or indeed if any risk adjustment should be used at all). Spector and Mukamel (1998) discuss numerous alternative risk-adjustment techniques that could be used in analyses such as ours. Preliminary analyses using some of these techniques did not significantly change the results (not reported). Because I cannot discount the possibility that the results are dependent on the risk-adjustment method used, I focus upon the most highly significant results. Even with this conservative approach, the results for physical restraint use and psychotropic medication use are consistently and highly significant—regardless of the risk-adjustment model used.

A further limitation of my approach is that the results for some facilities may only be representative at the two points of time. My analysis does not account for many of the fluctuations, trends, or patterns that could occur from 1998 to 1999 with the variables I use. The use of more data points would provide a more refined test of the effects of providing outcomes information to nursing homes. As Kane, Bell, Riegler, Wilson, and Keeler (1983) discuss, the use of more frequent data points might also be more useful to facilities and regulators. However, it should be noted that Porell and Caro (1998) have shown that examining outcomes information in this way can be problematic with the use of aggregate facility data. These authors identified only "modest" correlations between outcomes scores over time. Moreover, as Mukamel and Mushlin (2001) discuss, facilities are likely to react to the first outcomes report. It should also be noted that because of the randomized design of our intervention, secular trends would have also affected control facilities.

I believe that providing nursing homes with individualized outcomes information may influence the quality of resident care. However, another drawback of this study is that I do not investigate how outcomes information influenced the quality of nursing homes. Also, as some authors have pointed out (Cherry, 1991), in the nursing home setting researchers are often describing less poor care versus poor care, not necessarily good versus poor care.

An important consideration is the strength of the findings identified in this study. Examining the strongest effects found in Table 3, for intervention facilities, I find that the risk-adjusted use of restraints decreased by 5.7% and psychotrophic drug use decreased by 5%. Facilities are likely to weigh the costs and benefits of improving quality of care. The benefits are small, but when one considers the relative inexpense of producing outcomes reports, the cost to benefit ratio is very high. Clearly, this of course also assumes the reports are accurate and the risk adjustment is robust (Mukamel, 1997; Spector & Mukamel, 1998).

In summary, I believe that this study provides evidence that some outcomes initiatives, especially those providing individualized feedback to facilities, currently being pursued in the long-term care arena will positively affect quality of care. My results may be important for nursing homes and regulators trying to increase quality.


    Footnotes
 
1 RAND, Pittsburg, PA. Back

Decision Editor: Laurence G. Branch, PhD

Received for publication November 16, 2001. Accepted for publication September 6, 2002.


    References
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 




This article has been cited by other articles:


Home page
ptjournalHome page
L. Resnik, D. Liu, D. L Hart, and V. Mor
Benchmarking Physical Therapy Clinic Performance: Statistical Methods to Enhance Internal Validity When Using Observational Data
Physical Therapy, September 1, 2008; 88(9): 1078 - 1087.
[Abstract] [Full Text] [PDF]


Home page
GerontologistHome page
R. L. Kane, G. Arling, C. Mueller, R. Held, and V. Cooke
A Quality-Based Payment Strategy for Nursing Home Care in Minnesota
Gerontologist, February 1, 2007; 47(1): 108 - 115.
[Abstract] [Full Text] [PDF]


Home page
GerontologistHome page
J. A. Lucas, T. Avi-Itzhak, J. P. Robinson, C. G. Morris, M. J. Koren, and S. C. Reinhard
Continuous Quality Improvement as an Innovation: Which Nursing Facilities Adopt It?
Gerontologist, February 1, 2005; 45(1): 68 - 77.
[Abstract] [Full Text] [PDF]


Home page
Journal of Applied GerontologyHome page
N. G. Castle, T. J. Lowe, J. A. Lucas, J. P. Robinson, and S. Crystal
Use of Resident Satisfaction Surveys in New Jersey Nursing Homes and Assisted Living Facilities
Journal of Applied Gerontology, June 1, 2004; 23(2): 156 - 171.
[Abstract] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Services
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
PubMed
Right arrow PubMed Citation


HOME ARCHIVE SEARCH TABLE OF CONTENTS