| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
Correspondence: Address correspondence to Dana B. Mukamel, PhD, Department of Community and Preventive Medicine, University of Rochester Medical Center, 601 Elmwood Avenue, P.O. Box 644, Rochester, NY 14642. E-mail: dana_mukamel{at}urmc.rochester.edu
 |
Abstract |
|---|
 TOP Abstract Determinants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
Key Words: Quality of care • Competition • Quality indicators • Quality report cards • Risk-adjusted outcomes
Concerns about inadequate quality provided to nursing home residents have been discussed in the lay media and the professional literature for decades (Mendelson, 1974; Vladeck, 1980). In 1986, the Institute of Medicine (IOM; 1986) published its landmark report that called for major revisions in the way nursing home quality is monitored. It recommended the continuation of the existing system to periodically monitor quality through a survey process with deficiency citations. The report shifted the emphasis, however, to the assessment of care for the individual resident rather than a focus on policies and procedures, and recommended the strengthening of enforcement mechanisms. The report called for more emphasis on quality of life, as well as quality of care, and encouraged the use of outcome indicators to assess quality. The importance of risk adjustment of quality indicators was also emphasized.
The implementation of the IOM recommendations began in 1987 with the passage of the Nursing Home Reform Amendment to the Omnibus Budget Reconciliation Act (OBRA), which mandated a new system of standards of care, including increased minimum staffing regulations and quality of care monitoring (Harrington & Carrillo, 1999). (OBRA 87 requires nursing homes to provide "sufficient nurse staffing to attain or maintain the highest practicable ... well-being of each resident ..." For the first time, nursing facilities nationwide were required to have a registered nurse 8 hours per day and 24-hour licensed nurse coverage per day.) These efforts culminated in the 1998 nationwide implementation of the Resident Assessment Instrument (RAI) system, which is the cornerstone of the Centers for Medicare and Medicaid Services (CMS) Health Care Quality Improvement Program for nursing homes. The RAI is designed to improve quality by requiring nursing homes to develop individual care plans, which are to be reassessed periodically or whenever specific trigger events occur. It also includes protocols for follow-up care. The Minimum Data Set (MDS), a component of the RAI, is used to collect information about patients' physical and mental health status, as well as specific treatments, at regular time intervals. Because of the MDS, for the first time, we have access to patient-level clinical data that are collected with the same instrument for all nursing home residents around the country. (Facilities that do not wish to receive either Medicare or Medicaid payments are not required to collect MDS data.) Such data provide an opportunity to compare risk-adjusted health outcomes of nursing home residents across facilities and thus allow assessments of aspects of quality of care that were not feasible beforehand.
Despite these continuing efforts during the last decade, quality problems in nursing homes persist. Recent U.S. General Accounting Office reports find significant deficiencies in one fourth of nursing homes (U.S. General Accounting Office, 1999) and question the ability of state surveyors to detect serious care problems (U.S. General Accounting Office, 1998). Harrington and Carrillo (1999) report that whereas the number of deficiency citations for nursing homes have declined, many quality problems persisted over the 1991–1997 period. The medical literature also continues to identify many clinical problems that persist in nursing homes: malnutrition (Abbasi & Rudman, 1994; Crogan, Shultz, Adams, & Massey, 2001), dehydration (Kayser-Jones, Schell, Porter, Barbaccia, & Shaw, 1999), medication errors (Gurwitz et al., 2000), and pressure ulcer care (Spector & Fortinsky, 1998), among others.
Most recently, interest has been developing in addressing quality of nursing home care through the public dissemination of "quality report cards"—reports that provide information about various aspects of care (e.g., number of deficiency citations, rates of pressure ulcers) at individual facilities, often times in a manner that allows comparison to a peer group of nursing homes (e.g., all other facilities in the state). The medicare.gov/Nhcompare/home.asp Web site is being expanded to include quality measures for nursing homes based on outcomes from a pilot study in six states. This trend follows two decades of publishing quality report cards with information about managed care plans, hospitals, and physicians.
The preference of the Bush administration for the "quality report card" approach has recently been emphasized in a New York Times report (Pear, 2002) quoting the administration's response to a study of inadequate staffing levels in nursing homes to "... instead of imposing new rules..." [i.e., staffing standard], "... to publish data on the number of workers at each nursing home in the hope that staffing levels may simply increase due to the market demand created by an informed public."
This history and current policy trends beg two related questions: Why have we been unsuccessful in the past in ensuring appropriate quality of care in nursing homes? Is the current strategy to improve quality through the provision of quality report cards likely to make much of a difference?
The shortcomings of the regulatory surveillance system have been described and investigated previously (Walshe, 2001) and will not be addressed here. In this study, we focus on the latter question—the role that quality report cards can play in improving quality of nursing home care. We borrow from the economic literature to offer an explanation of the processes that influence the provision of nursing home quality and the potential contribution of quality report cards. We also review the empirical evidence in the health services literature to gain insights about the effectiveness of report cards and some of the issues that may influence it. We describe the content of existing nursing home report cards and conclude with a discussion of the important attributes of report cards.
|
Determinants of Nursing Home Quality of Care and the Role of Report Cards |
|---|
|
TOPAbstractDeterminants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
Role of Quality Report Cards
The demand for nursing home care reveals consumers' preferences for quality only if they are able to evaluate it. Unless consumers know that one nursing home offers better care than its competitors, they will be unwilling to pay more for its care. It is, however, very difficult for consumers to obtain comparative information about quality of competing nursing homes, a difficulty that is often compounded by the need to choose a nursing home under stressful conditions while trying to balance both patient and caregiver preferences. Quality report cards are intended to reduce substantially the cost of quality comparisons for consumers.
Assessment of quality of care can prove to be very difficult for most consumers for several reasons. First, quality is a multidimensional construct. Nursing home care has many aspects, any of which can be offered at different levels of quality. The most basic dichotomy is between hotel and medical aspects of the care nursing homes provide. Within each of these, there are numerous specific dimensions. Hotel quality can vary along dimensions such as the condition of the physical facility, the quality of the food, and the friendliness of the staff. The quality of the clinical care encompasses dimensions such as the quality of rehabilitative services for short-stay postacute patients, prevention of pressure ulcers, and treatment of urinary incontinence for long-term residents and, when appropriate, prevention of death. Furthermore, within clinical care, there are aspects of care that are part of the caring process that are important to consumers but difficult to measure, for example, how sensitive a clinician is to patient preferences (Feinstein, 2002). A testament to the complexity of the quality of nursing home care is the extent of the federal regulations survey requirements, which include 185 items covering both hotel and medical aspects of care. This multidimensionality makes it difficult for patients and their families to evaluate comprehensively the quality of the care provided by nursing homes.
Second, a meaningful assessment of quality requires that consumers compare performance across the facilities available to them in their area. This may entail substantial efforts and time commitment: it may require visits to several facilities, spending time in each, making observations and gathering information.
Third, some aspects of quality, in particular those that are related to the quality of medical care, require sophisticated knowledge and access to information that is not readily available to consumers. Assessment of quality of clinical care can be based on structure, process, or outcome measures (Donabedian, 1980). (Structure measures of quality relate to the physical and the organizational structure of the home and include, for example, staffing ratios, whether or not there is a full time medical director, and the training programs for personnel. Process measures relate to the specific processes applied to the patients, such as whether or not residents receive flu shots or whether bed-bound residents are turned at sufficiently close intervals to prevent development of skin ulcers. Outcome measures relate to the health [physical and mental] of residents, which can be influenced by the care [structure and process] provided to them.) Structure information is the easiest for consumers to obtain through questions to the facility's administrator or to other staff. Assessing it, however, is not as straightforward as the assessment of hotel quality—it requires some understanding of the needs of patients and how specific components of structure may be of importance to them. Process information may be less readily available and just as difficult to interpret. For example, a nursing home may have a policy to immunize its residents for influenza (a structure measure), but the policy may not be implemented or not implemented fully (a process measure), such that not all residents receive the required immunization. To ascertain the immunization status of residents, one would have to review medical charts and evaluate the appropriateness of cases in which immunization was not provided (because of counterindications), tasks that are beyond the scope of most consumers.
The most difficult to observe and evaluate are health outcomes. Health outcomes of individuals, such as mortality or decline in functional status, depend not only on quality of care, but also on patient risks and luck (Spector & Mukamel, 1998). Therefore, direct observation of health outcomes cannot be used to draw inferences about quality. For example, an individual may be more prone to develop skin ulcers because of his or her comorbidities, and may in fact develop them despite the best of care. Observing a large number of residents with skin problems, which is the type of information consumers may glean from a walk through the facility, cannot be interpreted to imply poor quality. To infer quality from outcomes one would need statistics about risk-adjusted outcomes for a large number of residents, such that risk-adjusted rates could be compared across facilities (DeLong et al., 1997; Mukamel, 1997; Spector & Mukamel, 1998). For example, Krumholz and colleagues (2002) suggest that hospital quality reports for acute myocardial infarction mortality do not discriminate well between any two hospitals because of limitations in the number of cases per hospital.
The complexity of the task facing consumers in assessing quality of nursing home care is demonstrated by the recommendations published by the American Association of Retired Persons (AARP; 2001) in a guide for choosing nursing homes. The AARP recommends a walk through the facility to assess things like cleanliness and grooming of residents. They also recommend assessing activities to determine if residents are being kept busy and stimulated; if residents unable to feed themselves are being fed and if the food quality is adequate; if residents are in restraints; what the staffing patterns are during the week, weekends, and at night; what staff turnover is; and attitude and training of staff. They also recommend getting information from the long-term care ombudsmen about complaints, reading the results of the Federal and State surveys; and talking with hospital social workers, medical doctors, family, and friends of people who live in the nursing home, as well as employees, to get opinions and information about the facility. These recommendations clearly demonstrate the difficulties that consumers face in performing such a complex assessment on their own. Most consumers are, therefore, forced to make choices based on incomplete and possibly inaccurate information.
Quality report cards, if they provide accurate information about many aspects of quality that are of importance to consumers in ways that they can understand (e.g., explaining the importance of influenza shots) and in a manner that allows comparisons across facilities, can greatly enhance consumers' ability to assess quality and make informed choices. They lower the cost of assessing quality by making information about it readily available. As a result, demand for high-quality care will better reflect consumers' preferences and will provide stronger incentives for nursing facilities to invest in high quality and provide quality at levels closer to the perfect competition optimum. Indeed, evidence from the airline industry shows that, following the reporting of on-time performance, demand increased and quality increased (Foreman & Shea, 1999). Evidence from the education sector suggests that the reporting of school rankings based on student test scores are associated with higher housing values in neighborhoods with better ranked schools (Figlio & Lucas, 2000).
Empirical Evidence on the Effectiveness of Hospital, Physician, and Health Plan Quality Report Cards
The evidence of the effectiveness of report cards used for evaluating the quality of hospitals, physicians, and health plans is mixed (Mukamel & Mushlin, 2001). (As there are no studies evaluating the impact of nursing home report cards, we review studies of report cards for other medical care providers.) To some extent, this may be because many of these studies represent assessments of early attempts at the development and dissemination of report cards. One might expect that with time consumers will become more knowledgeable about report cards and more facile with interpreting and using the information. Furthermore, many of the papers that evaluated the impact of report cards had methodological limitations. Consequently, studies may find a stronger effect when report cards are more established and methods for evaluating their impact are improved. Nevertheless, many of these early studies have positive findings.
The first study of quality report cards by Vladeck, Goodwin, Myers, and Sinisi (1988) found no significant increases in hospital discharges in New York City after publication of the Medicare hospital mortality reports. A later analysis by Mennemeyer, Morrisey, and Howard (1997) found small increases in admissions that were significantly associated with better reported risk-adjusted mortality rates (RAMR) in some, but not all, analyses. Chernew and Scanlon (1998; Scanlon & Chernew, 1999), studying employer-provided reports about health maintenance organizations (HMOs), found little and sometimes counterintuitive evidence on the effect of the reports on choice of health plan. Chernew, Gowrisankaran, and Scanlon (2001), in a more recent study, found a significant effect in the expected direction of an HMO report card on plan choice. Mukamel and Mushlin (1998) found that market shares of cardiac bypass graft (CABG) surgeons with reported low RAMR increased after publication of the information. Furthermore, availability of information about RAMRs was associated with a decline in the importance of traditional quality signals, such as price and surgeon's years of experience, suggesting that explicit information about quality made available in report cards is preferred by consumers (Mukamel, Weimer, Zwanziger, Mushlin, & Huang-Gorthy, 2002).
For the most part, these studies found a rather small effect. For example, Mukamel and Mushlin (1998) estimated that hospitals with RAMR that is 1 percentage point above the average (i.e., an RAMR of 4.65% instead of 3.65%) experienced a decline of 1.8% in CABG surgeries after publication of the report cards. The similar figure for surgeons was 7%. These studies suggest that the effect of report cards on consumers' choice may not be large. This may be partially because information about relative quality of providers was available in health care markets even before publication of the reports, through quality signals and informal channels. Thus, the value added by report cards may only be small, resulting in only small shifts in patient referrals.
These findings lead to the question of whether report cards change consumers' choices sufficiently to result in incentives for providers to offer high-quality care. There is less evidence to offer insights on this issue. A study of the publication of report cards on obstetrics services in Missouri hospitals (Longo et al., 1997) has found that approximately 50% of hospitals that did not have car seat programs, formal transfer agreements, or nurse educators for breast-feeding before the report, either instituted or planned to institute such services. This trend was stronger in more competitive markets. They also found that clinical outcomes improved. Studies of the New York State Cardiac Surgery Reports by Hannan, Kilburn, Racz, Shields, and Chassin (1994) and by Peterson, DeLong, Jollis, Muhlbaier, and Mark (1998) also suggest an overall improvement in quality levels. Peterson and colleagues (1998) found a much faster decline in RAMR of CABG patients in New York State (10.5% per year), compared with other states (5.8%) that they suggest might be attributable to the report cards. Anecdotal evidence about changes in practice styles and curtailment of privileges for high RAMR surgeons by hospitals also support the potential beneficial effect of these report cards (Gormley & Weimer, 1999). However, these findings have been controversial, with some suggesting that they may reflect changes in patient selection (i.e., turning away the sickest patients because of providers' concerns about their "ratings"), rather than improvement in average quality levels (Dranove, Kessler, McClellan, & Satterthwaite, 2002). Thus, the evidence that is available to us at this time on the effectiveness of quality report cards in improving market average quality levels seems equivocal.
The Potential Effectiveness of Report Cards in Nursing Home Markets
The effectiveness of report cards in nursing home markets may be more limited than in other segments of the health care system for two reasons: lack of competition and dominance of public payers.
Many nursing home markets are not competitive. In many areas, the number of facilities is small; as a result, nursing homes in such markets have significant market power. Furthermore, for residents who seek facilities with a particular religious or cultural affiliation, options might be limited even if there are many facilities in their geographic area. Market power typically translates into higher prices and lower quality. Mukamel and Spector (2002) estimated that nursing homes in some markets in New York State in 1991 were able to charge prices of 40% and more above the competitive price level (i.e., marginal costs). Nyman (1994) similarly found substantial market power and price markups. Although we are not aware of similar evidence in terms of quality, economic theory suggests that, in noncompetitive markets, nursing homes will provide lower levels of quality than they would in competitive markets. Nursing homes in these types of markets will not provide the competitive allocation of quality even if consumers had complete information and demand fully reflected preferences for quality. In such markets, quality report cards, although still providing positive incentives for quality and therefore having a positive impact, would fail to induce the level of quality that a perfectly competitive market with full information would. We should note, however, that there is some evidence to suggest that price markups are not always indicative of limited competition. Troyer (2002) shows that, in one quarter of Florida nursing homes, some of the markup may reflect the cross-subsidization of Medicaid residents by private pay residents because of the need to cover losses associated with the care of Medicaid residents when Medicaid payments are below costs. Nevertheless, Mukamel and Spector (2002) find that, in New York, Medicaid payment rates exceed marginal costs. Thus, price markups in this state are likely to reflect market power and limited competition.
The second factor that may limit the effectiveness of report cards is the dominance of nursing home revenues by public payers, primarily Medicaid (Rhoades & Sommers, 2000). Whereas the demand for quality by private pay patients can be expected to reveal their valuation for different levels of quality through their willingness to pay, the demand of Medicaid (and Medicare) patients does not. Medicaid rates are set by the state and do not vary by quality. Thus, Medicaid patients will seek the highest quality facility that would admit them. Under excess demand conditions, which prevailed in many areas for many years (Scanlon, 1980; Nyman, 1993), Medicaid patients typically have to accept placement in facilities when beds become available. Their preferences for quality, therefore, do not play a role. In markets without excess demand, Medicaid patients can shop for high quality. They thus provide incentives to nursing homes to compete on quality and to provide the level of quality with marginal costs equal to the Medicaid payment rate. Empirical evidence suggests that higher Medicaid reimbursement indeed is associated with increased staffing and higher quality only in markets without excess demand (Cohen & Spector, 1996; Grabowski, 2001; Nyman, 1985). In these markets, competition for Medicaid patients results in quality levels that are pegged to the Medicaid reimbursement rate. Because Medicaid rates are often determined by the political processes in each state and are subject to many influences besides concerns about quality of care, and in particular the need of Medicaid programs to keep their expenditures low, they are not likely to ensure high and uniform levels of quality across the country. Low Medicaid reimbursement rates limit the ability of nursing homes to attract adequately trained staff at levels that would afford high quality, as well as other quality-enhancing investments.
|
What Information Do Existing Nursing Home Report Cards Provide? |
|---|
|
TOPAbstractDeterminants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
CMS publishes the "Nursing Home Compare" on the Medicare.gov website. The site provides information about the number of beds and type of ownership of Medicare and Medicaid certified facilities, case-mix information (e.g., percentage of residents with pressure ulcers, percentage of residents with urinary incontinence), staffing, and results from the most recent state survey, including deficiency citations. Many states also use the web to publish report cards. Most include information about deficiencies; some also include information about complaints and enforcement action. Generally, the sites provide text to caution consumers on how to use the information. Recently, this report has been expanded to include additional quality indicators based on patient outcomes, some of which are risk adjusted.
The information about quality in different dimensions is presented differently in different report cards. One important aspect of information presentation is the level of aggregation at which data are presented because the relative importance of these dimensions varies for individual consumers depending on their preferences and health status. For example, the performance of a nursing home with respect to skin care may be of much more concern to residents with diabetes or those who are bed bound, whereas for residents with dementia, a program of mental stimulation for people with cognitive impairments may be more important. On the other hand, consumers find it difficult to deal with information about a very large number of quality dimensions, as we discuss herein. Most report cards do some categorizing of the different dimensions of care. In the Medicare Compare site, total number of deficiencies are compared with national and state averages per facility. These data do not weight deficiencies, thus assuming that all are of equal importance. Individual citations are also listed, allowing consumers to place their own preference weights. Massachusetts and New Jersey provide, in addition to information about individual citations, scores for five domains—administration, nursing, resident rights, food service, and environment—and a total score weighted by the severity of the deficiency.
Two states, Maryland and Texas, publish 32 quality indicators, some of which are stratified by risk group (the MDS-based indicators developed by the Center for Health Systems Research and Analysis [Zimmerman et al., 1995], but neither state publishes measures based on a comprehensive risk adjustment. To our knowledge, only LTCQI, a private company, and CMS publish report cards that include a more elaborate risk adjustment.
|
What Information Should Report Cards Include? |
|---|
|
TOPAbstractDeterminants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
Complex report cards with information about many attributes of care may, however, be difficult for consumers to process and effectively integrate into their decision-making process (Hibbard, Slovic, & Jewett, 1997). Experience with report cards about health plans, such as the Health Plan Employer Data and Information Set (HEDIS) report, which has more than 20 different quality indicators, suggests that consumers find it difficult to deal with complex information. An experiment in which consumers were asked about their preferences for health plan attributes and then were allowed to choose among plans with those attributes showed a discrepancy between professed preferences and actual choices (Hibbard & Jewett, 1996). This discrepancy can be attributed to the fact that making choices requires consumers to grapple with potentially difficult tradeoffs. For example, a nursing home that excels in terms of the facility and the environment may not be at the top of the list in terms of treating skin ulcers and functional decline. The consumer, for whom all these aspects of care are important, is forced to tradeoff one attribute for another.
It would have been easier for consumers, and would allow for less complex report cards, if quality indicators were highly correlated—e.g., if facilities that excelled in preventing functional decline were also the best in treating skin ulcers and had the best hotel services. If such correlations existed, report cards could provide summary statistics about a few indicators and assure consumers that they provide a comprehensive assessment of the care each nursing home provides. Empirical evidence, however, suggests that quality of different dimensions of care is often not or only minimally correlated. Mukamel and Brower (1998) found that, after risk adjustment, functional status, skin ulcer, and physical restraint outcomes were not significantly correlated. Mukamel and Spector (2000) report low correlations between risk-adjusted measures based on functional status, skin ulcer, and mortality. Porell and Caro (1998) found similarly no or only minimal correlations between nine measures. The correlations between nursing home expenditures related to hotel services and to clinical services are also generally small and often not significant (author calculations from the New York State residential health care facility data that reports annual expenditures for each nursing home by cost center). To the degree that higher expenditures enable higher quality, these data suggest that hotel and clinical quality are not highly correlated.
Some report cards attempt to simplify the information for consumers by presenting summary scores of quality indicators. Given the lack of correlation between individual dimensions of quality, such summary scores may be misleading. For example, adding up different nursing care-related deficiencies into one score assumes either that all the components are highly correlated (in which case the summary score only repeats the information about the individual categories), an assumption that based on the evidence we cite previously is not very likely true, or that all categories are of equal importance to the consumer. This latter assumption is also not likely, because people differ in their health status and preferences and would therefore place different importance on different aspects of the care. Providing such a summary score, especially if the information by category underlying the score is not included in the report as well, forces consumers into predetermined tradeoffs that may not be in line with their preferences.
Nursing home report cards, therefore, should be comprehensive and include many quality indicators. This in turns implies that consumers are likely to need help in incorporating the information into their decision-making process. Therefore, when designing report cards, efforts should be made not only to ensure their comprehensiveness and accuracy, but also to make them user-friendly and useful. The lessons learned from studies of the HEDIS (Gibbs, Sangl, & Burrus, 1996; Hibbard, Harris-Kojetin, Mullin, Lubalin, & Garfinkel, 2000) and consumer assessment of health plans (Goldstein & Fyock, 2001; McGee, Kanouse, Sofaer, Hargraves, Hoy, & Kleimann, 1999; Smith, Gerteis, Downey, Lewy, & Edgman-Levitan, 2001) report cards to determine optimal ways to present report card information should be incorporated into the design of nursing home report cards.
Quality Indicators
The examples we discuss show that several different types of quality indicators are currently in use in report cards. Each has been recognized to have limitations that impact on its validity and accuracy.
The most common quality indicators are those based on deficiency citations, which are primarily structure- and process-type measures. To the degree that specific structural arrangements and processes have been shown to improve outcomes or reflect professional consensus, these measures have face validity. They are, however, limited in two important ways. First, they identify only facilities that have not met a minimum standard. They do not provide information about those facilities that do meet the minimum standards, but who may still exhibit substantial variation in quality above the standard. Second, the reliability of deficiency citations has been questioned. Deficiency citations are generated by surveyors who inspect nursing homes. Survey practices, and particularly the threshold for issuing a citation, vary by state and even within states across regional offices that have responsibility for performing the survey (Mukamel, 1997; Spector & Drugovich, 1989). Thus the interrater and interregional reliability of these measures is questionable. Comparisons to state and national averages that are offered in some report cards may be misleading. This problem is mitigated to some degree by the fact that most consumers are interested in comparing facilities in close geographical proximity. Such facilities are likely to have been inspected by the same team of surveyors and thus intraregional comparisons may be reliable.
Some nursing home report cards include staffing information—typically hours of registered nurses, licensed practical nurses, and aides per resident day. Although the reports do not explicitly label these as quality indicators, it can be expected that the lay consumer would interpret them to imply that "more is better." Such conclusions would, however, be misleading because the association with improved outcomes has been demonstrated only for case-mix adjusted staffing level (Cohen & Spector, 1996). The same level of staff per day for a facility with a more debilitated resident population is likely to be associated with worse outcomes than its competitor with the same staffing level, but less debilitated population. Absent an adjustment for the different care requirements of residents (their case-mix), information about staffing is likely to be misleading.
Quality indicators based on health outcomes, such as rates of decline in functional status or prevalence rates of pressure ulcers and urinary incontinence, are gaining increasingly more attention. As discussed previously, health outcomes depend not only on processes of care (i.e., quality), but also on patient risks and luck (Spector & Mukamel, 1998). Therefore, inferring quality from data on health outcomes should be based on risk-adjusted rates calculated over a sufficiently large number of residents. Much research has been done to investigate the accuracy of outcome-based measures of quality over the last decade. These measures have been shown to be sensitive to the inclusion of specific risk factors (Iezzoni, Shwartz, Ash, Hughes, Daley, & Mackiernan, 1996; Mukamel & Brower, 1998). Using different risk factors can change the rank ordering of facilities, as well as designation as quality outliers. Rankings and outlier designation are also sensitive to the way the relationship between risks and processes of care are specified in calculating outcome-based measures, even when the risk factors are accurate. Mukamel, Dick, and Spector (2000) have shown that defining quality indicators as either the ratio of observed to expected rates or the difference between them leads to different rankings and outliers. Ranking and outlier designation are also sensitive to the number of patients used in the calculation of the measures (Localio, Hamory, Fisher, & TenHave, 1997). Samples of about 100, which are the typical size for a nursing home, may be too small to allow for sufficiently accurate estimates (Spector & Mukamel, 1998). Samples may even be smaller if quality indicators focus on subpopulations, such as rehabilitation outcomes for patients who had hip fractures. Although the trend toward chain ownership creates an opportunity to assess quality at the chain level, increasing the sample sizes from which indicators are derived, quality may not be homogeneous across all facilities within a chain. Consequently, there will always be a need for assessment at the facility level.
The concerns with the potential bias in quality measures, and in particular measures based on risk-adjusted health outcomes, have been discussed extensively in the literature (Chassin, Hannan, & DeBuono, 1996; Kassirer, 1993, 1994). The risks of publishing inaccurate and misleading measures are several. First, measures that lack credibility may result in backlash from the industry as the Health Care Financing Administration (HCFA) experience with publication of the Medicare hospital mortality reports has taught us. After several years of report publication, the HCFA stopped their publication because of lobbying by the hospital industry. Second, inaccurate measures will direct consumers to providers whose true quality is lower than what they were led to believe. Thus "true" average quality, as distinct from measured and published quality, will decline as more patients will receive care from lower quality providers. Third, measures that do not sufficiently account for differences in patient risks or case-mix across facilities will provide perverse incentives for nursing homes to avoid admissions of high-risk patients. If, for example, report cards include risk-adjusted measures based on decline in functional status and if the risk adjustment is biased such that some patient risks associated with decline are not accounted for, facilities may prefer to avoid admitting such patients. Facilities may fear that their "risk-adjusted functional decline rates"—i.e., the measure of their performance—will show them as providing poor quality care when in fact their high functional decline rates are because they treat a more debilitated patient population. Indeed, some of the criticism of the New York State Cardiac Surgery Reports centered around the perverse incentives they may provide surgeons to refuse treating the sickest patients (Green & Wintfeld, 1995).
|
Conclusions |
|---|
|
TOPAbstractDeterminants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
To increase the potential benefit from nursing home report cards, their development and public dissemination should take advantage of the lessons learned from two decades of publishing report cards for hospitals, physicians, and health plans; other industries; research on quality measurement; and research on how to design user-friendly reports. We should, however, be cognizant of the methodological difficulties in accurate measurement of quality and the potential for adverse effects on the market. Further research and development are needed to develop accurate and unbiased quality measures, especially those based on outcomes, to assess how consumers use report cards and to identify formats for presenting the information that would allow consumers to understand them and use them in their decision making.
|
Footnotes |
|---|
1 Department of Community and Preventive Medicine, University of Rochester Medical Center, Rochester, NY.
2 Agency for Healthcare Research and Quality, Rockville, MD.
Decision Editor: Laurence G. Branch,, PhD
Received for publication May 14, 2002. Accepted for publication September 17, 2002.
|
References |
|---|
|
TOPAbstractDeterminants of Nursing Home...What Information Do Existing...What Information Should Report...ConclusionsReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  E. A. Miller and V. Mor Balancing Regulatory Controls and Incentives: Toward Smarter and More Transparent Oversight in Long-Term Care Journal of Health Politics Policy and Law, January 1, 2008; 33(2): 249 - 279. [Abstract] [PDF]
|
|
|
|
 |
|
 D. C. Grabowski, A. J. O'Malley, and N. R. Barhydt The Costs And Potential Savings Associated With Nursing Home Hospitalizations Health Aff., November 1, 2007; 26(6): 1753 - 1761. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. G. Castle, J. Engberg, and A. Men Nursing Home Staff Turnover: Impact on Nursing Home Compare Quality Measures Gerontologist, October 1, 2007; 47(5): 650 - 661. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. A. Kash, C. Hawes, and C. D. Phillips Comparing Staffing Levels in the Online Survey Certification and Reporting (OSCAR) System With the Medicaid Cost Report Data: Are Differences Systematic? Gerontologist, August 1, 2007; 47(4): 480 - 489. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. B. Mukamel, W. D. Spector, J. S. Zinn, L. Huang, D. L. Weimer, and A. Dozier Nursing Homes' Response to the Nursing Home Compare Report Card J. Gerontol. B. Psychol. Sci. Soc. Sci., July 1, 2007; 62(4): S218 - S225. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. G Castle, J. Engberg, and D. Liu Have Nursing Home Compare quality measure scores changed over time in response to competition? Qual. Saf. Health Care, June 1, 2007; 16(3): 185 - 191. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Tsoukalas, C. Rudder, R. J. Mollot, M. Shineman, H. Y. Lee, and C. Harrington The Collection and Use of Funds From Civil Money Penalties and Fines From Nursing Homes Gerontologist, December 1, 2006; 46(6): 759 - 771. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. G. Stevenson Is a public reporting approach appropriate for nursing home care? Journal of Health Politics Policy and Law, August 1, 2006; 31(4): 773 - 810. [Abstract] [PDF]
|
|
|
|
 |
|
 C. Donoghue and N. G. Castle Voluntary and Involuntary Nursing Home Staff Turnover Research on Aging, July 1, 2006; 28(4): 454 - 472. [Abstract] [PDF]
|
|
|
|
|
|
D. B. Mukamel, H. Temkin-Greener, R. Delavan, D. R. Peterson, D. Gross, S. Kunitz, and T. F. Williams Team Performance and Risk-Adjusted Health Outcomes in the Program of All-Inclusive Care for the Elderly (PACE) Gerontologist, April 1, 2006; 46(2): 227 - 237. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Zinn, W. Spector, L. Hsieh, and D. B. Mukamel Do Trends in the Reporting of Quality Measures on the Nursing Home Compare Web Site Differ by Nursing Home Characteristics? Gerontologist, December 1, 2005; 45(6): 720 - 730. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. G. Castle Nursing Home Administrators' Opinions of the Nursing Home Compare Web Site Gerontologist, June 1, 2005; 45(3): 299 - 308. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. G. Castle and T. J. Lowe Report Cards and Nursing Homes Gerontologist, February 1, 2005; 45(1): 48 - 67. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Harrington, J. O'Meara, M. Kitchener, L. P. Simon, and J. F. Schnelle Designing a Report Card for Nursing Facilities: What Information Is Needed and Why Gerontologist, April 1, 2003; 43(90002): 47 - 57. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
