| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
Correspondence: Address correspondence to Nicholas G. Castle, PhD, University of Pittsburgh, Graduate School of Public Health, 130 DeSoto Street, Pittsburgh, PA 15261. E-mail: CastleN{at}Pitt.edu
 |
Abstract |
|---|
 TOP Abstract Report Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
Key Words: Quality • Report cards • Nursing home compare
Report cards are generally defined as quality-of-care information that is standardized and released to the public (Kroll Letwat, 1999; Scanlon, Chernew, Sheffler, & Fendrick, 1998). One of the earliest examples of a report card was the publication by the CMS of death rates for all U.S. hospitals in 1986 (Mennemeyer, Morrisey, & Howard, 1997). A second example report card is the Cardiac Surgery Reporting System developed by the New York State Department of Health. Beginning in 1989, this report published hospital and surgeon-specific mortality rates for coronary artery bypass graft surgery (Mukamel & Mushlin, 1998). Probably the most well-known report card is HEDIS—the Health Plan Employer Data Information Set. HEDIS includes quality measures on satisfaction, access, utilization, and financial indicators collected from health-plan members (Gibbs, Sangl, & Burrus, 1996). Because it was originally developed as a tool for large purchasers of health care, the NCQA does not make HEDIS information available to the public. However, employers may share measures of specific plans with their employees, and some states release the measures to the public (e.g., Minnesota, at http://www.health.state.mn.us/divs/hpsc/mcs/hedishome.htm).
As Relman (1988, p.1222) suggests, "the era of Assessment and Accountability is dawning at last." Report cards are becoming pervasive. For example, it would appear that several major health legislative initiatives proposed by the federal government in recent times have advocated the use of report cards. As reported by Wicks and associates (1999), the Health Equity and Access Reform Today Act (HR 3704/S 1770), the Managed Competition Act (HR 3222/S 1579), Affordable Health Care Now Act (HR 3080/S 1533), and the Consumer Choice Health Security Act (HR 3698/S 1743) have all advocated the use of report cards.
Report cards would seem especially relevant for potential nursing home consumers. Quality concerns have been particularly pronounced in nursing homes, where incidences of resident abuse, fraud, and understaffing have received considerable attention in the public press and government reports, and where frail elderly persons are particularly vulnerable (Institute of Medicine [IOM], 1986). In addition, unlike the information available for health plans through Quality Compass (http://hprc.ncqa.org/index.asp), or published in news media (e.g., The New York Times and Consumer Reports), quality and performance data on nursing homes are often difficult to obtain and interpret (Mukamel & Spector, 2003). Therefore, it is not surprising that, in November of 2002, the CMS released the report card titled Nursing Home Compare, or NHC (http://www.medicare.gov/NHCompare/home.asp), with its creation of a set of core quality measures for the nursing home industry based on the computerized Minimum Data Set (MDS). In recent years, many state government agencies responsible for elder care or nursing homes have also produced report cards on nursing homes. These are the subject of the research we present.
Very little is known about nursing home report cards and the impact they are having on consumers and providers. With the exception of a recent U.S. General Accounting Office (U.S. GAO; 2002) report describing the federal NHC initiative and two recent descriptive articles (Harrington, O'Meara, Kitchener, Simon, & Schnelle, 2003; Mukamel & Spector, 2003), we could not identify any research on nursing home report cards. Thus, despite the unquestionable need for report cards on nursing home care, some preliminary research in this area is clearly warranted. In this article, we focus on state initiatives. First we describe which states have produced nursing home report cards; second, we compare what information is provided in these report cards; third, we identify data sources used for the information presented; fourth, we examine seven factors previously shown to be associated with the usefulness of report-card information and provide suggestions for improving current state report cards.
Before we discuss our research on state-government-generated nursing home report cards, we present a brief overview of the potential benefits and the potential limitations of report cards. Most of this information on benefits and limitations does not come from nursing home research; therefore, we also discuss specific challenges report cards have to overcome if they are to be useful to the consumer in choosing a nursing home.
|
Report Cards: Benefits, Limitations, and Challenges |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
In order to make informed choices about nursing home care decisions, consumers are in need of information on quality that they can understand and find useful. This is important because, as Mukamel and Mushlin describe, "public dissemination of accurate information about quality restores the imbalance between quality and costs in the ‘value’ equation" (2001, p. 21). Providing quality information to consumers will enable them to choose a provider that best suits their needs.
Once an elder is admitted to a facility, family members can use report-card information to monitor facility performance over time. In a focus group study we conducted with families of nursing home and assisted living residents, participants identified the need for ongoing information to judge whether a facility was in the process of improving itself.
Report cards may educate nursing home care purchasers about the quality of care provided in a nursing home (Davies, Washington, & Bindman, 2002; Kroll Letwat, 1999). The long-term-care environment presents a complex array of terms, procedures, and cues that may prove both confusing and frustrating to those unfamiliar with it (Harrington, 1991). Report cards could assist in educating and sensitizing both residents and their families to those areas of resident care and life that are most important to clinical personnel and regulators. Report cards may also provide the opportunity for dialogue among residents, families, and professionals as to what constitutes quality care.
Quality improvement and increasing market efficiency are other potential benefits of consumer report cards. Several studies of hospitals and cardiac surgeons have concluded that performance-rating systems alter the behavior of providers and improve quality (e.g., Hannan, Kilburn, Racz, Shields, & Chassin, 1994; Longo et al., 1997), especially when providers are presented with data comparing themselves to their colleagues (Keller, Chapin, & Soule, 1990). Report cards have also influenced health care markets by influencing HMO-plan selection by employees (Scanlon & Chernew, 1999), HMO-contracting behaviors with organizations (Mukamel, Mushlin, et al., 2000), and patient referrals in fee-for-service plans (Mukamel & Mushlin, 2001).
Potential Limitations
Despite the growing number of report cards in use, they are potentially subject to several limitations. These potential limitations include a confusing array of information, difficulties in accurately reporting quality information, and unexpected provider reactions.
There is wide variation in the types of information provided to consumers. Some of these differences can be attributed to the particular needs of regulators, provider organizations, and consumer groups involved in report-card development. Some of these differences can also be attributed to the wide array of quality measures that could be used in report cards. For consumers, however, the wide variation in the types of information provided may make it more difficult for them to make trade-offs in their priorities (Harris-Kojetin, McCormack, Jael, Sangl, & Garfinkel, 2001).
Some impediments to the usefulness of report cards exist because of difficulties in accurately reporting quality information. For example, quality indicators based on health outcomes, without risk adjustment, have been criticized as potentially inaccurate and misleading (Mukamel & Brower, 1998). However, risk adjustment itself is not without limitations. Large differences in outcomes can result when specific risk factors are or are not included (Iezzoni, 1994) and when different methodologies are used for risk adjustment (Iezzoni et al., 1996). Report cards also have the nontrivial task of reporting to consumers why and how risk adjustment was used. Further, sample size has been found to effect the rank ordering of providers (Localio, Hamory, Fisher, & TenHave, 1997). Mukamel, Dick, and Spector (2000) have also found differences in rank ordering when different definitions of quality measures are used (e.g., the ratio of observed rates to expected rates or the difference between observed and expected rates).
Using mortality rates as a measure of quality, Hofer and Hayward (1996) found that the measures could not accurately distinguish between high-quality and low-quality hospitals. Such findings serve to illustrate the potential harm to consumers and providers when inaccurate and misleading data are published (Berwick & Wald, 1990; Hofer et al., 1999; Mukamel & Spector, 2003; U.S. GAO, 1994), and they have led many authors to the conclusion that the "science" of quality measurement used in report cards is not well developed (Berwick & Wald, 1990; Hofer et al., 1999).
Providers may not necessarily react to report cards by improving quality of care. As Green and Wintfeld (1995, p. 1230) report, "some surgeons have declined to operate on severely ill patients for fear that to do so could have lowered their standing in the mortality report" (although this was refuted by Hannan et al., 1994). Reports also exist of hospitals' turning away patients to maintain their quality statistics (Green & Wintfeld, 1995; Marshall, Sheklle, Leatherman, and Brook (2000) believe providers may be responsive to report cards, but they also cite several studies identifying the failure of providers to use report-card information.
Challenges
Clearly, as the previous sections describe, the use and usefulness of report cards is influenced by many factors. Some of these factors represent especially salient challenges for nursing home report cards, such as promoting market efficiency, providers' reactions to potential loss of clients, a confusing array of quality information, and difficulties in accurately reporting quality information. Several additional challenges are also present in the nursing home sector, including issues with defining who the nursing home consumer is and how the choice of provider is made.
As the IOM (1986) report has pointed out, nursing homes do not operate within an efficient market system. The nursing home market characteristically operates with limited competition between facilities, excess demand (most notably observed in facility waiting lists), dominance by public payers (the majority of which is Medicaid), and, frequently, the need for consumers to select a facility in an emergency (Castle, 2003; Mukamel & Spector, 2003). As some authors have pointed out, nursing home occupancy rates have declined in recent years (Bishop, 1999), and the number of alternative long-term-care providers, such as assisted living, has increased. However, it is still likely that the best nursing homes are operating at maximum capacity, and, despite competition from other providers, excess demand exists in many markets (Mukamel & Spector, 2003). Given these conditions, whether nursing home report cards will have any beneficial effects in the nursing home market is open to question.
Given high occupancy rates and excess demand, nursing homes may not respond to report cards because they may not perceive any loss of business (Harrington et al., 2003; Mukamel & Spector, 2003). Other reasons nursing homes may not respond (or be able to respond) to nursing home report cards readily come to mind, including understaffing and underfunding in many facilities. However, little research has been conducted on changes in nursing home behavior as the result of making performance measures publicly available.
We also know that nursing home quality is a multidimensional construct, with a wide array of available quality measures (Mukamel & Spector, 2003). The many dimensions of quality in long-term care were recently discussed by Zimmerman and Bowers (2000), including well-known taxonomies such as structure–process–outcome and major quality domains such as medical–clinical status, functional status, and general well-being. The wide array of available quality measures is evident in many quality initiatives. For example, in developing the MDS Quality Indicators (QIs), the team at the Center for Health Systems Research and Analysis (CHSRA) began with 175 indicators (Zimmerman, 2003). As part of the federal nursing home certification process, 185 indicators (deficiency citations) are used. In developing a satisfaction survey, Ejaz, Straker, Fox, and Swami (2003) identified 87 items of potential importance. Moreover, as noted by Mor and associates (2003, p. 41), "there is only a low level of correlation among the various measures of quality." Clearly, given this context, identifying a concise set of quality measures to include in a nursing home report card represents a considerable challenge.
As we have already described, the accuracy of quality measures may be an issue for report cards. Limitations of risk adjustment of nursing home quality measures may be especially problematic in this regard. Mukamel and Brower (1998) demonstrated the sensitivity of risk-adjusted measures using nursing home data. Spector and Mukamel (1998) expand on the difficulties involved in comparing nursing facility outcomes, and they have noted that the mean size of nursing homes nationally (approximately 100 beds) may be too small for accurate comparisons between facilities to be made—a conclusion echoed in at least one text on health statistics (McDowell & Newell, 1996).
We have limited information on how consumers choose nursing homes. Castle (2003) recently reviewed this literature and examined the choice processes used by approximately 600 residents and family members. Both this literature review and empirical results show that residents are not very influential in the process of choosing a nursing home. The consumer is most likely to be a family member. This prior work also shows that the choice of nursing home is seldom made in a proactive way by either residents or family members (Castle, 2003); rather, the choice of nursing home is most often made within a few days of hospitalization of the elder. This represents a considerable challenge for nursing home report cards, and at the same time makes them potentially extremely important. Because many consumers are placed in the situation of needing nursing home information quickly, it is especially important that report cards be accessible and understandable. These same time constraints are known to limit decision makers' ability to visit facilities (Castle, 2003), and they may also mean that report cards are the only source of information used to make a choice.
|
Methods |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
Some information was found quickly at some state Web sites. In other cases the search would best be described as laborious, as nursing home information was difficult to find. Because of the possibility of missing pertinent information at some of the more difficult Web sites to navigate, we visited some sites three times. Specifically, for states identified in our first search as having report cards, we visited the site once; for states with links to NHC but no report cards of their own, we visited the state Web site twice; and for states with no links to NHC and no nursing home report card, we visited the Web site three times.
The Web site did not have to define the nursing home information presented as a report card. Rather, if any information was presented that (a) listed quality-of-care information and (b) was standardized for all state nursing facilities, we classified the information as a report card. Most definitions of report cards also include the criterion that the information should be released to the public (Kroll Letwat, 1999; Scanlon et al., 1998). Because we examined sites on the World Wide Web, clearly this was also the case for the report cards we examined. As part of this process, we did not identify specific quality measures to be included in our search. This was because of the large number of quality measures available for nursing homes and the ongoing debate as to whether some quality measures actually measure quality. We more simply operationalized quality-of-care information to be any measure that could be used as a quality signal. In practice, most quality-of-care information came from accreditation or complaint surveys, MDS QIs, or MDS Quality Measures (QMs).
The research team that searched these Web sites and provided the evaluation of specific characteristics of the state reporting systems (described in more detail in the paragraphs that follow) included a health-services researcher with experience in long-term-care policy and practice, and a geriatric social worker and nursing home administrator with broad experience working with nursing home and assisted living residents, families, and state regulatory agencies. Both evaluators have advanced degrees in gerontology or health care policy, and, for one researcher, an advanced degree in education and communication sciences.
|
Analyses |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
|
Presentation Format
We also examine how the information is presented. First, the rating scale is described. The information may be presented as text, a number, a series of stars, or Consumer Reports-type ratings (circles with various degrees of shading). The information may be available for only one facility at a time, or comparisons between facilities may be incorporated into the report card. For example, all other facilities in the county may be listed. We examine the use of benchmarks that allow the consumer to compare scores between facilities and a reference group. Benchmarks are most often given as state or national averages. We also describe whether the report card presents the information graphically, or not.
Utility Measures
Rather than just present descriptive characteristics of these report cards, we also evaluated their potential usefulness. Harris-Kojetin and associates (2001) produced seven lessons that they believed were important fundamentals associated with the utility of report cards. These lessons were developed for health-plan choices, but if they are modified slightly for nursing home report cards, they: (a) are clear and easy to use; (b) address diversity among the target audience; (c) help consumers understand key fundamentals; (d) assist consumers to determine and differentiate among their preferences; (e) minimize cognitive complexity; (f) help consumers understand how and why to use quality information; and (g) present the material in short, manageable segments (Harris-Kojetin et al., 2001). We examine whether nursing home report cards have taken these lessons into account—although we caution that this evaluation is based on the opinions of one research team, and not on the opinions of the actual report-card designers or users.
To address whether the information is clear and easy to use, we determine whether early in the report card the structure and intent of the materials are stated. A second factor we examine is whether the materials are clearly explained all through the report. We also assess the use of navigational aids that can reduce the complexity of going through the report card. Navigational aids are defined by Harris-Kojetin and associates (2001, p. 455) as "instructions, section overviews, or graphics." Tumlinson, Bottigheimer, Mahoney, Stone, and Hendricks (1997) believe clarity is important in getting consumers to use report cards.
For the diversity among the target audience to be addressed, layering of information is recommended. Layering is a process whereby more detailed materials can be examined, if needed, but disinterested or less able consumers do not necessarily have to examine these materials. It is important to note that several studies have shown the need of some consumers for highly detailed information on facilities or services (e.g., Mukamel, Mushlin et al., 2000; U.S. GAO, 2002). A greater level of detail provides consumers with the ability to weigh different aspects of the quality data according to their own needs and preferences (Mukamel & Spector, 2003).
To address whether the information helps consumers understand key fundamentals, we determine whether basic information concerning the measures used are presented. This includes whether definitions of measures and definitions of data manipulation (e.g., risk adjustment) are presented.
To address whether the information assists consumers to determine and differentiate among their preferences, we examine whether the report cards present self-tests to help with decision making, or if trade-offs are mentioned at all in the text. This was considered important by Harris-Kojetin and colleagues (2001) because they identified that consumers often found it difficult to make trade-offs, or weights, in their priorities. A similar observation was made by Mukamel and Spector (2003) and Hibbard and Jewett (1996).
To address whether the information minimizes cognitive complexity, we determine if the report cards break the choice task into a series of smaller steps. This comes from decision theory, whereby information retention is believed to be limited. Smaller steps may improve decisions.
To address whether the information helps consumers understand how and why to use quality information, we examine if the report cards explain how the QMs can affect residents. Several QMs may be unfamiliar to decision makers. Therefore, this information is considered useful for many consumers.
Harris-Kojetin and associates (2001) recommend presenting the information in short, manageable sections on each page. This includes the judicious use of blank space and large fonts so that the information is not overwhelming. The presentation format of information may be especially important for elders, some of whom may have limited eyesight. Therefore, we extended this concept to include whether state report cards have incorporated a basic design to assist the disabled consumer—a condition required by Section 508 of the U.S. Rehabilitation Act (see http://www.section508.gov/). We do this by indicating whether the report card has received a Bobby Approved rating. This approval rating is given by the Watchfire Corporation (http://www.watchfire.com/solutions/accessibility.asp), which provides tools and recommendations to make Web sites user friendly for the disabled.
|
Results |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
As we show in Table 2, the information presented in these report cards differs quite substantially across states. Some state report cards report few quality data elements (e.g., AZ has 1 and CO, MS, NY, and PA have 2); some use more (e.g., MD gives 27 and OH gives 34). The data sources for report cards do not differ substantially. Most state report cards use data derived from the annual licensure and recertification inspection (deficiencies), and a few others use MDS data. Only a few state report cards (OH and VT) use primary data (satisfaction of residents or family) along with these other data sources.
|
|
|
|
Discussion |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
Descriptive Information Given in Report Cards
Almost all report cards included some facility information, such as location and number of beds. However, this is one area where a few report cards provided numerous potentially useful pieces of information for consumers making a nursing facility choice (e.g., IL, OH, and MD). For example, the Ohio report card provides a listing of facility services (such as special care units) and some background information on facility management. The Ohio report card also includes a map link, which could be very helpful to consumers. Several report cards provide cost information (FL, IL, and OH).
The California Advocates for Nursing Home Reform report card includes a comprehensive section of financial indicators (e.g., daily rate, wages for nursing assistants, and net operating income), including bankruptcy. No strong association between bankruptcy and closure has yet been identified, but with the spate of facility bankruptcies in recent years and the possible reduction in services caused by limited cash flows, this measure may be highly useful for consumers (Castle, in press).
Quality Information Given in Report Cards
Deficiency Citations
Most state report cards rely to some degree on deficiency citations. All nursing homes accepting Medicare–Medicaid residents must be certified each year, and minimum standards were established for this certification (42 CFR 483, Subpart B). If facilities do not meet these standards, deficiency citations are issued. Thus, a readily available and frequently updated data source for nursing home information exists. The intent of the survey and certification process is also simple: to monitor and ensure quality of care. This clearly makes deficiency citations attractive for use in report cards. However, although the intent of certification is simple, the process itself is complex and error prone. As an important quality signal, one desirable property of these citations would be low levels of variability and high reliability. Numerous reports indicate that this may not be the case (Nursing Home Initiative, 2000); the survey process and resulting deficiency citations are believed to be highly variable, and in some cases unreliable (Grabowski & Castle, 2004; U.S. GAO, 1999). For purposes of consumer choice, unreliability may be the greatest liability of deficiency citation use. Variability of deficiency citations is greatest between states (Grabowski & Castle, 2004; Mukamel & Spector, 2003), so this is likely less problematic for consumers who are known to often select facilities primarily from their county of residence (Gertler, 1989).
The data used to identify deficiency citations come from the Medicare and Medicaid certification data. This data source gives facility characteristics and aggregate resident characteristics of most nursing homes in the United States (see Harrington, Zimmerman, Karon, Robinson, & Beutel, 2000, for a description). It is worth noting that, in addition to deficiency citations, these data contain information that could be used as QIs. This includes the number of residents with physical restraint use, contractures, pressure ulcers, and catheter use. However, to be appropriately used as QIs, these data require risk adjustment, and, as with the deficiency citations, are likely to be error prone. Nonetheless, no state report cards presented this quality information. The data also contain information on the types of residents found in a facility (case mix), such as the number of residents with mental health problems. State report cards also did not provide resident information from this data, although the California Advocates for Nursing Home Reform, California Health Care Foundation, and Nursing Home Reports report cards do use this information.
Resident Quality Measures
The resident quality measures used by states are all based on the MDS. The MDS contains over 350 data elements in 17 main areas of care (such as mood and skin condition). The MDS data are variously used for determining Medicare payments, Medicaid payments, and quality improvement.
Several state report cards used the QIs created from the MDS data (MD, OH, RI, and TX). A description of the QIs and background to their development is provided on the CHSRA Web site (http://www.CHSRA.wisc.edu) and in several publications (Zimmerman, 2003; Zimmerman et al., 1995). In short, the QIs were developed for use by surveyors as indicators of potential quality problems, and 24 are used for this purpose. However, for use in report cards, two potential limitations have been identified: (a) the QIs were not developed for public reporting; (b) they only identify potential quality problems.
To address these potential limitations, in developing the California Health Care Foundation report card, a validation study was used to identify which QIs distinguished between good and bad facility care (Harrington et al., 2002). Weight loss, being in bed all or most of the time, and use of physical restraints were shown to be reliable for this purpose. Four other QIs are reported by the California Health Care Foundation report card to inform consumers about other common quality concerns: presence of pressure sores, problems controlling bowel or bladder functions, loss of functioning, and depression.
In developing NHC, the CMS examined both the QIs and other indicators to identify valid and reliable measures for use in public reporting. This development is described in a U.S. GAO report (2002). The CMS selected 10 measures for use in NHC—these are called the core QMs. Six measures are for long-stay residents (decline in activities of daily living; pressure ulcers, without facility-level adjustment; pressure ulcers, with facility-level adjustment; inadequate pain management; physical restraint use; and infections) and four measures are for short-stay residents (failure to improve and manage delirium, without facility-level adjustment; failure to improve and manage delirium, with facility-level adjustment; inadequate pain management; and improvement in walking). The QIs, loss of ability in activities of daily living, pressure sores, and physical-restraint use are included in both NHC and the California Nursing Home Search report card. These QIs are also part of the original 24 developed by CHSRA.
State report cards have attempted to address the potential limitations of QIs in two ways. First, some state report cards only report the NHC QMs (OH and MD). Second, other state report cards report all of the QIs (MD and TX). The Maryland report card reports 24 QIs and groups them into four domains (clinical, psychosocial, medications, and functional). The Texas report card reports 32 QIs drawn from the enhanced Resource Utilization Groups III quarterly instrument. It should be noted that both the Maryland and Texas report cards caution users on the limitations of the QIs.
Satisfaction Measures
Two state report cards (OH and VT) and one private report card in MI included information on resident or family satisfaction. Research on consumer preferences has demonstrated that consumers value ratings of quality and satisfaction by people like themselves (Hibbard & Jewett, 1995; Sofaer, 1997). Therefore, the inclusion of resident and family satisfaction data when available may be worthwhile. Ejaz and colleagues (2003) report on the OH initiative. A recent policy analysis by Lowe, Lucas, Castle, Robinson, and Crystal (2003) has shown a growing awareness and interest by several other states in developing and reporting satisfaction measures for use by long-term-care consumers. However, these authors also describe some of the limitations and costs involved in undertaking such initiatives.
Presentation of Report-Card Information
Comparing Facilities
We found that several state report cards only provided ratings for individually chosen facilities (CO, FL, MD, and NY). Clearly, this does not preclude comparisons between facilities, as the information-gathering process can be repeated for each facility of interest; however, for consumers this process can be tedious. We found the least user-friendly case of this to be the Arizona report card, where facility characteristics (names and location) were actually needed to search for information.
Benchmarks
Many report cards do not give benchmark information. Thus, the consumer has no way of knowing the relative quality standing of the facilities selected. For the most part, report cards giving benchmark information use state averages. Texas and Wisconsin give county averages (TX also gives ZIP code averages). As has been previously noted, consumers are known to often select facilities from their county of residence (Gertler, 1989). Thus, these latter benchmarks may prove more realistic and useful for consumers. We suggest the inclusion of tools to assist the consumer in comparing nursing homes to each other and to county and state averages. The Texas and Ohio report cards provide excellent examples of this. Several private report cards also provide historical benchmarks (e.g., Nursing Home Reports and HealthGrades), which may be particularly useful for consumers who are concerned with past performance.
Utility of Information Presented
Clear or Easy to Use
Some state report cards present a great deal of information describing deficiency citations. This may not be productive, and it makes the report cards difficult to use. An emphasis on describing deficiency citations may also be indicative of a belief that consumers may not fully understand deficiency citations. This, of course, further weakens their utility in report cards—although it should be noted that the opposite situation occurs in several report cards. Some report cards provide very little (or no) information on deficiency citations (e.g., NV).
Report cards using MDS QMs were structured much more clearly. For example, the Maryland report card provides detailed information on how to use the quality ratings, how they were created, and why the measures included are important.
Assists in Determining Preferences
Few report cards assisted consumers in determining preferences, yet most report cards did direct consumers to supplemental literature on "how to choose a nursing facility." This approach may help consumers determine their preferences. However, using two steps rather than one may be confusing for many consumers. The time available for choosing a facility may also be limited (Castle, 2003). Thus, for some consumers, creating an extra step may limit the usefulness of report cards.
Short Manageable Segments
The desirable properties of nursing home report cards include presenting the information in short, manageable portions with a basic design to assist the disabled consumer. Only four report cards indicated receiving the Bobby Approved rating. Although some report cards (e.g., FL and MD) appeared to have incorporated many of the recommendations noted by Harris-Kojetin and associates (2001) and by the Watchfire Corporation (e.g., large fonts and contrasting colors), the majority of report cards were especially difficult to read and included little or no help to direct the disabled consumer. We suggest that state report cards more carefully consider the readability of the Web pages for the disabled.
Study Limitations
Our investigation is clearly limited in that it is based on the opinions of one research team. Information on what consumers think of these report cards should be collected and analyzed—although this may not be as simple as it sounds. It is unclear who the consumer is in making nursing home choices. In many cases it is not the elder but a family member making the choice. In other cases, health care professionals are influential in this decision (Castle, 2003).
We used dichotomous scales (yes–no) to assess the utility of the report cards in seven areas. In some cases an analogue scale (such as 1–10) would have provided a greater level of detail in rating the report cards. For example, most report cards give navigation aids, but several were certainly more comprehensive in how they did this than others.
We also found that some of our utility measures were limited. For example, our evaluation of whether a report card indicated if it was Bobby Approved failed to take into consideration state Web sites that had implemented some of the recommendations provided by advocacy groups for the disabled, but that did not qualify for full Bobby Approved status. It is also possible that some state Web sites had applied for Bobby Approved status and were undergoing the review process.
An issue that we do not address is how much information should be provided in a report card. The current thinking, derived from health-plan choices, is that brevity is advantageous (Harris-Kojetin et al., 2001). The validity of this same approach has to be investigated for elders making nursing home choices. Health-plan consumers are typically young and are typically making decisions that probably will affect them very little, relative to elders choosing a nursing home. Elders in many cases are making nursing home decisions that may affect their care, health, location, and finances for the rest of their lives. With the exception of Maryland and Ohio, we found all other state report cards to be short. Private report cards were in general much longer (e.g., the California Health Care Foundation report card has approximately 100 items). It may be that many elders would want to examine all of these data elements.
Conclusions
One unexpected result of our research was the significant amount of effort required to find some state nursing home report cards within the state Web site. Perhaps this is an oversight on the part of the organizations responsible for publication of the information, but such conditions limit the usefulness of the information by creating access barriers for the very consumers who need the information.
It is clear that state report cards are produced for quite different purposes. First, some report cards include numerous quality measures, primarily using MDS data, and are clearly intended to inform consumers so that they can make nursing home choices. This includes report cards produced by Maryland, Ohio, Rhode Island, and Vermont. Second, other state report cards are clearly intended to inform consumers only of deficiency citations. This information may not be sufficient to fully inform consumers to make nursing home choices. These state report cards include Arizona, Colorado, Iowa, and Mississippi.
Recent reports (American Association of Retired Persons, 2001; Kaiser Family Foundation, 2002) support the notion that few elders are adequately able to judge the medical quality of nursing facility care. However, every year approximately 800,000 residents enter a nursing facility for the first time (Spence & Wiener, 1990), creating a situation whereby consumers may choose inappropriate facilities because of this disconnect between the information available to make an informed choice and the ability of consumers to make that choice. Most state report cards rely on reporting the medical quality of nursing facility care by using the MDS. Although important, MDS information may not be most relevant to consumers. Recent work on quality of life (Degenholtz, Kane, & Kivnick, 1997) and satisfaction (Ejaz et al., 2003; Lowe et al., 2003) is clearly driven by the need to encompass more than just clinical measures in the evaluation of nursing homes, and this should be incorporated into future report cards.
It is clear that current state nursing home report cards have taken several different routes in addressing the needs of consumers. When the effects of these different reporting efforts on consumer choice are studied, these different report cards may reveal what information is most useful for consumers and how this information is best presented. Some of this information could be used by NHC, for example. The NHC report card was touted as expanding "the availability of public information on nursing homes and the quality of care provided" (U.S. GAO, 2002, p. 1). However, concerns levied against NHC include the lack of a consumer-friendly format, the counterintuitive nature of some QM reporting, the lack of data precision, and confusion over how to interpret missing data (U.S. GAO, 2002).
Information from current report-card initiatives on what information is most useful for consumers and how best to present this information could also be used by future state report cards. Some authors believe that, rather than replace state report cards, NHC has stimulated more activity by states in this area (Sprague, 2003). Our examination of current state reporting efforts appears to confirm this belief.
In summary, many prior report-card initiatives have received considerable criticism. Much of this has focused on whether consumers would use the information. It may be worth questioning whether nursing home consumers will use, or are able to use, currently available report-card information (U.S. GAO, 2002). We identified 19 states as reporting nursing home quality on the World Wide Web. This, of course, is in addition to the federal NHC initiative and several private sites. With the exception of very specific conditions in a limited number of states (such as the Cardiac Surgery Reporting System in New York), there may now be more quality information available to consumers on nursing homes than for any other health provider. Even with the limitations of MDS and deficiency data, this may be due, in part, to the availability of these uniform data sources.
Report-card advocates believe that encouraging consumer choice will lead to improvements in quality of care. For these improvements to occur, consumers must use and providers must believe report-card information is being used to select nursing facilities. However, providing information in report cards does not necessarily guarantee their use, nor does it necessarily improve decision making. Evaluating and improving current nursing home report cards is therefore an important objective.
|
Footnotes |
|---|
1 RAND, Pittsburgh, Pennsylvania.
2 Institute for Health, Health Care Policy, and Aging Research, Rutgers, the State University of New Jersey, New Brunswick.
Decision Editor: Linda S. Noelker, PhD
Received for publication March 17, 2003. Accepted for publication October 28, 2003.
|
References |
|---|
|
TOPAbstractReport Cards: Benefits,...MethodsAnalysesResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  C. D. Phillips, M. Chen, and M. Sherman To What Degree Does Provider Performance Affect a Quality Indicator? The Case of Nursing Homes and ADL Change Gerontologist, June 1, 2008; 48(3): 330 - 337. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. S. Groenewoud, MSc, N. J. A. van Exel, MSc, M. Berg, PhD, and R. Huijsman, PhD Building Quality Report Cards for Geriatric Care in The Netherlands: Using Concept Mapping to Identify the Appropriate "Building Blocks" From the Consumer's Perspective Gerontologist, February 1, 2008; 48(1): 79 - 92. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. A. Miller and V. Mor Balancing Regulatory Controls and Incentives: Toward Smarter and More Transparent Oversight in Long-Term Care Journal of Health Politics Policy and Law, January 1, 2008; 33(2): 249 - 279. [Abstract] [PDF]
|
|
|
|
 |
|
 N. G Castle, J. Engberg, and D. Liu Have Nursing Home Compare quality measure scores changed over time in response to competition? Qual. Saf. Health Care, June 1, 2007; 16(3): 185 - 191. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. G. Castle and J. B. Engberg Nursing Home Deficiency Citations for Medication Use Journal of Applied Gerontology, April 1, 2007; 26(2): 208 - 232. [Abstract] [PDF]
|
|
|
|
|
|
R. L. Kane, G. Arling, C. Mueller, R. Held, and V. Cooke A Quality-Based Payment Strategy for Nursing Home Care in Minnesota Gerontologist, February 1, 2007; 47(1): 108 - 115. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Tsoukalas, C. Rudder, R. J. Mollot, M. Shineman, H. Y. Lee, and C. Harrington The Collection and Use of Funds From Civil Money Penalties and Fines From Nursing Homes Gerontologist, December 1, 2006; 46(6): 759 - 771. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. G. Castle and K. E. Sonon Internet Resources and Searching for a Residential Care Setting: What Information Is Available for Consumers? Journal of Applied Gerontology, June 1, 2006; 25(3): 214 - 233. [Abstract] [PDF]
|
|
|
|
|
|
N. G. Castle Nursing Home Administrators' Opinions of the Nursing Home Compare Web Site Gerontologist, June 1, 2005; 45(3): 299 - 308. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
