This Article Abstract Full Text (PDF) Services Download to citation manager Citing Articles Citing Articles via HighWire PubMed PubMed Citation

Home Care Quality Indicators (HCQIs) Based on the MDS-HC

Correspondence: Address correspondence to John P. Hirdes, PhD, Department of Health Studies and Gerontology, University of Waterloo, Waterloo, Ontario, Canada N2L, 3G1. E-mail: hirdes{at}uwaterloo.ca

	Abstract

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
Purpose: This study aimed to develop home care quality indicators (HCQIs) to be used by a variety of audiences including consumers, agencies, regulators, and policy makers to support evidence-based decision making related to the quality of home care services. Design and Methods: Data from 3,041 Canadian and 11,252 U.S. home care clients assessed with the Minimum Data Set—Home Care (MDS-HC) were used to evaluate a series of indicators suggested by international experts and by focus groups conducted in Canada and the United States. Risk adjustment methods were derived and validated using data from Ontario and Michigan. Results: Of the 73 original candidate HCQIs, 22 were retained for the final list of recommended indicators. All but three indicators include risk adjusters based on individual-level covariates. An agency-level risk adjustment was developed to correct for selection and ascertainment bias. Implications: The HCQIs are new tools providing a first step along the path of quality improvement for home care. These indicators can provide high-quality evidence on performance at the agency level and on a regional basis.

Key Words: Inter-RAI • MDS–HC • Quality of care • Home care • Risk adjustment

This article aims to provide an overview of a multinational effort to develop home care quality indicators (HCQIs) and associated risk adjusters for the Minimum Data Set—Home Care (MDS-HC), which was developed by an international research network known as interRAI. These indicators can be used to support a variety of quality applications including internal quality management initiatives, accreditation, and public report cards. The use of comprehensive assessment data from the MDS-HC has considerable promise for providing more meaningful information about the quality of home care agencies than would be available with standard health and sociodemographic measures (e.g., diagnosis, age, length of service); however, this potential is contingent on the ability to create benchmarking systems that are reasonable reflections of true underlying differences in quality. Mor, Berg, Angelelli, Gifford, Morris, & Moore (2003b) note that the state of knowledge regarding quality indicators is in its infancy, and many technical and conceptual issues remain to be addressed. Arguably, home care represents one of the most challenging contexts for translating this type of research into practice in real world settings.

	Performance Measurement for Home Care in the United States and Canada

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
In the United States, the 1997 Balanced Budget Act called for the establishment of a prospective payment system for home health services. In accordance with this requirement, as of July 1999, all U.S. home health agencies participating in the Medicare program or in certain states' Medicaid programs have been required to perform assessments using the Outcome Assessment Information Set (OASIS). The OASIS instrument measures patient outcomes and collects data on sociodemographic, environmental, support system, health status, and functional attributes of patients. Home care agencies are mandated to collect these data on all adult, nonmaternity patients at the time of referral into the program, at follow-up assessments every 60 days, and at discharge (Department of Health and Human Services, 1999).

OASIS is used primarily in two ways: to case-mix adjust prospective payment rates so as to reflect the differences in services required by patients with different diagnoses and severity and for developing outcome-based quality indicators for home care services (Shaughnessy, Crisler, Schlenker, & Arnold, 1997). Outcome measures are derived by comparing one or more assessments for a client and calculating rates of improvement or stabilization in areas of functional, physiologic, emotional/behavioral, and cognitive health status as well as rate of hospitalization. Previous analysis has shown that all but 2 of the 38 outcome measures derived from OASIS have acceptable inter-rater reliability (Shaughnessy et al., 2002b). At the time of writing, the U.S. government is mandating that the OASIS be reduced to include only those items that support these two uses: outcomes and case-mix measurement.

In Canada, the Canadian Institute for Health Information (CIHI) has been developing a series of performance indicators to assist provinces with planning, quality management, and public reporting for home care services (CIHI, 2001). Some of these proposed indicators are outcome based (e.g., summary scores combining activities of daily living [ADL] and instrumental ADL [IADL] impairments over time, disruptive behaviors), but many are descriptions of client types (e.g., proportion of rehabilitation clients, end-of-life clients) or sociodemographic characteristics that could not be considered to be quality measures (e.g., living arrangements). In addition, the proposed indicators generally do not include risk adjusters to take into account either client- or agency-level differences in factors that could affect the probability of an indicator being present. Therefore, these indicators may be vulnerable to the effects of selection biases across the regions being compared. CIHI has recently announced that it will develop a national home care reporting system that includes the MDS-HC as a source of clinical data for national statistical reporting. Wherever possible, CIHI will use the MDS-HC data elements to calculate its proposed performance indicators.

By the beginning of 2003, one-fifth of all U.S. states, five Canadian provinces/territories, and several other international jurisdictions (e.g., Iceland, Japan, Hong Kong, Switzerland) had begun implementation of the MDS-HC assessment tool (Morris et al., 1997, 1999a). The MDS-HC differs from the OASIS in that it is designed to serve multiple functions including care planning, eligibility screening, case mix, outcome measurement, and QIs (Hirdes et al., 1999a; Morris, Carpenter, Berg, & Jones, 2000). Reliability and validity of the MDS-HC data elements and its embedded scales have been demonstrated through a series of on-going international studies (e.g., Kwan, Chi, Lam, Lam, & Chou, 2000; Landi et al., 2000; Morris et al., 1997). For example, a five-country trial of inter-rater reliability showed an average weighted kappa of 0.74 for the MDS-HC items (Morris et al., 1997).

	Measuring Quality in Home Care Settings

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
Home care is a rapidly expanding sector of health care, often serving as a replacement for, or continuation of, facility-based acute care and/or LTC. In the Toronto area, the proportion of home care clients considered to be "acute" cases has risen steadily over time so they now compose about half of all clients (McClennan, Pawlenko, Teplitsky, Valiabi, Adam, & Foley, 2000). The increased emphasis on home care is a function, at least in part, of changing public preferences and a growing desire to place clients in the "least costly" setting possible, for at least some populations (Greene, Ondrich, & Laditka, 1998; Hux, O'Brien, Iskedjian, Goeree, Gagnon, & Gauthier, 1998; Weissert, Lesnick, Musliner, & Foley, 1997). The number of people receiving home care in a given jurisdiction is generally much larger than those receiving facility-based care, such that home care now represents one of the most important sectors for public health care expenditures. For example, whereas <1% of Canadians under age 65 receive home care, about 22% of those aged 80 and older received this type of care (Wilkins & Park, 1998). In 1985, <1% of Canadian per capita health expenditures were allocated to home care (Health Canada, 1990), and this rose to 4% by 1997–1998 (MacAdam, 2000) to just over $2 billion. Recently, the Commission on the Future of Health Care in Canada (Romanow, 2002) recommended further expansion of funding for targeted home care and its inclusion as a universal publicly funded service under the Canada Health Act.

In the United States, Medicaid LTC spending in home- and community-based services waiver program expenditures rose from 3% of all LTC spending in 1988 to 18% in 2000 (Wiener, Tilly, & Alecxih, 2002). By that year, noninstitutional services represented about one-quarter of all Medicaid LTC expenditures. However, there is considerable state-by-state variation in spending in home- and community-based services. For example, in 1998, Michigan's per capita Medicaid expenditures for these services in the elderly population were about three times higher than in Alabama and almost seven times higher than Indiana (Wiener et al., 2002).

In addition to serving large populations, home care is important because of the linking function it performs between primary care, acute care, LTC, and mental health services. For example, home care is often an intermediary care setting between acute hospitals and LTC facilities. Eligibility criteria for home care can have a direct impact on the types of individuals who are maintained at home and thereby the profiles of residents of LTC care facilities. For example, Hirdes and colleagues (1999b) showed that Italian home care clients, who are managed within a system that has few LTC beds, have considerably higher levels of impairment in cognition and ADL compared with U.S. and Canadian clients. This may reflect, at least in part, a stronger policy emphasis on maintaining older Italians in the community.

The clients receiving home care may be at a crucial turning point in their health. Depending on the care they receive, home care clients may be restored to a level of functioning that allows them to live relatively independently in the community, or, conversely, with problematic care, they may experience further health declines in a cascade of events leading to further dependency and eventual institutionalization. To evaluate the effectiveness of interventions by home care agencies at these potential turning points, it would be helpful to have standardized performance measures to document changes in client characteristics that can be compared between agencies.

Although a great deal can be learned from examining quality initiatives in other health care sectors, it should be recognized that many unique factors influence the quality of home care. Home care differs from hospital or LTC facilities in terms of the nature of formal service provision, the role of family members, and the characteristics of the individuals receiving care.

Most formal home care is provided out of the view of the general public and to a single client at a time. In addition, the amount of daily contact between home care staff and the client is substantially shorter than in facility-based care. Community-based individuals are also more likely to receive services from multiple agencies or professionals (e.g., home care agencies, volunteer organizations, private pay nurses, physicians). This means that home care agencies are not in a position to either fully control or directly audit all, or even most, of the direct care the community-based clients receive. There may also be a greater diffusion of responsibility with respect to the quality of care s/he receives. This may be of particular concern for individuals residing in the increasing number of unregulated special housing settings. Persons with dementia who live alone may be especially vulnerable.

The role of the family is another factor that may have a different effect on the quality of home care compared with facility-based care. Although hospital and LTC facility staff will be the main provider of care to in-patients, families account for as much as 80% of the care received by home care clients, according to some estimates (Chappell, 1992). Therefore, outcomes in at least some dimensions will be dependent on the quality of both the formal and the informal care they receive. For example, when meal preparation is done primarily by a family member, the nutritional quality of those meals could directly affect weight change and hydration. Alternatively, when caregivers experience high levels of distress, negative consequences for clients could include an elevated risk of depression and/or abuse. Formal home care agencies, therefore, can be expected to influence client outcomes directly through the services they provide to clients and indirectly through the assistance, guidance, and/or respite they give to informal caregivers. The implication of this is that client outcomes can be considered to be the responsibility of home care agencies, even if those outcomes are affected by the quality of informal support provided. In cases where client outcomes are closely linked to caregiver characteristics (e.g., depression), it might be appropriate to include those caregiver traits as individual-level risk adjusters.

The characteristics of home care clients can influence the quality of community-based services in a variety of ways. For example, individuals in the community are subject to less intensive monitoring by health professionals than in in-patient settings, meaning that there may be higher rates of nonadherence to pharmaceutical or treatment regimens. A related point is that home care clients can generally be regarded as having greater autonomy than the more dependent residents of LTC facilities, making home care outcomes relatively more dependent on client behavior than one might see in facility-based care. Community-based clients will tend to be less functionally impaired, meaning they will also probably have different rates of decline or transition to more impaired states than their counterparts in LTC facilities.

	Factors to Consider in Developing HCQIs

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
QIs can be used for a variety of purposes and can reach different target audiences. For example, they can be used for reports that guide internal quality improvement activities. Administrators and clinicians (both case managers and direct service providers) can use this information to monitor changes over time in their agency's practice patterns or to benchmark their organization against their peers with the hope of identifying opportunities for improved care. The audience for this type of report will be intimately familiar with the agency's clients and practice patterns. They will have ready access to the agency staff and clients to verify impressions left by the report. In that sense, the QI report can be a starting point in a process of identifying and responding to opportunities to improve care.

Accreditation agencies or regulators can also use QIs to evaluate the agency's track record related to the processes and outcomes of care being provided. In this case, the audience for the report is an external individual who is typically well trained in evaluating health service agencies. Although these individuals will be from outside the organization, they typically will have the ability to interview staff, administrators, clients, and/or family members and can directly gather additional information on the organization's care patterns.

In the United States and Canada, agency-specific quality reports are seen as essential new tools to address citizen demands for better public accountability of health care services. The intent is to develop reasonably comprehensive reports that consumers can access electronically (or in print form) that allow them to assess the care provided by home care agencies they or their family members are planning to utilize.

Public consumer reports differ from other applications of QIs because consumers will tend to (a) be less sophisticated in their knowledge of health care (b) be less sophisticated in the interpretation of QIs (specifically, they will be much more likely to treat QI results as definitive statements on the quality of care) (c) rely on the media to assist in the interpretation of quality reports, and (d) have limited access to the agencies in question to verify or refute the impressions they have gained from the report.

For these reasons, the accuracy of QIs is critical, particularly if they are to be used for public reports. They should be based on standardized measures with demonstrated reliability and validity (Mor et al., 2003b). However, each QI must also be considered to be a reasonable measure of quality. For instance, whereas variations in poverty across regions might be a relevant measure of the consequences of social policy, it is probably not reasonable to assume that this is a measure of home care quality. One must also consider what points of comparison to use in QI construction. Prevalence-based measures have the benefit of being relatively easy to construct and monitor, but they are not dynamic reflections of the effects of interventions at the individual level. Incidence-based measures consider changes in status for individual clients. They provide a clearer indication of the outcomes of care but are somewhat more burdensome to monitor because they depend on longitudinal record linkage.

Exclusion criteria for the numerator and denominator of a QI are important to consider because they can have a dramatic effect on the QI's overall rate. For example, if the numerator is defined too restrictively, it will be difficult to detect the quality problem. Conversely, if it is defined too liberally, it may overestimate the true rate of the quality problem. Definition of the denominator should be considered carefully, because small numbers can make the QI unstable and affect its generalizability.

Perhaps the most contentious issue in QI development is risk adjustment. Population differences arising from selection biases between organizations can result in some agencies being incorrectly labeled as having a higher, or lower, than average QI rate (Mor et al., 2003b). Under one argument, if agencies are to be compared on an equal footing, the QIs should control for these population differences. The counterargument is that agencies that accept "riskier" clients should put in place interventions that neutralize this risk and should not gain an advantage from the adjustment of the QI rates. It is clear, however, that any risk adjustment must not employ strategies that can obscure true differences in quality.

	Methods

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
The HCQIs were developed as a three-nation effort involving researchers, clinicians, and policy makers from Canada, the United States, and Japan. The research approach used multiple information sources to identify, evaluate, and refine quality measures for use with the MDS-HC. An extensive literature review was undertaken to identify QIs used in other sectors of health care (e.g., the CHSRA Nursing Home Quality Indicators, the Nursing Home and Post-Hospital Rehabilitative-Care Quality Indicators, developed by the Centers for Medicare and Medicaid Services, and the HRCA Expansion of Quality-of-Care Measure (Q-Span) Post-acute Indicators). These QIs could represent appropriate domains to be measured or even be candidate QIs for home care. In addition, new indicators designed specifically for home care were developed. In some cases, there were already available scales that combined items into a measure descriptive of a whole domain, such as the Cognitive Performance Scale (CPS) (Morris et al., 1994). When possible, the HCQIs used these validated scales rather than the individual items within the scale.

Focus Groups
One other starting point for the development effort was a series of focus groups held with the aim of generating candidate measures to be considered as possible QIs. Health professionals from six Canadian provinces and from Michigan's Home and Community Based Waiver Program met multiple times to provide input to the research effort at its various stages. In addition, a focus group with older adults was used to derive information on consumer perspectives on quality in home care. Draft quality measures from the focus groups were combined with the previously constructed indicators to create an initial list for expert review.

Data Sources
Data for derivation and testing of the QIs came from assessments done by trained case managers in gate-keeping organizations in Canada and the United States. The data sources are described below.

Canadian Data
The RAI Health Informatics Project (RAI-HIP) was a 2.5-year study in which 14 Ontario Community Care Access Centres (CCACs) implemented the MDS-HC on a pilot basis. Twelve of those agencies had enough cases available to allow analyses at the organizational level. CCACs are single-point entry agencies that determine eligibility for community and institutional services. They also act as case management organizations contracting with home care agencies to provide services ranging from homemaking to skilled nursing and therapies. CCACs are the agencies that would be considered to have ultimate responsibility for the quality of home care provided in Ontario, but the actual delivery of care is carried out by a variety of for-profit and not-for-profit provider agencies. In 2003, the provincial government mandated that all CCACs implement the MDS-HC for use with all long-stay home care clients.

CCACs participating in the pilot implemented the MDS-HC in their normal clinical practice for the duration of the study. New clients or persons scheduled for a reassessment were assessed with the MDS-HC, and copies of the assessments were sent to the researchers with personal identifiers removed or encrypted. All assessments were completed in the client's home by trained case managers (typically nurses or social workers). Although CCACs also assess hospital-based clients awaiting discharge, the current study was restricted to community-based adult clients only. Clients who were on service for <30 days or who were being seen for an initial assessment were excluded. This resulted in a sample of 3,041 individuals with single assessments and a subset of 203 cases with longitudinal data suitable for deriving HCQIs on the failure to improve or incidence of decline in various conditions. Age was not used as an inclusion/exclusion criterion; however, it should be recognized that the majority of subjects in this study and the majority of home care clients are elderly.

Table 1 provides a summary of the client characteristics. With use of the larger sample of clients meeting the conditions for prevalence HCQIs, the majority was female, with a mean age of 75.8 (SD 13.3). About one-third of clients showed some cognitive impairment based on the CPS (Morris et al., 1994), with about 12% being moderately to very severely impaired. About 72% of clients had no ADL impairment, based on the ADL Hierarchy Scale (Morris, Fries, & Morris, 1999b). The most common diagnoses were arthritis (45.7%), hypertension (35.9%), and diabetes (21.1%). A comparison of this long-stay sample with all Ontario home care clients (Dalby, 2003) suggests that the current sample has a similar proportion of clients aged 75 and over (64%) and a slightly higher proportion of females (73% compared with 71%). However, the distribution of diagnoses is comparable with the overall home care population. Also, although the Ontario sample is somewhat less impaired than the Michigan sample, Dalby (2003) reported it to be somewhat more impaired than home care clients in Manitoba.

In 1999, Michigan adopted the MDS-HC for use in both the MI Choice and the OSA programs, and the state supported a variety of training opportunities to enable local case managers (a nurse or social worker) to hone their skills with the new instrument. Like the Ontario CCACs, the Michigan agencies provide case management only and they contract with external agencies for care delivery. Wiener et al. (2002) provide a comprehensive comparison of home- and community-based services in seven U.S. states, including Michigan.

The Michigan sample included 11,252 individuals on service for 30 days who met the criteria for both the prevalence and the incidence HCQIs. The average age of the Michigan clients was 64.4 (SD = 20.6) years, and about two-thirds were female. The relatively young mean age of this sample reflects the inclusion of young disabled persons in the MI Choice program. About one-third of these clients are under 55 years of age. For that reason, it is important to evaluate the need for age adjustments in any comparisons with other jurisdictions. The prevalences of clients who were intact in cognitive function (42.1%) or in ADL (25.9%) were both considerably lower than noted in the Ontario sample. The most common diagnoses among the Michigan clients were arthritis (58.4%), hypertension (51.1%), and congestive heart disease (26.5%).

Analytic Strategy
The starting point for the selection of HCQIs was to consider four types of home care clients: long stay (i.e., clients who would be on service for 30 days), post acute (i.e., clients on service <30 days, discharged from acute care, with postacute care or functional rehabilitation as goals of care), mental health (i.e., clients with a psychiatric diagnosis or showing psychotic symptoms like delusions or hallucinations), and palliative care (i.e., clients reported to have end-stage disease and clients receiving hospice or palliative care services). The study team then prioritized the candidate HCQIs identified in the literature review and focus groups, according to their relevance to each of these groups. For example, unexpected weight loss could be an important indicator of poor nutritional status that must be addressed in a long-staying client, but it is unlikely that it will be detected with sufficiently high prevalence rates to be used with postacute clients. To narrow the analyses to be undertaken, investigators from each of the five main research centers collaborating in this effort ranked the individual HCQIs in terms of their appropriateness for the four types of patient subgroups. The average ranks were then used as the first step toward reducing the number of candidate HCQIs from the original set of 73 candidate indicators to the 22 HCQIs that were ultimately examined here.

Because the mental health and palliative clients were relatively small subgroups of the overall samples (e.g., the Ontario sample had only 277 mental health and 58 palliative clients), the current analyses differentiated only between long- and short-stay clients (representing about two-thirds and one-third of the overall sample, respectively). Mental health and palliative clients were left in the long-stay sample (defined as clients who had been on service for at least 30 days) for some HCQIs, but they were excluded from the denominator for others where there was a lack of consensus on the appropriateness of including those subgroups (e.g., the weight loss HCQI excludes end-stage clients).

The current article focuses on HCQIs for home care clients who had been on service for 30 days, because too few of the short-stay cases had reassessments done to permit analyses of their outcomes. The distributional properties of the HCQIs were examined in long-stay clients using the U.S. (n = 11,252) and Canadian (n = 3,041) data. Prevalence estimates for HCQIs excluded all clients being seen for their initial assessment, as an agency cannot be held responsible for the well-being of clients it has just accepted for care. Incidence HCQIs were computed using the changes in client characteristics between intake and follow-up assessments (intervals between assessments typically ranged between 60 and 120 days).

The review of the HCQIs focused on the following main considerations for their acceptance: (a) relevance of the measure to quality in home care; (b) prevalence or incidence rates (HCQIs that had rates below 5% or above 95% in all jurisdictions were usually excluded because of sample size concerns at the agency level; however, two HCQIs [dehydration and abuse] were retained because of their clinical importance despite low prevalence rates); (c) variability between agencies, as otherwise the indicator would be of little interest; (d) good underlying psychometric properties of the MDS-HC items used to construct the HCQI based on the previously published results of international reliability trials (Morris et al., 1997); and (e) the size of the denominator (individual agencies should have a minimum of 20 cases in their denominator to obtain stable estimates of QIs [Berg, Mor, Morris, Murphy, Moore, & Harris, 2002]).

For the reasons identified earlier, risk adjustment was a point of particular debate and careful evaluation. Risk adjusters were co-morbid conditions for prevalence HCQIs and baseline (intake) conditions for incidence HCQIs. A limited number of fundamental client characteristics were tested as potential risk adjusters for all HCQIs, including age (dichotomized into <55, 65+, and 75+ groups based on preliminary analyses with age as a continuous variable), baseline physical and cognitive functioning, clinical complexity based on the Changes in Health, End-Stage Disease, and Symptoms and Signs (CHESS) scales (Hirdes, Frijters, & Teare, 2003), and gender. The research team members and the focus groups suggested additional potential risk adjusters, taking care that they not represent risks potentially ameliorated by the agency.

Whereas other efforts at developing risk adjustments for QIs have used large numbers of risk adjustments (e.g., the OASIS measure for risk of hospitalization uses 49 risk adjusters to achieve an R² of.165 [Shaughnessy et al., 2002a]), the aim here was to develop parsimonious models rather than including an excessive number of adjusters that may or may not play a meaningful role in the occurrence of the potential quality issue of interest. Hagenaars (1990, p. 61) argued that "given a certain level of accuracy, a less complex explanation of the data is to be preferred above a more complex one and a model with fewer parameters should be preferred to a model with more parameters (other things being equal)." There is also a compelling practical reason to constrain risk adjusters to simpler models with significant terms only: These models must be translated into benchmarking systems that can be used in real world settings. Aside from the statistical problems associated with overfitting models (Hagenaars, 1990), unnecessarily complex models can quickly become impractical for use in day-to-day decision making in the health care system. Nonsignificant terms that have no substantive effect on QI should not be left in simply for the sake of their inclusion.

Candidate risk adjusters were evaluated quantitatively by using a combination of generalized estimating equation (GEE) and logistic regression models. For the first stage of analysis, stepwise logistic regression models were run separately using all intake and follow-up data available within each country to identify which of the risk adjusters should be evaluated further. Risk adjusters that were kept in the final model for either country were retained for consideration in further analysis.

In the second stage, all possible combinations of independent variables from the first stage were examined for each country to rule out order-of-entry or -deletion effects that can occur with stepwise approaches. Both GEE and logistic regression analyses were used to finalize all models for individual covariates. GEE analysis has two advantages here: (a) It takes into account the effects of clustering of observations within home care agencies; and (b) it provides a conservative test of the significance of the relationships by reducing the problem of artificial reduction of standard errors associated with the use of clustered observations. Both the Michigan and the Ontario data were combined into a single data set, the agency identifiers were used as the clustering variable for both settings, and an exchangeable error structure was specified in each model. Other options were examined in relation to the error term, but GEE models with alternative correlation structures either did not reach convergence or gave comparable answers to this approach.

In all but three cases, a p value of.05 was used as the cut-off for retention of an individual-level covariate in the GEE models. The exceptions to this rule were in the incidence HCQIs where the p values for three covariates were.06,.07, and.09. These were retained in the final models because (a) the number of cases available for the incidence HCQIs was much smaller, (b) there were only minor differences in the parameter estimates in the logistic and GEE models, and (c) there were strong substantive grounds for retention of the individual covariates in question.

Once the final GEE models were specified, they were replicated using logistic regression to determine whether the more complex procedure yielded meaningful differences in the parameter estimates. This is of practical interest because these parameter estimates would potentially be used as fixed values in subsequent benchmarking comparisons when the HCQIs come into routine use. Therefore, if one obtains parameter estimates of the same direction and approximate magnitude, one could use the less complex model to obtain adjusted HCQI rates for individual agencies.

The aim of developing a set of risk adjusters is not to maximize explained variance in the HCQIs by including all possible independent variables that might be related to the dependent variable of interest. The adjusters should include only factors associated with the HCQI that would themselves not be considered quality issues under the control of the agency, and it should be reasonable to expect them to play a role in predisposing the client to a higher or lower rate of the HCQI. Therefore, some risk adjusters that were strongly associated with the outcome or process measure in question were ultimately excluded when it could be argued that the adjuster itself could be considered a sign of a quality problem or if the relationship was likely to be spurious. For example, despite their highly significant associations with falls, psychotropic medications were not used as risk adjusters, because it is well known that these drugs elevate risk of falling and their use with the elderly may be considered inappropriate except in specific justifiable cases. A detailed summary of the HCQI specifications is available on the internet (www.interrai.org).

In addition to the exclusion criteria for the denominator and specification of client-level covariates for adjustment with multivariate models, an agency-level adjuster was also developed for most HCQIs. The agency intake profile (AIP) is intended to serve as a control for agency differences in the types of clients that they admit. The concern is that some agencies may admit more clients with a stronger predisposition for developing certain quality problems than others. For example, the risk of a decline in cognition probably varies as a function of the severity of baseline impairment. Therefore, agencies that tend to take in more clients with moderate cognitive impairment could be more likely to notice declines in those clients than agencies with a larger proportion of cognitively intact clients. Also, agencies may be more adept at identifying problems, potentially to the detriment of their HCQI scores. The AIP is calculated in one of two ways, based on the type of variable involved: a prevalence rate for categorical variables or a mean score for continuous scales. In all cases, the AIP is based on the initial assessments of clients on intake into the home care program.

	Results

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 
The final list of HCQIs is described in Table 2, and their unadjusted rates (at the individual and agency levels) in Ontario and Michigan are provided in Table 3. The five HCQIs with the lowest unadjusted agency-level rates in these two jurisdictions were prevalence of dehydration and prevalence of neglect or abuse (2.8%), prevalence of inadequate meals (5.5%), prevalence of no medication review by a physician (5.6%), and failure to improve or incidence of skin ulcers (6.2%). The five most common HCQIs were prevalence of ADL rehabilitation potential with no therapies (74.5%), failure to improve or cognitive decline (51.1%), failure to improve or decline in the ADL Long Form Scale (an additive scale that summarizes the severity of impairment in self-performance in eight ADLs; see Morris et al., 1999b) score (49.2%), failure to improve in bladder incontinence (40.9%), and prevalence of disruptive or intense daily pain (38.2%). Similar findings are evident if one examines prevalence and incidence rates based on the individual-level data reported in Table 3.

View larger version (34K): [in this window] [in a new window]   Figure 1. Adjusted and unadjusted negative mood scores, by Michigan agency

	Discussion

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

QIs of the type proposed here have numerous advantages that should make them attractive to any agency seeking to improve its quality-monitoring systems. First, these indicators can be derived directly from the MDS-HC with no further data collection, so agencies using the MDS-HC for clinical purposes (e.g., to support care planning for individual clients) will automatically have the information needed to calculate the HCQIs. Second, these indicators provide a mixture of process (e.g., lack of medication reviews, failure to receive influenza vaccinations) and outcome measures (e.g., failure to improve or decline in ADL). Third, clinical indicators of this type are more likely to provide a valid reflection of quality of care than satisfaction surveys, which are typically biased by poor response rates and social desirability bias—individuals saying what they believe is expected of them. Fourth, the risk adjustment methodologies proposed for these indicators can be used to take into account differences in agency admission patterns and ascertainment bias. This will be particularly important for comparisons between jurisdictions where eligibility criteria differ, as may be the case with the Michigan and Ontario samples included here.

Although the availability of HCQIs for agencies using the MDS-HC represents a valuable step forward in creating opportunities to improve the quality of home care, there are some important caveats to be noted. First, despite the availability of large multinational data sets to derive and test the HCQIs, it has not yet been possible to examine their behavior over prolonged periods with large populations. Such data would provide useful information about the stability of HCQIs over time as well as an indication of their responsiveness to changes in policy and practice patterns. Second, it must be recognized that the computation of the HCQIs requires a level of analytic expertise that may not be readily available to home care agencies. Although the software they use to implement the MDS-HC may have the capacity to generate agency reports on HCQIs, these may not necessarily contain information on other organizations that would be needed for meaningful benchmarking. Third, the HCQIs proposed here represent a reduced subset of the original list of candidate HCQIs, but they are still sufficiently numerous that it may be difficult for a single agency to address all HCQIs at once. Therefore, quality managers and home care managers will be required to exercise some judgment about which HCQIs their agency may target at a given point in time. Fourth, the availability of HCQIs does not guarantee that they will be implemented into daily decision making by home care managers and clinicians. It is not yet known, for example, whether feedback on quality performance actually changes behavior of the organization. In addition, consideration should be given to development of training strategies that would make HCQIs meaningful and useful to frontline staff.

An interesting question arising from the current research is: Who should be the target audience for HCQI results? The current study was based on gate-keeping agencies in one U.S. state and one Canadian province. In both settings, these agencies are the fiscally responsible party contracting home care services. In that sense, they represent consumers of the information who would be interested in determining the "value for money" of the services they are contracting. On the other hand, one could argue that the information must also be fed back to the actual home care provider agencies in order to allow them to identify and respond to quality issues. Indeed, the most reasonable position is probably that both of these parties should be recipients of the HCQI results.

An important practical challenge is how to best summarize relatively complex information on HCQIs for use by different target audiences. Although there may be considerable appeal in developing a composite measure of quality that combines all 22 indicators into a single measure, early work in this area has suggested that it may be difficult to develop a combined indicator that consumers, providers, and regulators could use to determine quality. For example, Mor and colleagues (2003b) found little evidence of a correlation among nursing home QIs, and they argued that quality should be best treated as a multidimensional construct.

If singular composite measures of quality are inappropriate, one must address the problem of how to report complex information in a way that allows target audiences to consume and respond to the findings in a meaningful way. The problems posed by a 22-dimensional quality report are further complicated by the possibility that multiple risk adjustment strategies would have to be considered.

One approach to providing an overview of the data is to report the absolute values of the unadjusted and adjusted rates in tabular form for all sites parallel with values for a specific agency. For many consumers, this level of numeric detail may be overwhelming. Figure 2 shows a graphic method of reporting the adjusted rate for an individual agency (open columns) as well as the median (black squares) and the 10^th and 90^th percentiles (lower and upper bounds of shaded columns) for the industry. This graph allows one to examine both absolute HCQI rates and performance relative to the industry. Other options include radar plots showing the percentile ranks, adjusted odds ratios, or ratio of agency rates to industry means. As each method has advantages and disadvantages, it is probably best to allow for a number of views of the data to support their interpretation. Clearly, additional research on reporting HCQI information to decision makers is needed.

View larger version (37K): [in this window] [in a new window]   Figure 2. Home care quality indicator rates for a sample agency compared with industry median, 10th and 90th percentiles. Incontin <--?PC1-->= bladder incontinence. ADL = activities of daily living. Loco = locomotion. Commun = communication

The next step in this research is to expand the analysis of the HCQIs to involve more extensive cross-national analysis. An important challenge that remains is the need to verify that the risk adjustment methodologies will control for international differences in the eligibility criteria for home care. To this end, Dalby (2003) has recently completed a dissertation that involved an extensive evaluation of the implications of alternative risk adjustment strategies (e.g., stratification methods, AIP adjustment, case-mix adjustment) using data from Ontario and Manitoba. Resolution of the issue of risk adjustment is a precondition to the use of the HCQIs in benchmarking the quality of home care internationally. It will also be important to expand the search for HCQIs to identify indicators that would be appropriate for specialized subpopulations such as clients who require mental health, postacute, or palliative care services. For the current study, there were insufficient data on these three populations to allow the HCQIs to be tested for their applicability. However, as large data sets become available from Canadian, U.S., and international implementation of the MDS-HC, it will be possible to do further testing of the HCQIs with these small but important clinical subgroups.

It is critical to do additional research comparing the HCQIs with other measures of quality, including outcomes and survey experiences. Experience with experimental use of the HCQIs will greatly expand knowledge of their utility. For example, further work is needed to develop formats for displaying HCQI results in a way that is understandable for a wide range of audiences including regulators, agency managers, policy makers, and consumers. Material also needs to be developed to help agencies address identified quality-of-care problems. One approach is to develop "best practice" protocols that provide pragmatic, real-life advice on approaches that work. It should be noted that the HCQIs can be used to recognize agencies whose performance identifies them as a source of best practice information.

In short, the HCQIs are new tools providing a first step along the path of quality improvement for home care. These indicators can provide high-quality evidence on performance at the agency level and on a regional basis. However, the ability to actually improve quality also depends on successful communication of those findings to the appropriate target audiences, the capacity of home care professionals to make evidence-based decisions, the availability of effective solutions to identified quality problems, and the resources to implement those solutions.

	Footnotes

 
The Home Care Quality Indicators (HCQIs) reported here were created for use with the Minimum Data Set—Home Care (MDS-HC) version 2.0 by interRAI, a 21-country, nonprofit network of researchers and clinicians responsible for the development and application of the RAI/MDS series of instruments. Italian data were made available courtesy of Prof. Roberto Bernabei. The authors are grateful for contributions to the development effort by Mary James, Nancy Curtin-Telegdi, Jeff Poss, Colleen Maxwell, Lori Mitchell, Trevor Smith, Paula Fletcher, Gary Teare, Michael Stones, Kim Voelker, and Judy Bowyer. The authors thank the following agencies for financial support of this research: Agency for Health Care Policy Research (Grant 5 U18 HS09455), the Health Transition Fund—Health Canada (ON 421), interRAI, the State of Michigan and the Robert Wood Johnson Foundation.

¹ Department of Health Studies and Gerontology, University of Waterloo and Homewood Research Institute, Ontario, Canada.

² University of Michigan and Ann Arbor VA Medical Center.

³ Hebrew Rehabilitation Center for the Aged, Boston (Roslindale), MA.

⁴ Department of Health Policy and Management, Keio University School of Medicine, Tokyo, Japan.

⁵ University of Wisconsin, Madison.

⁶ University of Michigan, Ann Arbor.

⁷ International Medical Corps, Los Angeles, CA.

Decision Editor: Linda S. Noelker, PhD

Received for publication August 9, 2002. Accepted for publication February 2, 2004.

	References

TOP Abstract Performance Measurement for Home... Measuring Quality in Home... Factors to Consider in... Methods Results Discussion References

 

• Berg, K., Mor, V., Morris, J., Murphy, K. M., Moore, T., & Harris, Y., (2002). Identification and evaluation of existing nursing homes quality indicators. Health Care Financing Review, 23, 19-36.[Medline]
• Canadian Institute of Health Information. (2001). Development of national indicators and reports for home care, phase 2: Proposed home care indicators. Ottawa: Author.
• Chappell, N. L., (1992). Social support and aging. Toronto: Butterworths.
• Dalby, D. M., (2003). An evaluation of risk adjustment methods for home care quality indicators in Ontario and Manitoba. Unpublished doctoral dissertation, University of Waterloo, Waterloo, Ontario, Canada.
• Department of Health and Human Services, Health Care Financing Administration. (1999). Medicare and Medicaid programs; comprehensive assessment and use of the OASIS as part of the conditions of participation for home health agencies. Federal Register, January 25.
• Greene, V. L., Ondrich, J., & Laditka, S., (1998). Can home care services achieve cost savings in long-term care for older people? Journal of Gerontology: Social Sciences, 53, S228-S238.[Abstract]
• Hagenaars, J. A., (1990). Categorical longitudinal data. London: Sage.
• Health Canada, Federal/Provincial/Territorial Working Group on Home Care. (1990). Report on home care. Ottawa: Health Services and Promotions Branch, Health and Welfare Canada.
• Hirdes, J. P., Fries, B. E., Morris, J., Steel, K., Mor, V., & Frijters, D. H., et al (1999a). Integrated health information systems based on the RAI/MDS series of instruments. Healthcare Management Forum, 12, 30-40.
• Hirdes, J. P., Frijters, D. H., & Teare, G. F., (2003). The MDS-CHESS Scale: A new measure to predict mortality in the institutionalized elderly. Journal of the American Geriatrics Society, 51, 96-100.[Medline]
• Hirdes, J. P., Morris, J. N., Fries, B. E., & Bernabei, R., (1999b). Society's adaptation to its own aging: A new challenge for community-based health care. In Aging and health: A global challenge for the twenty-first century (pp. 205–212). Kobe, Japan: World Health Organization.
• Hirdes, J. P., Zimmerman, D. R., Hallman, K. G., & Soucie, P. S., (1998). Use of the MDS quality indicators to assess quality of care in institutional settings. Canadian Journal of Quality in Health Care, 14, 5-11.
• Hux, M. J., O'Brien, B. J., Iskedjian, M., Goeree, R., Gagnon, M., & Gauthier, S., (1998). Relation between Alzheimer's disease and costs of caring. Canadian Medical Association Journal, 159, 457-465.[Abstract]
• Kwan, C-W., Chi, I., Lam, T. P., Lam, K. F., & Chou, K-L., (2000). Validation of minimum data set for home care assessment instrument (MDS-HC) for Hong Kong Chinese elders. The Clinical Gerontologist, 21, 35-48.
• Landi, F., Tua, E., Onder, G., Carrara, B., Sgadari, A., & Rinaldi, C., et al (2000). Minimum data set for home care: A valid instrument to assess frail older people living in the community. Medical Care, 38, 1184-1190.[Medline]
• MacAdam, M., (2000). Home care: It's time for a Canadian model. Healthcare Papers, 1, 9-36.
• McClennan, S., Pawlenko, N., Teplitsky, F., Vahabi, M., Adam, C., & Foley, J., (2000). Service utilization patterns in community care access centres in Toronto: 1996 to 1999. Toronto: Toronto District Health Council.
• Mor, V., Angelelli, J., Jones, R., Roy, J., Moore, T., & Morris, J., (2003a). Inter-rater reliability of nursing home quality indicators in the U.S. BMC Health Services Research, 3, 1-13.
• Mor, V., Berg, K., Angelelli, J., Gifford, D., Morris, J., & Moore, T., (2003b). The quality of quality measurement in U.S. nursing homes. The Gerontologist, 43, 37-46.[Abstract/Free Full Text]
• Morris, J. N., Carpenter, G. I., Berg, K., & Jones, R. N., (2000). Outcome measures for use with home care clients. Canadian Journal on Aging, 19, (suppl 2), 87-105.
• Morris, J. N., Fries, B. E., Bernabei, R., Steel, R. K., Ikegami, N., & Carpenter, G. I., et al (1999a). RAI—Home Care (RAI-HC) assessment manual for version 2.0. Boston: interRAI..
• Morris, J. N., Fries, B. E., Mehr, D. R., Hawes, C., Phillips, C. D., & Mor, V., et al (1994). MDS Cognitive Performance Scale. Journal of Gerontology: Medical Sciences, 49A, M174-M182.
• Morris, J. N., Fries, B. E., & Morris, S. A., (1999b). Scaling ADLs within the MDS. Journal of Gerontology: Medical Sciences, 54A, M546-M553.
• Morris, J. N., Fries, B. E., Steel, K., Ikegami, N., Bernabei, R., & Carpenter, G. I., et al (1997). Comprehensive clinical assessment in community setting: Applicability of the MDS-HC. Journal of the American Geriatrics Society, 45, 1017-1024.[Medline]
• Moty, C., Barberger-Gateau, P., de Sarasqueta, A. M., Teare, G. F., & Henrard, J. C., (2003). Risk adjustment of quality indicators in French long term care facilities for elderly people. A preliminary study. Revue d'Epidémiologie et de Santé Publique, 51, 327-338.
• Rantz, M. J., Popejoy, L., Mehr, D. R., Zwygart-Stauffacher, M., Hicks, L. L., & Grando, V., et al (1997). Verifying nursing home care quality using Minimum Data Set quality indicators and other quality measures. Journal of Nursing Care Quality, 12, 54-62.[Medline]
• Romanow, R. J., (2002). Building on values: The future of health care in Canada (pp. 171–188). Ottawa: Commission on the Future of Health Care in Canada.
• Shaughnessy, P. W., Crisler, K. S., Hittle, D. F., & Schlenker, R. E., (2002a). OASIS and outcome-based quality improvement in home health care: Research and demonstration findings, policy implications, and considerations for future change. Washington, DC: Center for Medicare and Medicaid Services.
• Shaughnessy, P. W., Crisler, K. S., Schlenker, R. E., & Arnold, A. G., (1997). Outcomes across the care continuum: Home health care. Medical Care, 11, 115-123.
• Shaughnessy, P. W., Hittle, D. F., Crisler, K. S., Powell, M. C., Richard, A. A., & Kramer, A. M., et al (2002b). Improving patient outcomes of home health care: Findings from two demonstration trials of outcome-based quality improvement. Journal of the American Geriatrics Society, 50, 1354-1364.[Medline]
• Teare, G., Hirdes, J. P., Ziraldo, M., Proctor, W., & Nenadovic, M., (2000). Provincial status report: The quality of caring, Ontario, April 1998–1999. Toronto: Canadian Institute of Health Information.
• Weissert, W. G., Lesnick, T., Musliner, M., & Foley, K. A., (1997). Cost savings from home and community-based services: Arizona's capitated Medicaid long-term care program. Journal of Health Politics, Policy and Law, 22, 1329-1357.
• Wiener, J. M., Tilly, J., & Alecxih, L. M. B., (2002). Home and community-based services in seven states. Health Care Financing Review, 23, 89-114.[Medline]
• Wilkins, K., & Park, E., (1998). Home care in Canada. Health Reports, 10, 29-37.
• Zimmerman, D. R., Karon, S. L., Arling, G., Clark, B. R., Collins, T., & Ross, R., et al (1995). Development and testing of nursing home quality indicators. Health Care Financing Review, 16, 104-127.

This article has been cited by other articles:

				N. Kerse, M. Boyd, C. Mclean, J. Koziol-Mclain, and G. Robb The BRIGHT tool Age Ageing, September 1, 2008; 37(5): 553 - 588. [Abstract] [Full Text] [PDF]

				C. Hawes, B. E. Fries, M. L. James, and M. Guihan Prospects and Pitfalls: Use of the RAI-HC Assessment by the Department of Veterans Affairs for Home Care Clients Gerontologist, June 1, 2007; 47(3): 378 - 387. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Services Download to citation manager Citing Articles Citing Articles via HighWire PubMed PubMed Citation