| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||
BOOK REVIEW |
Department of Epidemiology and Biostatistics Health Services Research Division School of Medicine Case Western Reserve University Cleveland, Ohio 44106
Risk Adjustment for Measuring Health Care Outcomes (3rd edition), edited by Lisa I. Iezzoni. Health Administration Press, Chicago, IL, 2003, 508 pp., $85.00 (cloth).
In recent years many academics, health care providers, and third-party payers have focused their attention on estimating and discussing the quality of health care. This focus is derived in part from concerns that the incentives of funding systems, such as prospective payments, result in cost/quality trade-offs that have a negative impact on the health of beneficiaries (Poland, Bollinger, Bedard, & Cohen, 1985; Thomas et al., 1986).
A high proportion of the research and literature investigating these issues has centered on older populations, either explicitly by analyzing administrative databases such as those of the U.S. Medicare system (Schneeweiss, Wang, Avorn, & Glynn, 2003; Yuan, Cooper, Einstadter, Cebul, & Rimm, 2000) or implicitly by focusing on treatments that are more common in older patients (Hannan et al., 2003; Luft, 2003; Rosenberg, 2002). The frequent use of age as a proxy for other factors in such analyses seriously biases the studies' conclusions.
A key issue in estimating the quality of health care, of course, is the quality of the analysis—especially how the researchers attempted to isolate the adverse consequences directly attributable to the quality of care provided. In medicine, these efforts are often referred to as risk adjustment (Inouye et al., 2003, Johnson, 2003; Render et al., 2003); other common terms include outcomes research (Dimick, Cowan, Upchurch, & Colletti, 2003; Silvet et al., 2003; Weir, Signorini, Dennis, & Murdoch, 2000) and quality-of-care research (Mor, Angelelli, Gifford, Morris, & Moore, 2003; Reuben, Shekelle, & Wenger, 2003; Scott et al., 2003). The ultimate aim of this body of research is to determine a statistical method that can clearly distinguish between adverse events that are attributable to a specific treatment choice, provider, or delivery venue and those that are due to intrinsic characteristics of the patient. This is a very difficult goal to achieve because of the complexities of human health, the impact of random events, and the unavoidable resource constraints and limitations in current information systems that often lead to less than optimal data being utilized.
The recently published third edition of Risk Adjustment for Measuring Health Care Outcomes, edited by Lisa Iezzoni, provides an excellent overview of these and other issues related to risk adjustment and quality-of-care research. The previous versions of this book have been widely accepted as the quintessential text for those who are interested in outcomes research, risk adjustment, or quality of care. As is stated in the preface, the aim of the third edition is "to introduce the issues underlying risk adjustment and to suggest important conceptual and methodological considerations in designing and evaluating risk-adjustment strategies" (p. xvii). Although the authors have updated their focus on methodological techniques in this edition, the book still is better described as a justification for why one should perform risk adjustment and some of the processes involved in doing so, rather than a manual for how to perform the often quite difficult statistical techniques involved in risk adjustment.
New Elements in the Third Edition
While much of the text is unchanged, Iezzoni now includes sections on conducting surveys, measuring the validity and reliability of risk adjustment strategies, general linear and logistic regression, propensity scores, instrumental variables, and hierarchical modeling. Perhaps the most significant change in the third edition, however, is the expansion in the scope of the discussion and examples provided. In addition to in-patient hospital stays, other health care settings are now incorporated. Notable among the additions are long-term care examples, including discussions of home care and nursing homes—both of which are given significant attention within gerontological quality-of-care research. Sections have also been added to identify specific issues in risk adjustment associated with mental health and disability.
Understanding the Concept of Risk Adjustment
In an illustration of the basic problem of outcomes research, Iezzoni provides a conceptual model of the summation of patient factors, treatment effects, and random events that produce health outcomes. This conceptual model is referred to as the "Algebra of Effectiveness," although most risk adjustment methods involve much more complicated mathematical and/or statistical approaches. Figure 1 illustrates the basic problem in risk adjustment and has a similar structure to the conceptual model presented by Iezzoni. In Figure 1, however, four basic sources of health outcomes are identified: patient characteristics (both clinical and nonclinical), treatment characteristics, organization characteristics (including the facility and providers), and random events. Like the Iezzoni model, Figure 1 assumes an additive relationship between the causes of health outcomes, although the true (and causal) relationships may be more complicated (i.e., involving interactions between the four components and possible reverse causations—for example health outcomes affecting patient and organizational characteristics).
|
The Interaction and Correlation of Variables
The patient factors described in Iezzoni's model include a holistic description of well-being, including physical functional status and psychological, cognitive, and psychosocial function. Many of these factors have either strong correlations with each other or may impact the health of the individual in a multiplicative fashion (Fang, Liu, Tang, Wang, & Ko, 2004; Kalantar-Zadeh, Kleiner, Dunne, Lee, & Luft, 1999). The model and methods presented in the book, and that are most commonly used, focus on the individual patient factors and not the correlations and non-linearities that are known to exist. This oversight is not due to ignorance of these interactions and effects but, as Iezzoni observes (pp. 179–180), from the trade-offs made in risk adjustment between content validity for a specific intervention/population and the desire to compare results across interventions/populations by using compatible methodologies and to minimize the resources devoted to the development of new risk adjustment strategies. The issue of patient factor interactions is of specific importance in aging populations. Almost all risk adjustment methods include age as a factor to serve as a proxy for the various changes in health status associated with aging that are now explicitly included in Iezzoni's model.
Bias Effects of Using Age as a Proxy
Consider that we are looking at some important outcome for patient i treated at institution j, denoted Dij, where a simple risk adjustment equation uses the patient's age (Ai) and the volume of the patients treated at that institution (Vj)
|
|
This simple model demonstrates many of the potential problems with risk adjustment. First, although age is an important variable in most risk adjustment models, it is also correlated with many other health and institutional variables that may not be included. Thus, if we simply estimate the coefficient for age it will be biased because age will reflect the significance of these other missing variables (i.e., missing variable bias). The same can be said for volume. Unless you account for all relevant organizational variables, you will inevitably have biased parameter estimates. Further, the quality/volume relationship may also suffer from endogeneity. Specifically, if the quality of an institution is correlated over time, then patients may be attracted to facilities with better quality. Thus, the presence of quality may in fact drive volume, rather than institutions with higher volumes resulting in provision of higher quality health care.
If all relevant factors are not included in a risk adjustment model, the unexplained variance will weight correlated factors that are included. In these circumstances, the use of age as a proxy for age-related changes in health status results in a bias against healthier older persons. For instance, treatment guidelines may be developed based on patient age, when issues of social support, physical functioning, cognitive status, and patient attitudes and preferences are more efficient predictors of health outcomes. Unfortunately, however, age is more easily measured and quantified than these more amorphous and less stable factors.
These observations lead to our next consideration, specifically the influence of patient factors on treatment choice (and compliance), with treatment choice an important moderator of treatment effectiveness and health outcomes. Differences in rates of interventions among different age cohorts are typically explained by cost–benefit arguments due to expected remaining lifetime and/or inability to tolerate aggressive treatment due to frailty (Cameron & Williams, 1996; Silverman, McDowell, Musa, Rodriguez, & Martin, 1997; van der Steen, Ooms, Ader, Ribbe, & van der Wal, 2002). These arguments may be made by the clinician, the patient, or both when choosing among intervention options (Baig et al., 2003). An extension of our previous argument can then be made. Specifically, that the use of age-based clinical guidelines based on outcomes research that used chronological age as a proxy for age-related changes in health status, results in a further bias against healthier seniors. These healthier individuals may not even receive the opportunity to respond to a treatment associated with the greatest probability of positive outcome.
Is Age Really Important?
Perhaps we, as a society and as health researchers, have focused too closely on chronological age and have ignored not only the greater picture of current well-being, but also the actual process or time-dependencies of aging. In addition to individual-specific variations of risk over time, various birth cohorts of individuals live through different historical periods and through some of the same periods, but at different ages. The mixture of these factors can influence their current or future health. A 70-year-old individual born during the 1930s may not have the same health foundation as a 70-year-old individual born during the periods of relative affluence in the decades immediately preceding or following the Depression. In risk adjustment, it is necessary to consider global temporal changes in health and medicine and their influence on cohorts of individuals, as well as individual fluctuations in risk profile.
The lack of a solid theoretical consideration for the intricacies and complexities of risk adjustment raises much more important questions: Who is ensuring that the practitioners of risk adjustment are doing it correctly? Are we encouraging uniformity of method over construct validity? While these may sound like technical questions that only academic researchers may be interested in, the quality of risk adjustment is a significant issue with ramifications for all participants in a health care system, from consumers to providers to payers. For example, a poor quality scorecard can cause a physician or hospital to lose clientele or funding or may even affect the types of care that they are willing to provide.
The Art and Science of Risk Adjustment
Given the complexity of risk adjustment, we applaud Iezzoni's volume for its thoughtful and thorough discussion of the intricacies and complexities of risk adjustment and of the increasing importance placed on outcome and quality-of-care research. As a result of this increasing attention, risk adjustment methods are rapidly changing and knowledge is increasing exponentially. Perhaps the most important aspect of Iezzoni's third edition is her continued restraint from claiming to provide the pathway to risk adjustment's holy grail—a method that perfectly describes the probability of a health outcome from an intervention, holding all other factors constant.
This of course leaves risk adjustment as part art and part science. There is much skill needed to set up the problem (for which the Iezzoni text is a great resource) and much more skill in devising an appropriate statistical model. It is on this front where one needs to look beyond the Iezzoni text. This is not surprising though, given that to perform unbiased and/or efficient risk adjustment, the most recent advances in statistics, biostatistics, and econometrics are needed. This is because of the complexity of the problem of isolating the marginal effects that the four components expressed in Figure 1 (patient characteristics, treatment characteristics, organization characteristics, and random events) have on the health outcomes of patients.
A Final Word
When assessing the impact of organizational characteristics on health outcomes, even if we could derive the perfect risk adjustment methodology and isolate the true quality effects, we may not arrive at the optimal solution to improving health delivery. First, as stated in the beginning of this essay, the rationale behind risk adjustment and quality-of-care research is often estimation of the trade-offs required in balancing quality demands in the face of resource constraints. As yet, there has been very little literature specifically focused on this relationship, that is, when is quality improvement worth the additional costs. While some have looked at the cost-quality relationship in aggregated data, their use of data on average cost and average quality does not focus on the true marginal costs of attempting to achieve better quality. Here we might be better served to develop a hybrid between quality analysis and cost effectiveness analysis (which is normally used on experimental data) to focus on the true cost–quality trade-off.
More importantly, perhaps, by making judgments solely on risk-adjusted health outcomes alone, we may be ignoring potentially significant nonhealth or nonoutcomes (i.e., the process of care) factors, leading to suboptimal resource allocation decisions. By focusing purely on health outcomes, we may be making trade-offs that lead to overall lower welfare in the community (Graff Zivin & Bridges, 2002). In cost effectiveness analysis, an outcomes-based analysis, the "Extra-Welfarist" perspective is employed when only health outcomes are considered, and the "Welfarist" perspective considers both health and nonhealth outcomes (Birch & Donaldson, 2003). What is missing in quality-of-care research is the more holistic perspective for patients' preferences—for health, nonhealth and process consequences of health care. This type of approach to measurement is especially relevant to older populations who may place a higher value on the process of successful aging, through maintenance of various health and nonhealth attributes of life quality, as compared to indiscriminant increases in longevity (Drewnowski & Evans, 2001). We have argued that we must move beyond age as a proxy for aging in risk adjustment. Similarly, we must expand our investigations beyond mortality as the sole outcome of interest.
References
| ||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
