| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
Correspondence: Address correspondence to Barbara Resnick, University of Maryland School of Nursing, 655 West Lombard Street, Baltimore, MD 21201. E-mail: barbresnick{at}aol.com
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Recruitment • Consent • Nursing homes • Research participants
One of the challenges in conducting nursing home research is the ethical issue surrounding consent. Nursing home residents are generally considered frail and vulnerable, largely because at least 50% of them have some degree of cognitive impairment (Gruber-Baldini, Zimmerman, Mortimore, & Magaziner, 2000). Thus, issues related to decision-making capacity are common and questions often arise related to participation in interventions. As previously defined by Karlawish, Casarett, and James (2002), "capacity" refers to a person's performance on measures of decision-making abilities (such as understanding). Taking this one step further, Karlawish and colleagues define "competency" as a judgment that the individual's capacity is adequate to make the decision he or she has made. Capacity issues and the ability to provide informed consent extend to the residents' participation in any type of research, but this concept has not been critically examined in the literature. Considering that recent studies have determined that even moderately cognitively impaired individuals have the capacity to respond to questions about end-of-life care decisions (Mezey, Teresi, Ramsey, Mitty, & Bobrowitz, 2000) and quality of life (Brod, Stewart, Sands, & Walton, 1999), we find it appropriate to consider if cognitively impaired individuals likewise have the capacity to consent to participate in research. Issues relevant to this matter have been set forth in the Omnibus Budget Reconciliation Act (OBRA) of 1987, in guidelines from the National Institutes of Health, through case law, and from Institutional Review Boards (IRBs). It is helpful to review these current recommendations and guidelines to best understand how to consider and measure capacity to consent to research among nursing home residents.
Omnibus Budget Reconciliation Act of 1987
A component of the OBRA of 1987(OBRA Guideline), The Residents Bill of Rights, directly addresses research. Specifically, this section of the OBRA Guideline provides guidance for how residents should be treated, which includes their involvement in research studies. For example, residents have the right to choose their treatment and to have their needs accommodated; certainly, participation in or refusal to participate in clinical trials should be considered each resident's right.
Researchers in nursing homes must carefully address the rights afforded to all human subjects in the research process: informed consent without coercion, the ability to withdraw from participation in a research study without any negative ramifications, assurance of protection from harm and financial burden, and a guarantee to uphold dignity and privacy (Touitou, Portaluppi, Smolensky, & Rensing, 2004). Researchers must ensure disclosure and comprehension when presenting information to a potential participant, and potential participants must be competent and have the ability to decide voluntarily. Therefore, the actual process of obtaining informed consent requires that all relevant information gets conveyed, the environment remains noncoercive, and the participant has the capacity to make a decision on his or her own behalf. It is this latter component that is the focus of this article.
Recommendations From the National Institutes of Health
In an attempt to protect vulnerable research participants, such as those living in nursing homes, the National Institutes of Health established recommendations for researchers to follow when doing research with cognitively impaired individuals (Table 1). These guidelines provide protection for residents at the time of the consent process and throughout the course of the research study. For example, consideration is given to possible changes in participants' cognition over the course of the study and a process for reevaluation and a plan for reassessing the individuals' understanding are suggested.
|
Although selected neuropsychiatric tests and cognitive screening tests have been associated with decision-making capacity (Casarett, Karlawish, & Hirschman, 2003), there are no cognitive screening tests that clearly establish residents' decision-making capacity (Allen et al., 2003; Kim, Karlawish, & Caine, 2002; Mezey, Mitty, & Ramsey, 1997; Moye, Karel, Azar, & Gurrera, 2004). Moreover, a combination of screening tests and vignette-based measures has been found to result in errors in determining capacity in one third of residents (Pruchno, Smyer, Rose, Hartman-Stein, & Henderson, 1995). Physicians tend to be more lenient in their judgment of resident capacity, whereas family members are more stringent (Vellinga, Smit, Van Leeuwen, Van Tilburg, & Jonker, 2004). This discrepancy could be because clinicians evaluate capacity on the basis of cognitive status (Williams & Engles, 1995), whereas family members depend on less clinical information. Moreover, there may be differences in the determination of capacity because of the different methods and tests of determination that are used. Three such measures are described here.
In an attempt to specifically evaluate the capacity to consent to research, researchers developed the MacArthur Competence Assessment Tool—Clinical Research Version (MacCAT-CR; Applebaum & Grisso, 2001; Kim, Caine, Currier, Leibovici, & Ryan, 2001). This tool contains 21 disclosure elements of informed consent that fall under the four abilities of capacity previously described (understanding, appreciation, reasoning, and choice). For each item, there is a response rating of 0 (inadequate), 1 (partial), or 2 (adequate). The MacCAT-CR is administered by means of a semistructured interview that is adapted to reflect the specific research protocol being addressed. Of the 21 items, 13 focus on understanding of the research, 3 focus on appreciation or the participants' ability to apply what they have been told about the study to themselves, and the remaining 5 address reasoning and the participants' ability to compare research participation with other treatment options. This tool has been used with individuals with schizophrenia (Carpenter et al., 2000; Dunn, Palmer, Keehan, Jeste, & Appelbaum, 2006), depression (Appelbaum, Grisso, Frank, O'Donnell, & Kupfer, 1999), Alzheimer's disease (Kim et al.; Karlawish et al., 2002), and cancer (Aaronson et al., 1996; Casarett et al., 2003; Simes et al., 1986), and it has provided some evidence of internal consistency and validity (Dunn et al.; Karlawish et al.; Kim et al.).
Saks and colleagues developed the California Scale of Appreciation (CSA; Saks et al., 2002) to specifically assess the appreciation aspect of capacity, and it establishes if potential participants can form adequate beliefs about how the information provided applies to them. The CSA involves having participants respond to 13 close-ended questions and 5 open-ended questions after they have read a consent form. Although there is some evidence of the reliability of this measure, there are concerns about its validity, as Saks found that the majority of participants were fully capable to consent (Saks et al.).
The Evaluation to Sign Consent (ESC) is a brief five-item questionnaire that evaluates the individual's capacity (DeRenzo, Conley, & Love, 2001). It was developed in response to previous findings that suggested that study participants could not correctly state the purpose of the research study in which they had consented to participate (Dunn & Jeste, 2001; Sugarman, McCrocry, & Hubal, 1998; Appelbaum & Roth, 1982). The intent of the ESC therefore is to assess whether a participant's factual understanding of information is sufficient to provide ethically valid consent to participate in a specific research study. No psychometric properties have been published on this tool, although it has been used in multiple studies with schizophrenia patients (DeRenzo et al.). The focus of this report is on the use and psychometric testing of the ESC.
Institutional Review Boards' Recommendations
In light of concerns about capacity to sign consent, some IRBs are beginning to require that a set of questions be asked of potential study participants to establish capacity. If an individual can demonstrate an understanding of the research study and the impact that participation will have on her or him, then capacity is assumed and consent to participate can proceed, regardless of underlying cognitive ability. Because of the high rate of cognitive impairment in some nursing home residents, having a standardized assessment is a welcomed addition in nursing home research (Magaziner et al., 2000). Although different approaches to ascertain the residents' capacity have been suggested (Cleary, 2004; Powers, 2003), there is no gold standard that fully ensures this.
In this article we report on the reliability, validity, and utility of the ESC (Appendix A), one of several assessment tools permitted by the IRB of the University of Maryland School of Medicine. For studies employing the ESC, the IRB requires potential study participants to complete the ESC prior to completing the consent process. Residents who pass the ESC (answer all of the items correctly) are invited to provide their own consent to participate. Those who are unable to pass the ESC but verbally consent and express a willingness to participate in the study are asked to sign an assent form. The proxy of such residents is then contacted to complete the consent process. The ESC focuses on only an understanding of the research, and although it has been used in clinical research there is no published evidence of its reliability and validity (DeRenzo et al., 1998). Thus, our purpose in this article is to evaluate the reliability and validity of the ESC. In addition, we assessed the predictive ability of each ESC item and the predictive ability of cognitive status and demographic variables on the residents' capacity to sign consent. Findings are useful in providing guidance to researchers working with residents in any long-term-care or community setting.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
Participants
The analyses in this report include data from the first 346 residents in six nursing homes who were approached and consented to participate in the Res-Care study. Residents completed the ESC and, if we obtained consent from a resident or her or his proxy, we administered the Mini-Mental State Exam (MMSE) to determine if the resident was eligible to participate in the Res-Care study (a cutoff point of 11 was a criterion of study eligibility). Residents not approached to take the ESC, as they were not eligible for the Res-Care study, were those who had a life expectancy of less than 6 months based on documentation by the resident's primary health care provider, or those who were admitted to the facility for short-stay rehabilitation (anticipated stay less than 4 months).
Measures
In addition to using the ESC, we evaluated the resident's cognitive status (using the MMSE) and abstracted information regarding age from the nursing home chart.
The ESC
The ESC is a five-item measure with an opening question that is not scored, which is used simply to indicate if an individual is alert and able to communicate. The five items reflect the participant's ability to name at least two potential risks incurred as a result of participating in the study; name two things that will be expected of him or her related to participation; explain what he or she would do if no longer interested in participating in the study, or if distress or discomfort was experienced associated with study participation; and explain the randomization process. For the purposes of our research, evidence of ability to sign consent was based on correct responses to all five items on the ESC.
Mini-Mental State Examination
We evaluated participants' cognitive status by using the MMSE (Folstein, Folstein, & McHugh, 1975). Because we did not know the education level of the participants, in our analyses we used a cutoff score of less than 20 to dichotomize the MMSE score to be indicative of those with or without moderate impairment (Murden, McRae, Kaner, & Bucknam, 1991).
Data Analysis
We performed descriptive analyses to describe the sample, the cognitive status of participants, and the results of the ESC. Our reliability and validity testing of the ESC was based on Rasch analysis, and we conducted this by using the Winsteps program. We used the Rasch analysis over traditional methods of reliability and validity testing, because this analysis is able to match the individuals' ability with the difficulty of the item and the individual standard error is considered versus the sample or test average. This results in a more accurate evaluation of internal consistency. Rasch analysis is based on the assumption that individuals with high or low scores have less error variance than those with scores near 50%. That is, individuals are more likely to get a challenging item incorrect than they are a less challenging item. Validity estimates are also improved by the use of model testing. The unidimensionality of the model can be evaluated along with the fit of each item, while matching the individuals' ability and the item difficulty. A detailed description of Rasch measurement is available elsewhere (Smith & Smith, 2004).
Validity
We based validity on evidence of the unidimensionality of the measure, the fit of each of the items to the overall tool, and item mapping (Smith & Smith, 2004; Waugh & Chapman, 2005). Item mapping lists the items from most difficult to endorse (i.e., to get correct) to those that are easiest or most likely to be endorsed in the sample tested. In addition, item mapping provides a distribution of individuals responding correctly to questions along the continuum from the easiest to the most difficult, so that the researcher can evaluate how well items discriminate those individuals who might be particularly high or low on a trait (i.e., capacity). The Infit and Outfit reflect the fit of the items to the model. These statistics are acceptable if they are in a range from 0.6 to 1.4 (Smith & Smith). An Infit and Outfit value of less than 0.6 indicates that the item does not provide additional information beyond the rest of the items on the scale. An Infit and Outfit value greater than 1.4 indicates that the item does not define the same construct as do the rest of the items in the instrument, is poorly constructed or misunderstood, or is ambiguously defined (Linacre, 2004). Outfit statistics are sensitive to a few off-targeted unlikely responses (i.e., a large logit difference between an item's difficulty and the person's ability). The Infit statistic weighs the squared standardized residual by the individual variances and is therefore not as influenced by unanticipated responses; thus, Infit statistics are more relevant to the utility of a measure.
Reliability
We based reliability of the ESC on an estimate of the person-fit statistics that consider each person's "true" ability. Item-fit statistics, which are similar to the traditional measurement of internal consistency, provide a measure of the "true" difficulty of each item. We based inter-rater reliability of the ESC on consistency in scoring by the initial evaluator and a second trained evaluator.
The Predictive Ability of Demographic Variables, Cognitive Status, and ESC Items
We tested two logistic regression models of capacity. One model considered the percentage of cases for which capacity was correctly predicted by each individual ESC item. The other considered the percentage of cases for which capacity was correctly predicted by demographic variables (age and gender) and cognitive status. We used binary logistic regression using a forward conditional entry method to obtain the positive predictive value and the negative predictive value of each item or variable, and we used a value of p <.05.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
A total of 218 residents (63%) did not pass the ESC. Responses to individual items are shown in Table 3. Approximately 80% of the participants could state what to do if they wanted to withdraw from the study or if they were experiencing pain or discomfort associated with participation. Approximately 50% of the participants could name two things that were expected of them as participants in the study and state two risks associated with participation; 57% could explain treatment assignment.
|
|
Predictive Ability
Results of the logistic regression analyses are presented in Table 4. The item that asks the patient to explain what he or she would do if he or she is experiencing distress or discomfort (Item 4) did not significantly enter the model or predict being able to sign consent. All other items did predict capacity to sign consent. For example, the item asking the patient to explain at least two things that will be expected of him or her in terms of participant cooperation during the study (Item 2) had a positive predictive value of 99%, a negative predictive value of 78%, and an overall percentage of agreement of 93%. Cognitive status, based on MMSE score, was the only demographic variable to significantly predict capacity (
2 = 49.9, p <.0001), with a positive predictive value of 84%, a negative predictive value of 52%, and an overall percentage of agreement of 65%. Age (2 = 0.02, p =.99) and gender (2 = 0.31, p =.57) did not significantly enter the model.
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
The completion of the ESC does add additional time to the consent process for a group of individuals who may have limited ability to concentrate as a result of both physical and cognitive problems. The time involved, however, is less than that required for the 21-item MacCAT-CR or the 18-item CSA. In addition to the time for completion, the three available tools to measure capacity differ with regard to the inclusion of specific elements that predicate capacity: understanding, appreciation, reasoning, and choice. The MacCAT-CR has the advantage of considering understanding, appreciation, reasoning, and choice related to capacity; the CSA solely addresses appreciation; and the ESC addresses understanding. In response to prior findings (Appelbaum & Roth, 1982; Dunn & Jeste, 2001; Sugarman et al., 1998) that the majority of research participants are unable to correctly describe the research to which they consented (i.e., did not seem to be understanding), the ESC intentionally focuses on understanding. Future research needs to compare of these measures and establish if all elements of capacity should be considered or whether a measure such as the ESC, which focuses only on understanding, is sufficient to determine capacity to consent to participate in research.
In addition to providing some evidence for the reliability and validity of the ESC, the Rasch analysis provided some support for the utility of the items in the ESC. The items fit the model particularly well with regard to Infit statistics, or when the item matched the individual's ability. The Outfit statistics are generally believed to be less critical to the establishment of validity, as they are focused on sensitivity in instances when there is a large difference between an item's difficulty and the person's ability. The larger Outfit statistics noted in the use of the ESC may be due to the situation in which individuals guessed correctly at the answer, which is likely to be prevalent in this population (Nedjam, Devouche, & Dalla Barba, 2004). According to the Rasch analysis, the most difficult items to answer correctly were asking the patient to name at least two potential risks incurred as a result of participating in the study (Item 1) and asking the patient to name at least two things that will be expected of him or her in terms of patient cooperation during the study (Item 2). It is these more difficult items to endorse that also seem to be the best predictors of capacity based on logistic regression. Specifically, Items 1 and 2 had the largest percentage of agreement with the full ESC. Future research should test a more parsimonious measure of capacity using only these two items, as it is possible that they may be sufficient to determine capacity. In addition, researchers who solicit consent might be advised to more clearly and sufficiently describe these components during the consent process.
It should be recognized, however, that the negative predictive value, reflecting the percentage of people with a negative test who do not lack capacity, was higher in Item 3 (knowing what to do to withdraw from a study) and Item 5 (understanding random assignment). This suggests that these items may provide an important addition to the measure, and also that they, too, should be more clearly explained by researchers during the consent process. Further, Item 5 may not be relevant to all research and certainly should be eliminated if there is no group assignment involved in the proposed research protocol.
Almost two thirds of the individuals who were approached to participate in the research project were unable to answer all five of the ESC items correctly. We standardized the correct responses to each item to help the evaluators' scoring, but they used their own language to explain the study to the residents. It is possible that the sections that were most often answered incorrectly were not explained as simply as possible on the consent form and during the consent process. Future work should standardize the study explanation so that each evaluator provides the same information to each potential participant. Increased understanding and integration of optimal methods for teaching new information to older adults, particularly those with cognitive impairment, could be used to establish the scripted procedure.
Study Limitations
As with all projects, there were some limitations to this project. The first is the homogeneity of the sample: all participants were in nursing homes and most were female and Caucasian, not known to have a life expectancy of less than 6 months, and not admitted for short-stay rehabilitation. Moreover, although the mean MMSE score was 18, indicative of moderate cognitive impairment (Hirschman, Xie, Feudtner, & Karlawish, 2004), only 18% of those individuals approached were severely impaired (MMSE < 11). Future research should consider using the ESC among a more heterogeneous sample of older adults with regard to demographic characteristics and cognitive ability.
Despite these limitations, this study provides useful information about the ESC. It indicates a reason and a method to move beyond cognitive testing to appropriately evaluate an individual with regard to his or her capacity to consent to participate in research. It is possible that a single item from the ESC ("name at least 2 things that will be expected of her or him in terms of patient cooperation during the study") may be sufficient to correctly predict capacity to consent, based on understanding of the research, at least 93% of the time. If used in this manner, it could constitute a simple and standard screen in an area that sorely lacks guidance.
|
Footnotes |
|---|
We thank all of the evaluators on the Res-Care Project for their dedication to helping residents in nursing homes have the opportunity to understand and consent to participate in research projects.
1 University of Maryland School of Nursing, Baltimore.
Decision Editor: Linda S. Noelker, PhD
Received for publication March 10, 2006. Accepted for publication August 29, 2006.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
203 (1987).This article has been cited by other articles:
 |
|
 |
|
  Bibliography for Ethics, Professionalism, and End of Life Care Focus, January 1, 2007; 5(4): 417 - 419. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
