| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||
Correspondence: Address correspondence to Lené Levy-Storms, Department of Social Welfare, School of Public Affairs, Box 951656, 5226 Public Policy Building, Los Angeles, CA 90095-1656. E-mail: llstorms{at}ucla.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Satisfaction • Preferences • Assessment • Quality of care • Quality of life
The results of two recent studies showed that the format of the satisfaction question may affect the tendency to provide acquiescent responses among long-term-care residents (Levy-Storms et al., 2002; Simmons & Schnelle, 1999). In both studies, closed-ended direct satisfaction questions (i.e., questions that explicitly used the term satisfaction and yielded a yes/no response) produced the highest reported rates of satisfaction, or met need, among residents. In comparison, a discrepancy index, wherein two corollary closed-ended questions asked about a resident's perceived and preferred care frequency, produced the lowest rates of met need (Levy-Storms et al., 2002; Simmons & Schnelle, 1999). Thus, some types of questions may be more resistant to acquiescence and, more sensitive to improvements in care quality.
One controlled intervention trial designed to improve incontinence and mobility care evaluated the sensitivity of both resident and family member reports to improvements in care. Independent observational data confirmed that the intervention resulted in significant objective improvements in the frequency and quality of care delivery as well as associated clinical outcomes (Simmons & Ouslander, 2005). Three important findings emerged from this study. First, the reports of neither residents nor family members were sensitive to improvements in care quality based on closed-ended direct satisfaction questions (i.e., questions that explicitly used the term satisfaction and yielded a yes/no response). Second, only the reports of residents were sensitive to care quality improvements based on closed-ended discrepancy indices, wherein two corollary closed-ended questions asked about perceived and preferred care frequencies. Third, family members' responses to both types of closed-ended questions were insensitive to care quality improvements. Moreover, the majority of family members responded "don't know" to the closed-ended questions, which suggests that family members may not feel qualified to respond to closed-ended questions about their relative's incontinence and mobility care (Simmons & Ouslander, 2005). These findings suggest that an alternative method of questioning may be necessary for this long-term care consumer group.
The purpose of the current study was to extend the results of this previous study (Simmons & Ouslander, 2005) by analyzing family members' responses to open-ended questions. We addressed two primary research questions:
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Participants and Setting
Two long-term-care facilities participated in this study, one of which was proprietary. Detailed inclusion criteria and sampling procedures for the clinical intervention trial have been published elsewhere (Schnelle et al., 2002; Simmons & Ouslander, 2005). The research team obtained informed consent, or the assent of the resident and the consent of a responsible party designated in the medical record, from 145 (70%) of the 206 eligible residents. They randomized residents into intervention and control groups following the completion of baseline incontinence and mobility assessments.
Intervention Protocol
Detailed descriptions of the intervention protocol and the significant positive intervention effects on multiple physical functioning (e.g., endurance, strength) and continence status clinical outcome measures have been reported elsewhere (Schnelle et al., 2002; Simmons & Ouslander, 2005). Briefly, research staff implemented the intervention every 2 hours, on 5 week days, between the hours of 8 a.m. and 4:30 p.m., for a possible total of four care episodes per day. As part of each care episode, research staff changed the resident (if wet or soiled) and provided toileting and walking assistance. These same research staff socially interacted with the resident throughout each care episode with verbal cueing and encouragement. The intervention group was then maintained for a total of 8 weeks, after which both groups completed postintervention interviews.
Usual Care
Residents who did not receive the intervention received usual nursing home care.
Family Members of Study Participants
In all, 115 (87%) of the 132 respective responsible party members were willing to complete a baseline interview, which required an average of 23 (± 9) min. In addition, 82 family members who completed a baseline interview and whose relative completed 8 weeks of intervention also completed a postintervention interview. These 82 family members comprised the family respondent sample for baseline to post-8-week comparisons between intervention and control groups (n = 41 per group). Family members included adult children (71%), spouses (9%), and other relatives (20%). Their average reported visitation frequency was one to two times per week, which did not change over the course of the 8 study weeks, and 59% reported that their visitation occurred during week days.
Measures
Interview Questions
Trained research staff interviewed family members using a standardized script via telephone. Based on previous research (Simmons & Schnelle, 1999), staff posed interview questions to family members regarding incontinence (toileting assistance) and mobility (walking assistance) care frequencies. We describe open-ended interview questions here. The results of the closed-ended questions have been reported elsewhere (Simmons & Ouslander, 2005). The current study focused on an analysis of family members' responses to the following set of open-ended questions, which varied slightly depending on experimental group (intervention vs usual care) and used a branching question format: "Do you see any difference in the [incontinence care] your relative receives when University of California, Los Angeles, staff: do the care [intervention]/are present [control]? If yes, is care better or worse? If better or worse, what have you noticed that is different?" Interviewers asked the exact same question for mobility care. The data in the current study included family members' responses to this set of questions posed at postintervention only. The interviewer recorded all family member responses verbatim, including reports of no noticeable difference, don't know, and no response.
Coding Protocol for Open-Ended Comments
Figure 1 outlines the five steps of the coding protocol applied to family members' responses to open-ended questions posed at postintervention only (e.g., "If better or worse, what have you noticed that is different?"). If family members replied that they did not know or did not provide any answer, then no further coding was possible. If we coded their responses, these responses may have pertained to staff or the resident or both. The coding protocol focused on whether the family member actually reported having noticed an improvement in care, not whether he or she wanted a change in care. This coding protocol entailed yes/no judgments by two independent raters about whether family members' open-ended responses (one or more statements) indicated that they noticed (a) any improvement (i.e., related to staff, resident, or both), (b) improvement related to staff, and/or (c) improvement related to the resident for each care domain (incontinence and mobility). In terms of staff, we further coded open-ended comments indicative of improvement into two categories: (a) relates to frequency of care delivery, and/or (b) relates to manner of care delivery (e.g., not feeling rushed). In terms of residents, we further coded open-ended comments indicative of improvement into two categories: (a) relates to the physical functioning of the resident, and/or (b) relates to the psychological well-being of the resident.
|
Data Analyses
For each care domain (i.e., incontinence and mobility), we compared the proportions of family members who responded "I don't know" to the average proportions who provided similar responses to closed-ended questions pertaining to that care domain. We calculated interrater reliability statistics for the coding protocol applied to all other open-ended comments (described in Figure 1) using Cohen's kappa. For each care domain (i.e., incontinence and mobility), we compared the proportion of family members who noticed an improvement based on their responses to the open-ended questions (posed at postintervention only) between the intervention and control groups using t tests for independent proportions. Finally, descriptive analyses presented the absolute differences between the intervention and usual care groups by the type of staff care processes and resident outcomes. The low sample sizes for these final analyses resulted in a lack of power to test for differences statistically.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
=.76), (b) noticing improvement in staff care processes ( =.60), and (c) noticing improvement in residents' outcomes ( =.65).
Proportion of "Don't Know" Responses by Family Members to Open- and Closed-Ended Questions
The proportions of family members who responded "I don't know" to the open-ended questions were not significantly different between the intervention and control groups for the two care domains: 36% versus 33% in the intervention compared to control for incontinence, 
2(1, N = 40) = 0.04, p >.05, and 18% versus 15% in the intervention compared to control for mobility, 2(1, N = 45) = 0.07, p >.05. Likewise, the proportions of family members who responded "I don't know" to the closed-ended questions that asked about satisfaction directly were not significantly different between the intervention and control groups for the two care domains: 43% versus 28% in the intervention compared to control for incontinence, 2(1, N = 73) = 1.90, p >.05, and 33% versus 18% in the intervention compared to control for mobility, 2(1, N = 52) = 1.48, p >.05; Simmons & Ouslander, 2005). Furthermore, the proportions who responded "I don't know" to the open- and closed-ended questions were not statistically different in both care domains: 45% versus 31% for open- and closed-ended questions, respectively, for incontinence, 2(1, N = 35) = 0.73, p >.05, and 19% versus 29% for open- and closed-ended questions, respectively, for mobility, 2(1, N = 33) = 0.29, p >.05.
Group Differences Based on Open-Ended Responses
Table 1 shows the results of the coding protocol (see Figure 1) by care domain for family members who provided responses indicative of noticeable improvements in care. Specifically, the three columns in Table 1 show the proportion of family members in the intervention and usual care groups that expressed a comment indicative of noticing any improvement (Column 1), improvement in staff manner of care and/or frequency of care delivery (processes; Column 2), and improvement in the resident's psychological well-being and/or the resident's physical functioning (outcomes; Column 3). Columns 2 and 3 reflect a subset of Column 1.
|
Figure 2 shows the absolute differences (i.e., unadjusted effect sizes) between the intervention and control groups by subtypes of staff care processes and resident outcomes. For incontinence care, the largest difference that family noticed was the improvement in residents' psychological well-being (i.e.,.33). For mobility care, the largest differences that family noticed were the improvements in residents' psychological well-being (i.e.,.21) and staff's frequency and manner of care delivery (i.e.,.18 and.21, respectively).
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
The results of a previous study (Simmons & Ouslander, 2005) showed that family members' responses to direct satisfaction and discrepancy index closed-ended questions were not sensitive to objective improvements in incontinence and mobility care. In the current study, family members' responses to open-ended questions did show significant discrimination between intervention and control groups in the expected direction for both incontinence and mobility care domains. These data suggest that important differences in quality may be missed if researchers measure families' perceptions of quality with only direct satisfaction and discrepancy index closed-ended questions. This finding has immediate implications for recent efforts by the Centers for Medicare and Medicaid Services as well as state-level quality improvement organizations to include consumer perceptions as an indicator of long-term-care quality. If experts measure these perceptions with only direct satisfaction and discrepancy index closed-ended questions, they may not identify important quality differences. Although the findings from this research pertain specifically to an intervention within a randomized controlled trial, the findings suggest that further research is needed to establish the usefulness of coding family members' open-ended comments about their satisfaction with care as they may relate to broader organizational differences in culture and staff care processes. That is, in the context of a randomized controlled trial, the mere presence of research staff may be enough to get family members to notice improvements in care. During such efforts, reactivity of the nursing home staff to the observers remains an issue for the initial period, so experts recommend longer observation periods (Schnelle, Osterweil, & Simmons, 2005). It remains to be seen if the same phenomenon occurs when part of a naturally occurring quality-improvement effort.
Studies conducted in other health care settings and populations have shown that open-ended questions yield information that is different from and more specific than that produced by closed-ended questions in terms of desired improvements in care quality (Billings & Kolton, 1999; Chesney, Lindeke, Johnson, Jukkala, & Lynch, 2005; Lehman, Reed, & Possidente, 1982). Unlike these previous studies, the results from the current study indicate a reliable method for coding open-ended responses that quantifies the scope of open-ended comments into two types of improvement (staff care processes and resident outcomes). Moreover, family members may notice nuanced differences in the subtypes of care processes and resident outcomes. These differences include the possible effects of staff's interpersonal manner of care delivery on residents' emotional well-being, as well as the frequency of care delivery and its possible effects on the residents' physical functioning. Research suggests that interpersonal aspects of care (e.g., manner of care delivery), in particular, most strongly affect patients' overall satisfaction with care (Chesney et al., 2005).
There are several important limitations of the current study. First, interviews with family members were conducted in only two intervention sites, which limited the sample sizes available for analysis and may also pose limits to the generalizability of study results. The limited sample sizes also contribute to the lack of power for multivariate analyses. However, we found significant differences in this study between experimental groups despite the small sample size, which increases the internal validity of the results. A second important limitation of this study is the high proportion of family members with "don't know" or "no response" to both closed and open-ended questions. These results suggest that many family members do not feel that they can adequately judge the quality of their relative's incontinence and mobility care. One reason for their lack of confidence may be the variation in their frequency of visits. Thus, the results may be biased in the following way: Family who visit more often are more likely to respond to the questions than family who do not visit as often. Thus, readers must consider the results from the current study as representative of frequent family visitors and possibly as overestimating the proportion who noticed an improvement. These results may not be generalizable to other aspects of daily care (e.g., medical or nursing care, food service). Future research to evaluate intervention effects or quality-improvement efforts in any aspect of long-term-care quality from the perspective of families should include both open- and closed-ended questions as well as probes (e.g., "Tell me more," "Why?") to increase family members' response rates and, thereby, the external validity of the findings. Although both open- and closed-ended questions may produce valid data for a subset of respondents, an analysis of open-ended comments allows for the identification of specific quality-improvement efforts most visible or salient to families, which has implications for individualizing care (Walker, Porter, Gruman, & Michalski, 1999) and improving quality of care and quality of life (Kane, 2001; Kane et al., 2003).
In summary, the results of this study showed that open-ended questions were more sensitive to the effects of an incontinence and mobility intervention than direct satisfaction and discrepancy-based closed-ended questions among some family members, because an analysis of only their responses to direct satisfaction and discrepancy-based closed-ended questions missed statistically and clinically significant findings (Simmons & Ouslander, 2005). Although the intervention emphasized an improvement in the staff's frequency of daily care delivery, families also noticed improvement in staff members' manner of care delivery, residents' psychological well-being, and residents' physical functioning. Inherently, interpersonal interaction occurs during the implementation of most behavioral interventions. Such interpersonal interactions have implications for residents' quality of life (Kane, 2001; Kane et al., 2003; Levy-Storms et al., 2005); thus, experts should routinely evaluate behavioral interventions for consumer perceptions of interpersonal as well as technical improvements in care quality.
|
Footnotes |
|---|
1 Department of Social Welfare, School of Public Affairs, University of California, Los Angeles.
2 Department of Geriatrics, School of Medicine, University of California, Los Angeles, and Borun Center for Gerontological Research, Los Angeles, CA.
3 Jewish Home for the Aging of Greater Los Angeles, Reseda, CA.
4 School of Medicine, Division of General Internal Medicine & Public Health, Vanderbilt University, Nashville, TN.
5 Department of Veterans Affairs Geriatric Research Education and Clinical Center and the Center for Quality Aging, Nashville, TN.
Decision Editor: Linda S. Noelker, PhD
Received for publication April 10, 2006. Accepted for publication September 14, 2006.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
