| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
a University of California, Los Angeles, School of Medicine and the Los Angeles Jewish Home for the Aging/Borun Center for Gerontological Research, Reseda, CA
b Veterans Administration, Greater Los Angeles Healthcare System, Geriatric Research, Education, and Clinical Center, Sepulveda, CA
Correspondence: Lené Levy-Storms, PhD, MPH, Jewish Home for the Aging, 7150 Tampa Avenue, Reseda, CA 91335. E-mail: llstorms{at}ucla.edu.
Decision Editor: Laurence G. Branch, PhD
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionAppendixReferences |
|---|
Key Words: Satisfaction • Preferences • Long-term care • Quality of care • Qualitative analysis
Long-term care services should reflect the preferences of consumers, and consumers' perceptions should be taken into account when judging the quality of long-term care (Burgio et al. 2001
; Wunderlich and Kohler 2001). However, collecting and interpreting consumer data from nursing home (NH) residents is problematic. Residents' advanced ages, varying levels of cognitive impairment, and fears of criticizing NH staff complicate such efforts (Bond and Thomas 1992). Despite the importance of directly asking NH residents about their care, it is not clear what interview methodology works best with this population. That is, we do not know how different interview methodologies produce information that is useful for either detecting unmet needs or designing improvement interventions.
Current approaches to assessing NH residents' perceptions of care quality include measuring their satisfaction with care, preferences for care, and experiences with care (Wunderlich and Kohler 2001). Direct (i.e., closed-ended questions) measures of satisfaction in which NH residents explicitly indicate their level of satisfaction (e.g., using a multiple category response scale such as "very satisfied" or "not at all satisfied") are hampered by a tendency for responses to skew toward greater satisfaction (Simmons and Schnelle 1999), possibly as a result of NH residents' reduced expectations for care and/or fear of staff retaliation (Grau, Chandler, and Saunders 1995; Wunderlich and Kohler 2001). For example, the results of one recent study showed that NH residents responded positively to direct satisfaction questions about the frequency of incontinence care while researchers' direct observations of care revealed that residents received assistance to the toilet fewer than once and had their pad changed fewer than twice in an 8-hour period during the day (Simmons and Schnelle 1999). Many NH residents in this same study reported being satisfied when no toilet assists occurred although these same residents reported a preference to receive multiple toilet assists each day. Despite such disadvantages, direct questions about satisfaction have face validity and are relatively easy to design and use. These advantages may explain their widespread popularity in multiple health care settings, including long-term care.
In contrast to direct satisfaction measures, other measures reflect a specific behavioral approach to assess unmet needs. For example, Simmons and Schnelle 1999 evaluated a discrepancy measure with a NH population in which residents were asked two corollary questions: "How often would you like someone on the staff to help you walk?" and "How often does someone on the staff help you to walk?" The results showed an average discrepancy of -0.9 between the number of times per day NH residents preferred walking assistance and the number of times per day residents perceived such assistance to be provided to them by staff. This relatively large discrepancy indicated unmet need in this activities of daily living (ADLs) care domain (i.e., residents preferred on average to be assisted one additional time per day). One benefit of using the discrepancy measure or other methods that directly ask NH residents about very specific aspects of ADL care (e.g., Did you have to wait too long for assistance to the bathroom this morning?) is that these methods produce more variation in responses than direct satisfaction questions (Cleary and McNeil 1988; Simmons and Schnelle 1999). However, these methods have been used primarily to measure unmet needs with regard to technical aspects of ADL care (e.g., the frequency of care) rather than interpersonal aspects of ADL care, which assess the quality of interactions between direct-care staff (i.e., nursing aides) and residents. Researchers know relatively little about the importance of interpersonal and technical aspects of ADL care in relation to satisfaction in institutional as compared to noninstitutional populations.
Among noninstitutionalized populations in outpatient settings, for example, research has shown that patients focus on interpersonal aspects of care when rating their satisfaction (e.g., a lack of warmth, failure to demonstrate real caring or concern, unprofessional conduct), often regardless of the technical (e.g., promptness of provider, perceived competency of provider) quality of care (van Servellen 1997). Objective assessments of the quality of ADL care in the institutional setting that focus only on technical aspects of care (e.g., frequency or occurrence) might be less relevant to the resident in terms of their satisfaction with care quality or unmet needs. It is, thus, important to assess NH residents' perceptions of both interpersonal and technical aspects of ADL care. However, the full range and nature of interpersonal and technical issues related to the delivery of ADL care are not known and may include a variety of subtle issues.
In this study, we describe an approach that measures residents' perceptions of the quality of both interpersonal and technical aspects of ADL care. We use open-ended questions to ask residents what they wanted to change about ADL care or the way staff help them during ADL care. This approach includes a methodology that permits the reliable coding of residents' open-ended comments into quantitative measures of unmet needs as indicated by whether the residents wanted a change or not in ADL care. Studies show that residents often make spontaneous comments during interviews that illustrate major problems with interpersonal aspects of care (Cohn and Sugar 1991; Grau et al. 1995; van Servellen 1997). For example, a resident may report satisfaction with the frequency of toileting assistance but then comment, "The staff just don't seem to have that much time for me. They are just too busy." This comment may indicate that staff are communicating to residents that they should not ask for toileting assistance because of time pressures. This example from a participant in this study suggests an approach to measuring perceptions of ADL care that either analyzes NH residents' spontaneous comments or that analyzes their comments in relation to open-ended questions (e.g., What would you like to see changed?) during an in-person interview.
The purpose of this study was to compare open-ended questions to direct satisfaction and discrepancy questions in estimating unmet needs and to discuss the usefulness of each interview methodology for designing improvement interventions. Specifically, for each ADL care domain, the methods included: (a) direct measures of residents' satisfaction with the frequency or occurrence of ADL care using closed-ended questions, (b) discrepancy measures based on a comparison of residents' preferences for the frequency or occurrence of ADL care versus their perception of the ADL care received (Bond and Thomas 1992; Davis, Sebastian, and Tschetter 1997; Simmons and Schnelle 1999), and (c) open-ended questions that ask residents what they wanted changed about ADL care. The three major questions that this research addresses are: (a) How do the three methods compare with respect to measuring unmet needs? (b) What aspects of interpersonal and technical care emerge as important according to NH residents' open-ended comments? (c) Which methods generate information most useful for the design of improvement interventions?
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionAppendixReferences |
|---|
). The CPS is derived from five MDS items with scores ranging from 0 (cognitively intact) to 6 (severe cognitive impairment or comatose; Hartmaier, Sloane, Guess, and Koch 1994). CPS scores correlate with other established measures of cognitive status including the Folstein, Folstein, and McHugh Mini-Mental Status Examination (Hartmaier et al. 1995). Of the 111 residents, 17 (15%) either failed the cognitive screen on two separate days or had CPS scores of 5 or 6 (indicating severe cognitive impairment). The remaining 94 NH residents, who were cognitively intact (CPS = 0) or had mild (CPS = 1 or 2) to moderate (CPS = 3 or 4) cognitive impairment, were eligible for interviews. Seventy (74%) of these 94 residents successfully completed interviews; 24 residents either refused to be interviewed, had died, or were not able to be located for an interview. There was no difference in the proportion of NH residents with CPS scores of 0–2 (n = 43) versus 3–4 (n = 51) who completed at least one interview (84% and 69%, respectively, 
2 = 2.88; p = .09). There was also no difference in the average total number of interviews completed by each of these two groups (CPS: 0–2, mean = 3.93 ± 2.29; CPS: 3–4, mean = 3.41 ± 2.83; t = 0.98; p = .33). For each ADL domain, we also compared these two CPS groups on the number of don't know, missing/no response, or nonsense responses and found no difference. These results support previous research that has shown that most NH residents with CPS scores of 0–2 and a substantial proportion of those with CPS scores of 3–4 can provide reliable information to closed-ended questions about their perceptions of ADL care received (Simmons et al. 1997).
Interview Protocols
We developed separate interview protocols relevant to assistance in each of seven ADL care domains: walking, mealtime, dressing, showering, getting in and out of bed, toileting, and pad changes. Each interview required an average of 10 min to complete. For each ADL domain, we attempted to interview only those residents who were rated as needing assistance (i.e., supervision to total assistance) in that particular domain as indicated by their most recent MDS records (MDS, Section G: Physical Functioning Items: 1) ADL Self-performance: d) walking, g) dressing, h) eating; and Section H: 1) Continence Items: b) Bladder; Health Care Financing Administration 1999). For example, if residents were rated as completely continent (MDS, Section H: 1b, rated 0), they were not interviewed about pad changes. We did not interview some residents in all ADL care domains, even if the domains were applicable to them, because of interviewee burden. Our balancing interviewee burden and the goal to interview 50 persons per ADL care domain resulted in a total of 70 residents completing at least one interview. The only ADL domains that were not applicable to all residents were the pad changes and walking domains (not applicable to 14 and 15 residents, respectively, out of 94). The distribution of the number of interviews completed for the 70 residents who participated in at least one interview was: 1 (7%), 2 (3%), 3 (8%), 4 (28%), 5 (14%), 6 (14%), and 7 (25%). Thus, most residents participated in four to seven interviews related to ADL care domains for which residents required assistance from staff.
The participating NH considered these interviews as internal quality assessment, and the facility's Institutional Review Board approved the interview protocols; thus, consent was provided at the facility level. Research staff used a standardized script to introduce the interview protocol, which the Institutional Review Board approved, such that residents had an opportunity to refuse to participate at the beginning of the interview. Trained research staff conducted the one-on-one interviews in a private room where NH staff or other NH residents were not present.
There were three types of interview questions (Table 1 ): (a) direct satisfaction questions about the frequency or occurrence of ADL care, (b) discrepancy-based questions about the frequency or occurrence of ADL care, and (c) open-ended questions in which residents' comments were recoded into quantitative measures of wanted a change or not. The direct satisfaction questions asked NH residents for yes/no responses to questions that used the word satisfaction (e.g., "Overall are you satisfied with how often someone helps you to walk?"). The discrepancy measures did not use the term satisfaction but, instead, asked residents two related questions for each ADL care domain. We first asked nursing home residents to express their preferences for care in a specific ADL domain (e.g., "How many times during the day would you like staff to help you walk?"). Then, we asked them how often they received care in this ADL domain (e.g., "How many times during the day do staff help you to walk?"). With this information, we calculated the discrepancy between NH residents' preferences for and their perceptions of the frequency or occurrence of ADL care received. For example, if NH residents reported that they would like walking assistance three times per day but only received one walking assist per day, then such responses resulted in a discrepancy of -2. This discrepancy measure works best in ADL care domains in which the frequency of assistance is the most important dimension of perceived quality. For ADL domains where this was not the case, we made modifications for the occurrence of assistance with dressing, mealtime, and getting in or out of bed (see the A for a complete listing of all three types of questions for each ADL care domain). The third type of question to assess unmet needs was open-ended (e.g., "If you could change something about the way staff help you with your food, what would it be?") and from spontaneous comments made by the resident after any closed-ended question. For this open-ended measure, two raters independently coded residents' responses as to whether the responses indicated a desire for change (e.g., regarding toileting assistance: "When I want to go, I want to go!"). Inter-rater reliabilities for wanted a change using Kappa coefficients averaged .73 across all ADL care domains and ranged from a low of .50 (get in or out of bed domain) to a high of .88 (mealtime assistance domain).
|
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionAppendixReferences |
|---|
The results in Table 1 show that the direct satisfaction questions produced the lowest estimates of unmet needs (i.e., percentage of residents who were dissatisfied) with responses ranging from only 3% who were dissatisfied with the frequency of pad changes to a high of 24% who were dissatisfied with the frequency of walking assistance. Column 2 lists the discrepancy-based estimates of unmet needs. Specifically, estimates of unmet needs based on the discrepancy measures were significantly higher as compared to column 1 for each ADL care domain (McNemar's; p < .05–p < .01) with a range from 36% in the showering and dressing domains to 76% in the walking domain. Column 3 illustrates the percentage of residents who reported that they wanted a change in response to the open-ended questions, which produced estimates of unmet needs that were between those estimates produced by the direct satisfaction and discrepancy measures. In most ADL care domains, estimates of unmet needs based on NH residents' open-ended responses were significantly higher (McNemar's; p < .05–p < .01) than estimates based on direct satisfaction questions, with a range from 18% in the mealtime domain to 33% in the getting in or out of bed domain.
Any methodology that attempts to measure NH residents' perceptions of the quality of ADL care should have as a primary goal the identification of specific ways to improve it. In this regard, the discrepancy measure provided more specific information related to designing interventions to improve the frequency or occurrence of ADL care than the direct satisfaction approach. The direct satisfaction question failed to identify specific ways that dissatisfaction or unmet needs could be addressed. For example, it is helpful for improvement purposes to know that a resident would like someone to render walking assistance three times a day and toileting assistance four times per day in order to have their needs met with respect to these two ADL care domains. Direct satisfaction questions do not capture this type of specific preference information.
Differences also exist between the discrepancy and open-ended measures. Table 2 shows a comparison of the responses between the discrepancy-based and open-ended questions. It displays the prevalence of NH residents who reported unmet or met needs in response to discrepancy-based questions and the percent of NH residents within these two groups who wanted a change according to analyses of their open-ended comments. Two types of information emerge from these results. First, among residents with unmet needs from the discrepancy-based questions, the results indicate additional aspects of unmet needs based on what percentage of residents wanted changes in ADL care. Second, among residents with met needs from the discrepancy-based questions, the results indicate what percentage of residents wanted a change in aspects of ADL care not captured in the discrepancy questions. For example, the highest percentages of residents who wanted a change among those with met needs occurred in the dressing and pad change domains (37% and 38%, respectively). These results suggest that residents have unmet needs beyond the frequency or occurrence of ADL care that include subtle aspects of ADL care. The next section describes how specific open-ended comments from residents were used to identify problems with ADL care that were not identified with the other measures.
|
Among residents with unmet needs according to the discrepancy-based measures, a summary of exemplary issues from the open-ended comments included overassistance from staff, a lack of continuity of care, perceived incompetence of staff, a desire for more encouragement from staff, a frustration with having to prompt staff for ADL care, and a lack of sensitivity from staff (see Table 3 , rows A, C, E, G, I, K, and M).
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionAppendixReferences |
|---|
; Grau, Chandler, and Saunders 1995; Simmons and Schnelle 1999; Wunderlich and Kohler 2001). The results of this study suggest that a combination of open- and closed-ended questions is more useful for planning and evaluating interventions to improve the quality of ADL care for several reasons. First, the direct satisfaction approach produces extremely low estimates of unmet needs, which are contradicted by the other two measures. Furthermore, the direct satisfaction questions do not provide information that can be used to design specific interventions. For example, even in the rare case where NH residents report dissatisfaction, their responses are not indicative of what they would like to see changed. In addition, NH residents who report satisfaction with this approach give responses to the open-ended questions indicating that they would like specific changes in ADL care. In contrast, the discrepancy measure provides information about residents' preferences concerning the frequency or occurrence of care (e.g., the number of times they would like walking assistance or when they would like to get up in the morning) that could be directly translated into individualized care plans. The open-ended questions supplement these preference data with information about different aspects of ADL care that could not have been easily gained from the closed-ended questions a priori (e.g., when, how, or by whom the residents would like to be walked). The open-ended questions also provide information about interpersonal aspects of care that are not directly solicited by the closed-ended questions (e.g., desire to feel less rushed or more respect from nursing aides). Thus, the preference and open-ended questions produce complementary information that reinforces the contention that ADL care can be improved and provides the specific information necessary to implement improvement.
The analyses of the open-ended comments and recoding into wanted a change serve to illustrate how important, detailed information can be obtained from residents in their own words. That we were able to reliably code residents' responses to the open-ended questions into wanted a change (i.e., unmet needs) or not is an important extension of previous research (Grau et al. 1995). Also, consistent with the results of previous studies (Simmons et al. 1997; Simmons and Schnelle 1999), residents with mild to moderate cognitive impairment (MDS-derived CPS scores of 0–4) are able to complete the interviews and provide reliable information in response to all three methodologies.
The primary disadvantage of the open-ended approach, however, is that it requires significantly more time and skill to both record and code NH residents' open-ended comments than either of the other methodologies. This is less of a limitation in a research project than it is in practice. This problem could be potentially solved by using the open-ended questions during the initial phases of an improvement project when it is most important to complete a thorough assessment of all aspects of residents' unmet needs, especially unmet needs that are specific to interpersonal aspects of care. Once all critical aspects of unmet needs are identified with the open-ended questions, they can be converted into specific behavioral or preference questions that would be useful for tracking changes in unmet needs once the improvement activities are implemented. For example, if feeling rushed by nursing aides is a consistent theme then a behavioral question that would be easier to score could be phrased as follows: "Did you feel rushed this morning when you were helped out of bed?" Open-ended questions could then be used periodically to assess if new dimensions of unmet needs occur or to identify aspects of care that constitute unmet needs for a particular NH resident.
In conclusion, efforts to incorporate long-term care consumer perspectives into assessments about ADL care quality in NHs should incorporate both direct questions to measure residents' preferences in combination with open-ended questions, at least initially, and only residents with severe cognitive impairment should be excluded from this process. Open-ended comments can either be descriptively analyzed, as we do in Table 3 , or an in-depth qualitative analysis could be conducted to identify themes (see Strauss and Corbin 1990). These themes could then be the basis for developing specific, closed-ended questions related to aspects of ADL care or daily quality of life within NHs. On the basis of the results of this study, we conclude that open-ended questions may be the most efficient method to elicit information about interpersonal care issues, although not exclusively since the open-ended responses also provide insight into technical issues. The importance of both technical and interpersonal aspects of ADL care as highlighted in this research provides a basis for future investigations regarding relationships between NH residents' unmet needs, quality of ADL care, and their quality of life (Kane 2001).
The Forum
Book Reviews
Practice Concepts
|
Acknowledgments |
|---|
Received for publication October 22, 2001. Accepted for publication March 20, 2002.
|
Appendix |
|---|
|
TOPAbstractMethodsResultsDiscussionAppendixReferences |
|---|
Toileting
Direct Satisfaction: Overall, are you satisfied with how often you get help to use the toilet, bedpan, or urinal?
Discrepancy: (a) How many times during the day does someone on the staff help you to use the toilet, bedpan, or urinal? (b) How often would you like to be helped to use the toilet, bedpan, or urinal?
Want a Change: If you could change something about the toileting schedule and/or the way staff help you to use the toilet/bedpan/urinal, what would it be?
Walking
Direct Satisfaction: Overall, are you satisfied with how often someone helps you to walk?
Discrepancy: (a) How many times during the day does someone on the staff help you to walk? (b) How many times would you like to walk each day?
Want a Change: If you could change something about the walking schedule and/or the way staff help you walk, what would it be?
Pad Changes
Direct Satisfaction: Overall, are you satisfied with how often your pad is changed?
Discrepancy: (a) How many times during the day does someone on the staff (the people who work here) change your pad? (b) How many times during the day would you like for your pad to be changed?
Want a Change: If you could change something about the pad changing schedule and/or the way staff changes your pad, what would it be?
Dressing
Direct Satisfaction: Overall, are you satisfied with the help you receive to get dressed or get ready for the day?
Discrepancy: (a) Do you need help getting dressed or changing clothes? (b) Does someone on the staff help you get dressed or change?
Want a Change: If you could change something about the way staff help you to get ready and/or the things they do for you, what would it be?
Bathing/Showering
Direct Satisfaction: Overall, are you satisfied with how often you receive a shower or bath?
Discrepancy: (a) How often do you have a shower or bath? (b) How often would you like to have a shower or bath?
Want a Change: If you could change something about your shower or bath schedule and/or the way staff help you to take a shower or bath, what would it be?
Mealtime or Feeding Assistance
Direct Satisfaction: Overall, are you satisfied with how much someone helps you with your food or during meals?
Discrepancy: Would you like someone to help you with your food more or less often (than they do now)?
Want a Change: If you could change something about the way staff help you with your food, what would it be?
In or Out of Bed
Direct Satisfaction: Overall, are you satisfied with when you [get out of bed in the morning or go back to bed at night]?
Discrepancy: (a) About what time do you get out of bed in the morning or go back to bed at night? (b) About what time would you like to get out of bed in the morning or to go back to bed at night?
Want a Change: If you could change something about your bedtime schedule and/or the way staff help you in and out of bed, what would it be?
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionAppendixReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  A. N. Rahman and J. F. Schnelle The Nursing Home Culture-Change Movement: Recent Past, Present, and Future Directions for Research Gerontologist, April 1, 2008; 48(2): 142 - 148. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Levy-Storms, J. F. Schnelle, and S. F. Simmons What Do Family Members Notice Following an Intervention to Improve Mobility and Incontinence Care for Nursing Home Residents? An Analysis of Open-Ended Comments Gerontologist, February 1, 2007; 47(1): 14 - 20. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. E. Fisher, L. D. Burgio, B. E. Thorn, and J. M. Hardin Obtaining self-report data from cognitively impaired elders: methodological issues and clinical implications for nursing home pain assessment. Gerontologist, February 1, 2006; 46(1): 81 - 88. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Levy-Storms, S. F. Simmons, V. F. Gutierrez, D. Miller-Martinez, K. Hickey, and J. F. Schnelle A Refined Protocol for Coding Nursing Home Residents' Comments During Satisfaction Interviews J. Gerontol. B. Psychol. Sci. Soc. Sci., November 1, 2005; 60(6): S326 - S330. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. M. Carroll, K. Vetor, S. Holmes, and K. P. Supiano Ask the consumer: An innovative approach to dementia-related adult day service evaluation American Journal of Alzheimer's Disease and Other Dementias, September 1, 2005; 20(5): 290 - 294. [Abstract] [PDF]
|
|
|
|
|
|
S. F. Simmons and J. G. Ouslander Resident and Family Satisfaction With Incontinence and Mobility Care: Sensitivity to Intervention Effects? Gerontologist, June 1, 2005; 45(3): 318 - 326. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Levy-Storms and D. Miller-Martinez Family Caregiver Involvement and Satisfaction With Institutional Care During the 1st Year After Admission Journal of Applied Gerontology, April 1, 2005; 24(2): 160 - 174. [Abstract] [PDF]
|
|
|
|
 |
|
 J. F. Schnelle Improving Nursing Home Quality Assessment: Capturing the Voice of Cognitively Impaired Elders J. Gerontol. A Biol. Sci. Med. Sci., March 1, 2003; 58(3): M238 - 239. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
