| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
a School of Public Health, Curtin University of Technology, Perth, Western Australia, Australia
b Freemasons Centre for Research into Aged Care Services, Division of Health Sciences, Curtin University of Technology, Perth, Western Australia, Australia
Correspondence: Duncan P. Boldy, PhD, Department of Health Policy and Management, School of Public Health, Curtin University of Technology, GPO Box U 1987, Perth 6845, Western Australia. E-mail: dboldy{at}health.curtin.edu.au.
Decision Editor: Laurence G. Branch, PhD
 |
Abstract |
|---|
 TOP Abstract Measuring SatisfactionMethodsResultsDiscussionReferences |
|---|
Key Words: Long-term care • Quality • Questionnaire • Reliability • Validity
The quality of aged care is a fundamental concern for elderly people and their families, as well as policy makers at both state and federal levels. With the aim of improving quality of residential aged care in Australia, the federal government developed and introduced outcome standards (31 grouped under seven broad headings) in 1987 for nursing homes (
Gibson 1998
). Subsequently, outcome standards (25 grouped under six broad headings) were also introduced for hostels in 1991. A hostel is a low-care facility in which residents are more independent and receive personal but not nursing care. These standards have been used as a basis for the external monitoring and assessment of residential aged care facilities by the federal government. However, this system did not systematically take residents' views and satisfaction into consideration, so residents were not being given full opportunity to have their say or to participate in the provision and planning of their care.
An Aged Care Structural Reform Package was introduced in late 1996 as an attempt to address these problems, and outcome standards were revised to reflect a number of changes within aged care provision. The resulting accreditation standards mainly build on previous ones, and embody a strong new focus on continuous improvement, which includes the requirement for appropriate resident input (
Aged and Community Care Division 1998). Overall, the Australian federal government has moved toward a "partnership quality improvement process" with more of a focus on provider-initiated (proactive) management systems and on incorporating the views of residents. However, despite resident input having been formalized as a key aspect of the new accreditation process for residential aged care facilities, no specific approach has been prescribed for how residents' views can best be incorporated into the service delivery process (
McDonald 2000). Resident satisfaction surveys provide one such approach (
Boldy and Grenade 2001). Clearly, residents' assessments of the quality of care of the services they receive will play an increasingly important role in the provision of aged care.
|
Measuring Satisfaction |
|---|
|
TOPAbstractMeasuring SatisfactionMethodsResultsDiscussionReferences |
|---|
;
Pascoe and Atkinson 1983;
Rubin, Ware, Nelson, and Meterko 1990;
Ware and Hays 1988). Of the few relevant aged care studies identified, resident satisfaction is not often the primary focus (e.g.,
Namazi, Eckert, Kahana, and Lyon 1989;
Pearson, Hocking, Mott, and Riggs 1993). Moreover the focus of interest has usually been general life satisfaction (e.g.,
Thomas and Hayley 1991) or broader aspects of quality of care (e.g.,
Phillips-Doyle 1992,
Phillips-Doyle 1995;
Rhys-Hearn 1986). Other researchers adopted resident satisfaction measures specifically for nursing home residents based on an existing satisfaction instrument that was originally designed for acute care, for example, the Satisfaction with the Nursing Home Instrument (SNHI;
Ryden et al. 2000).
One of the primary problems associated with the measurement of satisfaction relates to its conceptualization, in terms of the dimensions to be measured and specific issues to be explored. Other problems include ignoring the measurement properties of an instrument, such as its reliability and validity (e.g.,
Pearson et al. 1993;
Thomas and Hayley 1991;
Weihl 1981), using a summated or averaged overall satisfaction score (assuming all items have equal contribution to the latent construct and ignoring its multidimensional nature; e.g.,
Kruzich, Clinton, and Kelber 1992), and using only a global overall satisfaction item (e.g.,
Duffy and Ketchand 1998). These issues are discussed further below.
Relevant Context
Schneider 1991 noted that one of the common problems with consumer satisfaction measures is that customers are asked to respond to questions that are contextually irrelevant. Hence it is critical that the measurement of customer service quality perceptions (such as customer satisfaction) is targeted to the specific service facets that are relevant for that particular market segment.
It is clear that the dimensions covered must be relevant and important to the care recipients. For example, in high care facilities (nursing homes) it is irrelevant and inappropriate to ask residents if the discharge planning is appropriate; a more relevant question might be whether the staff members are responsive and friendly. The relevant service facets in nursing home facilities, as compared to high technology hospitals, are obviously quite different. Therefore, a generic satisfaction instrument cannot be applied across different types of health care settings, such as acute care and aged care.
Consultation With Consumers
Lack of consultation with consumers themselves, in the initial identification of the dimensions and issues to be incorporated within an assessment instrument, is another common issue (
Carr-Hill, Dixon, and Thompson 1989;
Freytag and Rossiter 1986;
International Hospital Federation 1988;
Kelman 1976;
Lebow 1983). Focus groups have been suggested as a means of incorporating consumers' points of view (
Dickens, Browning, Jellie, and Thomas 1993) and for identifying the words, concepts, and language of clients as a basis for developing questionnaires (
Strasser and Davis 1991). This approach was adopted in a recent Australian project in the development of a resident satisfaction assessment package (
Boldy and Grenade 2001).
Multidimension Nature of Satisfaction.
Although there is increasing consensus that satisfaction with health care is a multidimensional construct (e.g.,
Avis, Bond, and Arthur 1995;
Dickens et al. 1993;
Lebow 1983;
Pascoe and Atkinson 1983), in practice many researchers still use a global "overall" satisfaction item or unidimensional measure.
Levkoff and DeShane 1979 noted that the overall reaction to health care services is commonly used to represent the level of satisfaction, and discrete service components are frequently neglected or totally ignored. The obvious problem is that clients may rate a health service overall as satisfactory even though they may not be happy with some specific components (
Corrigan 1990).
Elwell 1984 also noted that high standards of cleanliness, safety, and health, for example, may not be necessarily related to beautiful features, diversional activities, or other quality-of-life indicators. It is clear that only limited information can be obtained from a single "overall" item.
Importance of Measurement Properties
Lack of validity and reliability testing of satisfaction measuring instruments is another major criticism in the health arena (
Ware and Hays 1988). Issues of measurement error are also rarely addressed, and only few researchers have empirically assessed measurement properties of their questionnaires (
Hall and Dornan 1990;
Rubin 1990). A valid and reliable resident satisfaction instrument is therefore required to more accurately assess residents' satisfaction in residential aged care. The purpose of the present study was firstly to examine the dimensionality of resident satisfaction, via the Resident Satisfaction Questionnaire (RSQ) developed by
Boldy and Grenade 1998, and secondly to develop a reliable and valid short form RSQ for regular monitoring.
|
Methods |
|---|
|
TOPAbstractMeasuring SatisfactionMethodsResultsDiscussionReferences |
|---|
Research Design
A cross-sectional resident satisfaction survey was designed to collect information over a period of approximately 12 months (April 1998–April 1999) via a self-complete questionnaire (RSQ). An invitation letter with an information sheet, an overview of the proposed research, and an agreement form with a postage-paid reply envelope were mailed to all directors of nursing or managers of the selected facilities. A facility-specific report of the survey findings was offered to the participating facilities upon request.
Sample and Procedure
Selection of Aged Care Facilities
A list of residential aged care facilities in Western Australia was obtained from the Commonwealth Department of Health and Family Services. The sampling frame for the study included all private, public and charitable aged care facilities—a total population of 294 facilities with 12,012 beds. Stratified sampling was employed to ensure that a wide variety of facilities were included, by first categorizing institutions by size of facility, then by type and location. The size of the aged care facilities was divided into the following categories: small (
30 beds), medium (31–59 beds), and large (
60 beds). Facilities were categorized as "high care (nursing home)" and "low care (hostel)" and location according to "metro" and "nonmetro."
When a refusal occurred, a replacement facility was selected randomly from the same stratum. Facilities were asked to nominate a coordinator to help identify eligible residents and assist in the distribution and return of the questionnaires.
Selection of Residents
To qualify for selection, a resident had to understand English, be sufficiently cognitively competent, have an energy level sufficient to participate in the survey, and have lived in the facility for more than 4 weeks. The staff or manager determined the eligibility of each resident based on the selection criteria and in consultation with the researchers. All residents satisfying the selection criteria were invited to participate.
Residents were first informed by the facility manager or survey coordinator regarding the research activities approximately 1 week before the survey commenced. Residents were then either approached by the facility manager, survey coordinator, or the researchers. To minimize the tendency of respondents in satisfaction research to give overly positive responses (
Jimmieson and Griffin 1998;
Ware and Hays 1988), complete confidentiality and anonymity were assured via an information sheet given to each resident. Such assurance was also stated clearly on the first page of the questionnaire. This is particularly important because of the acknowledged reluctance of older people to criticize services on which they are dependent (
Goldberg and Connelly 1982;
Pearson et al. 1993).
Where possible, residents completed the questionnaire themselves. If assistance was needed, help was offered, either to read the questions aloud for residents or to assist with completion. Alternatively, families, friends or volunteers could also be invited to assist residents. Approximately two thirds of participating residents required some form of assistance. Although it could be argued that providing such assistance could bias responses, survey coordinators strongly emphasized to all individuals who assisted (family, friends and staff) that the questionnaires should be completed based solely on the residents' view. The specific issue of possible influence of assistance on responses is addressed later, under resident demographic characteristics.
The strength of permitting resident assistance is that a wider range of residents were able to participate (see later), particularly those with a mild degree of cognitive impairment, as was strongly desired. A "passive consent" approach was adopted; that is, if participants completed the questionnaire and returned it, this was taken to imply that they consented to participate.
Measurement Instrument: RSQ
The RSQ is part of the resident satisfaction assessment package developed by
Boldy and Grenade 1998 for assessing residents' satisfaction in aged care facilities. This package was developed based on a review of relevant literature and extensive consultations with service providers and consumer representatives in a wide range of residential aged care facilities (covering both nursing homes and hostels) around Australia.
A focus group approach, involving as many as 129 residents chosen so as to represent as wide a range as possible, was used to identify relevant and important dimensions from the residents' perspective, which then formed the basis for the self-complete questionnaire. The RSQ consists of 10 dimensions—namely, moving to the home (settling-in process), the resident's room, the home, passing the time, social life, links with the community, resident services, staff care, resident involvement, and other issues. A 3-point scale response (no = 1, depends = 2, yes = 3) or 4-point scale response (poor = 1, fair = 2, good = 3, excellent = 4) format was used for the 50 items. All "unsure/can't remember" responses were coded as missing values. An overall rating item, using a 4-point scale response, was also included at the end of each dimension (
Boldy and Grenade 1998).
Content validity of the questionnaire was addressed by testing and refining the instrument several times. Facility staff and administrators were also consulted as part of the instrument development and testing process. Feedback from a variety of "stakeholders" (industry and consumer groups) was also taken into account. Interrater reliability was found to be acceptable (
Boldy and Grenade 2001). The resident satisfaction manual gives more detailed information regarding the instrument's theoretical background and scale development (
Boldy and Grenade 1998).
Exploratory Factor Analysis
Exploratory factor analysis (EFA) was first used to summarize the factor structure of resident satisfaction contained in the observed variables (items). The analysis was performed based on the principal axis factoring (PAF) method with varimax rotation on the correlations of the observed variables, using SPSS (
SPSS, Inc 1999). Weak items (i.e., loadings < 0.3), or items that cross-loaded on two different factors and had a loading of > 0.3 on the second factor, were deleted from further analysis. The context and meaning of items were also taken into account during the process.
Confirmatory Factor Analysis
Confirmatory factor analysis (CFA) was conducted to test the viability of a hypothesized structure that had been formulated from theoretical considerations and results of the EFA. CFA was conducted using LISREL (
Joreskog and Sorbom 1999). Satisfaction scores for each dimension were obtained using factor score regressions generated from the CFA as proportional weights to combine item scores.
In addition to theoretical and practical considerations, assessment of model adequacy was based on the following goodness of fit criteria. Normed Chi-square (
2/df) < 3, the root mean square error of approximation (RMSEA) < .05, non-normed fit index (NNFI) > .90, the comparative fit index (CFI) > .90, the goodness of fit index (GFI) > .90 and the adjusted goodness of fit index (AGFI) > .90 (
Byrne 1998;
Joreskog and Sorbom 1999;
Kline 1998;
Schumacker and Lomax 1996).
|
Results |
|---|
|
TOPAbstractMeasuring SatisfactionMethodsResultsDiscussionReferences |
|---|
Due to confidentiality, it cannot be determined whether residents who did not return the questionnaire differed from the participants in any systematic way. Response rates were improved by implementing extensive follow-up procedures within each facility (e.g., reminder notices given out by the survey coordinator at resident meetings and via notice boards and announcements during meal times) about 10–14 days after the distribution of the questionnaires.
Resident Demographic Characteristics
The resident sample comprised 284 men and 849 women. Residents ranged in age from 45 to 103 years, with a mean of 83.2 years (SD = 8.7). The majority of residents were female (75%), born in Australia (63%), spoke English as their first language (94%), and had their own room and bathroom (65%). Overall, 39% of study residents had lived in the facility for more than 3 years. Thirty-seven percent were assessed as "high care," which includes some residents with a mild degree of cognitive impairment.
The 37% of residents who were born overseas is demographically comparable to that of all residential aged care facilities in Western Australia (36.5%;
Australian Institute of Health and Welfare 1998). This fact, together with the high resident response rate, means that any resident response bias is likely to be minimal.
Although 72% of residents required assistance to complete the questionnaire, help was mostly provided by family and friends. Staff provided assistance for 11% of residents. As it was anticipated this might happen in some facilities, and to minimize any undue influence, the survey coordinators were asked to ensure that all the questionnaires were completed based solely on the residents' view. The overall satisfaction item "Taking everything into account, how would you rate the home?" was used as an indicator to assess whether there was any association between "who filled out this questionnaire" and resident satisfaction. No significant association was found between these two variables, 2 (6, N = 1123) = 10.07, p = .122, providing some evidence that residents' views had been reasonably captured and did not vary according to whether anyone assisted them in completing the questionnaire.
Exploring the Factor Structure of Resident Satisfaction
The sample size of 1,146 residents is sufficient for the required EFA. Most items were found to be only minimally skewed or kurtotic. A considerable number of item correlations also exceed .3. The Bartlett test of sphericity (2=20456, df=1225, N=1146, p=.00) is significant and the Kaiser-Meyer-Olkin measure of sampling adequacy is .95, which justifies proceeding with factor analysis (
Coakes and Steed 1997).
Ten factors were extracted with eigenvalues greater than 1, accounting for 51.2% of the total item variance. Loadings of variables on factors, communalities, and percentage of variance explained are shown in Table 1 , with loadings under .3 omitted to facilitate interpretation. According to the scree plot and percentage of variance explained, the last four factors may be dropped. The extracted 10 factors are:
|
Resident Satisfaction Constructs and Dimensionalities
The EFA results above, together with the evidence gained from previous research (
Boldy and Grenade 1998,
Boldy and Grenade 2001), provided a guide for selecting items for each satisfaction component and specifying an empirically based factor structure. All satisfaction components were tested independently in the form of a one-factor congeneric model (
Byrne 1998;
Joreskog and Sorbom 1996). To keep the measurement model simple, only 3–5 observed variables (items) were retained per factor in the CFA. Items were excluded on the basis of relatively large measurement errors (> .5), weak factor loadings (< .5), high correlation between items (> .85), low item reliability (< .5), and large modification index or residual (> 2.58), following the recommendations of
Kline 1998 and
Byrne 1998. The meaning and context of individual items were also taken into account. After deleting all "noisy" items, the selected 24 items were then tested jointly in the form of a one-factor model and a multifactor measurement model.
Although it has been recognized that satisfaction is best represented by a multidimensional model, many researchers still aggregate all items to obtain an overall satisfaction score. This implicitly assumes that satisfaction is a unidimensional concept. The following analysis examines the appropriateness of such an assumption.
Single Factor Satisfaction Measurement Model
The one-factor model, whereby all 24 items were loaded onto a single factor "general resident satisfaction," was tested first. Table 2 shows that the 2 (252, N = 935) value (1367.55) is significant at the .001 level, although this is not surprising given the large sample size (935 cases with complete data). More informative is the 2/df ratio, which for this model is greater than 3. In addition, the NNFI is less than .90 and RMSEA is greater than .05. Because three of the six criteria are not met, it appears that a one-factor, "general resident satisfaction" model does not fit the data adequately.
|
The various indices of fit shown in Table 2 suggest that satisfaction structure can be adequately described by the 6 correlated factors which are graphically presented in Fig. 1. Latent constructs (satisfaction components) are shown as ellipses and questionnaire items measuring these latent constructs are represented as rectangles. Arrows pointing from the latent constructs (ellipses) to the observed variables (rectangles) exhibit the corresponding factor loadings. All loadings are greater than .5.
|
). As shown in
Fig. 1, all factor loadings are greater than .5 and are significant at the .001 level, suggesting that convergent validity is supported.
Discriminant validity refers to the distinctiveness of the factors measured by different sets of indicators (
Kline 1998). Poor discriminant validity is evidenced by high factor correlations (> .85). Here, the estimated correlations among the 6 factors range from .558 to .802.
A second-order factor was next fitted, with factor loadings for each of the 6 factors being presented in Table 3 . Although the resulting GFIs are satisfactory (see Table 2 ), the first-order 6-factor solution is preferred to measure the underlying domains of resident satisfaction and for the practical development of a short form RSQ.
|
). Coefficients for all satisfaction component composite variables are high (ranging from .81 to .90; see Table 3 ), and only a few measurement errors are slightly greater than .5 (see
Fig. 1), suggesting that the observed variables are reliable measures of the underlying latent constructs. The test–retest reliability of the RSQ was also assessed. Two identical RSQs were administrated to 33 residents within a 2-week interval in one large hostel. The observed intraclass correlation coefficients range from .74 to .90 (Table 3 ), suggesting that the RSQ has good test–retest reliability in hostel residents.
|
Discussion |
|---|
|
TOPAbstractMeasuring SatisfactionMethodsResultsDiscussionReferences |
|---|
The 6-factor model via the RSQ is a psychometrically sound and valid instrument. Findings revealed good composite reliability and satisfactory convergent and discriminant validity. This represents an important contribution to satisfaction research in residential aged care settings, where a "good" tool is needed so that residents' views related to improving service quality and/or for the purpose of accreditation can be incorporated. It has added significant value to the resident satisfaction assessment package developed by
Boldy and Grenade 1998.
Short Form RSQ
Many participating residents commented that the RSQ is too long and some tended to get tired and frustrated, and lost concentration. For the purpose of regular monitoring, a valid and reliable yet shorter but comprehensive version of the questionnaire would reduce the residents' burden. Such a short form of the RSQ can be developed based on the 6-factor resident satisfaction measurement model. It will provide a practical and useful instrument for regular internal (e.g., in-house quality improvement) and external (e.g., accreditation) use.
Multidimensional Versus Unidimensional Satisfaction Construct
The superiority of the 6-factor over the 1-factor model implies that residents distinguish between different components of residential aged care services. Therefore resident satisfaction as assessed via the RSQ is best represented by six dimensions. The inappropriateness of applying a general resident satisfaction score is thus reinforced. This finding is congruent with the need for a multidimensional approach to resident satisfaction (
Corrigan 1990).
Limitations
Various statistical criteria have been used to guide the selection of items for CFA. It should be emphasized that the meaning and context of individual items also needs to be taken into account in the selection process. Similarly, the choice of competing measurement models should not be based solely on statistical grounds. Some related discussion on model selection can be found in
Breckler 1990.
The RSQ was developed based on the views of the residents in Australian aged care facilities. If the instrument is to be used in another country, then it would be advisable to undertake some sort of "pre-study" to assess whether the dimensions and questions included all relevant aspects, pertinent to that country's aged care resident mix.
Implications
Customer satisfaction can be viewed as an advance warning system (
Bernhardt, Donthu, and Kennett 2000). The continuous monitoring of residents via the short form RSQ can detect early warning signals in terms of declining satisfaction scores on particular components. A facility can then choose to take action to reverse the decline before residents' quality of life and the facility's performance deteriorate further. Such a process has long-term benefits by reducing complaints handling and increasing employee retention, consequently saving the costs associated with hiring new employees (
Bernhardt et al. 2000).
Information from regular application of the short form RSQ could form the basis of a resident satisfaction database for quality improvement, identifying best practices, searching for most influential factors, and other management, research or policy purposes. The collection of information across different facilities would enable comparisons among facilities benchmarking for both short- and long-term planning. Therefore, further work and resources for the development of the short form RSQ is warranted, in terms of its layout, format, the flow of the question items, and instructions to complete the questionnaire.
The Forum
|
Acknowledgments |
|---|
Received for publication February 2, 2001. Accepted for publication May 9, 2001.
|
References |
|---|
|
TOPAbstractMeasuring SatisfactionMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  C.-m. Hsieh Using Client Satisfaction to Improve Case Management Services for the Elderly Research on Social Work Practice, November 1, 2006; 16(6): 605 - 612. [Abstract] [PDF]
|
|
|
|
 |
|
 P. Edelman, M. Guihan, F. B. Bryant, and D. J. Munroe Measuring resident and family member determinants of satisfaction with assisted living. Gerontologist, October 1, 2006; 46(5): 599 - 608. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. C. Evans and N. L. Crogan Using the FoodEx-LTC to Assess Institutional Food Service Practices Through Nursing Home Residents' Perspectives on Nutrition Care J. Gerontol. A Biol. Sci. Med. Sci., January 1, 2005; 60(1): 125 - 128. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. P. Robinson, J. A. Lucas, N. G. Castle, T. J. Lowe, and S. Crystal Consumer Satisfaction in Nursing Homes: Current Practices and Resident Priorities Research on Aging, July 1, 2004; 26(4): 454 - 480. [Abstract] [PDF]
|
|
|
|
|
|
N. L. Crogan, B. Evans, and D. Velasquez Measuring Nursing Home Resident Satisfaction With Food and Food Service: Initial Testing of the FoodEx-LTC J. Gerontol. A Biol. Sci. Med. Sci., April 1, 2004; 59(4): M370 - M377. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S.-C. Chou, D. P. Boldy, and A. H. Lee Factors Influencing Residents' Satisfaction in Residential Aged Care Gerontologist, August 1, 2003; 43(4): 459 - 472. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S.-C. Chou, D. P. Boldy, and A. H. Lee Resident Satisfaction and Its Components in Residential Aged Care Gerontologist, April 1, 2002; 42(2): 188 - 198. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
