| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||
Correspondence: Address correspondence to Rachel A. Pruchno, PhD, New Jersey Institute for Successful Aging, UMDNJ-SOM, 42 East Laurel Rd., Suite 2300, Stratford, NJ 08084. E-mail: pruchnra{at}umdnj.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Sampling • Caregiving research • Random digit dial
Sampling and generalizability are methodological issues fundamental to the social and behavioral sciences (Hultsch, MacDonald, Hunter, Maitland, & Dixon, 2002), yet researchers often overlook their significance. Schaie and Hertzog (1985) suggested that sampling strategies can be grouped into three major categories: (a) samples drawn at random from some specified population, (b) structured convenience samples, and (c) explicitly biased samples with particular characteristics.
Random (i.e., probability) sampling offers the best chance of minimizing selection effects because, theoretically, each person in the population has a known chance of being chosen for participation. Even though RDD samples can be biased because not all eligible persons agree to participate, these samples should result in significantly less bias than those developed using convenience methods (Lindenberger et al., 1999; Nesselroade, 1988). However, random samples are rare in psychological research on aging because of their higher costs and other logistical concerns. In their analysis of articles published between 1990 and 1998 in the Journal of Gerontology: Psychological Sciences and Psychology and Aging, Hultsch and colleagues (2002) found that fewer than 10% of articles described studies that used random selection.
Use of structured convenience samples and selection of persons based on particular characteristics are the sampling strategies typically used in caregiving research as well as in broader social and behavioral research on aging. Although these procedures are pragmatic and cost effective (Fredman et al., 2004), they yield nonprobability samples that are likely to be unrepresentative and biased (Atkinson, 2000; Dickinson, 2002; Williamson, 2003). Indeed, structured convenience samples typically recruit people who are well connected to their communities, as they solicit interest at support groups, from service providers, or from other gatekeepers. Although investigators often attempt to compensate for nonrandom selection by including quotas of individuals with particular demographic characteristics (e.g., gender, age, race, and/or income), generalizability remains a concern because it is impossible to know whether and how population characteristics are associated with study findings.
Studies contrasting the characteristics of people recruited using RDD and convenience methods are rare. In one of the few studies to examine this issue, Ganguli, Lytle, Reynolds, and Dodge (1998) found that eligible participants identified by convenience methods were younger, more likely to be female, and better educated than those identified using random selection. They also had higher functional ability, were less likely to have been hospitalized during the past 6 months, and were less likely to have used a home health care service in the past year than participants recruited using random selection. Similarly, Hultsch and colleagues (2002) found significant differences on almost half of the demographic and psychological measures they examined, with respondents identified using convenience methods being better educated and having higher income and occupational status than those identified using random selection.
Relatively few caregiving studies have used RDD methods to identify caregivers. Some exceptions include research by Farberman and colleagues (2003), Scharlach, Li, and Dalvi (2006), Levine and associates (2000); the AARP Traveler's Survey (American Association of Retired Persons & Traveler's Companies, 1988); and the National Alliance for Caregiving and AARP Survey (2004). To date, however, no study has contrasted characteristics of caregivers identified using RDD methods with those of caregivers responding to various community outreach strategies.
The following analyses contrast sample performance on a range of demographic, psychological, and social variables for a sample of community-dwelling caregivers identified by RDD with that of a convenience sample recruited through networking strategies commonplace in caregiver research. We hypothesize that samples identified using convenience strategies will be less heterogeneous than samples recruited by RDD, and that participants identified using convenience strategies will have higher levels of education and income and lower levels of psychological well-being and social support than those generated using RDD procedures. Finally, following the strategies described by Hultsch and colleagues (2002), we contrast correlations among variables frequently examined in caregiving research for people identified using RDD procedures and those identified using the convenience approach.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
Study eligibility criteria specified women between the ages of 50 and 64 who were currently working for pay for 35+ hr weekly on average. These women were also required to be providing unpaid assistance with personal care tasks or household duties on a regular basis to someone aged 50 or older living in the community and unable to manage on his or her own without help. A professional interviewing staff conducted all interviews by telephone between April and December 2006.
Measures
Measures selected for inclusion in the project are those commonly used in caregiving research. All have excellent reliability and were validated in other studies. Table 2 provides details regarding all measures. For analyses, we coded relationship between caregiver and recipient as spouse (1) versus other (0); living arrangements as same household (1) versus separately (0); marital status of caregiver as married (1) versus not married (0); race as Black (1) versus other (0); education as high school or less (1), some college (2), college degree (3), and more than 4-year degree (4); income as less than $25,000 (1), $25,000 to $49,999 (2), $50,000 to $100,000 (3), and greater than $100,000 (4); diagnosis of Alzheimer's disease as present (1) or absent (0).
|
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
We found significant mean differences for 63% of the variables examined. Respondents identified using convenience methods were more likely to be the spouse of the care recipient and more likely to be living with the care recipient than those recruited using RDD. They were more likely to have higher levels of education and income. Respondents identified using convenience methods were more likely to be caring for someone diagnosed with Alzheimer's disease, to indicate greater cognitive problems experienced by the care recipient, and to report providing significantly more hours of weekly help than those identified by RDD. Convenience sample respondents reported fewer other people helping the care recipient and indicated more negative relationships and fewer positive exchanges with their care recipient than RDD respondents. Finally, they also reported significantly higher levels of caregiver burden and depression than did RDD recruits.
We examined the magnitudes of statistically significant effects by calculating effect sizes and power statistics. As indicated in Table 3, values of Cohen's d ranged from.37 to.82, indicating that the majority of effect sizes were of moderate to large magnitude (Cohen, 1988, 1992). It is interesting that caregiving burden showed the largest effect size. Power statistics for variables exhibiting statistically significant differences ranged from.50 (hours of help provided by the caregiver) to.99 (caregiver burden).
Correlations Among Variables: RDD Versus Convenience Samples
We examined differences between the samples in the magnitudes of relationships among variables by computing zero-order correlations separately by sample. We tested the significance of the differences between the two correlation coefficients with the Fisher r-to-z transformation. We computed confidence intervals on the difference between the two independent correlations and computed the z value using standard tables. This test avoids the common mistake of assuming that a significant difference in coefficients across samples can be determined by the significance tests for coefficients performed within a sample. Table 4 reports these results. The correlations between care recipient depressive symptoms and caregiver burden (z = 2.05, p <.05), care recipient depressive symptoms and caregiver satisfaction (z = –1.67, p <.05), and receipt of paid help and depressive symptoms (z = –2.03, p <.05) were significantly stronger among the RDD sample than among the convenience sample. In contrast, the correlation between receipt of paid help and caregiver burden was significantly stronger for the convenience sample than the RDD sample (z = –2.12, p <.05).
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
We found differences between the samples identified through convenience and RDD methods in the demographic characteristics of respondents, care recipient characteristics, and outcome variables. Respondents identified using convenience methods reported higher levels of education and income than RDD respondents. They were more likely to be married to and living with the care recipient than those identified using RDD methods. They were also more likely than those identified by RDD methods to be caring for a recipient diagnosed with Alzheimer's disease and to characterize the care recipient as having greater cognitive difficulties. These demographic differences are important, with each having the potential to modify conclusions drawn.
Respondents recruited using convenience strategies reported significantly higher levels of caregiver burden and increased depressive symptomatology relative to those identified using RDD, suggesting that caregivers drawn to research studies through convenience sampling strategies may be more needy and overburdened than the general population of caregivers. Consistent with these higher levels of burden and depression, caregivers recruited by convenience methods reported having more negative and fewer positive relationships with their care recipient than those drawn through RDD methods. They reported providing significantly more hours of help to their care recipient and were more likely to be the sole provider of care than those recruited using RDD strategies.
In addition to means and variance, correlations between study variables differed as a function of sample recruitment method. Correlations between burden and subjective health, caregiver satisfaction and depressive symptoms, and subjective health and depressive symptoms were stronger for respondents identified using convenience methods than for those identified through RDD. The consistently stronger magnitudes of these relationships among the convenience sample suggest that the effects of the caregiving role among caregivers recruited via convenience methods are experienced more intensely than the effects of the role among caregivers generally. Thus, reliance upon results from convenience samples to identify needs of caregivers or to develop policies to assist caregivers may result in programs that do not adequately reflect and address the circumstances of the broad population of caregivers. If differences between participants recruited using RDD and convenience strategies are replicated in other studies, there are important implications for how best to improve the lives of caregivers.
Limitations on the generalizability of findings from this study include the relatively small sample sizes for both the RDD and convenience groups. Respondents recruited using RDD and convenience methods were part of a larger study designed to address conceptual issues not discussed here. Limited resources precluded inclusion of additional respondents for the present analyses. The extent to which results can be generalized may also be influenced by the way in which this particular convenience sample was generated. Although the broad-based strategies used to identify members of the convenience sample were similar to those used by others (e.g., Atienza & Stephens, 2000; Atienza et al., 2004; Brummett et al., 2006), and most of the demographic characteristics of our convenience sample were comparable to those reported by others studying women providing care to an older relative (e.g., Atienza & Stephens, 2000; Cicirelli, 2003; King, Atienza, Castro, & Collins, 2002; Stephens, Townsend, Martire, & Druley, 2001), our failure to recruit through local service providers (e.g., home health agencies, support groups, hospitals) may have affected findings. Moreover, the overrepresentation of persons in this convenience sample who were caring for a relative with Alzheimer's disease and those who were coresiding with the care recipient suggest that there may be bias to this convenience sample. Finally, because eligibility criteria can limit the generalizability of results, it is important to note that study eligibility criteria sought only women aged 50 to 64 who were working full time and providing care to an older person who lived in the community. Hence, it is unclear whether our results are generalizable to other groups of caregivers.
Although we treat RDD here as a gold standard of comparison, it is important to acknowledge that there are significant challenges to using these methods. Declining participation rates, self-selection bias, and increasing numbers of households lacking landlines all have the potential to bias samples generated using RDD methods (Simon, Mercy, & Barker, 2006). It is also important to acknowledge the relatively low rate at which RDD strategies yielded completed interviews. It is possible that people who completed the interview were different from those who were eligible but did not complete the interview. Whereas some researchers have contended that low response rates selectively bias findings (O'Rourke, 2000), others (Curtin, Presser, & Singer, 2000; Keeter, Miller, Kohut, Groves, & Presser, 2000) have questioned the extent to which this is the case. Thus, it remains unclear whether and how the low response rate affected findings.
Additional research designed to overcome the limitations of this study is essential. Although preliminary, our findings show the critical need for research that contrasts findings from samples identified using RDD and convenience methods using larger samples. Moreover, studies contrasting findings from convenience samples identified using different strategies (e.g., those learning about the study through various media approaches, those learning about it from providers) are needed in order to provide important information regarding the generalizability of findings. Studies that contrast RDD and convenience strategies with strategies such as list-assisted RDD that approximate representation within a community, yet are less expensive than RDD, are also needed. Finally, meta-analyses contrasting findings from studies that identify participants through RDD methods with those from studies using convenience sampling methods have the potential to further clarify researchers' understanding of the caregiving experience and serve as the foundation for intervention research. Given that the volume and quality of caregiving research have grown exponentially over the past several decades, it becomes imperative that future research more closely examine the ways in which sampling and recruitment strategies affect study findings.
|
Footnotes |
|---|
1 New Jersey Institute for Successful Aging, University of Medicine and Dentistry of New Jersey, Stratford, NJ.
2 Carroll School of Management, Boston College, Chestnut Hill, MA.
Decision Editor: William J. McAuley, PhD
Received for publication July 30, 2007. Accepted for publication December 7, 2007.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
