This Article Abstract Full Text (PDF) Services Download to citation manager Citing Articles Citing Articles via HighWire PubMed PubMed Citation

E-mail Versus Conventional Postal Mail Survey of Geriatric Chiefs

^a Department of Medicine, Division of Internal Medicine, University of Pennsylvania, Philadelphia
^b Department of Medicine, Leonard Davis Institute on Health Economics, and Institute on Aging, University of Pennsylvania, Philadelphia
^c School of Arts and Sciences, University of Pennsylvania, Philadelphia
^d Department of Medicine, University of Pennsylvania, Philadelphia
^e Division of Geriatric Medicine and Institute on Aging, University of Pennsylvania, Philadelphia

Correspondence: Donna Brady Raziano, MD, Fellow, Geriatric Medicine, Department of Medicine, University of Pennsylvania, Ralston-Penn Center, 3615 Chestnut Street, Philadelphia, PA 19104. E-mail: Donna.Raziano{at}uphs.upenn.edu.

Decision Editor: Laurence G. Branch, PhD

	Abstract

TOP Abstract Methods Results Discussion References

 
Purpose: This study compared the response time, response rate, and cost of two types of survey administration techniques: e-mail/web-based versus conventional postal mail. The main aim of the survey was to collect descriptive information on the existence of Acute Care for Elders units and their characteristics by surveying geriatric division chiefs. Design and Methods: Two randomized cohorts of geriatric division chiefs were formed to receive a survey either by electronic mail (n = 57) or by conventional postal mail (n = 57). If there was no response to the initial mailing, two follow-up mailings were sent to both groups using the original modality; a third follow-up was performed using the alternative modality. For each group, response rate and response time were calculated. The average total cost was computed and compared across two groups. Results: The aggregate response rate was 58% (n = 31) for the e-mail group versus 77% (n = 44) for the postal mail group. The overall average response time was shorter in the e-mail group, 18 days compared with 33 days for the conventional postal mailing group. The cost comparison showed that average cost was $7.70 for the e-mail group, compared to $10.50 per response for the conventional mail group. Implications: It appears that although the web-based technology is gaining popularity and leads to lower cost per response, the conventional postal method of surveying continues to deliver a better response rate among the geriatric medicine division chiefs. The web-based approach holds promise given its lower costs and acceptable response rate combined with the shorter response time.

Key Words: Electronic mail • Mail surveys • Survey techniques

Recently, Internet-based survey techniques have been gaining popularity for health care research and medical applications (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000; Thomas, Stamler, Lafreniere, and Dumula 2000). The Internet provides a new survey technique to the geriatric research community in targeting the geriatric populations in a more cost-effective way. Despite the growing enthusiasm for Internet-based surveys (Lakeman 1997; Schleyer and Forrest 2000), the geriatric medical community seems to have fallen behind in the effective use of this technique. The current literature indicates that when using conventional survey techniques, such as postal surveys, researchers can expect a response rate ranging from 70% to 75% (Davis 1999; Jones and Pitt 1999). On the other hand, the response rates for electronic mail surveys have been found to vary from 34% to 76% (Eley 1999; Jones and Pitt 1999). These numbers, however, should be viewed with caution because there is a paucity of substantial comparison studies, and none are noted in the geriatric research community. Additionally, the nature of the sample or the target group, the conditions under which the survey was administered, and the content of the survey can have profound implications on response rate and effectiveness of Internet-based survey techniques (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000). Table 1 displays some of the benefits and drawbacks of a postal survey versus an e-mail/web-based survey (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000; Thomas et al. 2000).

	Methods

TOP Abstract Methods Results Discussion References

 
The study population consisted of 114 chiefs of established geriatric medical divisions and geriatric teaching programs across the United States. The names, mailing addresses, and e-mail addresses for the study population were obtained by reviewing the following sources: the American Geriatrics Society (AGS) board of directors and task force committees, American Directory of Geriatric Academic Programs (ADGAP), and Geriatric Education Centers (GEC). Two groups, each containing 57 institutions, were formed by random assignment of the study population. The institutions were numbered from 1 to 114; then, with the help of random number tables, an institution was assigned to either e-mail group (group 1) or the conventional postal mailing group (group 2). The process was repeated until each group had 57 institutions. We obtained e-mail addresses for chiefs in group 1 via the following sources: Internet institution directory sites, professional medical society directories of the AGS, American College of Physicians, The Gerontological Society of America, GEC lists, ADGAP program directories, and the Association of American Medical Colleges Graduate Medical Education directory. We were able to obtain e-mail addresses for 40 chiefs by using these methods. For the remaining 13 chiefs, whose e-mail could not be obtained by these methods, we obtained e-mail addresses from their office assistants without disclosing the study. Four individuals who did not have an e-mail address were excluded from the e-mail survey group; thus, the adjusted denominator for group 1 was 53. For group 2 (n = 57), the most recent postal mailing addresses from the sources mentioned were used.

Survey Design
The survey instrument was designed to determine the prevalence and characteristics of ACE units among academic geriatric programs. The survey (see the Table A1 1) consisted of 17 multiple-choice questions, including questions on the existence, size, length of stay, and patient characteristics of the ACE unit and demographics.

Survey Administration
Respondents in the e-mail group received a letter stating the purpose of the survey and our contact information. The survey could be accessed through the e-mail letter as an attachment in an HTML format and/or as a hyper link to our survey on the World Wide Web. The Uniform Resource Locator address for the location of the survey was not stated in the letter, but an access button was created for convenience and to restrict unsolicited responses. Thus, the e-mail group could respond to the survey by completing and returning the attachment, by accessing and completing the survey on web page, or by printing out the survey and mailing back the completed survey via postal mail or fax. In total, three follow-ups were done for the nonrespondents. For those who did not respond to the first e-mail survey request, surveys were re-sent twice at 5-day intervals. The third and final follow-up for the nonrespondents was performed through conventional postal mail (the alternative method).

Respondents in the conventional postal mail group received an introductory letter similar to group 1. The chiefs received a hard copy of the survey and a self-addressed, stamped return envelope. They were also given an option to return the survey via fax. For nonrespondents in group 2, three follow-ups were done in total. They were re-sent the postal mail survey twice, allowing about 20 days between each attempt. The third and final follow-up for the nonrespondents was performed by e-mail (the alternative method). It is important to note here that the allocated time space between subsequent follow-ups is different for the two groups. We had to take into account the longer institutional processing time for conventional mail. For each group, response rate and response time were calculated. The response time was defined as the number of days between the date the survey was sent out and the date it was received. Hence, the first mailing and subsequent follow-ups were treated as discrete events.

The overall costs included labor, supplies, and postal costs. Costs of the final follow-up by the alternative method were not included. The average total cost was computed and compared across two groups. Finally, paired t-test and chi-square statistics were used to compare the average group response rate and response time.

	Results

TOP Abstract Methods Results Discussion References

 
The e-mail group had four individuals with either incorrect or no e-mail address, thus reducing the total number of participants in this group to 53. Although these individuals were sent a postal mail survey, their responses were excluded for the purpose of this study. The first attempt received maximum responses in both groups, with a 39% response rate in the e-mail group and a 63% response rate in the conventional postal mailing group. For both groups, the response rate showed a decreasing trend during the second and third follow-ups. Key findings are summarized in Table 2 .

Following the completion of the study, the e-mail nonresponders were contacted to better understand the nonresponder behavior and influencing factors. Individuals who did not respond to all three e-mail attempts (n = 6), but completed a conventional postal mail survey, were contacted personally by telephone and asked why they did not respond via e-mail. The reasons reported included a higher level of comfort with the conventional mail survey (n = 1), unavailability of e-mail accounts (n = 2), and lack of technical savvy with the Internet and attachments (n = 3). Therefore, in group 1, in addition to four for whom we could not identify an e-mail address, three more individuals had but did not use e-mail accounts. Thus, approximately 12% of the e-mail group was inaccessible electronically.

Response Time
There was a significant difference in the response time between the two groups (p < .001). The average response time for the e-mail group was 18 days, with 30% of the responses arriving within 24 hours after request. On the other hand, for the conventional postal mailing group, the average response time was 33 days. The first response from the conventional postal mailing group was received on the 15th day. This suggests that e-mail surveys are capable of generating much quicker responses compared with conventional postal surveys. For the e-mail group, the average electronic response time for the first attempt was 3.3 days. The average electronic response time for the second attempt was 4.7 days and for the third attempt was 2.3 days. For the conventional postal group, the average response time for the first mailing was 21 days. The average response time for the second mailing was 15.5 days and for the third mailing was 21 days.

Cost Analysis
We calculated average cost per response for both groups. The total costs included the labor costs involved in survey preparation, follow-up, survey administration, data input, stationary cost, and postage cost. We derived these costs based on the cost estimates for our institution letterhead, large and regular mailing envelopes, hard copies of the survey, and postage for both our mailing of the envelopes and prepaid return envelopes. Long-distance telephone calls and Internet connection costs were not included. The labor cost differed for each activity: The development of an Internet-based e-mail survey required a research assistant with reasonable computer skills at $40.00 per hour, whereas for all other activities (data input, postal mailing survey administration, and survey preparation) the cost was $20.00 per hour.

The average cost per response was $10.50 for conventional postal mail group and $7.70 for the e-mail group. Thus, the conventional postal mail survey costs were 27% higher than the e-mail survey. This higher cost of a conventional postal mail survey can be attributed to the extra costs of stationary, postage, administration, and follow-up. However, for longitudinal surveys and surveys of large populations, cost of survey administration and follow-ups can be reduced significantly by using e-mail/web-based survey techniques. Table 3 presents the cost analysis for both groups.

	Discussion

TOP Abstract Methods Results Discussion References

 
Although the postal mail survey had a higher response, e-mail questionnaires are a promising way to survey the division chiefs of geriatric medicine because of lower response time and costs. Currently there are many reports available that describe the methods involved in conducting an e-mail or web-based survey. However, little knowledge exists about the effectiveness of this technique among geriatric chiefs and the barriers faced in implementing these techniques.

Earlier studies have indicated the ways in which e-mail or web-based surveys could be used across different health professionals. Researchers at Temple University, Philadelphia, developed a web-based survey with 22 questions to study the use of the Internet in clinical practice. They recruited 450 dentists from an e-mail address list to receive this web-based survey. Participants were sent one e-mail, and a maximum of three e-mail follow-ups were sent to the nonresponders. E-mail response rates, strictly defined as an e-mail only, were 32.9%, 50.2%, 57.1%, and 64.4% for the chronological attempts. The addition of alternative responses, such as fax or mail, raised total response rate to 74.2%. The cost comparison indicated almost 50% reduction in total cost for e-mail surveys compared with conventional mail surveys ($1,916 compared with a hypothetical postal cost of $3,092). A minimum of 275 e-mail responses were recommended to break even on the costs of e-mail and to make it a more economical method than postal mail (Schleyer and Forrest 2000). This study differs from our study because recruited participants were established e-mail users.

The process of obtaining e-mail addresses through mailing address lists was extremely labor-intensive, despite the availability of a number of Internet search engines. This finding was less of an issue in other studies, such as the dental study described, and suggests that geriatrics is lagging other medical subspecialties in creating searchable electronic databases of its physicians. The existence of such a database would have made the electronic surveys even more cost-effective.

A study of university staff done in the United Kingdom compared the response timing and rates using electronic and paper methods. E-mail health surveys that required web-based submission were sent to 200 people, and 100 other people received traditional postal surveys. The postal mail survey received a response rate of 72%, followed by 32% with respect to e-mail and 19% for the World Wide Web. Although e-mail and web-based surveys were quicker and cheaper by half, the authors concluded that the higher response rate made postal mail more preferable (Jones and Pitt 1999). Caution must be exercised, because the overall cost is extremely sensitive to the type and scale of the target population. If the Internet usage of the target population is low, a great deal of effort would be expended trying to locate e-mail addresses for people who are not reachable electronically; among those who could be reached, response rates could be poor as a result of inadequate Internet skills.

An experiment comparing the response rates and response time for different approaches to mailing highlighted the strengths and weakness of e-mail surveys. In this study, the respondents recruited from the Internet received postal mail or e-mail surveys. The respondents were randomly assigned to one of five groups. Group 1 consisted of regular mail with no prenotification, no incentives, and no reminders (n = 202); group 2 consisted of regular mail with prenotification, incentives, and reminders (n = 107); and groups 3 (n = 60) and 4 (n = 122) were e-mail replications of groups 1 and 2. The final group (n = 172) is an international group, otherwise the same as group 4. The results showed that e-mail surveys had much lower response time (2–3 days vs 3 weeks) and higher response rates (40% in group 1 vs 45% in group 3). In the prenotified groups the rates increased to 63% and 83%, respectively, and were much less expensive compared to conventional mail surveys (Mehta and Sivadas 1995).

None of the previously reported survey research has directly compared the postal mail and the electronic web-based mail in a random selection method in the geriatric community. In addition, all previous studies utilized established e-mail address lists, implying that the potential respondents were known to have both a working e-mail address and Internet access. Open recruitment of participants as well as guaranteed Internet users as participants has commonly occurred in other published research (Eley 1999; Mehta and Sivadas 1995). Our study differs in this respect since we had no prior knowledge of the computer skill level of the participants or their level of Internet knowledge.

The higher response rate for the conventional mail group was countered by a longer response time in our study. On the other hand, the administration of the e-mail surveys resulted in faster responses and was more cost-effective albeit with a lower overall response rate. It should be noted that, from the first three attempts, we received 31 responses from the e-mail group and 44 responses from the conventional mail group. The electronic response was acceptable but still not as high as expected. The response rate of 58% for the e-mail group is less than other published results that indicate a response rate closer to 70% (Lakeman 1997; Mehta and Sivadas 1995). This discrepancy may have been due to the fact that our target population is unlike the other research populations that used existing Internet or e-mail user lists. Also, it is important to note that our response rates, response time, and related cost for the chiefs may not be generalized to other physicians. To minimize sampling bias in our small preselected population, randomization of survey recipients was used to make the two groups comparable. Despite moderate response rates, the rapid turnaround time could make e-mail the superior method for eliciting data in the future as more and more geriatric chiefs access electronic mail accounts. In tune with earlier findings, the electronic mail response rate for our survey was comparatively lower. We conclude that, even with this limitation, e-mail should be the preferred method if a prompt response is a primary goal. There may also be better control over the respondents and quality of the data with e-mail. Especially if traditional postal mailing is used as a salvage step, the e-mail/web-based survey follow-up reminders were easier and more cost-effective. The cost saving associated with electronic surveys must be weighed against the poorer expected response rate.

Practice Concepts

The Forum

Book Reviews

	Acknowledgments

 
Support for this study was provided by an American Federation on Aging research geriatric fellowship.

Received for publication February 19, 2001. Accepted for publication July 23, 2001.

	References

TOP Abstract Methods Results Discussion References

 

• Davis R. N., 1999& Computers, , 577. Web-based administration of a personality questionnaire: Comparison with traditional methods. Behavior Research Methods Instruments:31-572.
• Eley S., 1999. Nutrition research using electronic mail. British Journal of Nutrition 81:413-416. [Medline]
• Houston J. D., Fiore D. C., 1998. Online medical surveys: Using the Internet as a research tool. MD Computing 15:116-120.
• Jones R., Pitt N., 1999. Health surveys in the workplace: Comparison of postal, email and World Wide Web methods. Occupational Medicine (Oxford) 49:556-558.
• Lakeman R., 1997. Using the Internet for data collection in nursing research. Computers in Nursing 15:269-275. [Medline]
• Mehta R., Sivadas E., 1995. Comparing response rates and response content in mail versus electronic mail surveys. Journal of the Market Research Society 37:429-439.
• Schleyer T. K. L., Forrest J. L., 2000. Methods for the design and administration of web-based surveys. Journal of the American Medical Information Association 7:416-425.
• Thomas B., Stamler L. L., Lafreniere K., Dumula R., 2000. The Internet: An effective tool for nursing research with women. Computers in Nursing 18:13-18. [Medline]

This article has been cited by other articles:

				T.-H. Shih and Xitao Fan Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis Field Methods, August 1, 2008; 20(3): 249 - 271. [Abstract] [PDF]

				K. E.A. Burns MD MSc, M. Duffett BScPharm, M. E. Kho PT MSc, M. O. Meade MD MSc, N. K.J. Adhikari MDCM MSc, T. Sinuff MD PhD, D. J. Cook MD MSc, and for the ACCADEMY Group A guide for the design and conduct of self-administered surveys of clinicians Can. Med. Assoc. J., July 29, 2008; 179(3): 245 - 252. [Full Text] [PDF]

				P. D. Converse, E. W. Wolfe, Xiaoting Huang, and F. L. Oswald Response Rates for Mixed-Mode Surveys Using Mail and E-mail/Web American Journal of Evaluation, March 1, 2008; 29(1): 99 - 107. [Abstract] [PDF]

				L. Foster Thompson and E. A. Surface Employee Surveys Administered Online: Attitudes Toward the Medium, Nonresponse, and Data Representativeness Organizational Research Methods, April 1, 2007; 10(2): 241 - 261. [Abstract] [PDF]

				J. L. M. McCoyd and T. S. Kerson Conducting Intensive Interviews Using Email: A Serendipitous Comparative Opportunity Qualitative Social Work, September 1, 2006; 5(3): 389 - 406. [Abstract] [PDF]

				S. T. Cole Comparing Mail and Web-Based Survey Distribution Methods: Results of Surveys to Leisure Travel Retailers Journal of Travel Research, May 1, 2005; 43(4): 422 - 430. [Abstract] [PDF]

				W. D. Pace and E. W. Staton Electronic Data Collection Options for Practice-Based Research Networks Ann. Fam. Med, May 1, 2005; 3(suppl_1): S21 - S29. [Abstract] [Full Text] [PDF]

				T. Vertinsky and B. Forster Prevalence of Eye Strain Among Radiologists: Influence of Viewing Variables on Symptoms Am. J. Roentgenol., February 1, 2005; 184(2): 681 - 686. [Abstract] [Full Text] [PDF]

				D. Braithwaite, J. Emery, S. de Lusignan, and S. Sutton Using the Internet to conduct surveys of health professionals: a valid alternative? Fam. Pract., October 1, 2003; 20(5): 545 - 551. [Abstract] [Full Text] [PDF]

				F O'Leary Is email a reliable means of contacting authors of previously published papers? A study of the Emergency Medicine Journal for 2001 Emerg. Med. J., July 1, 2003; 20(4): 352 - 353. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Services Download to citation manager Citing Articles Citing Articles via HighWire PubMed PubMed Citation