The
University of Texas at Austin
Skip to page content.
Information Technology Services
|
|
|
||
About ITS ITS Services Contact ITS ITS Departments ITS Employment ITS News ITS Help Desk
|
|
|
|

| | | | |
|
| header |
|

Getting Good Data

|
|

Introduction

Advantages of Online Surveys

Disadvantages of Online Surveys

Designing the Questionnaire

Creating the Questionnaire

Data Collection

Data Analysis

Getting Good Data

Ethical Issues and "Netiquette"

Putting it all Together

References

|

Recruiting Subjects

Methods of Recruitment

Once you have created your online survey, you will need to recruit your sample. (Please note that your sample recruitment techniques and advertisements, as well as your other study methods, must be approved by your ethical review board. For more information, see the section of this document regarding human subjects approval at UT.) There are three major ways to recruit online participants from specific subpopulations: creating a web presence, e-mailing a sampling frame, or posting to newsgroups and listservs.

Creating a web presence. While it is certainly desirable to have a web page that explains the purpose of your survey, the simple presence of the page on the web is unlikely to attract respondents, unless it is heavily advertised at various sites across the Internet (which may be quite an expensive proposition). The web presence tactic, then, is probably best pursued in conjunction with e-mail, newsgroup, or listserv recruitment.

E-mailing a specific sample. Many survey researchers begin the sampling process by creating a sampling frame: a specific roster of the population of people from which the researchers will select a final sample, and to whom the researchers wish to generalize the results. For example, in the fall of 2000, the Research Consulting group at the University of Texas conducted a survey of students’ usage of information technology. The sampling frame was the 49,828 students enrolled for the fall semester. 2,167 specific students were then randomly selected from the sampling frame to receive the survey. Each of the 2,167 students was e-mailed an invitation to visit the survey web page and fill out the online questionnaire. Because the students who responded were part of a specified sample, researchers could determine the proportion of the sample that actually responded to the survey, and could get a sense of how well their final sample represented the population of interest.

If you are particularly concerned about sample quality, you may wish to follow up the online survey with less high-tech methods. For the student information technology survey, researchers sent a paper survey through the U.S. mail to respondents whose e-mail invitations bounced. For those who did not respond to either e-mail or mail, a subsample of 292 completed the survey via telephone.

Posting to newsgroups and listservs. Posting an invitation to a listserv is a common subject recruitment tactic for online surveys. This tactic is particularly useful for recruiting rare or deviant subpopulations. For example, in his study of illicit drug dealers, Coomber (1997) posted requests to 23 English-language drug-related newsgroups, and received 80 responses, including 19 respondents for whom drug sales were their primary source of income (for more information about studies of illegal phenomena, see the section of this document on data confidentiality and security). A pitfall of the newsgroup and listserv recruitment method, however, is that individuals who are not part of the population of interest may belong to the listserv; in addition, your invitation may be forwarded to other listservs and newsgroups. As a result, the researcher must deal with two problems:

First, the researcher cannot ascertain how many members of the population of interest received the invitation, and thus cannot assess the rate of response to the survey. As a result, the generalizability of the results is unknown. (However, if the population of interest has known demographic characteristics, one can attempt to establish the study’s generalizability by gathering demographic data on each respondent and comparing the sample characteristics to those of the larger population.)

Second, the researcher must develop a strategy for discarding data from individuals who are not among the population of interest. Many online surveys contain a set of screening questions at the beginning of the survey to ascertain which respondents are eligible to complete the study. For example, because research involving children is subject to more ethical restrictions, online surveys often ask respondents to certify that they are over the age of 18.

"Spamming" and Junk Mail

If you send out e-mail invitations for your survey, do not send out massive volumes of e-mail at random to a large number of newsgroups or recipients. As UT's rules for acceptable use of information technology state: "Do not interfere with the activities of others or use a disproportionate share of resources. Send messages only to those who may be interested in the content."

If you have an e-mail list of thousands of people who qualify for your survey, you may wish to send e-mail only to a random sample of these people, or send e-mail in batches of several hundred at a time, rather than thousands at once (note: if you do send e-mail to more than one person at a time, conceal the recipients' e-mail addresses by placing them in the BCC field of your e-mail). If you wish to post your survey invitation to a newsgroup or listserv, there are four steps to take to avoid the perception that you are posting random and unwanted "junk mail."

First, lay the groundwork for your request before you post the survey invitation. If the list is moderated, communicate with the moderators first about the appropriateness of your request. If the list is unmoderated, send an initial post informing the readers of your intent to post the request and asking for their feedback.

Second, send an e-mail to each group separately. If you e-mail several groups at the same time, your post is more likely to be perceived as spam.

Third, defuse the assumption that your invitation is an advertisement in disguise. Smith & Leigh (1997) provide good guidelines about how to write your invitation:

"Newsgroup readers tend to be quite skeptical of nontopical requests and dislike advertisements of any kind because they are often of dubious quality. Therefore, the face validity of the notice needs to be high. Our recruitment notice contained the following features: It mentioned that the moderator considered the request acceptable or that there were no objections from any readers; it contained enough information to allow potential subjects to be fully informed before they participated; it provided the ethical approval code, institutional affiliation, the e-mail address of the researcher, and the mailing address of the ethics committee, so that anyone could direct questions or comments about the research to a third party without the researcher's knowledge; and finally, it included detailed instructions on how to participate."

Fourth, if someone responds rudely to your e-mail in the public forum, resist the urge to respond in kind. If you do respond to the complaint publicly, be polite and professional, restating the reasons why you believe the survey is valuable.

Incentives

Some researchers feel that providing incentives (for example, $20 for each respondent who completes the study) helps to increase the response rate for their study. If you have a specific sampling frame, it is possible to calculate how much you can afford to spend on incentives, divide that by the number of people who are expected to respond, and promise each respondent the resulting amount of money. If, however, you are recruiting respondents at large from the Internet community, it is impossible to predict how many responses you may receive. Thus, it may not be a good idea to promise respondents a certain sum of money for participating in your study. Another tactic is to offer respondents an entry in a prize drawing if they complete the survey. For example, if 2,000 people reply to your survey, each person has a 1 in 2,000 chance to win the prize drawing (please note, however, that this type of incentive may be illegal in certain areas). If you wish to pursue the "prize drawing" tactic, you will need to collect an e-mail address from each respondent; for more information on issues pertaining to the collection of e-mail addresses online, see the section of this document on confidentiality and data security.

Top

|

Information Technology Services. Infrastructure. Innovation. Integrity.

Last updated February 27, 2008.
Copyright © 2001-14, Information Technology Services at The University of Texas at Austin.
All rights reserved. For privacy concerns read our privacy policy.

To submit questions or comments regarding this page, use the online Comment Form.