THE IMPORTANCE OF RANDOM SAMPLING

 
It is very important to develop a sampling plan that permits legitimate generalization from the survey results to the population of interest (unless the study specifically calls for a non-probability sample design). The key is the use of statistically derived random sampling procedures. These ensure that survey results can be defended as statistically representative of the population. Surveys that do not follow these procedures can produce results that lead to misguided market research, strategic, or policy decisions. Any so-called "survey" in which no attempt is made to randomly select respondents, such as call-in readers' or viewers' "polls", is likely to produce results that in no way reflect overall public opinion--even if many thousands of individuals participate.
SAMPLE SIZE

Surprisingly, the accuracy of a given sample size in reflecting the characteristics of a population usually is not greatly affected by the size of the population. (Note that the concept of sampling error assumes that the sample was selected following statistically random procedures.) If the sample comprises less than about a fifth of the population, then the size of the population is not important in determining how accurate a given sample size will be. For example, as shown in the table below, the maximum sampling error for a random sample of 500 is ±4.4 percentage points. This is true regardless of whether the sample was selected from a population of 5,000 people or one of 5,000,000 people.

 
MAXIMUM SAMPLING ERROR AT THE 95% CONFIDENCE LEVEL FOR DIFFERENT SAMPLE SIZES (For Populations at Least Five Times the Size of Each Sample)  
50 
Sample Size  Sampling Error 
50
±.138*
100  .098
200  .069
300  .057
400 .049
500  .044
750  .036
1,000 .031
2,000  .022
3,000  .018
4,000  .015
5,000  .014
 
*i.e., With a random sample of 50, the percentage who provide a given response will be within 13.8 percentage points of the true population percentage in 95 surveys out of 100.
If the sample size for each group of interest is more than about 200, the accuracy of the survey results is often more seriously affected by low response rates than by small sample sizes. Thus, funds usually should be invested to enhance the response rate rather than to increase the sample size beyond the minimum number necessary to achieve the desired level of sampling accuracy.
One must also remember that sampling accuracy is dependent on the sample size of the particular group whose responses are being reported. Thus, a random sample of, say, 1,000 respondents has an overall sampling error of 3.1%, but if one is looking at the responses of white males (of whom there might be about 400 if this is a national probability sample), then the sampling error is 4.9%. And if one is looking at the responses of black males (of whom there are probably only about 60, unless blacks were oversampled), then the sampling error jumps to 12.6%. Therefore, the subgroups in which one is interested, and their frequency of occurrence in the population, often are important considerations in determining the optimal sample size.

Reference:

Adopted with permission from the Survey Research Program. Principal Investigator: Susan H. Russell, Ph.D.