Page 4
Understanding the Methodological Issues (cont’d)
Total Sample Size = 4341
Margin of Error on Total = +/- 1.5 percentage points,
19 out of 20 times. 
The total figure is most strongly influenced by on-line survey respondents as they account for 55% of the total figure.  Telephone respondents make up 40% of the total sample and street level respondents only 5%.
On-line Survey
Telephone Survey
Street Level Survey
Margin of error = +/- 2.0 (95% confidence)
Strengths
Very robust sample size.  The large sample size creates a small margin of error.
Weaknesses
The margin of error really only has merit if the sample was drawn randomly, and is reflective of the larger population.  This sample was not random.  Individuals “opted in” to complete the survey. 
Another major weakness is that there were no controls placed upon participation.  It would have been possible for the same person to complete the survey dozens, even hundreds, of times.
Finally, there is no way of telling if survey respondents actually saw the bin in question before responding to the survey.
Margin of error = +/- 2.4 (95% confidence)
Strengths
Very robust sample size.  The large sample size creates a small margin of error.
Weaknesses
The same criticisms of the on-line survey hold true for the telephone survey.
The sample was not random.  Individuals “opted in” to complete the survey. 
Similarly, there were no controls placed upon participation.  Individuals could have called in multiple times to complete the survey.
Again, there is no way of verifying that the respondents actually saw the new bin format.
Margin of error = +/- 6.9 (95% confidence)
Strengths
This sample was collected in a random manner.  Individuals passing by bins in multiple areas throughout the city were stopped and asked about the bin.
The days, hours and locations of interviewing differed, increasing the likelihood of obtaining “different” people in the sample
Due to proximity of interviewing close by the new bin, we can say with reliability that respondents answered based on actually seeing the bin in question.
Weaknesses
Only 200 surveys were completed using this approach.  This relatively small sample size results in a broad margin of error.