



Understanding the Methodological
Issues (cont’d)
Total Sample Size = 4341
Margin of Error on Total = +/- 1.5 percentage points,
19 out of 20 times.
The total figure is most
strongly influenced by on-line
survey respondents as they account for 55% of the total figure. Telephone respondents make up 40% of the total sample and street level respondents only 5%.
•Margin of error = +/- 2.0 (95% confidence)
Strengths
•Very robust sample size. The
large sample size creates a small margin of error.
Weaknesses
•The margin of error really only has merit if the sample was drawn randomly,
and is reflective
of the larger population. This sample was not random. Individuals “opted in” to complete the survey.
•Another major weakness is that there were no
controls placed upon participation. It would have been possible for
the same person to
complete the survey dozens, even hundreds, of times.
•Finally, there is no way of telling if survey respondents
actually saw the bin in question before responding to the survey.
•Margin of error = +/- 2.4 (95% confidence)
Strengths
•Very robust sample size. The
large sample
size creates a small margin of error.
Weaknesses
•The same criticisms of the on-line survey hold true for the telephone
survey.
•The sample was not random. Individuals “opted in” to
complete the survey.
•Similarly, there were no controls placed
upon participation. Individuals could have called in multiple
times to complete
the survey.
•Again, there is no way of verifying that the
respondents actually saw the new bin format.
•Margin of error = +/- 6.9 (95% confidence)
Strengths
•This sample was collected in a random
manner. Individuals passing by bins in multiple
areas throughout
the city were stopped and
asked about the bin.
•The days, hours and locations of interviewing
differed,
increasing the
likelihood of obtaining “different” people in the sample
•Due to proximity of interviewing close by the new bin, we can say with reliability that respondents
answered based on actually seeing the bin in question.
Weaknesses
•Only 200 surveys were completed using this approach.
This relatively small sample size results in a broad margin of error.
•
•
•