Understanding the
Methodological Issues (cont’d)
Total Sample Size = 4341
Margin of Error on Total = +/- 1.5
percentage points,
19 out of 20 times.
The total figure is most strongly
influenced by on-line survey
respondents as they account for 55% of the total figure.
Telephone respondents make up 40% of the total sample and street level respondents only 5%.
•Margin of error = +/- 2.0 (95%
confidence)
Strengths
•Very robust sample size. The large sample size
creates a small margin of error.
Weaknesses
•The margin of error really
only has merit if the
sample was drawn randomly, and is reflective
of the larger population. This sample
was not random. Individuals
“opted in”
to complete the survey.
•Another major weakness is that
there were no controls placed upon participation. It would
have been possible for the same person
to complete the survey dozens, even hundreds,
of times.
•Finally, there is no way of
telling if survey respondents actually saw the bin in question
before responding to the survey.
•Margin of error = +/- 2.4 (95%
confidence)
Strengths
•Very robust sample size. The large sample
size creates a small margin of error.
Weaknesses
•The same criticisms of the
on-line survey hold
true for the telephone survey.
•The sample was not random. Individuals
“opted in” to complete the survey.
•Similarly, there were no
controls placed upon participation. Individuals could
have called in multiple times to complete
the survey.
•Again, there is no way of verifying
that the respondents actually saw the new bin format.
•Margin of error = +/- 6.9 (95%
confidence)
Strengths
•This sample was
collected in a random manner. Individuals passing
by bins in multiple areas throughout
the city were stopped and
asked about the bin.
•The days,
hours and locations of interviewing differed, increasing the
likelihood of obtaining “different” people
in the sample
•Due to proximity of
interviewing close
by the new bin, we can say with
reliability that respondents answered based on actually seeing the bin in question.
Weaknesses
•Only 200 surveys were
completed using
this approach. This relatively small
sample size results in a broad margin
of error.