Monday, June 22, 2009

Designing useful questions

I received a survey recently that presumably was trying to get a sense of what the community thought:

Should we have an ordinance about land preservation?

(1) Yes
(2) Not sure
(3) No

The problem with yes/no questions like this is obvious – the topic is more complex than the question allows for, and the opposite of “no” isn’t always “yes.” Results from such a poll will be meaningless at best, which is fine for CNN's topic-of-the-day-ratings-boosting result, but not so good at understanding what a group actually might think about some complex topic. “Highest response” in this case is not the same as “majority opinion.”

A more useful question might have asked about different types of land preservation ordinances, listing them from extremely prescriptive and limiting to very suggestive and open to interpretation, as well as including a "no" choice and the ability to comment.

Polls and surveys are rarely useful in delivering the the sort of clarity needed to make good decisions or come to consensus. They are tools to help focus the questions, but not answer them.

No comments:

Post a Comment