Newsletter, July 2006

Contents this month

• Main article: Wise Crowds, a new fashion in engagement
• Why qualitative methods are usually best for engagement work
• Project news

Why qualitative methods are usually best for engagement work

Most of Dialogue by Design’s work is qualitative but recently we have been using a quantitative approach with representative samples of a population. This has helped us to clarify why we usually advocate a qualitative approach and the particular situations in which quantitative methods are better.

The established arguments are that:

• Quantitative methods are objective and make the results of engagement with a representative sample easy to extrapolate to the general population. They usually use closed yes/no or multiple-choice type questions where the responses for the whole sample can be expressed as percentages.
• Qualitative analysis on the other hand, depends more on the skill of the facilitator or analyst and although counts of responses of one type or another can be produced this is an intrinsically subjective process and it is hard to generalise the results to an entire population. This approach is also more labour-intensive and therefore usually more expensive than quantitative analysis.

Despite these drawbacks, qualitative methods have some key advantages over quantitative analysis:

• they generate rich, detailed responses that communicate participants’ real perspectives and concerns;
• they allows participants the freedom to say exactly what they think and avoid boxing them into saying something a little different from what they really want to say;
• and – most important of all – qualitative methods can provide answers to the question “why”.

The last point is crucial. Closed questions can obscure or disguise underlying assumptions that only become clear when participants complain that they are being coerced into saying things that they do not intend. This is a particular problem when the facts surrounding an issue are not widely accepted, for example the pros and cons of incineration technology or the safety of the MMR vaccination.

Secondly, quantitative methods often raise the expectation that the results will be interpreted as a binding vote and participants are regularly disappointed when their views do not determine the eventual decision.

Quantitative methods do have their place, for example, when the results from a sample must be representative of a population as a whole or when budget is limited, and we use them whenever they offer the best solution. But in most cases involving consultation or public engagement, qualitative methods provide a much better foundation for informed decision-making.