Getting to the heart of the question

Questions feature in most evaluation tools, but it can be difficult to know how and what to ask in order to evoke the desired response. The questions which are asked are so important as they shape the response, and therefore the quality of the data received. This article provides some tips and ideas on how to plan and write good questions for your evaluation.

Easy peasy evaluation strategy

  1. Consider what sort of questions do you want to ask? These could take the form of a survey, an interview, instant voting, an online poll, comment cards, or another evaluation method.
  2. Next, define specific aims and objectives, considering budget and resources, as well as timeline from setting research objectives, planning, writing, distributing, collecting data back in, and analysing results.
  3. Think about how you will analyse the data in advance as this will shape the design of the evaluation method. Make sure you know exactly when, how, and who will analyse the data.

Find out what works

It’s always worth seeking out similar evaluation types which have been carried out before, to see what has or hasn’t worked. This can provide you with ideas for question design, formatting, and any indications of things to avoid or be careful with. Question banks (links to Constant Contact) and similar resources (links to Questionnaire design document) are a great place to look, as well as seeing what has already been used in your organisation.

Question types

Once you are clear about the objectives of the survey and what you want to ask, you need to decide on a few more details surrounding the design. An extremely important part of writing better questions is ensuring that you have the correct question types (links to Surveymonkey) for your survey.

Question types:

  • Open questions are used when descriptive answers are required and can provide qualitative data. They allow respondents to answer freely and provide detailed responses. However, they may produce unclear responses and are harder to analyse and categorise.
  • Closed questions are simple and quick to answer. The data is easier to analyse and quantify than data from open questions, however the questions may result in box ticking without really thinking about answers.
  • Scales and rankings are another type of closed question, which ask the respondent to rate something on a scale, for example asking ‘did the day meet your expectations’ with response options ranging from ‘it exceeded my expectations’ to ‘it did not meet my expectations’.

Formatting

The formatting of the questions is another important thing to consider. The question order should flow logically. Always avoid leading questions, asking two questions in one, double negatives, and making assumptions, as these may all lead to respondents selecting a particular response due to the question phrasing and not due to their actual thoughts and feelings. Don’t forget to thank respondents at the end of the questions.

Tips and suggestions

– It is important to speak the ‘language’ of your participants, including using age appropriate language and not using technical terminology for a group of participants who are not acquainted with technicalities in your field. Try reading your questions out loud to see how they sound.

– Questions should always be easy to understand, and equally as easy to answer. They should remain relevant, and when drafting them you should keep referring to the agreed aims and objectives to ensure your questions are valid. A small pilot study could be distributed to the target demographic for a review of questions and to locate any gaps, as well as to check that the questions flow logically.