Design a survey
In designing a survey, you may find it useful to view the process as occurring in three phases: preliminary, survey construction, and testing and revising.
1. Preliminary phase
In the first phase, you identify the survey’s purpose, topics, overarching questions, as well as identifying the stakeholders, audience, and how will the information be used.
Your thoughts in each of these areas shape the design of the survey in terms of length, question type, question sequence, and survey tone. The first phase is mainly reflection and brainstorming. You are simply constructing a preliminary design —a sketch that helps you envision the survey.
To begin, develop an outline that includes sample questions for each topic. Arrange topics in an order that is coherent, logical, and contributes to the flow of the survey’s narrative. Repeating the process several times sharpens your intent, clarifies your expectations, and provides a clearer understanding of how the survey should take shape during the construction phase.
2. Survey construction phase
In this phase, you match the questions you want answered with an appropriate question format.
In choosing the format, consider question content, survey time constraints, and how the data and findings will be used, and also represented in reports, presentations, or articles.
Be vigilant of the factors that that will cause participants to lose interest in the survey, reduce their motivation in responding thoughtfully and accurately, and increase their fatigue. These factors include survey length, a turgid writing style, or a chaotic sequence of questions. The more participants find your survey uninteresting and tedious, the greater the likelihood they will disengage from the survey by responding haphazardly, not reading the questions carefully, or respond on auto-pilot.
3. Revising and pilot testing
As you develop your survey, especially if you are working in a survey application such as Qualtrics, your survey expectations may change in a variety of ways. An expanding range and depth of topics, or even the abandonment of some topics, may cause your questions, the sequence of questions, and length of the survey to change. The more revisions that occur, the more important it is to review your preliminary thoughts expressed in step 1. While revising, make sure that some core questions have not been lost and that the survey direction has not veered off course.
Moreover, the greater the stakes of the survey findings, the greater the importance to pilot test your survey.
Design an interview
Before you consider potential questions to ask participants, there are two sets of questions you should ask yourself. The first set of questions relate to the survey’s context:
- Why are you conducting these interviews?
- What is the problem, situation, or challenge that is driving the need for a survey?
- Who wants this interview?
- Who are stakeholders?
- What are their expectations?
- How will the data be reported and in what form?
- How will the findings be used?
The second set of questions relate to the participants:
- Who are they?
- Why are they participating?
- What is the context in which they are participating?
- What are their expectations?
- What are the survey’s context, setting, purpose, topics, or questions that might affect how openly, motivated, thoughtfully, or accurately they respond?
Answers to these questions shape what questions you ask, how you ask them, and at what point in the interview you ask them. The sequence of questions is important for developing openness, stimulating memory, and building a logical flow.
If you find yourself generating a large number of questions, you may want to consider a survey.
Interviews are most fruitful when rapport is established between interviewer and interviewee. A large number of questions may impede establishing rapport or, even worse, transform an interviewee into an automaton.
Interview structure
Decide on whether your interview will be structured or semi-structured.
A structured interview involves the same questions to all participants, presented in the same sequence. You may wish to use prompts, which can provide a degree of standardization, a consistent structure and will be easier to manage later during analysis.
In contrast, a semi-structured interview is more exploratory and may also be more engaging for the participant.
Regardless of the structure you choose, be sure to frame the interview. Explain briefly why you are conducting the survey, for whom, and how the findings might be useful.
The sequence of questions plays a role in making the respondent experience the interview as meaningful and of value. A coherent and logical line of questioning encourages openness, stimulate memory, and engages the respondent more deeply.
Design a focus group
The first step in designing a focus group is to identify its purpose, overarching questions, and function. While the purpose and overarching questions appear as a natural first step, the function may be less evident.
The purpose and overarching questions determine the function, and the function shapes focus group design and structure. Function influences the level of formality/informality, the flexibility of and sequence in which questions are asked, the choice of facilitator(s), and how the discussion is recorded.
In educational settings, focus groups may serve a range of functions, such as brainstorming. You may wish to gain a quick overview of a topic you intend to pursue at a deeper level. Or, you may wish to speak with students about their experience with educational innovation, in which their responses will serve as a source of topics and questions for a survey. You could also use a focus group to provide supportive data or explanatory data for a study that involves several different methodologies.
Questions that guide first steps:
- What is the purpose?
- What is the context for the focus group?
- What are the overarching questions?
- Is the purpose to brainstorm, explore, confirm?
- Is the focus group the only methodology involved in the study, or are there other methodologies also included?
- How high are the stakes?
- Who are the stakeholders?
- Who will be the participants?
- How many?
- What do you want to have gained by the end of focus groups?
- How will be information from the session be reported?
- How will the information from the session be used?
Guidelines for conducting a focus group
1. Make the focus group a safe environment, create a spirit of openness, and respect differing viewpoints.
2. Be mindful of participants who tend to dominate or act aggressively. In these situations, it’s best to address these situations early, diplomatically, and, if possible, with humor. Otherwise, you may lose control of the meeting (i.e., some participants remain silent or self-censor what) and, with it, you lose the opportunity to gather viable data.
3. Prepare well how you will frame the focus group at the start of the session in terms of context, purpose, and objectives. What you say and how you say it influences how effective the session will be.
4. One value of a focus group lies in individuals’ hearing other perspectives that, in turn, stimulate their thinking. Also, interactions among the participants increase the group’s productivity. Collectively, participants hearing different perspectives. Their interactions among one another create a body of thought that is greater than the sum of participants’ personal views.
5. Recording a session may occur by a facilitator’s taking notes, another individual taking notes, or a voice recorder. The greater the stakes, role, or importance of the focus group(s), the more important it will be to provide accurate documentation of what was discussed during the session. In such situations, use a voice recorder.