Develop Research Design
With well-defined goals, suggestions for metrics and methods are easier to make. If quantitative methods are used, published tests and questionnaires are recommended. These instruments have the advantage of established validity and reliability and can be discussed in terms of previous uses and findings. It also is possible to custom tailor and/or create measures for the needs of specific research questions or educational goals. For example, TLL educational researchers have developed a set of Learning Behavior Surveys designed to measure student attitudes about those aspects of the educational environment that contribute to their learning. In many cases, qualitative measures are appropriate. These include focus groups, interviews, classroom observations, and “think-aloud” protocols.
A mixed method (quantitative and qualitative) approach is often the most useful, and both direct and indirect approaches can be employed to assess learning. The direct approach uses quantitative performance assessments such as:
- Oral presentations
- Problem sets
- Pretest-posttest comparisons of learning
The indirect approach uses qualitative and quantitative approaches, such as:
- Naturalistic descriptive observations
- Focus groups
- Structured and open-ended interviews
Where it is possible, historical or matched comparison groups and experimental procedures are used. However, because we are an applied laboratory and our research settings are generally “real” classrooms, the preferred measures and methods adjust to whatever the best approach is for answering the questions being asked in a specific situation.