Overview

CourseKata enables randomized controlled experiments, allowing researchers and educators to rigorously test and improve the learning experience for students using our interactive curriculum. Experiments are created by improvement teams, which include at least one researcher, an instructor, and a core team member. Once an experiment is set up, classes that have been created in CourseKata can be associated with an experiment, and students within those classes will be randomly assigned to different versions of the curriculum. This allows for the testing of variations in content, questions, or other elements to evaluate their impact on student learning outcomes.

The experiment system currently supports random assignment of students within the same class to different experimental versions of the release branch: This design controls for class-level effects and thus maximizes statistical power. In this approach, you create an experiment, add multiple variations, and associate one or more classes with the experiment. Students within each class are then randomly assigned to different variations. 

The rest of these documents will explain how to create experiments. It assumes you have a basic understanding of the experiment system (see paragraphs above), a basic understanding of some CourseKata-specific terms, and a basic understanding of git and its terms (see the Glossary).

How to Create Experiments