The Design of Experiments: Quasi-Experimental Design

It is not always possible to achieve true experimental control. Donald Campbell & Julian Stanley have systematically explored ways to relax some of the requirements of a true experiment (they call them “quasi-experiments”), and the threats to validity which might weaken one’s conclusions.

The quasi-experimental designs are (“O” means “observation,” “X” means “experimental manipulation,” and the passage of time is represented by the movement from left to right):

Time Series: O1 O2 O3 X O4 O5 O6

In this design, periodic measurement before and after the experimental condition create a kind of internal control group. It does not, however, control for validity threats from history.

Equivalent Time Samples: XO1 O2 XO3 O4 O5 XO6….

This is an extension of the time-series design, with the experimental condition inserted at random intervals. This design controls for all the threats to internal validity, although it does not control for threats to external validity (interaction effects).

Nonequivalent Control Groups: O1 X O2

O3 O4

This design is similar to the classic experiment, but individuals were not randomly assigned to the experimental and control groups. This is probably the most common form of experimentation in social science. It can be susceptible to regression effects, and does not control for interaction effects or threats to external validity.

Counterbalanced Designs: X1O X2O X3O X4O

X2O X3O X4O X1O

X3O X4O X1O X2O

X4O X1O X2O X3O

This design also employs nonequivalent control groups, but randomizes the presentation of different interventions. This design can be susceptible to internal interaction effects and may be susceptible to threats to external validity.

Separate sample pretest/posttest: R O (X)

R X O

This design is often used when one has access to large enough populations (such as cities) that one can create a random pool with a pre-measure and then a random pool with a post-measure. The design has several internal weaknesses (history, maturation, mortality, interaction), but it handles the threats to external validity.

Separate Sample Pretest/Posttest Control: R O (X)

R X O

R O

This design is simple to the previous one, except that the same people are not retested (and thus interaction effects and threats to external validity are avoided).

Multiple Time Series: O1 O2 O3 X O4 O5 O6

O1 O2 O3 O4 O5 O6

Similar to several of the previous designs, this design collects data from an equivalent control group to increase the certainty of the interpretation. It avoids all of the threats to internal validity, although it is still subject to threats to external validity.

Institutional Cycle Design: X O1

O2 X O3

This designs combines elements of both “longitudinal” and “cross-sectional” approaches. Campbell & Stanley call it a “patched-up” design for field research in which one starts out with an inadequate design and then adds additional features to achieve greater control.

Next Section

Back to Syllabus

609