Experiments
Experimental
- The “gold standard” of proving cause and effect (Beans in window vs closet)
- 2 or more treatments
- Random assignment to treatment
- All else kept the same
- Cause and effect established
- Often very artificial
- Very difficult to do in social science in real world contexts
Quasi-experimental
- A weaker test of cause and effect (Teaching strategy A vs B)
- 2 or more treatments
- Random assignment????
- Students choose which class sections to take. Not random choices.
- All else the same????
- Other aspects of class might vary. If two professors, they vary in other ways.
Strategies for Quasi-experimental:
- Pre-test and Post-test (so same students; look for change. But other things intervening too)
- This semester and last semester (diff students; other things may have changed.)
- Two sections this semester with different methods (Diff students, possibly dynamics)
- Your class vs that of another instructor (many things vary between the two of you)
- ABAB design in one section (Unit 1 uses A; Unit 2 uses B; Unit 3 uses A….)
Triangulation of methods:
Do several different ways and see if results are the same every time.
E.g. Two instructors in same semester. Each do A in one section and B in the other.
E.g. Pre- and post-test in two different sections this semester, with A in one section and B in the other.
SOURCES OF DATA (University)
Triangulate by using several different types of data:
Different sources (students, teachers, parents); different methods (e.g. observations, grades); and/or different times (before and after)
Existing data
(Easiest to gather)
Records
- Student grades*
- Attendance
- Teaching evaluations
- Retention rate
- % of students in school-sponsored organizations
- Standardized test results
- Demographics
Archival data
- Research literature*
- Local policies, rules, and regulations about education
- Syllabi and curriculum
- Accreditation reports
|
Conventional sources
(Require selection or development of data-collection instruments. Still relatively easy to acquire.)
Behavioral data (best)
- Teacher journal, or field notes*
- Number of books read
- Library use
- Writing samples
- Grades on classroom tests*
- Variety of materials used
- Teacher observations of student participation, interaction, etc.*
- Outside observations of classroom$
- Student journals
- Minutes from meetings
- Staff development (hours, types)
- Audio or videotapes of classroom$
Perceptual data (attitudes, opinions)
- Surveys
- Simple interviews
|
Inventive sources
(Products or performances. May be difficult to acquire and can be difficult to evaluate)
- Exhibits$
- Portfolios$
- Expositions$
- Videotapes$
|
* = excellent source of data
= may be expensive in time or money to do well
(adapted from Calhoun, 1994)
Resources
Materials from a workshop on quasi-experimental design
http://www.ipr.northwestern.edu/workshops/past-workshops/quasi-experimental-design-and-analysis-in-education/2012/
Good overview of quasi-experimental design
http://en.wikipedia.org/wiki/Quasi-experiment
A more detailed explanation
http://www.csulb.edu/~msaintg/ppa696/696quasi.htm
Mettetal, Mack @ FACET, 2014
Defining operationalization of variables