University:
Massachusetts Institute of Technology
Course:
15.501/516 | Accounting
Academic year:

2023
Views:

293

Pages:

20
Author:

Jeremy Mcclain

Experimental design Experiments are the most common and useful way to determine causality. The question is: how to design them? What is a good experiment? Hard to say... art form! Experimentation is partially an But, in general, a good experiment is one that controls for everything else -- testing only what you want it to test Planned well in advance Controlling for other plausible hypotheses The ﬁrst experiment An Early Experimental Design:“Dew it both ways” Judges (Shoftim) Chapter 6 And Gideon said to G-d: “If You will save Israel by my hand, as You have said, look, I will put a ﬂeece of wool on the threshing-ﬂoor; if there be dew on the ﬂeece only, and it be dry upon all the ground, then shall I know that You will save Israel by my hand, as You have said.” And it was so; for he rose up early on the next day, and pressed the ﬂeece together, and wrung dew out of the ﬂeece, a bowlful of water. And Gideon said to G-d:“Do not be angry with me, and I will speak just this once: let me try just once more, I ask You, with the ﬂeece; let it now be dry only upon the ﬂeece, and upon all the ground let there be dew.” And G-d did so that night; for it was dry upon the ﬂeece only, and there was dew on all the ground. An experiment vs. experiments Any experiment is a step in a dialog Experiments build on each other No experiment is the ultimate one The art of setting experiments Industry standards Elegance Symmetry Many ways to measure the same idea What is convincing evidence? Single cell designs Not an experiment -- a study Parameter estimations Theory test Examples: economics, physics Creating a control group from the single cell Examples: ordering simulation in variety seeking Random assignment Random assignment is the central aspect of experiments By randomly assigning subjects to groups we can assume no systematic differences between groups Assume they are the same on everything but the manipulation. The importance of manipulations By using a manipulation you can determine the direction of causality Based also on random assignment we can link the outcome to our manipulation One-factor designs One-factor designs DV Level 1 Level 2 XX.XX XX.XX One-factor designs 80 80 60 60 40 40 20 20 0 0 Level 1 Level 2 Level 1 Level 2 Level 3 Level 4 Multiple-factor designs Multiple-factor designs Factor 1 Level 1 Level 2 Level 1 XX.XX XX.XX Level 2 XX.XX XX.XX Factor 2 The importance of interactions No interaction 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Ordinal interaction Assumptions? 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Dis-ordinal interaction Assumptions? 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Ordinal Disordinal Interactions help us rule out alternative explanations Examples….. Within vs. between subject designs Which is better? Why What are the advantages of each? What are the disadvantages of each? Experimental design: summary Experimental design is about the design of the structure of the experiment The origins of this is in agriculture Without a good design it is hard to know for sure what we have learned Summary Experiments Random assignment Controls Types of experiments Once cell, one factor, multiple factors The role of interactions Within and between subjects designs