Experimental design 1 Experiments are the most common and useful way to determine causality. What is a good experiment? Hard to say... Experiments are an art form! But in general it is an experiment that control for everything else and tests just what you want it to test. Planned well in advance Controlling for other plausible hypotheses The ﬁrst experiment An Early Experimental Design:“Dew it both ways” Judges (Shoftim) Chapter 6 And Gideon said to G-d: “If You will save Israel by my hand, as You have said, look, I will put a ﬂeece of wool on the threshing-ﬂoor; if there be dew on the ﬂeece only, and it be dry upon all the ground, then shall I know that You will save Israel by my hand, as You have said.” And it was so; for he rose up early on the next day, and pressed the ﬂeece together, and wrung dew out of the ﬂeece, a bowlful of water. And Gideon said to G-d:“Do not be angry with me, and I will speak just this once: let me try just once more, I ask You, with the ﬂeece; let it now be dry only upon the ﬂeece, and upon all the ground let there be dew.” And G-d did so that night; for it was dry upon the ﬂeece only, and there was dew on all the ground. An experiment vs. experiments Any experiment is a step in a dialog Experiments build on each other No experiment is the ultimate one The art of setting experiments Industry standards Elegance Symmetry Many ways to measure the same idea What is convincing evidence? Single cell designs Not an experiment -- a study Parameter estimations Theory test Examples econ, physics Creating a control group from the single cell Ordering simulation in seeking variety Random assignment Random assignment is the central aspect of experiments By randomly assigning subjects to groups we can assume no systematic differences between groups Assume they are the same on everything but the manipulation. The importance of manipulations By using a manipulation you can determine the direction of causality Based also on random assignment we can link the outcome to our manipulation One-factor designs DV Level 1 Level 2 XX.XX XX.XX Multiple-factor designs Factor 1 Level 1 Level 2 Level 1 XX.XX XX.XX Level 2 XX.XX XX.XX Factor 2 The importance of interactions No interaction 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Ordinal interaction Assumptions? 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Dis-ordinal interaction Assumptions? 100 80 Gender 1 Gender 2 60 40 20 0 Low High The importance of interactions Ordinal Disordinal Interactions help us rule out alternative explanations Examples…..