Managerial Psych Lab - Experimental Design Notes

Managerial Psychology Recitation 7: Experimental Design Start day – why experimental design so important Imagine you work for Slimfast, trying new diet product line Milkshake, ice, powder, milk replace entire diet for month What is market share? You in charge – figure out what demand will be in market. How going to do it? -similar products Nothing similar This only where mix milk and ice, but supposed to lose 2x weight -hire research firm Ok, you’re research firm now, what are you going to do? -not know about food products What would you ask, how would you ask people? How would you do it -ask about consumption of milk, if want to They say love milk, want to lose weight. Going to buy product? -no, but starting pt Give $10 ask if willing to trade for product Initial purchasing, not long term -give sample Predict before have for market, spend money to develop Work marketing firm or not, Somebody come to you at some pt in life, say we asked people how much they would buy this product for, they said very likely, and for $500 You have to think carefully about how survey was designed, what are questions, what does it mean, how were people interviewed, who wrote it, what was the sample Think all these considerations, take numbers you get, transfer to something meaningful Common challenge, Much like technology Figure out how do it, how interpret numbers Hope you get better idea of how to take raw numbers and how to interpret them Today, watch a struggle to present DVD on screen☺ We might be successful Ad, stand in front of group of people, nothing coming Audience shouting “try config.sys,” etc… One says– get an apeoplee How many of you doing surveys vs experiments? Surveys? Experiments? What is the difference between survey, experiment? -Experiment, not measuring preferences, but actions no -survey observations, experiment variables? No, like example, experiments observe -experiment, actually observing? No, My advisor used to say, couldn’t be more wrong. I haven’t had a chance to use this in class yet, but I’m going to. So, the big difference Survey, one state one cell design, measuring only 1 thing. Look – how large is portion of pasta in cafeteria. Experiment? No, a study, measure only 1 thing, only 1 parameter In experiment, we change something. See if weather outside changes portion of pasta in cafeteria If people know cook, changes portions? Survey -1 cell Experiment – multiple cells Reason to have multiple cells, want to manipulate cells, gives us a way to know causality of the effect Here is a study, one cell. Read and quickly answer. (survey) … Rate Hershey’s Kiss liking Which animal? How many to trade for animal? … What do you think this study is about? Anybody? -quantify preference through Hershey kisses Yes In general, is this a worthwhile study? Say dif is 8. is it a worthwhile study? So, this is not sensible study if want estimate parameter What we want to see here is if amount of Hershey’s kisses correlated to how much one likes Hershey’s kisses? People who like them a lot ask for more or less in trade for the animal? Who thinks ask less? More? Same amount? Why more? -try to get more out of trade Assuming everyone likes stuffed animal better, people who like Hershey’s Kisses more will get more out of each one, value higher, so ask for less Even though single cell design, might be able to make some sense from it. Why? We had a strong theory that people who like Hershey’s kisses more will ask for less kisses Single cell design Useful to extent parameter interesting or have strong theory Strong theory, have prediction. If doesn’t follow, disprove theory. Most useful about Experiment – show causality. The question is how to design them Hard to say What is an elegant experiment? When see it, know it’s elegant Like a picture, you look and say, I like that. Very hard up front to say what to do to make it beautiful In general good experiment controls for everything else, only change the one thing you are interested in. rest about aesthetic value Example from book of Judges, chapter 6 “the first experiment” (slide) Gideon wakes up, hears God telling him to save Israel He says, this sounds like a dangerous task, it’s not every dream that I have that I follow it right away Be a bit More cautious, let’s do the experiment. God, prove to me that this is the case. He is Designing first condition of experiment Puts wool on floor. Says, if wool wet and rest dry, we are getting somewhere Got up next day, Experiment worked – was he happy? Not yet Why? He has not tried this before, maybe it always turns out that only the wool will be wet with dew. He was quite sophisticated experimentally. Ok, let’s do opposite condition, that would be the most discriminating, wool dry, rest wet with dew Now Gideon happy Controlled, did condition, opposite condition, worked out Now I know something else must be causing this 2 conditions, 2 cells This is Example for a good exp Many times, want do multiple exp Series of exp – try get logic across to figure out pt No experiment can be perfect Build case through multiple attempts Single cell designs So we said single cell designs not experiments, but studies Can set parameters, test theories Think about way single cell design, measuring one thing, but can say something Example: I first came to MIT, following experiment Went to Chapel Hill where I studied psychology Brewery, let us be waiters for 1 week We were tasting waiters Beers to taste, described, what would you like to try? Table by table Some gave menus, told to mark Brought beer, ask how much like beer and if they regret making that choice Wanted to look at if when people make decisions one after other, if influenced by the other people More or less like other people? Seeking variety or seeking conformity? How many people think conformity? variety? variety Same experiment in Hong Kong – conformity That’s not the ptoin, it’s if people before you can change what beer you get, and you might not like it very much Here’s what happened People who ordered out loud less happy Why? If you ordered Summer Wheat, and I don’t order Summer Wheat because of that, I order something I like less, I just took a hit on my preferences Wasn’t true 1st person 1st person ordered happiest 2 conditions Also ask another ? People who ordered regularly – too much or too little variety? What’s the standard? Created Simulation Took all people’s data, assigned to random tables, what would be average variety? Compare to actual variety in table Think about it in a way, Take experiment, create some standard Turns out, people had too much variety Paper on my web site Example of take single cell and create some rational standard Random Assignment One of the most important things. Talked about it 1st week w/ causality, correlation Experiment, people doing 1 or 2 things Want to be sure 2 groups differ only on manipulation Say sing for 5 min or poetry 5 min, then ask about careers See if singing out loud changes what we think about possible careers If don’t have random assign, instead let them choose, cannot attribute effect on career choice to anything about the act of singing Might be self selection Random assignment very important Make sure groups same on everything else Most importantly, because of that, direction for causation Everything same, controlled, thing different caused by manipulation Now, do experiment Will be a great experiment Fast, pay attention Face you prefer – 5s look, 2s choose Mark on paper, series of options … Ok, what did you notice happened here? -faces recycled -same comparison made No, not same, but against others Why did that? -if prefer somebody, see if prefer over third What we are looking at is called transitivity When prefer a to b, b to c, but c to a In general when prefer a to b, b to c, also have to prefer a to c Wanted to see if people violate transitivity If violate, means preferences are very hard to come by We asked people to answer 1 of 2 ?s Who do you like more, or who is happier What we want to look at, is What is easier, what do people make less mistakes in? Like more, or who is happier? If one more collective transitivity in one than other, can say which is more difficult Consult own preferences, or look in external world? One factor design – manipulate only 1 thing Have level one, level 2, effective measure. Could be more levels, but one factor Could be, see if people violate transitivity more levels than 2, but one factor -I don’t know Wearing hat backward Could ask if people smile at you more w/ hat backward or forward. One factor design. Why? Manipulate one thing. Backward vs forward -3 types testimonials, neutral, positive, balance Think about ad for presidential election, negative and positive ads. Could look at which is more effective Could say, what motivates students to study, possibility of A or fear of F? Manipulate 1 thing, considered 1 factor, any number levels Students study, can have parents call, peer pressure, fear of failing, all kinds of things Can test all things to affect 1 thing – one factor designs A lot of manipulators, but one factor design In contrast, multiple factor design Take hat example, factor 1 wear hat fwd or bkwd. Factor 2 - we have the hat be Red Sox or Yankees Full factorial design Motivate students do well by frightening or encouraging, Add something else to it Classical music or war marches When cross all levels, multi cell, Multi factor design Single fact des not capturing complexity of real life Studying not only driven by grades, but social pressure Could say Fear vs hope and Parents call y/n Think 2 things could work together Parents call and fear worst performance? Want multiple Story: PhD joint computer, psychology, in same lab. My professor comes in with wife Ask beer, hello Dan, how are you, graduate? Yes. Something wrong, right? This guy graduated w/ PhD, only get job serving beer? Had to write him afterwards, explain Say did something, high energy, low energy music Theory, low energy better performance Gender 2 more than gender 1 2 effects, both work, work independently Could test separately Ordinal interaction – both increase, but more for 1 gender Not just additive Small difference in low energy, but big w/ high energy Woman and high energy, Also have disordinal – one effect one way, other effect other way Men positively affected by music, women negatively Difference for how interpret results All these interactions is the reason we do multi factorial designs Interested to find out why we have reactions so much? Not only tell us effect works, but also when effect doesn’t work Usually feel we understand something better when we know when it works and we know when it doesn’t work Example, motivate people to come to class, all kinds of things we can do What could I do to have no effect? Have effect working and have it not working Another topic within vs between subject designs within – subject or subjects in multiple conditions experiment we did, between subject design - some asked about liking, some asking happy could have been within subject design, 15 pairs happier, 15 who like more -within design, becomes apparent to subject what experiment is about Disadvantage, what is the advantage? -lot of variables get automatic controls All kinds of variances, all have preferencess for like and not like Could Run experiment, give shock, erase memory, try again – not very common These are advantages and disadvantages, there is not one that is better Experimental design is the design of structure of the experiment What cells we have, what factors, what are we measuring? Started in agriculture, figure out what gave more yield Fields of wheat – had to fig out what worked Fertilizer, no fertilizer, sun, no sun Divide field into small parts, cells, try different things Reason for cells in design Figure out which one worked, how they worked Summarize, then show you a great movie Experimental design is about laying groundwork, rules for how exp will work Think about single factor/ multiple factor All interactions? All cells? Next week Will talk hypothesis testing Do experiment, one of cells different Nothing would come up exactly same twice Statistically significant – how decide when something is significantly different? Short survey 2 experiments Pen and Teller conducting, find out if Feng Shui works or not Second, bottled water Episodes: Pen & Teller, Bullshit -Feng Shiu -Bottled water