Learning and Conditioning Study Guide

Simplestudies.org Learning and Conditioning Study Guide Studying Learning • Present a stimulus • Arrange a relationship between two stimuli • Arrange a relationship between a stimuli and a behavior Tolerance & Drugs • Tolerance: a decrease in the response to a drug over the course of repeated administrations • Your body responds to drug effects with compensatory responses to reach homeostasis • If a drug is delivered again and again, the compensatory response increase in strength and thus decrease the reactions • These responses can be conditioned o External OR internal cues • Changing the environment in which you dose can change the ability to respond to it • Professor Siegel’s experiment showed that the ability to tolerate morphine is a learned response o Habituation o Experiment summary: Simplestudies.org • Learned tolerance effect—the idea that we become tolerant to something when we see that stimulus in the same environment Forms of learning • Habituation: present a stimulus o Ex: Flower needs eye drops that are uncomfortable. She doesn’t want her eye meds (which are the stimulus). • Pavlovian learning: arrange a relationship between two stimuli o Ex: Ex: Flower needs eye drops that are uncomfortable but she gets a treat after. She enjoys her eye meds. • Instrumental learning: arrange a relationship between a stimuli and a behavior o Aka operant conditioning- they do something to get something o Ex: lab rats press a lever when a green light is on, they receive a food pellet as a reward. When they press the lever when a red light is on, they receive a mild electric shock. As a result, they learn to press the lever when the green light is on and avoid the red light. Stimulus Learning • Basic t1-t2 design for stimulus learning o Vary experience for learning at t1 o Give common test at t2 o Look for difference in behavior at t2 T1 T2 Experimental S1 S1 Control No S1 S1 o Food imprinting example: three groups of turtles were fed one of three diets. Each group were tested after 12 days and showed that they preferred whatever they’d been eating. Simplestudies.org Habituation • Habituation: an operation in which a stimulus is repeatedly presented • A common outcome of this operation is a decrease in the response elicited by the stimulus o Ex: a rat hears a loud noise and is startled but every time it repeats, it is less startled • What is the most effective procedure to produce habituation? o better with strong stimuli over weak ▪ Davis and Wagner studies the sequence of stimulus intensity o punishment resistant ▪ Punishment: the act or process of imposing and/or applying a sanction for an undesired behavior when conditioning toward a desired behavior. • How do we remove habituation? o Time (spontaneous recovery) ▪ It weakens over time o Application of novel stimulus (dishabituation) • ▪ The novel stimulus removes the effect that was learned ▪ Important for eliminating non-learning accounts of response decrement: • Sensory adaptation • Motor fatigue Case of the missing towels o The problem: ▪ A patient who had been in the hospital for nine years took towels from random places and stored up to 30 in her room. ▪ The staff would remove the towels but the patent would replace them. o The solution: ▪ Give her a ton of towels and don’t remove them ▪ It took four weeks to get 625 problems but during the next ten weeks, the patient started getting rid of them. ▪ It stopped her habit of getting more towels Simplestudies.org Acoustic Startle Response (ASR) • Acquisition o a myth and bad science—neuroscientists report that weak stimuli produce greater habituation than strong stimuli but this is not true ▪ Behavior is an ordinal scale. It is not meaningful to compare the magnitude of changes at different points on the scale. • Ordinal scale—order matters but not the difference between values o ex: pain on a scale of 1-10 is ordinal because 5→7 has the same numerical differences as 3→5 but the actual pain difference is difference • Interval scale—one where the difference between two values is equidistant o ex: the difference between 5 and 6 feet is the same as the difference between 2 and 3 feet o the Davis studies & stimulus intensity ▪ four groups of animals, two of which got 300 stimulus and two of which got 700. two got 108 decibels and two got 120 decibels so there would be 1 of each combo ▪ Davis' demonstration that habituation is greater with strong stimuli than with weak stimuli ▪ there is more habituation with an intense stimulus o little Albert ▪ little A was shown a white rat and when he tried to touch it, there was a loud scary sound ▪ If the experimenter planned to present the white rat and the noise, it’s Pavlovian. If the experimenter waited for little A to reach out, then the stimulus of the rat was just meant to elicit the behavior, which would mean it’s instrumental learning (ambiguous scenario) o implications for exposure therapy Simplestudies.org Good Theories • A good theory will do 3 things: o Explain the data we already have o Make novel predictions o Be falsifiable • Understanding may be best advanced with 2+ theories Theories of Habituation • Groves and Thompson’s dual process theory (biological): habituation and sensitization both occur and cancel out to a certain extent but one exists more than the other o Presentation of a stimulus (S1) o → The habituation process and a decreasing response to S1 ▪ This occurs in the S-R pathway (pre-synaptic side of the junction) o → The sensitization process (increase in response to S1) ▪ This occurs in the system that controls arousal o The response to S1 is the combination of the two above processes o Predictions: ▪ A dishabituator must active the state system ▪ Habituation fades with time (spontaneous recovery) ▪ A short inter-stimulus interval (ISI) will yield better habituation than a long ISI ▪ • A weak stimulus will yield better habituation than a strong stimulus Wagner’s memory model (psychological) o associating a stimulus with context o explaining dishabituation: ▪ the novel stimulus (the dishabituator) triggers sensitization ▪ → Response to the previously habituated stimuli increases o Habituation is explained by retrieval-generated priming. o There will be more habituation to an intense stimulus presented with a long ISI than to a weak stimulus presented with a short ISI. o Three core assumptions of the model: Simplestudies.org ▪ Association is learned between the context and S1 ▪ Limited capacity short-term memory (STM) ▪ Surprise • S1 is surprising if its representation isn’t active in STM when it’s presented • S1 elicits response • Context-S1 association is strengthened and then stored in longterm memory (LTM) o Representation of S1 activation ▪ Self-generated priming: When S1 is presented, it leaves a trace in STM • ▪ The trace decays exponentially over time Retrieval-generated priming: After the context-S1 association is established, context will activate representation of S1 in STM o Explaining dishabituation ▪ The novel stimulus/dishabituator displaces the trace of S1 from STM ▪ → S1 will then be surprising and elicit a response ▪ A dishabituator can work if: • It activates the state system • It increases a response to the habituated stimulus and to a nonhabituated stimulus ▪ If the state system isn’t activated: • The dishabituator increases response to the habituated stimulus but not to a non-habituated stimulus • • Displacement of trace from STM Wagner Memory Model > Dual-Process Theory ISI Studies • Short ISI → trace doesn’t have time to decay so S1 won’t be surprising • Long ISI → trace decays so the stimulus can be surprising o Surprising → learning Simplestudies.org Dear Enemy Effect • Two neighboring territorial animals become less aggressive toward one another once territorial borders are well-established • They expend less time and energy on defensive behaviors directed towards one another • Aggression towards unfamiliar neighbors, however, remains the same • The effect supports Wagner's Memory Model because it shows that context matters for habituation. Pavlovian Conditioning • Classical Conditioning: Ivan Pavlov found that dogs learn to salivate to the sounds that they regularly hear before being fed even without the foot, purely due to association o Pavlov used a metronome, not a bell, to condition dogs to salivate at the sound of it o Developed classical conditioning: neutral stimuli associated with stimuli such as food will produce similar responses as the old stimuli o Classical Conditioning Process: 1. unconditioned stimulus (US or UCS) is presented-the natural thing 2. it is presented with the neutral stimulus 3. the unconditioned response (UR or UCR) occurs naturally Simplestudies.org 4. if the continually paired with a neutral stimulus, they will be associated • Operation + rules o Invented the operation o Developed the vocab o Uncovered the principles: ▪ temporal contiguity: CS and US must occur together in time • ▪ * CS should precede US by .5 seconds stimulus substitution: when the conditioned stimulus is said to have taken on the functions of the unconditioned stimulus • Documented phenomena: o Acquisition: learning has occurred once the animals respond to the CS without the US; repeated pairings of CS and US yield a stronger CR o Conditioned inhibition: a stimulus that predicts the absence of an otherwise expected outcome comes to control an organism's responding. o Extinction: the CS no longer elicits the CR; achieved by presenting the CS without the US repeatedly ▪ Spontaneous Recovery: after extinction, the CR briefly reappears upon presentation of the CS sometimes ▪ Disinhibition: novel stimulus is presented with the CS and there is an increased response to CS (extinction is disrupted) ▪ Rapid Reconditioning o Taste aversion learning is an example • In summary o Conditions: when does association develop? ▪ temporal contiguity-- CS should precede US by .5 seconds o Content: what is learned? ▪ association between CS and US o Expression: what does the CR look like? ▪ Principle of stimulus substitution: CS will elicit the same response as US because the CS is replacing the US • Impacts on us: Simplestudies.org o Impacts our behavior o Neuro-immune associative conditioning ▪ Metalmikov + Chorine (1926): Injected foreign agents → antibodies → boost their immune system → paired injections with lights → light + cholera injection → animals with conditioning had better survival than controls with no conditioning T1 T2 Experimental Light + injection Light + cholera Control Unspecified Light + cholera Pavlovian conditioning in the t1-t2 design he should have done: T1 T2 Experimental CS-US CS Control CS/US CS Conditioning Vocabulary • Conditioned stimulus (CS): a neutral stimulus that is paired with an unconditioned stimulus to elicit a conditioned response (CR) even in the absence of the US o whatever you want to have meaning to the subject • Unconditioned stimulus (US): a stimulus that is not paired with any conditioning (ex: the food in Pavlov’s experiment) o does something to your behavior that is instinctive • Unconditioned response (UR): response elicited by US o instinctive reaction • Conditioned response (CR): response elicited by CS o goal = achieved • Ineffective learning methods: o trace conditioning: presentation of CS, short break, presentation of US o simultaneous conditioning: CS and US presented at same time o backward conditioning: presentation of US, presentation of CS Simplestudies.org • Generalization: the tendency to respond to stimuli that is similar in some way to the CS, but the subjects can be trained to discriminate • Discrimination: learned ability to distinguish CS • Explicitly unpaired: the CS and US are presented but not together (CS/US) Rescorla’s Contingency View Rescorla’s contingency view: association between CS and US depends on the contingency, not on their pairing • learning happens if the CS is informative about if the US will occur • if there is a positive contingency, CS will be associated with the US • p(US|CS) > p(US|no CS) • 1968 experiment o Phase 1: establish steady bar for pressing for food (instrumental) o Phase 2: CER training to establish fear to CS + vary CS-US contingency while maintaining pairings (Pavlovian, t1) o Phase 3: test the CS alone while rats press the lever for food (t2) o → calculate suppression ratio (closer to 0 the better) o All rats were exposed to the same CS (2 tones) and the probability of the US/shock (.4) every 2 min with an average 8 min in between o Groups differed only in the probability of shock during the inter-tone interval • CS-US contingency o positive contingency: p(US|CS) > p (US|no CS) o zero contingency: p(US|CS) = p(US|no CS) o negative contingency: P(US|CS) < p(US|no CS) • Proper control for Pavlovian conditioning is not independent of theory • Contingency study reveals a flaw in Pavlov’s principle of temporal contiguity • Contingency study raises perplexing questions about the learning mechanism o Rate operates over time o How are the events represented? o How is the contingency between events calculated? Simplestudies.org • Rescorla's demonstration that CS-US contingency, not CS-US temporal continguity, determines learning Conditioned Inhibition • Conditioned inhibitor: a stimulus that signals the omissions of an expected US o cancels excitation • Symbols: o A+: US is delivered o AX-: nothing happens after presentation of stimuli • Skinner’s challenge: why do we need to invoke a new learning process (inhibition) to explain what looks like a simple lack of excitation? o A+, AXA+, X- X X o Two ways to test: summation and retardation tests • Summation test: a CI acts on two separate CSs and proves that it acts on the US o A and B are excitatory stimuli, X and Y are inhibitory o Phase 1: A+, AX-, Y-, B+ o Phase 2: B, BX, BY o Prediction: BX 0 • Excitatory conditioning occurs when there is a positive CS-US contingency and inhibitory conditioning occurs when there is a negative CS-US contingency • Learning will occur if the US is surprising • CSs conditioned in compound will lose associative strength if their Vsum > λ • There is no overshadowing on the first compound conditioning trial • In the Rescorla-Wagner model, the associative strengths of all CSs present on a trial will change in the same direction Simplestudies.org • In the Rescorla-Wagner model, λ represents the maximum learning/conditioning supported by the US. Durlach • autoshaping: a method of conditioning in which the conditioned response has not been reinforced by reward or punishment, but is a modified instinctive response to certain stimuli. o AutoShaping can occur with CS-US contingency was zero (random). • Durlach Exp 2 o Used autoshaping in pigeons o L→ food with .25 probability reinforcement (25%) o T = tone • Over the course of training, the context becomes more important • Context = the environment in which is takes place o static cues • Possible that the tone itself conditions the context o second-order conditioning: a form of learning in which a stimulus is first made meaningful or consequential for an organism through an initial step of learning, and then that stimulus is used as a basis for learning about some new stimulus ▪ Ex: you like candy → you like money because you can buy candy → your uncle gives you money → you like your uncle because you can get candy o has value because of the food • In the Durlach autoshaping study, signaling the noncontingent outcomes with a tone, allowed learning about the light-food association to occur even though the CS-US contingency was zero (random). Group .25 T → food L → Food Group unsignaled T → food L → Food, Food (.25) Group signaled T → food L→ Food, T→ Food (.25) Simplestudies.org Instrumental Conditioning • EL Thorndike o Cats were put in a puzzle box with food available outside of it. o The cats moved around until they accidentally manipulated a latch to open a door. o The latency (delay in learning) to escape and get food decreased with time o law of effect: behavioral responses that were most closely followed by a satisfying result were most likely to become established patterns and to occur again in response to the same stimulus. ▪ IOW: if the response is followed by a satisfying event, the S-R association is strengthened. If the response is followed by an annoying event, the S-R association is weakened. • ▪ S-R association = the connection between stimulus and response they don’t learn about consequences because of the response—they learn BECAUSE of the consequences ▪ • Thorndike → two scientific fields emerged: o 1) one focused on an analysis of the underlying processes of behavioral change brought about between a behavior and stimulus relationship o 2) applied behavior analysis: one focused with applying behavior-stimulus contingencies to solving real world problems • Punishment vs Reinforcement o Reinforcement: trying to encourage a behavior o Punishment: trying to discourage a behavior • Positive vs Negative o Negative: subtracting or weakening a stimulus o Positive: adding or strengthening a stimulus • P/N P/R o Positive reinforcement: adding a stimuli to encourage a behavior ▪ ex: get a cookie for doing something good ▪ aka positive reinforcement Simplestudies.org o Negative reinforcement: taking away a stimuli to encourage a behavior ▪ ex: stopping a loud noise as a result of completing a task ▪ aka omission training ▪ CAT: constructional aggression treatment • has a dash and a letter after it for whatever animal (CAT-H would be for horses) • removal of an aversive stimulus as a reward used for animals scared of humans where the person finds the threshold, marks that threshold, and holds that position, steps back on cue, and repeats when stable o Positive punishment: adding a stimuli to discourage a behavior ▪ ex: get shocked for touching a button ▪ aka punishment o Negative punishment: subtracting a stimuli to discourage a behavior • ▪ ex: taking away food as a result of pressing a lever ▪ aka escape avoidance Shaping: step-by-step training that is used to teach a behavior that is not in the subject’s repertoire o systematically and differentially reinforcing successive approximations of a terminal behavior • Tolman (the first cognitive psychologist) o Reinforcement is not necessary for learning ▪ IOW: Thorndike and law of effect are wrong o Latent Learning: subconsciously acquiring information, becomes obvious only once a reinforcement is given for demonstrating it o Performance depends on motivation o Edward Tolman’s experiment had three groups of rats: ▪ never got a reward ▪ always got a reward for completing the maze ▪ not rewarded during the first half of trials, rewarded in second half Simplestudies.org • during the first half, their performance was very similar to the group that never got a reward but performance improved dramatically and suddenly once they began to be rewarded for finishing the maze ▪ → conclusion: the rats must have learned their way around the maze in the first half, the dramatic improvement in the second half resulted from latent learning • Hull: S-R Theory (habits are reinforced) o wanted laws and principles and postulates to make psychology more like a science o The reward is responsible for learning but is not a part of what is learned. o Outcome devaluation will not affect performance of a response that had been previously trained with that outcome. o The S-R association is strengthened when the reward follows the response o Reinforcement = drive reduction o E = D X H (Performance = Drive x Habit) o Drive Reduction Theory: our behavior is motivated by biological needs (requirements for survival) ▪ Drives: our impulses to act in ways that satisfy our needs/desire to reach homeostasis • Primary drives: biological needs (ex: thirst) • Secondary drives: learned drives (ex: wanting money to satisfy primary drives) ▪ Ex: need is food, drive is hunger, drive-reduction: snacking o Homeostasis: a balanced internal state our body strives to be in: when we are out of it, we have a need that creates a drive o Incentives: positive or negative stimuli that entice or repel us • Garcia and Koelling studied selective Pavlovian conditioning o Phase 1 ▪ Group 1: Sick + light and noise → illness ▪ Group 2: Sick + light and noise → shock (fear conditioning) Simplestudies.org o Phase 2 ▪ ½ Sick, ½ light and noise for each group o Results: ▪ when the US was shock, they drink the saccarine (even if it makes them sick) ▪ they associated different elements of the compound stimulus that’s paired equally with stimuli ▪ • more likely to blame the audiovisual stimuli for not feeling well Foree and LoLordo: selective instrumental learning o between subjects design using pigeons o L= red light, T=tone, R = treadle press o Outcome for group 1 = positive reinforcement with food o Outcome for group 2 = negative reinforcement with avoidance of shock • Selective sensitization: what you do is you amplify a subset of responses o ex: if you get sick, you become sensitized to internal cues but it doesn’t sensitize you to external cues (sounds/lights) o ex: if you get a shock, you become sensitized to external cues but not internal cues o aka selective learning aka accelerated learning • Optimal Arousal Theory: we seek an optimum level of excitement or arousal o we are motivated by activities that help us achieve our needed level o balance of stimulation and relaxation, motivated to avoid boredom and stress • Yerkes-Dodson Law: we might perform well at an easy task with a very high arousal level o the same high level would worsen our performance on a difficult task • Opponent-Process Theory: people are usually at a baseline (normal) state. We might perform an act that moves us away from this state that may be pleasurable at first, but eventually we feel an opponent process and want to return to baseline state o with physically addictive substances, we get withdrawal and are required to have more to return to new baseline state o explains addictive behaviors Simplestudies.org • Incentive Theory: we learn to associate stimuli with rewards and others with punishment/motivated to act due to incentives • Maslow’s Hierarchy of Needs o Physiological needs: to satisfy drives for hunger, thirst, and shelter o Safety needs: to feel safe, secure, and out of danger o Belonging and love needs: to be accepted and belong o Esteem needs: to achieve and to gain approval and recognition/respect o Self-actualization needs: to achieve your unique potential, spiritual growth • Associative structure of instrumental conditioning o What do we learn about the outcome? ▪ S-R Theorists would say nothing. Positive outcomes reinforce the S-R association and make learning happen but aren’t learned themselves. ▪ Tinklepaugh—expectancy violation: behavior is different when stimuli are substituted • ex: monkeys acted differently when lettuce is expected and found vs when a banana is expected but lettuce is found ▪ Crespi—contrast effects • vary # of pellets across groups in phase 1 but then give the same # of pellets in phase 2 • S-R theory would predict they all converge to the same level BUT running speed decreases when the new reward is less than the old one • → indicates encoding of prior rewards because that’s necessary for comparison to new rewards • elation effect: more of a response by increasing the reward magnitude • depression effect: less of a response by decreasing the reward magnitude • Klossek et al to see what children understand about their behavior’s consequences o 72 children were divided into 3 10-month age bands Simplestudies.org o they touched a red or green butterfly icon on a touch-screen to see different cartoon videos for each color o devalue through outcome satiation—the butterflies disappear and the kids watch one of the cartoon videos over and over until they are sick of it o extinction test—they are then given the butterflies again and they will press the butterfly for the one they didn’t just see a lot (the one that was devalued) o then retraining o conclusion: the actions of the older children’s behavior depended on the current values of the outcomes but younger children did not ▪ goal-directed action develops between 2 and 3 years (hence the “terrible twos”) • OD Design (where Rs are responses and Os are outcomes like sugar water/pellets, + denotes devaluation, - means no devaluation) o Training: R1-O1, R2-O2 o Devaluation: O1+, O2o Test: R1 v R2 o Possibilities: • ▪ The R-O associations aren’t sufficiently established ▪ The O1+ was not devalued enough, might still be equal to O2- Outcome mediated transfer o within subject design Choice • The science of wise decision making: o the matching law: reduce or increase alternative sources of reinforcement ▪ Rate of a response depends on its own rate of reinforcement and the reinforcement rate of other available responses. ▪ 2. If a pigeon earns 10% of its reinforcers for pecking key A, it will emit 10% of its responses on key A. ▪ 3. Behavior cannot be viewed in isolation. o delay discounting: too many people choose the immediate small reward than the larger delayed reward ▪ IOW: lower value is assigned to remote rewards Simplestudies.org ▪ pathological gamblers and substance abusers tend to make impulsive choices, possibly by discounting the value of delayed rewards ▪ Rachlin and Green introduced this term ▪ indifference point: the point at which people are unsure if they would rather wait for a better reward or get a smaller immediate one • ▪ get an indifference point for each length of delay v = V / (1+ kD) • v = perceived value • V = actual reward value • k = discounted rate • D = delay • 1 is there in case there is 0 delay • In the formula v = V / (1 + kD), v represents the discounted value and D represents the delay in reward • Reinforcement Schedules o Continuous reinforcement: rewarding the behavior each time, best when first teaching a new behavior o Partial-reinforcement effect: behaviors will be more resistant to extinction if the animal has not been reinforced continuously; they yield higher response rates than continuous ▪ Ratio schedule: reinforcement is delivered based on number of responses made (ex: after doing it 2x) ▪ Interval schedule: reinforcement is delivered based on the passage of time (ex: weekly) o The pattern of reinforcement: ▪ Fixed schedule: constant ▪ Variable schedule: changing (more resistant to extinction than fixed schedules) • → The four reinforcement schedules: o Fixed-ratio (FR) schedule: provides reinforcement after a set number of responses (ex: FR-5 schedule: subject will be rewarded after the fifth response) Simplestudies.org o Variable-ratio (VR) schedule: provides reinforcement based on a varying number of responses (ex: VR-5 schedule: average number of responses required to get a reward is five) o Fixed-interval (FI) schedule: requires that a set amount of time elapse before a response results in a reward (FI-3 minute schedule: rewards the first response that occurs after three minutes) o Variable-interval (VI) schedule: varies the amount of time required to elapse before a response will result in reinforcement (VI-3 minute schedule: subject will be rewarded for the first response made after an average of three minutes) • Herrnstein’s experiment on concurrent schedules o pigeon looks at two keys, one VI 10 and VI 30 (VI = variable interval, one of four kinds of schedules of reinforcement) • Implications of matching: o Rate of R1 depends on: ▪ its own reinforcement rate AND ▪ the reinforcement rate of other behaviors o Consider behavior in a reinforcement-barren environments vs an environment with other reinforcement options Instrumental Conditioning Studies • Thorndike → habits o Association • Study between stimulus and response is strengthened by the outcome 1: Expectation violation (Tinklepaugh’s monkeys) – demonstration w candy in class • Study o Animal expects lettuce, picks up lettuce cup o Animal expects banana, picks up banana cup o Animal expect banana, finds lettuce → furious o Animal expects lettuce, finds banana → excitement 2: Vary reward magnitude (Crespi) Simplestudies.org o Rats learned to get 64 16, or 1 pellets during acquisition. The ones running for 1 run slower than the ones running for 64 pellets. S-R theory predicts bigger rewards have stronger S-R associations, (and bigger responses?) o S-R theory predicts they all converge on the same speed when they all are met with 16 pellets. Group that learned with 1 pellet gets faster, group that learned with 64 run slower. 16 group stays the same o In reality, group that was trained on 1 runs faster than group 16 (elation effect) and group trained on 64 runs slower than group 16 (depression effect) • Study 3: Outcome devaluation (Colwill & Rescorla) o Acquisition: Animals taught to press lever for pellet and pull chain for sucrose. (VI 60 schedule) o Devaluation o Test: → pellet: LiCL. Sucrose: no aversive stimulus Lever (devalued) vs chain (devalued) ▪ S-R theory says that the animal has an association between the context and the lever and the context and the chain. Thus, they respond the same. (devaluation is ineffective) ▪ If you have R-O associations, then the lever would be devalued. These are the results we see, thus S-R is wrong o The results from Colwill and Rescorla's (1988) transfer study support Animals learn both S-O and R-O associations • Study 4: Transfer o Contemporary data ▪ Outcome mediated transfer (within subject design) Sd ▪ S1: np - O1 ▪ S2: np – O2 ▪ NP ▪ Lever ▪ Test is nosepoke. ▪ S-R theory says: S1-np association will form ▪ S-O theory says: S1-O1 association will form press (R1) gets pellet (O1). Chain pull (R2) gets sucrose (O2). transfer: Simplestudies.org ▪ S-R theory predicts equal responding to R1 and R2 in the presence of S1. Equal responding to R1 and R2 in the presence of S2. ▪ Reality: ▪ R1 is same as S1. R2 is diff from S1. ▪ R1 is diff from S2. R2 is same as S2. ▪ S1 amplifies performance of R1, S2 promotes performance of R2. S1 has no effect on R2 and S2 has no effect on R1. ▪ In S1, animal does R1. In S2, animal does R2. What the animal does depends on ▪ ITI response rate: when no stimulus (noise/light) present ▪ Stimulus influences behavior even if it hasn't been trained with behavior. ▪ Suggests o Possible we form S-O and R-O associations. outcomes: ▪ S-R theory doesn’t explain behavior ▪ Better: • Study o ▪ S-O and S-R associations exist (2-process theories) ▪ R-O and S-O associations 5: Switching Design L and N are discriminative stimuli, R1 and R2 are responses (lever press and chain pull), O1 and O2 are outcomes (pellets and sugar water), + is LiCI (reduce value of outcome) o Training: o S-R ▪ L: R1-O1, R2-O2 ▪ N: R1-O2, R2-O1 associations: ▪L → R1 ▪L → R2 ▪N → R1 Simplestudies.org ▪N o S-O associations: o R-O ▪L - O1 ▪L - O2 ▪N - O1 ▪N - O2 associations: o What • → R2 ▪ R1 - O1 ▪ R1 - O2 ▪ R2 - O1 ▪ R2 - O2 if O1 becomes aversive, but not O2? (O1 → LiCl, O2 → nothing) ▪ What happens in L: R1 vs R2 ▪ What happens in N: R1 vs R2 Hierarchical association (3 term contingency) o If S then R leads to O aka S(R-O) Rewards • The Premark Principle o aka the grandma principle because Grandma says: “If you eat your peas, then you can have cake.” o high-probability behaviors can be used to reinforce low-probability behaviors ▪ high-probability behaviors: behaviors performed frequently under conditions of free choice o premack? • Overjustification effect o Extrinsic reward reduces intrinsic motivation o When you introduce an extrinsic reward, it weakens the strong intrinsic motivation that drove the original high performance o ex: Lepper, Greene, and Nisbett said women should not be paid to donate blood because payment → wanting to do it for money rather than just to be good Simplestudies.org ▪ white male and female children told to paint pictures ▪ 3 groups: • expected award (told they’d get a good player certificate) • unexpected award (surprise with a good player certificate) • no award ▪ procedure: observe → experimental treatment → observe ▪ kids that expected and got an award were not likely to do it without the award Differential Outcomes Effect Control of Behavior • Identifying the stimulus o Stevens says that “What is the stimulus?” is the most important question in psych o Reynolds used a white triangle on a red background to signal VI 3min for pecking in pigeons ▪ tested white triangle without red background and red background without triangle (separated the parts to see what responses the animal would have to the presentation of the parts) ▪ the pigeons saw the same thing but some picked the white triangle and some picked the red background • this indicates that the stimuli that controlled their behavior was different ▪ compound stimulus ▪ elemental theorists would say it’s a white triangle with red behind it; gestalt theorists would see it as a pattern, not as its parts • Differential Outcomes Effect (DOE) o Trapold used 3 groups of rats that were trained to know the tone meant to click the right level and clicker meant left lever (instrumental task) ▪ G1 (experimental): T-RL-F, C-LL-S ▪ G2 (food control): T-RL-F, C-LL-F ▪ G3 (sucrose control): T-RL-S, C-LL-S ▪ two process theory ?? Simplestudies.org o Discrimination ratios ▪ DR=1 means that there are no incorrect responses, DR=0 means no correct responses ▪ Usually starts at DR=.5 (50%) and improves as subjects learn o Association develops between sample S and outcome → Generates a unique expectancy to control choice → Outcome expectancy easier cue to remember Discrimination vs generalization • Discrimination: differential responding to two or more stimuli o Discrimination training ▪ used to determine sensory abilities ▪ Bach vs Hindemith – pigeons and classical music • Bach means key leads to food, Hindemith means key does not lead to food • • If pigeons could discriminate, they would only peck for Bach Generalization: similar responding to two or more stimuli o Stimulus generalization gradient—gradually changing the stimulus in one direction • Discrimination and generalization in consumer psychology o discrimination is used for separation from competition o generalization is used for branding, copying ▪ when a particular brand uses similar packaging design for its products to extend goodwill to all its products ▪ when a competing brand uses a similar packaging to an established brand, they want to steal customers • Peak shift- Stimulus discrimination training produces it in a generalization test o behavioral response bias arising from discrimination learning in which animals display a directional, but limited, preference for or avoidance of unusual stimuli. Simplestudies.org Absolute or relative • Spence- absolute value of stimulus learned o excitation gradients around S+ o inhibition gradients around So you pick the stimulus most like the S+ you were trained with • Kohler- relational approach o when you give a subject two stimuli, one that is reinforced and one that isn’t, the subject compares the two o ex: brighter than, louder than, dimmer than, etc • transposition tests prediction • intermediate size problem o with three stimuli (2 S- with a S+ in the middle) o Spence says excitation will occur with the S+ in the middle and inhibition for the S- on either side → teepee looking graph o animals follow the prediction rom Kohler and pick the middle one (the immediate) Extinction • Extinction means original learning is masked, not erased o Five phenomena to support: ▪ spontaneous recovery (Pavlov) ▪ disinhibition (Pavlov) ▪ rapid reacquisition (Pavlov) - retraining happens faster because you are building on retained knowledge • ▪ renewal ▪ reinstatement Elaborating on renewal o renewal: the return of responding to a previously extinguished CS when that CS is tested in a context different from the extinction context. Simplestudies.org o Bouton and King taught 3 groups to press a lever for food and taught them tones lead to shocks ▪ group 1: tone in familiar context - extinction ▪ group 2: tone in new context – extinction ▪ group 3: no extinction ▪ over the course of extinction, groups 1 and 2 find that they are less and less scared ▪ group 1 was much less scared because extinction occurred in a familiar context ▪ group 2, when extinguished in context b but exposed to context a, they were still shocked ▪ context is a cue that helps with the interpretation of a meaning of ambiguous stimulus o renewal studies conclude that extinction is context specific ▪ ex: a drug addict relapses when released from rehab ▪ extinction environment should be similar to the environment in which the stimulus usually exists o renewal – extinguishing in a different context • Elaborating on reinstatement o reinstatement: the return of a response to an extinguished conditioned stimulus due to exposure to an unconditioned stimulus o train and extinguish in the same context, present US/food reward and responding comes back o can cause relapses o ex: For example, a person who gets a headache from using earbuds may develop an aversion to using them. This aversion may become extinguished after the person tries using earbuds again a few times without getting a headache. However, if the person then gets a headache (for any reason), the aversive response to using earbuds may return (or be reinstated). • Resurgence- you train one response to a stimuli, then extinguish it in the same context, then present a similar stimuli Simplestudies.org o the first extinguished response recovers when you start extinguishing a second response in that context o renewal but in the same context • Bouton conclusions o For rats that did not receive extinction, it did not matter whether the reinstatement shocks were given in the same context or a different context ▪ contextual conditioning did not summate with the suppression elicited by the CS o For rats that did receive extinction, reinstatement shocks given in the same context produced more suppression than shocks given in a different context ▪ Contextual conditioning facilitates the reinstatement effect o A trained and extinguished stimulus is ambiguous (it has two meanings); context is used to resolve that ambiguity; an excitatory context retrieves the CS-US memory whereas a neutral context retrieves the CS-no US memory • Learned in extinction o inhibition accounts for response decrement and protection of original association o Rescorla did an experiment on extinction of a Pavlovian CS to see if there is inhibition • Extinction of an instrumental response o acquisition of an inhibitory association between the stimulus or context and the nonreinforced response o differential outcomes effect: the increase in speed of acquisition and accuracy that occurs in conditional discrimination learning when two or more stimuli are correlated with a particular outcome. • 2-process theorists o S-R (stimulus-response) and S-O (stimulus-outcome) o → expectancy of the outcome Spence’s Discrimination Learning • Spence's discrimination learning o (a) learning is continuous Simplestudies.org o (b) discriminative stimuli in simple discrimination tasks of both simultaneous and successive types are compounds with position elements as well as elements from the relevant visual dimension. • Excitatory strength is increased when a response to a specific stimulus is reinforced • Inhibitory strength is increased when a response to a specific stimulus is not reinforced • Excitatory and inhibitory strength generalize to surrounding stimuli • The tendency to respond to any stimulus is based on its net excitatory strength Applied Behavior Analysis • Most widely used for autism, now also used for weight loss • Using techniques such as reinforcement, prompting, operant conditioning, shaping, and contingencies to teach new behaviors or decrease undesired ones • Reasons might be for sensory, attention, escape, or gain Misc • Contiguity vs contingency o Contiguity: CS co-occurs with the US: they are contiguous, or close together, in space and time. o Contingency: the CS predicts the US: the occurrence of the US is contingent on the prior occurrence of the CS. • Response fatigue: exhaustion to responding • Things have to be present at the same time/same trial to compete • Sensitization acts on the response itself • Sign-tracking = autoshaping o propensity of individuals to engage with a conditioned stimulus (CS) that has been paired with a rewarding unconditioned stimulus (US), even when the location of the eventual reward is not colocalized with the CS o IOW: behavior that is directed towards a stimulus as a result of a learned association between that stimulus and a reward • Goal-tracking = a CR that is directed towards the location of reward delivery (i.e., the goal) rather than the signal for reward Simplestudies.org • AVN: anticipatory vomiting and nausea • PVN: post-treatment vomiting and nausea • Two CSs are presented together and paired with a shock US on four separate occasions. In a subsequent extinction test, the CSs are presented separately while the subjects lever press for food. The suppression ratio to one stimulus (A) is .4 and the suppression ratio to the other stimulus (B) is .2. This is an overshadowing study and B is more salient than A o more suppressed → more salient • Conditioned suppression: a phenomenon that occurs during an operant performance test when a conditioned response to a positive stimulus is reduced by another stimulus that is associated with an aversive stimulus. • Long-delay learning • Selective learning happens and to account in the Rescorla Wagner Model, add another rate parameter with alpha and beta: gamma o between 0-1 o emphasizes the importance of how we choose CSs • The study by Wolf et al. (1964) successfully used what procedure to teach a little boy to wear his glasses via shaping • elemental theorists focused on the makeup of things (?) • gestalt theorists saw things are configurations o sum is greater than its parts • discrete trial—signaling when a stimulus is going to come because the stimulus marks the start of a trial • instrumental transfer is outcome mediated transfer o Pavlovian instrumental transfer: when a CS that has a classically conditioned association with rewarding or aversive stimuli alters motivational salience and operant behavior. • two process theory- assumes that there are two distinct types of learning: (S-O) and instrumental conditioning (S-R) • R-O: response-outcome o when you do this, this happens • S-R-O: stimulus-response outcome Simplestudies.org o when you do this, this happens and usually in a specific context/with a specific stimuli • outcome devaluation: when an outcome/reward loses value