Decision Making under Uncertainty — Experiments and Value of Information

Decision Making under Uncertainty — Experiments and Value of Information • So far, we’ve studied how individuals choose among a given set of lotteries. • Here we are concerned with choices contingent on information that comes from an “experiment”. • Basic questions: - What’s the optimal contingent decision rule? - What’s the value of information? - Can information systems/experiments be ranked regardless of utility function? 1 Basic Structure θ - state of nature a - (final) action u(a, θ) - utility payoff if a chosen and state is θ. Could come from composition of monetary payoff x(a, θ) and utility function over money u e(x): u(a, θ) = u e(x(a, θ)) y - information signal/experimental outcome. 1 Decision Tree: θ1 a1 θ2 y1 a2 θ1 u(a1 ,θ 1 ) u(a1 ,θ 2 ) u(a1 ,θ 1 ) a1 θ2 y2 a2 Figure 1 Example: y — sales forecast a — production decision θ — realized demand x(a, θ) — monetary payoff Note: θ needs to include all payoff relevant events (but nothing more). One and only one state of nature should “happen” in the end. One and only one signal y should occur. Strategy/decision rule: {a(y)} What you decide if you observe signal y. 2 Priors and Posteriors We take a Bayesian view: θ and y are random variables with a joint distribution. p(y, θ) Often, we think of this joint distribution as stemming from two distributions: i a prior distribution p(θ) (= marginal distribution of θ) ii a set of likelihoods {p(y | θ)} ⇒ p(y, θ) = p(θ)p(y | θ) 2 The likelihoods {p(y | θ)} describe the full statistical characteristics of the experiment. For purposes of decision making, an experiment is identified with the set of likelihood functions {p(y | θ)}. The Law of Total Probability states: Z p(y) = p(y | θ)p(θ)dθ θ Example: A medical test with signals {positive, negative} for identifying conditions {healthy, sick}. Such tests are described by two numbers, e.g., p(negative | healthy) p(positive | sick) From these we get the other two likelihoods: p(positive | healthy) = 1 − p(negative | healthy) p(negative | sick) = 1 − p(positive | sick) Bayes rule states: · ¸ p(y | θ)p(θ) p(y, θ) = p(θ | y) = R p(y) p(y | θ)p(θ)dθ θ Probabilities p(θ | y) are called posteriors. Given y, p(θ | y) updates beliefs from the initial prior. Every experiment induces a distribution over posteriors! For any fixed θ, p(θ | ye) is a random variable driven by the distribution of y’s. The function of p(· | ye) is a random vector if there are a finite number of θ− outcomes. This will be conceptually important. Example: θ = {θ1 , θ2 } Priors (and posteriors) are single numbers. Prior: p = Pr(θ = θ1 ) ⇒ 1 − p = Pr(θ = θ2 ) Posterior: p0 (y) = Pr(θ = θ1 | y) 3 Suppose y = L or R p = .5 p(R | θ1 ) = .8 ⇒ p(L | θ1 ) = .2 p(L | θ2 ) = .6 ⇒ p(R | θ2 ) = .4 ⇒ p(R) = (.5)(.8 + .4) = .6 p(L) = (.5)(.2 + .6) = .4 (.8)(.5) = 2/3 (.6) (.2)(.5) = 1/4 p0 (L) = (.4) p0 (R) = Note: E(p0 ) = 2/3 · (.6) + 1/4 · (.4) = .5 = p L 0 p ′( L ) R p = .5 p ′( R ) 1 Figure 2 By the Law of Total Probability: Ey [p(θ | y)] = p(θ) ⇒ p(θ | ·), viewed as a random vector is a martingale. Very important feature of the stochastic process taking priors into posteriors. 4 Sequential Updating. Suppose y1 and y2 are outcomes from two separate experiments. We can view y = (y1 , y2 ) as the outcome of a single experiment and update beliefs about θ based on likelihoods p(y | θ). Or we can update beliefs sequentially: first incorporate the evidence from y1 to go from p(θ) to p(θ | y1 ) and then use the evidence from y2 to go from p(θ | y1 ) to p(θ | y1 , y2 ). Both procedures result in same final posterior. Example: θ1 = healthy yi = + or − θ2 = sick p = prob(θ = θ2 ) − + 0 + − i = 1, 2 p + − 1 Figure 3 On Sufficient Statistics In General, y is multi-dimensional. For instance, it may be a collection of facts or a large sample from an experiment (e.g., to test the effectiveness of a drug). A statistic is any (vector-valued) function T (y). For instance, the mean or average is a statistic. So is variance of a sample, median, etc. Suppose p(y | θ) = p(y | T (y))p(T (y) | θ) (1) where the operational assumption is that the conditional probability p(y | T (y)) does not depend on θ (we can always write p(y | θ) = p(y | T (y), θ))P (T (y) | θ)). When (1) holds we call T (y) a sufficient statistic. 5 The reason is this. Bayes rule gives p(θ | y) = R θ p(y | T (y))p(T (y) | θ)p(θ) p(T (y) | θ)p(θ) =R p(y | T (y))p(T (y) | θ)p(θ)dθ p(T (y) | θ)p(θ)dθ θ ⇒ posterior only depends on y through T (y). That is, for purposes of forming posteriors, it is enough to learn T (y) (rather than all of y). Very often, sample averages are sufficient statistics for the mean of a distribution. Note: The posterior {p(· | y)} is a sufficient statistic. Actually, it is a minimal sufficient statistic (the least one needs to know to form posteriors). The reason sufficient statistics are of interest is that optimal decisions will only depend on posteriors. 3 Decision Analysis A person can find an optimal decision rule or strategy a(y) in one of two ways: Ex Post: Z max u(a, θ)p(θ | y)dθ → a∗ (y) a θ Ex Ante: max a(·) Z Z y u(a(y), θ)p(y, θ)dθdy → a∗ (·) θ Both give the same answer, because ex ante optimality holds if and only if decision a (y) is optimal ex post for every y. Note: Ex post program can be written Z p(y | θ)p(θ) max u(a, θ) R dθ a p(y | θ)p(θ) θ θ Z ∼ max u(a, θ)p(y | θ)p(θ)dθ ∗ a θ v(a, p) ≡ Z u(a, θ)p(θ)dθ v is linear in probabilities regardless of shape of u(a, θ). a(y) = arg max v(a, p(· | y)) a V (p) ≡ max v(a, p) a 6 V is convex, because it is the upper envelope of linear functions. Z VI ≡ V (p(· | y))p(y)dy y This is the maximal expected utility that a person can achieve with information system Y = {p(y | θ)}. Value of information system Y : ZY ≡ VY − V (p0 ) where p0 is prior. Value of Y is the difference between maximal payoff with Y and payoff without Y (i.e., payoff achieved by choosing best action given prior p(·)). Example. Two states: θ1 θ2 Two signal outcomes: y = L or R Two actions: a1 or a2 1 − p = Pr(θ2 ) p = Pr(θ1 ) V (•) u ( a 2 ,θ 2 ) v(a 2 , p) v(a1 , p ) u (a1 ,θ 1 ) VY Z V ( p) u ( a 2 ,θ 1 ) u ( a1 ,θ 2 ) 0 θ = θ2 p0 p′(L ) prior p′(R ) 1 θ = θ1 Figure 4 v(a, p) = pu(a, θ1 ) + (1 − p)u(a, θ2 ) u(a1 , θ1 ) > u(a2 , θ1 ) u(a2 , θ2 ) > u(a1 , θ2 ) 7 p Based on graph, the best decision without Y is: a1 = arg max v(a, p0 ) a V (·) is the squiggly line that identifies upper envelope. According to the graph, if L is observed, a2 will be optimal decision. If R occurs, a1 will be optimal: a(L) = a2 a(R) = a1 Given this rule and considering the probability of L and R, which can be calculated from Law of Total Probability: pL · p0 (L) + (1 − pL ) · p0 (R) = p0 ⇒ pL = Pr(L) we get VY as the average of the value of V (·) at p0 (L) and p0 (R). Z then is the distance between this average and V (p) evaluated at p0 . Perfect information system: p0 (L) = 0, p0 (R) = 1 Totally uninformative information system: p0 (L) = p0 (R) = p0 (prior) Value of perfect information is graphically: Value of perfect information p Figure 5 8 4 Comparison of Information Systems We will consider only the case with two experimental outcomes: yA = L or R yB = B or G Immediate from the graph is that if posteriors from YB “brackets” posteriors from YA , then YB is at least as valuable as YA . (Need not be parallel) R L p ′B (B ) p B p ′A (L) G p ′A (R ) p ′B (G ) Figure 6 Note: The distribution of posteriors from YB is a mean-preserving spread of distribution of posteriors from YA . Given convexity of V (·), this explains (Jensens’ inequality) why YB is more valuable than YA (as can be seen from the graph). More generally, the information system YB is (weakly) preferred to YA by all decisionmakers (i.e., all utility functions u(a, θ)) if and only if posteriors p(θ | yB ) form meanpreserving spread of posteriors p(θ | yA ) for all θ. (Note: This allows both multidimensional θ, a and y.) Mean-preserving spread is better because of Jensen and convexity of V (·). Going the other way, find utility functions such that in one case YA is better, in the other case YB is better. Illustration: 9 v(a1 , p) v(a 2 , p) L B R p G Figure 7 Here YA is better than YB . Flipping payoff functions around gives the opposite conclusion ⇒ we cannot universally compare YA and YB , except when one distribution of posteriors is a mean-preserving spread of the other. Garbling Alternative characterization of information order can be obtained using the notion of garbling. YA is a garbling of YB if PA = MPBT where PA = [pA pA ij ] ij = Pr[yA = i | θ = j] B B PB = [pkl ] pkl = Pr[yB = k | θ = k] M = [mik ] mik = “ Pr[yA = i | yB = k]” M is a Markovian matrix, that is, its columns add up to 1. (The conditional probability interpretation of mik is natural, but the garbling definition does not per se rest on that.) Blackwell: YB is more informative than (i.e., every decision-maker prefers YB to YA (weakly)) if and only if YA is a garbling of YB . Intuitively easy in one direction: Signals yA can be construed as arising out of a two stage process: First, yB signal observed, then independently of θ, but conditional on yB , the signal yA is generated (so yA , given yB , is pure noise). Garbling ⇐⇒ MPS (mean-preserving spread) of Posteriors Easy to see in two-outcome systems YA , YB . 10 Garbling ⇒ p(L | B), p(L | G) are independent of θ. p(θ1 | L) = = = p(L | θ1 )p(θ1 ) p(L) [p(L | B)p(B | θ1 ) + p(L | G)p(G | θ1 )]p(θ) p(L) p(L | B)p(B)p(θ1 | B) p(L | G)p(G)p(θ1 | G) + p(L) p(L) = αp(θ1 | B) + (1 − α)p(θ1 | G) ⇒ p(θ1 | L) is convex combination of posteriors from YB . Similarly true for p(θ1 | R). ⇒ Posteriors of YB bracket posteriors of YA when garbling condition holds. R L p G B Figure 8 To prove result in other direction, note that given posteriors, we find p(L), p(R), p(B), p(G) from Law of Total Probability, (i.e., jump-probabilities in previous graph fixed by the location of the end points/posteriors). Can then run argument in reverse to get Markov matrix. (Note again there is no presumption that yeA is the result of a draw conditional on observing yB outcome.) One implication of Blackwell’s Theorem: Randomization is sub-optimal. 11