Ambiguous Correlation Research

Ambiguous Correlation Abstract Many decisions are made in environments where outcomes are determined by the realization of multiple random events. A decision maker may be uncertain how these events are related. We identify and experimentally substantiate behavior that intuitively re‡ects a lack of con…dence in their joint distribution. Our …ndings suggest a dimension of ambiguity which is di¤erent from that in the classical distinction between risk and “Knightian uncertainty.” 1 Introduction Individuals often make decisions when there is limited information about the stochastic environment and hence where there may be incomplete con…dence that any single given probability law accurately describes it. Ellsberg (1961) identi…es behavior that intuitively re‡ects such a lack of con…dence, and shows thereby that the distinction between risk (where information is perfect and con…dence is complete) and ambiguity is empirically meaningful. In this paper, we consider a setting where uncertainty is generated by the realization of multiple random events and we study ambiguity about how these random events di¤er or are related to one another. Paralleling Ellsberg, we identify behavior that intuitively re‡ects a lack of con…dence in (or uncertainty about) their joint distribution. Then we conduct a controlled incentivized laboratory experiment the results of which support the empirical signi…cance of this new dimension of ambiguity. In Ellsberg’s (1961) classic two-urn thought experiment, there are two urns, each containing 100 balls that are either red or black. You are told the exact color composition for one (50-50), the risky urn, and nothing at all about the composition of the other one, the ambiguous urn. Intuition suggests, and many subsequent laboratory experiments con…rm, that many people prefer to bet on drawing red (black, respectively) from the risky urn as opposed to from the ambiguous urn, thereby demonstrating a behavioral distinction between risk (known composition) and Knightian uncertainty (unknown bias or composition). We consider a setting where there are two urns and you are told the same about the composition of each, so that you have no reason to distinguish between them. However, you are told very little; for example, only that each urn contains two balls each of which is either red or black. Accordingly, you are not given any reason to be certain that the compositions are identical, nor are you given any reason for being con…dent that the urns’compositions are unrelated or related in any particular way. We study choice between bets on the colors of two balls, where simultaneously one ball is drawn from each urn, and we identify behavior that intuitively reveals a lack of con…dence concerning the relation between the urns. The key idea is that ambiguity about how the compositions of the two urns might di¤er or be related is not relevant to bets on a single urn. This leads us to focus on the choice between bets on the color of the ball drawn from one urn versus bets on the colors of the balls drawn from both urns (speci…cally, on whether the 2 two balls have the same color, or whether they have di¤erent colors). The behavior we identify is then tested in a controlled laboratory experiment which provides evidence of sensitivity to the lack of information concerning the relation between the urns’compositions. By considering also bets on an urn known to contain an equal number of red and black balls, we …nd that this new form of ambiguity aversion is only partially related to Knightian ambiguity aversion as measured in Ellsberg’s experiment, and we study at a behavioral level the three-fold distinction between risk, uncertain bias (or composition) and the uncertain relation between biases. The experimental design developed to study these preferences identi…es strict preference using pairwise choices between bets. To check the robustness of the results, we also employ standard choice lists that elicit an approximation to the certainty equivalent of each bet. The preceding summarizes the main contributions of the paper: highlighting ambiguity about correlation and providing supporting experimental evidence. The potential relevance for economic applications is addressed next in this introduction (and also in the concluding section). Our thought experiment is described in more detail in Section 2, and the following two sections describe the experimental implementation and results respectively. Section 5 turns to the question of how to model the observed behavior. Finally, a concluding section elaborates on the economic relevance of ambiguity about correlation and discusses some related literature. Economic signi…cance: Betting on the draws from several urns is intended as a canonical example of choice problems where payo¤s to an action depend on the (simultaneous) realization of multiple random events. A textbook example is the choice between bets on multiple tosses of a coin of unknown bias. Optimal portfolio choice is another important example. Here we indicate that the behavior we identify is relevant (albeit indirectly) to the potential gains from portfolio diversi…cation, one of the central principles of …nancial economics, to the “limited stock market participation”puzzle, and to the pricing of idiosyncratic uncertainty in a cross-sectional setting. The concluding section describes other motivating examples. It is well-known (Dow and Werlang 1992) that given a safe asset and a single uncertain stock, then nonparticipation in stocks is a knife-edge property under subjective expected utility maximization, but that it can be robustly optimal, (that is, optimal for a range of expected excess returns), if there is ambiguity aversion. This begs the question whether in the more realistic 3 situation where there are many uncertain assets, if diversi…cation can diminish the e¤ect of ambiguity on participation and possibly even restore the expected utility result asymptotically when the number of stocks is large. As a concrete example, suppose that there are I securities available and that the investor’s model of returns is a linear factor model as in arbitrage pricing theory. That is, the ith return si takes the form si = i X + i , i = 1; 2; :::; I; the vector X gives factor returns and i gives the betas or factor loadings of security i. Typically, strong assumptions on the idiosyncratic terms i are adopted, say that they are identically and independently distributed (i.i.d.). However, suppose that the investor is not con…dent that X captures all relevant factors. She may not be able to identify missing factors–if she could, then X could be expanded to include them–but she may nevertheless wish to take into account their possible existence when choosing a portfolio. Then there is no basis for taking a stand on how the i s may di¤er or are related across securities–if rates of return are in‡uenced also by omitted factors, then the distribution of the residuals i depends on the nature of the omitted factors, which, by assumption, is poorly understood. In particular, if the investor is sensitive to the resulting ambiguity about the di¤erences and relation between returns, she may perceive only limited gains from diversi…cation which may limit her degree of stock market participation. Indeed, Epstein and Seo (2015) show, using a formal model of preference that can accommodate the new ambiguity-sensitive behavior that is our focus here, that the optimality of nonparticipation is robust to the presence of many stocks even if I grows without bound. In the literature on optimal portfolio choice, Fouque, Pun and Wong (2016), in a continuous-time model, and Liu and Zeng (2016) and Huang, Zhang and Zhu (forthcoming), in a static mean-variance framework, show that limited diversi…cation is optimal given suitable ambiguity about the correlation of returns (see also Jiang and Tian (2016)). Readers are referred to these papers and the references therein for arguments that estimation of the correlation of returns can be di¢ cult, more so than for moments associated with any single asset, thus supporting the assumption of correlation ambiguity as opposed to correlation risk.1 1 One reason cited for the di¢ culty given high-frequency data, is that because di¤erent assets are traded at di¤erent times it is necessary to synchronize asset returns and hence remove some data points (Ait-Sahalia et al 2010). 4 There are e¤ects also on the pricing side–ambiguity about how returns are related implies that idiosyncratic uncertainty can have a positive price in equilibrium even in the limit as I goes to in…nity (Epstein and Schneider 2010). Though this may strike some as contrary to the intuition based on the classic Law of Large Numbers, the point is that the latter does not apply when there is ambiguity about security returns (that is, of the i s) of the sort studied here (Maccheroni and Marinacci 2005). 2 A Thought Experiment Two urns, numbered 1 and 2, each contain two balls, each of which is either red or black; no additional information about the urns’ compositions is given.2 One ball is to be drawn from each urn simultaneously. Thus the set of possible outcomes is fR1 B2 ; B1 R2 ; R1 R2 ; B1 B2 g, where the characters correspond to the color of the ball (red or black) and the subscript to the urn (1 or 2). Its subsets are called events. Before the balls are drawn, an individual is asked to choose between speci…ed bets on the colors of the two balls. Denote by R1 B2 both the obvious event and the corresponding bet that yields the (positive) prize x if that event is realized and the prize 0 otherwise; similarly for other events and bets. Prizes are denominated in dollars. Both the events fR1 B2 ; R1 R2 g and fB1 R2 ; B1 B2 g and the corresponding bets on the color of the ball drawn from urn 1 are sometimes denoted simply R1 and B1 respectively. Let denote a preference relation on the set of bets. As outlined in the introduction, our thought experiment, which we term One vs Two, o¤ers the individual the choice between betting on the color drawn from one urn as opposed to betting on the colors drawn from both urns. More precisely, consider the following rankings between the bets Same = fR1 R2 ; B1 B2 g, Dif f = fR1 B2 ; B1 R2 g and the bets R1 and B1 on the draw from urn 1: R1 Same and B1 Dif f . (2.1) Why would an individual exhibit these rankings? The intuition is that only the bets on the draws from both urns are subject to ambiguity about how urns di¤er or are related, which may, depending on the degree of aversion to 2 In Ellsberg’s two-urn experiment, the individual is given di¤erent information about the two urns. In particular, since one of the urns in that experiment has known composition, there is no scope for ambiguity about how the two compositions may be related which is our focus here. 5 such ambiguity, lead to the preference for the bets R1 and B1 . To elaborate, consider the choice between R1 and Same. Betting on the same color being drawn from both urns is attractive if it is believed that the compositions of the two urns are similar, which would make “positive correlation” between the colors drawn likely. Since you are not told anything to the contrary, this belief is plausible but no more so than the belief that the two compositions are di¤erent–one urn is biased towards red and the other towards black–which would make “negative correlation” between the colors drawn more likely and render Same an unattractive bet. Given a conservative attitude, this uncertainty would act against choosing Same. Of course, there is also reason for a conservative individual to discount the bet R1 because the composition of each urn is ambiguous. Conclude that a preference for R1 may arise if there is greater aversion to ambiguity about the relation between urns than to ambiguity about the bias of (the …rst urn and hence, presumably) any single urn. A simpler rationale for a strict preference for R1 is that the individual believes that the …rst urn has many more red than black balls, which rationale does not rely on sensitivity to ambiguity of any sort. To rule out this alternative rationale, we follow Ellsberg and the ensuing literature in considering also another choice–in our case, that between B1 and Dif f . A strict preference for B1 can be understood as above as a re‡ection of an aversion to ambiguity about the relation between urns, (here the unfavorable scenario for Dif f is that the two urns might both be biased towards the same color), which thus “explains”both rankings indicated in (2.1). In contrast, the pair of rankings is inconsistent with beliefs that can be represented by any probability measure: there does not exist a measure P on the four possible pairs of colors satisfying P (R1 ) > P (Same) , P (B1 ) > P (Dif f ) P (R1 ) + P (B1 ) = 1 = P (Same) + P (Dif f ) . More formally, the rankings contradict probabilistic sophistication as de…ned by Machina and Schmeidler (1992).3 Note that such a contradiction exists also if (2.1) is weakened so that at most one of the rankings is weak. Such behavior is abbreviated below as One Two. 3 Familiarity with their formal de…nition is not needed in the sequel. The reader can take probabilistic sophistication to mean simply that there exists a probability measure such that, when choosing between bets, the individual always prefers to bet on the event having higher probability. 6 A contradiction to probabilistic sophistication exists also if both rankings in (2.1) are reversed, with at least one of them strict, which we abbreviate by Two One. In the Ellsberg experiment, where the composition of one urn is known, a preference to bet on the unknown urn is naturally understood as ambiguity seeking. Here, however, because both urns have unknown compositions and all four bets considered above are ambiguous, the behavior we study does not justify taking a stand on whether the individual likes or dislikes ambiguity about correlation in an absolute rather than relative sense. As a result, just as we interpreted (2.1) above in terms of relative ambiguity aversion, we interpret the pair of reverse rankings as indicating a lesser aversion to (or a greater a¢ nity for) ambiguity about the relation between urns than to ambiguity about the bias of any single urn. Besides those discussed thus far, all other choices in One vs Two are consistent with probabilistic sophistication. For example, the rankings Same R1 and B1 Dif f (2.2) can be rationalized by any probability measure satisfying P (B1 B2 ) P (R1 B2 ). A Bayesian with an i.i.d. prior uniform within each urn would be indi¤erent between all four bets indicated. We turn now to describing our experimental investigation of One vs Two. 3 Experimental Design We conducted three experiments (in 2013, 2014 and 2015) that study the behavior in One vs Two. We report the two earlier studies in Appendix A.1.4 We concentrate here on the …nal experiment which included two experimental designs. Subjects were presented with two (ambiguous) urns and were told that each contained two balls, each ball being either red or black, and also a third (risky) urn that contained one red and one black ball.5 4 In 2014 we included important control treatments that tested alternative explanations for failure of probabilistic sophistication in One vs Two. 5 The language in the experiment used jars and marbles that were blue or green. 7 3.1 Pairwise choices In the main experimental design, which was also employed in the earlier experiments, subjects were presented with versions of the following six choice problems. Red vs Di¤ erent: Choose between a bet that pays if the color of the ball drawn from a single urn is red and a bet that pays if the balls drawn from the two urns have di¤erent colors. Black vs Di¤ erent: Choose between a bet that pays if the color of the ball drawn from a single urn is black and a bet that pays if the balls drawn from the two urns have di¤erent colors. Black vs Same: Choose between a bet that pays if the color of the ball drawn from a single urn is black and a bet that pays if the balls drawn from the two urns have the same color. Red vs Same: Choose between a bet that pays if the color of the ball drawn from a single urn is red and a bet that pays if the balls drawn from the two urns have the same color. Standard Ellsberg Red: Choose between a bet that pays if the color of the ball drawn from one of the two ambiguous urns is red and a bet that the color of a ball drawn from a third urn containing one red ball and one black ball (a risky urn as in Ellsberg) is red. Standard Ellsberg Black: Choose between a bet that pays if the color of the ball drawn from one of the two ambiguous urns is black and a bet that the color of a ball drawn from a third urn containing one red ball and one black ball (a risky urn as in Ellsberg) is black. Note that the …rst four choice problems included two variations of (2.1), where each pair of problems (1/3 and 2/4) allowed the experimenter to detect violation of probabilistic sophistication. The choice problems above were organized in triplets, which allowed us to infer strict and weak rankings from choices by slightly varying the prizes, assuming monotone and transitive preferences. For example, problem 3’ asked the subject to choose between a bet paying $25 if the ball drawn from urn 1 is black and a bet paying $25 if the balls drawn from the two 8 urns are of the same color. Choice made in this problem reveals only weak preference. Problem 3 (3”) was similar except that the winning prize on the latter (former) bet was increased to $26. The choice to bet on the single urn in problem 3 implies strict preference when the two prizes are equal; and symmetrically if one chooses in problem 3”to bet on the two colors being the same. Choice of the two bets (in both 3 and 3”) that pay $25 is inconsistent with monotone and transitive preference. Choice of the two bets that pay $26, together with either choice in 3’reveals only weak preference between the bets.6 The rationale behind this design was explained to subjects before they answered any questions. In the same way, the experimental design can reveal strict and weak ambiguity attitudes through versions of the two standard Ellsberg choice problems. Note that a subject who weakly prefers bets on the color of the ball drawn from the risky urn to the corresponding bets on the color of the ball drawn from an ambiguous urn does not necessarily violate probabilistic sophistication. However, if at least one of the above preferences is strict, we can infer that she must be strictly ambiguity averse and not probabilistically sophisticated. This approach yields lower and upper bounds on the frequency of ambiguity attitude. Finally, a similar procedure is applied to versions of the …rst four problems above (concerning One vs Two). Here also the distinction between weak and strict preference is important. For example, a subject who weakly prefers R1 to Dif f and B1 to Same is only weakly more averse to ambiguity about correlation than towards bias and may be probabilistically sophisticated. However, if she strictly prefers either R1 or B1 , then One Two and there is necessarily a violation of probabilistic sophistication. In a symmetric manner we de…ne a subject who is strictly (weakly) more averse to ambiguity about bias than about correlation, and denote this preference by Two ( )One. It may be helpful to “behaviorally”classify subjects into those who always choose the higher prize in every pair of problems in which the prizes are di¤erent, and those who sometimes choose the bet with the lower prize. The …rst group never exhibits strict preference, and therefore is consistent with probabilistic sophistication. Subjects in the second group may exhibit 6 It is important to remember that the choice of the two bets paying $26 in the problems in which prizes di¤er is consistent with indi¤erence. However, it could be that lowering the higher prize to $25 + " (for example, $25:1) in these problems might cause a subject to choose the same bet in all three comparisons, thus revealing strict preference when prizes are equal. 9 behavior that is inconsistent with probabilistic sophistication (Ellsbergian ambiguity aversion or seeking, One Two or Two One), or make choices that are consistent with “non-symmetric” probabilistic beliefs (for example, R1 R3 and B3 B1 in the Ellsberg case, and Dif f R1 and B1 Same in One vs Two). This design of choice problems, which we have not seen used previously, allows us to identify a strict ordinal ranking without using a cardinal valuation. Methods based on elicitation of cardinal valuations of bets (“matching tasks”), such as Becker-DeGroot-Marschak (1964) used in Halevy (2007) or a discrete version using a choice list used in Abdellaoui et al (2011), can obviously identify strict ranking. The problem is that often the response modes (matching vs. choice tasks) disagree. One may be worried, however, that our new design increases the complexity of the choice problems subjects are required to consider. To partially answer this concern, we included in the experiment a choice list design such as has been used in many experimental studies. 3.2 Choice lists In addition to the main pairwise choice design, we included in 2015 a standard choice list design. The latter elicits an approximation to the certainty equivalents of the six bets (each with winning and losing prizes $25 and $0 respectively): Red, Black, Same, Di¤erent, Risky (50%) Red, Risky (50%) Black. In each choice list the subject was asked to make 20 pairwise choices between the bet and a sure amount, which varied in increments of at least $1 from $0 to $25. The …rst comparison in which a subject chooses the sure amount approximates the subject’s certainty equivalent, and can serve as a cardinal index for utility. Thus, the comparison made in One vs Two and in the Ellsberg standard problem may be achieved by comparing the switching points of the di¤erent bets. In order to mimic the strict preference elicitation achieved in the pairwise choice design, we elicited strict preference between two bets only if the di¤erence between their switching points is at least two lines.7 7 A common practice is to use a single line di¤erence as indicating strict preference. However, it could be that a subject is indi¤erent between two bets and a sure amount that appears on the choice list, and when forced to choose breaks the indi¤erence di¤erently in the two lists. The use of two line di¤erences makes this scenario impossible (a similar approach was taken in Halevy (2015)). 10 3.3 Incentives and other details Payment was determined by randomly selecting a single choice problem for payment before subjects made their choices, by distributing (before the actual experiment started) sealed envelopes containing a number of a randomly selected choice problem (and a randomly selected line in the case of choice lists) among the participants in the experiment. The envelopes were opened only after subjects made all their choices and the balls were drawn from the urns. This version of the Random Incentive System (RIS) is theoretically incentive compatible in eliciting ambiguity attitudes (Baillon, Halevy and Li 2014), because the order suggests that choices are between lotteries over Savage acts wherein ambiguity cannot be hedged. We do not …nd evidence that subjects used the RIS to hedge the di¤erent sources of ambiguity. See Appendix A.2.4 for further discussion and examples, and for detailed evidence. Subjects were recruited from UBC’s Vancouver School of Economics subject pool using ORSEE (Greiner 2015) to an experiment that promised participants a chance to earn up to $36 during a one hour experiment in decision making (including a show-up fee of $10). After consent forms were signed, the instructions were read aloud. In order to eliminate a potential suspicion that the experimenter could manipulate the composition of the urns, each subject was asked at the beginning of the experiment to choose the urn (1 or 2) on which to bet in all those bets that involve a single urn. To simplify exposition of the results, we proceed as though all individuals chose urn 1. Thus, for example, the bet on red from the single urn is represented by R1 . Throughout the experiment the language used in the description of the di¤erent bets was completely symmetric, enumerating the states in which each bet pays. Complete instructions may be found in Appendix B.2. 4 Experimental Results This section describes the main results of the experiments described above (and conducted in 2015). The results of previous experiments (conducted in 2014 and 2013), which had higher stakes and included additional treatments that support our interpretation of the behavior, are included in Appendix A.2. The latter also contains some additional information regarding the 11 2015 results. Out of 153 subjects who took part in the pairwise choice experiment, 19 made choices inconsistent with monotone and transitive preferences in at least one (out of six) triplet of choices. An additional 5 subjects made intransitive choices between triplets of choices problems.8 The results reported below are restricted to the remaining 129, although adding (to the extent possible9 ) some of these subjects does not substantially change any of the …ndings. The average payment was $17.01 with a standard deviation of $12.79. The choice list treatment included 77 subjects, 3 of whom had multiple switching lines or did not respond in some of the decision problems. The reported results are restricted to the remaining 74 subjects. The average payment was $19.62 (including a show-up payment of $5) with a standard deviation of $9.20. There was no overlap between the subjects who completed the pairwise choice experiment and those who completed the choice list experiment. Note that there are many ways in which “random”choices in the pairwise choice design would lead to cyclical rankings (since subjects made choices in two di¤erent variations of (2.1)). We …nd no empirical evidence of such behavior.10 Similarly, the low frequency of multiple switching in the choice list design (only 3 subjects), and the symmetry in the valuations of di¤erent colors (only 10 subjects had color valuations that vary in more than $1) lead us to conclude that choices were deliberate. 8 For example, a subject may exhibit R1 Dif f , Dif f B1 , B1 Same, and Same R1 . 9 13 subjects made choices inconsistent with monotone or transitive preferences in two or more triplets. For 8, we could not impute any ranking since there were too many missing data; 3 were Ellsbergian ambiguity averse but there was no conclusive evidence for strict preference between bets on One and Two. Another 6 subjects made choices inconsistent with monotone or transitive preferences in only a single triplet. Three of them were Ellsbergian ambiguity averse, and two of the three made choices consistent with One Two (the other 3 made choices consistent with probabilistic beliefs in both domains). Five other subjects made choices inconsistent with transitive preferences across triplets. None of them was Ellsbergian ambiguity averse, but 3 violated probabilistic choices in at least one pair of triplets. 10 The code checks for all such cycles and is available from the authors. 12 Ellsbergian Ambiguity Attitude Pairwise Choice # % strictly averse (nonPS) 60 46.5 strictly seeking (nonPS) 3 2.3 weakly averse 37 28.7 weakly seeking 14 10.9 neutral – – non-symmetric beliefs (PS) 15 11.6 Total 129 100 Choice List # % 30 40.5 10 13.5 5 6.8 5 6.8 20 27.0 4 5.4 74 100 Table 4.1: Ellsbergian ambiguity attitude 4.1 Ellsbergian ambiguity Table 4.1 presents the attitude towards Ellsbergian ambiguity as derived from choices between bets on the color of a ball drawn from the ambiguous urn and bets on the color of the ball drawn from the risky urn. In the pairwise choice, 63 out of 129 subjects (48.8%) exhibit strict ambiguity attitude and hence are not probabilistically sophisticated (nonPS); all but 3 are ambiguity averse. The corresponding numbers in the choice list elicitation are 40 out of 74 (54%), and 75% of them are strictly ambiguity averse. The choices made by the remaining subjects are consistent with probabilistic sophistication (PS). It is important to remember that in the pairwise choice elicitation method we cannot elicit indi¤erence, but only weak preference. It is possible that some of the 51 subjects who revealed weak ambiguity preference, (those who chose the risky bet when prizes are equal and otherwise chose the bet with the higher prize), are indi¤erent between bets on risky and ambiguous events, and that for others the premium we used ($1) was too large to reveal strict preference. In particular, it is possible that these 51 subjects all violate probabilistic sophistication.11 The categories labeled as “weak preference”in the choice list correspond to a di¤erence of $1 in the switching lines (as noted in Section 3.2), while ambiguity neutrality 11 The fact that 72.5% of the subjects who exhibited weak preference chose the risky bets when the prizes were equal is consistent with the hypothesis that the proportion of ambiguity averse subjects among them is higher than the proportion of ambiguity seeking subjects. 13 One vs Two Pairwise Choice # % One Two (nonPS) 25 19.4 Two One (nonPS) 10 7.7 One Two 30 23.3 Two One 15 11.6 One Two 15 11.6 non-symmetric beliefs (PS) 34 26.4 Total 129 100 Choice List # % 25 33.8 18 24.3 5 6.8 1 1.3 17 23.0 8 10.8 74 100 Table 4.2: One vs Two corresponds to exactly the same switching line. The …nal category in Table 4.1 (non-symmetric beliefs) includes subjects who, when comparing bets on one color preferred the ambiguous urn, and when comparing bets on the other color preferred the risky urn.12 Seven subjects made choices consistent with strictly non-symmetric beliefs and another eight subjects made choices consistent with weakly non-symmetric beliefs. The frequency of this behavior in the choice list design is much lower (only 4 out of 74 subjects). Comparing the two elicitation methods, they detect ambiguity aversion and neutrality at roughly similar rates, pairwise choice classi…es proportionally more subjects as having non-symmetric beliefs over the composition of the ambiguous urn, and the choice list design …nds relatively more evidence for ambiguity seeking. 4.2 One vs Two Table 4.2 presents the attitude in One vs Two as derived from choices between bets on the color of a ball drawn from a single ambiguous urn and bets on the colors of the balls drawn from the two ambiguous urns. In the pairwise choice design, 35 subjects (out of 129) made choices in One vs Two that are inconsistent with probabilistic sophistication, 71% of them strictly preferring a bet on the bias to a bet on the relation between 12 Note that these choices must be consistent with choices made in the …rst four decision problems that study rankings in One vs Two. 14 the urns. Another 45 subjects made choices that reveal weak preference to bet on bias or on correlation (two thirds of them weakly preferred bets on bias). As discussed above, some of them may hold probabilistic beliefs, but for others the $1 di¤erence in prizes may have been too high to reveal strict preference. Altogether, between 35 and 80 subjects (out of 129) made choices consistent with violation of probabilistic sophistication in One vs Two. In the choice list elicitation, probabilistic sophistication was contradicted by 43 subjects (out of 74), of whom 58% strictly preferred betting on bias rather than on correlation between the urns. An additional 6 subjects showed weak preference ($1 di¤erence in the switching points), and may have shown strict preference if the di¤erence between lines in the choice list were smaller. The category One Two in Table 4.2 includes subjects who did not exhibit any strict preferences between the 4 ambiguous bets and who were not included in the two previous lines. This includes 15 subjects in the pairwise choice and 17 subjects in the choice list. The last category in Table 4.2 includes subjects who made choices consistent with probabilistic beliefs that assign a strictly higher likelihood that the urns are positively rather than negatively correlated (or vice versa); for example, Same R1 and B1 Dif f (where at least one ranking is strict). This pattern of choices is quite common in the pairwise choice design as 34 subjects (26.4% of 129) are classi…ed in this way. It is not as frequent in the choice list treatment (only 10 subjects, 10.8% of 74). Comparing the two elicitation methods, some di¤erences emerge. First, the proportion of subjects that are not probabilistically sophisticated is much higher in the list elicitation than in the pairwise choice (58.1% vs. 27.1%). However, the di¤erence is much smaller when considering also subjects that exhibited weak preference in One vs Two. Even among the probabilistically sophisticated subjects there are di¤erences between the two elicitation methods. The pairwise choice treatment yields relatively more subjects exhibiting non-symmetric attitude between Same and Dif f (26.4% vs. 10.8%), and the choice list treatment identi…es more subjects as being indi¤erent between all ambiguous bets (23% vs 11.6%). We …nd these di¤erences, like the di¤erences in the elicitation of Ellsbergian ambiguity attitude, intriguing and deserving independent experimental investigation. We conjecture that some of the di¤erences can be attributed to the fact the choice list compares (approximation to) certainty equivalents (which integrates payments and beliefs), while in pairwise choice no such integration is necessary. It could also be that the relatively large “tick”of $1 contributed to these di¤erences. Fi15 nally, it is possible that the pairwise choice method we have proposed here is cognitively more taxing relative to the choice list method, leading to more noise in measurement of underlying preferences.13 4.3 Comparison In the pairwise choice design, the proportion of subjects whose choices are consistent with probabilistic sophistication is higher in One vs Two (72.9%) than in the standard Ellsberg problem (51.2%). This di¤erence may not be surprising, in hindsight, because, as we suggest next, there is more scope for probabilistically sophisticated behavior in One vs Two than in Ellsberg, even for a subject who dislikes (or alternatively likes) ambiguity. In the choice list design, however, we did not observe such a di¤erence in the relative frequency of probabilistic sophistication: choices consistent with probabilistic sophistication were made by 34 subjects in the Ellsberg problems and by 31 subjects in One vs Two. In the classic Ellsberg problem, the choice of the risky urn leaves the subject with a purely risky bet in which the probability of winning is 50%. Thus, the preference to bet on the risky urn, and hence violation of probabilistic sophistication, arises given only aversion to uncertainty about the bias of the ambiguous urn. In contrast, in the choice problem One vs Two all alternatives (the bets R1 , B1 , Same and Dif f ) are ambiguous. Accordingly, as explained when discussing our thought experiment, choices depend on which is more important–ambiguity about bias or ambiguity about how urns di¤er or are related. If these opposing motives exactly o¤set one another, then lack of strict preference between R1 ; B1 ; Same and Dif f (classi…ed under One Two, Two One, or One Two in Table 4.2) could result. This rationale includes 20 subjects in the pairwise choice design and 8 in the choice list design whose choices are not probabilistically sophisticated in the Ellsberg problems (total of 63 and 40 in the pairwise choice and choice list designs) but do not exhibit strict preference in One vs Two. Hence, this behavior is consistent with all bets being perceived equally ambiguous. Our categorization of this behavior as being probabilistically sophisticated in Table 4.2 re‡ects the conservative (and demanding) approach we are adopting throughout to identifying empirical support for the e¤ects of ambiguity about correlation. 13 Noise should be distinguished from random choices, as the latter will lead to cyclical choice patterns - that we do not observe here. 16 There is another possibility consistent with probabilistic sophistication in One vs Two, namely the rankings Dif f R1 and B1 Same, which is a variant of (2.2). Such rankings can arise from a strong belief about how the urns were constructed. As an extreme example, suppose that the subject’s hypothesis is that the experimenter drew two balls without replacement from an auxiliary urn containing one red ball and one black ball, and if a red (black) ball was drawn …rst, then urn 1 was …lled with 2 red (black) balls; the composition of urn 2 was determined in a similar fashion. Then it is certain that the balls drawn from urns 1 and 2 have di¤erent colors and thus Dif f pays x (25 or 26) and Same pays 0, each with certainty, so that R1 is ranked between them. The latter might arise also more generally from the feeling that “there are only so many red balls to go around,”say because the urns are thought to have been constructed by drawing without replacement from an auxiliary urn containing n 2 balls of each color. The description of urns given to the subjects does not suggest this perception but there is no reason to rule it out. Probabilistic sophistication is consistent also with the rankings (2.2) as indicated at the end of Section 2. This might arise if urns are perceived to have a common component, so that a red draw from urn 1 indicates that a red draw is more likely also from urn 2; for example, if the preceding construction is modi…ed so that draws from the auxiliary urn are made with replacement. Out of the subjects who are not probabilistic sophisticated in the Ellsberg problem (63 and 40 in the pairwise choice and choice list elicitations, respectively) there are many subjects who exhibit this pattern (23 and 4 in the two methods, respectively). The substantial di¤erence in the patterns across the methods suggests that at least part of the interpretations above depend on the speci…c incentive system used to elicit rankings. In a similar vein, not all subjects who violate probabilistic sophistication in One vs Two, violate it also in the Ellsberg problem. Out of 35 and 43 in the pairwise choice and choice list methods, respectively, only 20 and 25 violate probabilistic sophistication in the Ellsberg problem. For example, some subjects are ambiguity neutral or even ambiguity seeking when betting on bias, but are averse to the ambiguity concerning the correlation between the urns. For the reasons given above, we do not expect behavior in One vs Two to mirror behavior in the Ellsberg problems. Indeed, although the association between probabilistic sophistication in the two problems is signi…cant in the choice list elicitation (Fisher exact test p-value 3.4%), the association is not 17 signi…cant when measured in the pairwise choice design (Fisher exact test pvalue > 10%).14 When partitioning the set of probabilistically sophisticated subjects in One vs Two into those who expressed or did not express strict preference, we …nd a signi…cant association between the two domains (Fisher exact test p-value 0.3%): subjects who did not express strict preference in one domain (Ellsberg / One vs Two) tended not to express strict preference in the other domain as well.15 5 Models Consider how the behavior exhibited in the experiment can be modeled. As shown earlier, both One Two and Two One contradict probabilistic sophistication, and hence, in particular, subjective expected utility theory. Thus we are led to consider generalizations of the latter that were developed in response to Ellsberg’s experiments; the maxmin expected utility model (Gilboa and Schmeidler 1989) is one prominent example, and others include Choquet expected utility (Schmeidler 1989), the smooth model (Nau 2006; Klibano¤, Marinacci and Mukerji 2005; Seo 2009), and variational utility (Maccheroni, Marinacci and Rustichini 2006). All of these models have many free parameters and thus can accommodate a broad range of ambiguity-sensitive behaviors. For an extremely simple example, both Ellsbergian ambiguity aversion and One Two are implied by a Choquet expected utility preference if we specify the representing capacity (or non-additive probability measure) so that (R1 ) = (B1 ) = :3 and (Same) = (Dif f ) = :2; (5.1) and Two One is accommodated instead if :2 is replaced by :4. Similarly the new behavior identi…ed here does not pose a challenge for the other models. However, our objective is not merely to model the behavior observed in the idealized laboratory setting with Ellsberg urns. We wish also to determine what the experimental results suggest about modeling behavior outside the laboratory when uncertainty is due to multiple random events (as we have seen, portfolio choice is one example; statistical decision problems are typically of this form; see Section 6 for more). These applications suggest a 14 This is due to the fact that many subjects who are Ellsbergian ambiguity averse make choices in One vs Two that may be rationalized by the rankings (2.2). 15 See further discussion in Appendix A.2.3. 18 Cartesian product state space and the desirability of preference models that are designed to address and exploit this special structure. Rationalization through (5.1), or similar exercises, do not satisfy the latter requirement. The Bayesian benchmark is the special case of subjective expected utility, due to de Finetti (1937), where the Savage predictive prior on the state space has the well-known “conditionally i.i.d.”form.16 With the preceding motivation, we consider two classes of models in this section. The …rst builds on de Finetti by adding a role for ambiguity. For the second, we adapt an approach to modeling Ellsberg-style behavior that centers on multiple “sources” (Tversky and Fox 1995; Tversky and Wakker 1995). This model captures naturally and simply a main point of the paper, namely the distinction between three kinds (or sources) of uncertainty: risk, bias and correlation. 5.1 Conditionally i.i.d. models The four possible outcomes of the two draws lie in the state space S1 S2 , where, for i = 1; 2, Si = fRi ; Bi g. For any subset A of outcomes, A denotes also the corresponding bet with prizes x and 0. According to the maxmin model, the utility of the bet on A is given by U (A) = min P (A) , P 2P (5.2) where P is a set of probability measures on S1 S2 , the set of predictive priors.17 (The model includes also a vNM utility index u, which here we have normalized to satisfy u (x) = 1 and u (0) = 0.) More generally, consider all utility functions over bets of the form U (A) = W (P (A))P 2P , (5.3) where P is a set of predictive priors and where W is weakly increasing in the sense that U (A0 ) U (A) if P (A0 ) 16 P (A) for all P 2 P. See Kreps (1988, Ch. 11) for a description and for a discussion of its importance as a normative guide to decision-making. Note that i.i.d. is an abbreviation for “identically and independently distributed.” 17 We follow common terminology in that “predictive prior” refers to beliefs about the payo¤ relevant state space and “prior” refers to beliefs about unknown probability laws or “parameters.” 19 We consider alternative speci…cations for P and W . In the benchmark Bayesian model, each urn is parametrized by a common unknown parameter p, 0 p 1, representing the proportion of red balls in the urn. In addition, the two draws are taken to be i.i.d. conditional on the true but unknown p, and uncertainty about p is modeled by a (single) prior over its possible values. Because beliefs are probabilistic, this model cannot accommodate the preference for One over T wo (or the reverse) – as noted, one needs to permit a role for ambiguity. The simplest way to do so is to generalize the preceding by positing a nonsingleton set M of priors about the value of p. Each prior in M induces a predictive prior P in the familiar way described by Z P = (p p)d (p) , (5.4) where p p denotes the i.i.d. product of the measure on fR; Bg that assigns the probability p to the outcome R. By varying over all priors in M, one obtains the set P exch of predictive priors. Using this set of predictive priors in (5.3), without any further restrictions on W , constitutes a seemingly natural generalization of de Finetti’s model. (Models of this form are studied in Epstein and Seo (2010, Model 1), Al Najjar and De Castro (2014), CerreiaVioglio et al (2013) and Klibano¤, Mukerji and Seo (2014)). However, this generalized conditionally i.i.d. model cannot simultaneously accommodate both One Two and Ellsbergian (strict) ambiguity aversion: 1 for all p in [0; 1], infer that for every P from the fact that p2 + (1 p)2 2 of the form (5.4), Z P (Same) = p2 + (1 p)2 d (p) 12 . It follows, given only that W is monotonic, that Same is weakly preferred to betting on red in a 50-50 urn. Adding Ellsbergian ambiguity aversion yields Same R1 , contrary to One Two. The intuitive reason for this limitation is clear: though there is ambiguity about p, the fact that every predictive prior in P exch has the form in (5.4) expresses certainty that draws are conditionally i.i.d., which is contrary to the rationale for preferring to bet on the single urn. This intuition suggests, and it is readily con…rmed,18 that the conditionally i.i.d. model can accommodate 18 q Let 0 < q < 1=2 be …xed and let P consist of the two i.i.d. predictive priors q and (1 q) (1 q). Let W be as in maxmin. Then U (R1 ) = U (B1 ) = q, 20 both Two One and Ellsbergian ambiguity aversion.19 The reason is that Two One indicates the predominance of ambiguity about the composition of each urn and this can be captured adequately by multiple priors about p. 5.2 A source-based model We consider next a model that relaxes the restriction that the set P contains only conditionally i.i.d. measures, while retaining structure that re‡ects the distinction between risk, uncertain bias and uncertain correlation. A streamlined version of the model is described diagrammatically in Figure 5.1. (A more general and precise speci…cation is provided in Appendix A.3.) Notationally, the composition of a single urn is described by a probability vector of the form (p; 1 p), where p denotes the proportion of red, and the joint composition of the two urns is described by a probability vector of the form (pRB ; pBR ; pRR ; pBB ). If urns 1 and 2 are described by (p; 1 p) and (q; 1 q) respectively, then (p; 1 p) (q; 1 q) denotes the joint distribution given by the product measure, (p; 1 p) (q; 1 q) (p(1 q); q (1 p) ; pq; (1 p) (1 q)) . Figure 5.1: Hierarchical beliefs on the two ambiguous urns U (Same) = q 2 + (1 q)2 > 1=2 and U (Dif f ) = 2q (1 the desired rankings. 19 q) > q, which implies all Of the 60 (30) subjects in the pairwise choice (choice list) design who were strictly ambiguity averse in the Ellsberg sense, 12 (9) exhibited One Two and 8 (13) exhibited Two One. 21 Issue 1 Uncertainty about the relation between urns takes the form of two alternative hypotheses. One possibility entertained by the decision-maker is that the urns are i.i.d. according to (p; 1 p) for some (unspeci…ed) p 2 0; 12 ; 1 , that is, the two urns can be described by a measure in the set P ident in the …gure. The alternative hypothesis is that the urns are “complementary” in the sense that their joint distribution lies in the set P compl in the …gure. This hypothesis is justi…ed, for example, by the following perception of how the urns are constructed: there is a set of two red and two black balls–two are drawn without replacement to …ll urn 1 and the remaining two are put into urn 2. The two hypotheses are assigned subjective probabilities and 1 respectively. Issue 2 Conditioning on either of the above hypotheses, the composition, or bias, of each urn is uncertain. The decision-maker assigns (conditional) probability 13 to each possible value of p. Issue 3 After conditioning on both the relation between urns and on the bias, there remains uncertainty about the colors of the two drawn balls. However, resolution of Issues 1 and 2 implies a unique probability distribution over fR1 B2 ; B1 R2 ; R1 R2 ; B1 B2 g. Thus the last issue concerns risk. Each bet (or act) f associates a (dollar) payo¤ to each terminal node, depending on the colors of the two balls drawn. Thus it induces a 3-stage lottery, denoted Df , which is evaluated recursively along the lines of Kreps and Porteus (1978): an expected utility function is used at each stage, with di¤erent utility indices for di¤erent issues, and compound lotteries are evaluated recursively.20 The utility of a bet on Ellsberg’s risky urn is computed by identifying the (single-stage) lottery induced by the bet with a three-stage lottery that is resolved completely at the third stage. Denote the utility indices by u3 (used to evaluate risk or one-stage lotteries), u2 (used to address uncertainty due to the unknown bias), and u1 (used to address the …rst issue). For simplicity, take u3 to be linear. The model is a special case of (5.3), where the set P is equal to the union of the sets P ident and P compl de…ned in the …gure; hence, P contains measures (all those in P compl ) that do not conform to (5.4). The recursive structure of utility de…nes the corresponding function W . 20 For more on such recursive models see, for example, Segal (1987, 1990), and Ergin and Gul (2009). Segal advocates using non-expected utility functions at each stage. 22 Turn to behavior. Because the bets of interest depend on di¤erent issues, the model’s predictions depend largely on the curvatures (both absolute and relative) of u1 and u2 . In particular, Ellsbergian ambiguity aversion (a¢ nity) is implied if and only if u2 is concave (convex). In comparing One vs Two, the bets R1 and B1 are subject to uncertainty about the bias, but their payo¤s do not depend on the …rst issue because they concern only a single urn. In contrast, the payo¤s to both Same and Dif f depend crucially on the relation between urns. This suggests that the choices in One vs T wo depend on both the relative curvatures of u1 and u2 and on the magnitude of . If we take = 21 , then One ( )Two is implied if u1 is more (less) concave than u2 . Consequently, the model can handle all preference patterns observed in our experiments by allowing su¢ cient heterogeneity in the curvatures of subjects’utility indices and in beliefs over the relation between the urns (see Appendix A.3). 6 Concluding Discussion The literature stimulated by Ellsberg, particularly the experimental literature, has focused on the two-fold distinction between risk and ambiguity, or “Knightian uncertainty.” In Ellsberg’s two-urn experiment, the latter is embodied in the uncertain bias (or composition) of the unknown urn. In a setting with multiple ambiguous urns, we have introduced a second source of ambiguity–the relation between urns. Thus we have studied the three-fold distinction between risk, bias and the relation between biases. The results of our laboratory experiments provide support for the empirical relevance of the new dimension, and its relation to Knightian uncertainty. To conclude, we describe concrete instances of decision-making where ambiguity about correlation is potentially relevant, and then we consider some related literature. 6.1 More on economic signi…cance As noted in the introduction, betting on the draws from a sequence of urns is intended as a canonical example of choice problems where payo¤s to an action depend on multiple random events (or variables). Portfolio choice is one such problem as discussed in the introduction. Here we describe other instances where ambiguity about correlation is plausibly a concern for a decision-maker. 23 Suppose, for example, that the outcome si of the i-th random event is given by an equation of the form si = xi + i , i = 1; 2; :::; I. (6.1) There may be di¤erences between the underlying mechanisms–these are captured by the vectors xi which describe the observable heterogeneity (in the urns context, there is no observable heterogeneity). The decision-maker chooses between bets on the outcomes of the I random variables. Her choices depend on her model of the residuals or unobserved heterogeneity i , which are the source of the uncertainty she faces. If all sources of heterogeneity of which she is aware are included in the xi s, then it is natural that she be indi¤erent between any two bets on the realization of residuals that di¤er only in a reordering of the random variables. However, the individual may not be con…dent that the xi s describe all relevant di¤erences, in which case she may not be certain that residuals are identical, or that they are related in any particular way. Though she may not be able to describe further forms of heterogeneity, she may be worried that there are gaps in her understanding that could be important and thus she may wish to take into account their possible existence when making choices. A number of studies have argued for the importance of such a lack of con…dence. In all the examples below, the decision-maker should be thought of as a policy maker, where policies can be identi…ed with (Savage-style) acts that pay o¤ according to the outcomes of the I random variables. In the context of the cross-country growth literature where each random variable corresponds to the growth rate for one of I countries, Brock and Durlauf (2001) point to the open-endedness of growth theories as a reason for skepticism that all possible di¤erences between countries can be accounted for (p. 231), and they emphasize the importance of “heterogeneity uncertainty.” King (2001) makes a similar critique in an international relations context where each random variable corresponds to a pair of countries and its outcome is con‡ict or lack of con‡ict; he refers to “unmeasured heterogeneity.”21 This paper complements these critiques by translating them into behavioral terms and thus giving more precise meaning to a concern with “heterogeneity uncertainty”or “unmeasured heterogeneity.” 21 He writes (p. 498) that “in international con‡ict data are neither powerful nor even adequate summaries of our qualitative knowledge”, so that the common assumption of exchangeability is usually violated. 24 The applied IO literature provides an example of a di¤erent sort.22 Here there is a cross-section of markets in each of which an entry game is played. Thus each random variable corresponds to one of I markets and its outcome is the number and identity of entrants in a pure strategy Nash equilibrium. The di¢ culty faced by the policy maker is that there may be multiple equilibria and she has little understanding of how equilibria are selected, and accordingly how selection mechanisms may di¤er or be related across markets. A fourth example arises in repeated English auctions when, as in Haile and Tamer (2003), because of the free-form nature of most English auctions in practice, one makes weak assumptions about bidders’ behavior. Then equilibrium behavior in each auction is multiple-valued and can be narrowed down and related across auctions only via heroic assumptions. This has implications for an auctioneer who is choosing reserve prices (Aryal and Kim 2013). Though our laboratory experiment does not investigate behavior in the above speci…c settings, the results lend support to the hypothesis that decision-makers may care about poorly understood di¤erences across markets or auctions. 6.2 Related literature We have already contrasted Ellsberg’s classic two-urn experiment with ours. Eliaz and Ortoleva (2016) and Eichberger et al (2015) modify Ellsberg’s experiment in order to study di¤erent dimensions of ambiguity; in addition to ambiguity about the probability of winning, both papers consider also ambiguity about the prize to be won, and the former considers in addition ambiguity about the time delay for receipt of the prize. Their experiments have in common with ours the investigation of multiple dimensions of ambiguity and the association between (ambiguity averse) behaviors in di¤erent dimensions, but the “dimensions” studied are di¤erent; this leads also to completely di¤erent experimental designs. Multiple urns are used in experiments exploring learning and dynamic consistency. These issues are not involved in our study because we consider only ex ante choice. Epstein and Seo (2010, Model 2) and (2015) present axiomatic models that generalize de Finetti’s Bayesian model in a way di¤erent from the 22 We are referring to the literature on entry games and partial identi…cation (see Tamer 2010, and the references therein). For an explicit choice-theoretic perspective, see Epstein and Seo (2015). 25 generalizations described in Section 5. Their capacity to accommodate the behavior studied here is intermediate between the capacities of the two models examined above; more detail would take us too far a…eld. We add only that though they are axiomatic, their axioms are not nearly as simple and transparent as the choice between betting on one urn or on two. Another important di¤erence is that the axioms are formulated in terms of horserace/roulette wheel (Anscombe-Aumann) acts which are less natural objects of choice than are the Savage-style acts considered here.23 Correspondingly, while the former acts are prominent in axiomatic work and are used in some laboratory experiments, we are not aware of any applied studies where they have been used to model behavior in the …eld or to explain …eld data–objects of choice are universally formalized as lotteries or as Savage acts. This disconnect in the literature between Anscombe-Aumann acts and descriptive modeling in the …eld suggests (to us) that tests of preference models that refer only to Savage-style acts are more relevant to the potential usefulness of these models outside the laboratory.24 Finally, the role played in decision-making by the individual’s perception of correlation is the focus also in a literature on “correlation neglect,”including, for example, experimental studies (Eyster and Weizsacker 2010, Enke and Zimmerman 2013), applications (De Marzo et al 2003, Levy and Razin 2015), and decision-theoretic foundations (Ellis and Piccione, forthcoming). These papers concern environments where there is a “true” probability law describing the uncertainty (that is, the law observed by the modeler), and where the decision-maker is Bayesian but where she has wrong beliefs. Because of limited attention or other cognitive constraints, she misperceives the connections between key random variables (for example, between returns to di¤erent securities in …nancial decision-making). A common hypothesis is that she wrongly treats key correlated variables as independent. There is no sense in these models that awareness of the complexity of her environment and self-awareness of her cognitive limitations lead the decision-maker to doubt that her wrong beliefs are correct. Such doubts are what we are try23 Savage-style acts can be thought of as consisting of the horse-race alone followed by monetary prizes as opposed to the spin of a roulette-wheel; see Kreps (1988), particularly Chs. 4 and 7. 24 In discussing if/how the use of axioms involving Anscombe-Aumann acts makes sense, Kreps (p. 101) writes “In descriptive applications, axioms are supposed to concern behavior that is observable, so what sense does it make to pose axioms about preferences/choices that are never observed ...?” 26 ing to detect here, through behavior (such as One Two or Two One) that contradicts probabilistic sophistication.25 Note that the Eyster-Weizsacker and Enke-Zimmerman experiments are not designed to test for violations of probabilistic sophistication. In a very recent paper, Levy and Razin (2017) model agents who are selfaware in the above sense and accordingly are averse to ambiguity about the correlation between di¤erent sources of information. Correlation aversion is modeled with a speci…c single-parameter (“correlation capacity”) functional form which the authors apply to models of …nancial investments and CDO ratings. Thus preferences are assumed known to the modeler and some e¤ects of changes in preferences are explored in speci…c settings. We see such work as complementary to ours, which takes behavior alone to be observable and asks “which behavior would reveal a concern with ambiguous correlation?” 25 We emphasize that violation of probabilistic sophistication rules out beliefs that can be represented by any probability measure, including one expressing stochastic independence or being “wrong” in any other way. 27