A Study Eclipsed by Confirmation Bias

1 MIT Student Professors David Jones and David Kaiser STS.003: The Rise of Modern Science Final Paper, Option A: 8 December 2010 A Study Eclipsed by Confirmation Bias How does one become an overnight celebrity? It takes a theory of gravitation rivaling Newton’s and a solar eclipse. In 1919, when Sir Arthur Eddington reported that starlight had been deflected by the Sun’s gravity by an amount predicted by Albert Einstein’s general relativity after expeditions to Sobral and Principe, major international newspapers and scientific societies (The New York Times and the Royal Society, among others) congratulated Einstein. 1 But Eddington’s methods were anything but a cause for celebration. Since he let his personal and political bias influence his work and took advantage of his experiment’s interpretability and his status in the scientific community, I argue that his case was one of scientific misconduct involving suppression of data and ipsedixitism. Eddington’s motivations to undertake the experiment were not hidden; he was already a supporter of Einstein’s general theory of relativity. Eddington, a British astrophysicist and pacifist, performed this experiment wanting a victory for Einstein, a German physicist and pacifist, because he believed it would signify post-World War I political reconciliation between the UK and Germany. 2 The rift between Allied and German scholars began when well-known German scientists like Max Planck signed the 1 Alan Lightman, “Relativity and the cosmos,” Nova (Jun 2005), available at http://pbs.org/wgbh/nova/einstein/relativity (accessed 8 Dec 2010). 2 Daniel Kennefick, “Testing relativity from the 1919 eclipse—A question of bias,” Physics Today 62 (March 2009): 37–42, on 37. 2 Manifesto of German Professors and Men of Science, “defending German war policy and action.” Allied scientists retaliated by condemning and severing their ties with German science. In response, Eddington wrote that German scientists were simply supporting their country as the Allied ones did theirs, and thus he wrote to end British selfrighteousness. In 1919, he came to believe that the eclipse experiments would jump-start reconciliation between the two groups, saying, “It is the best possible thing that could have happened between England and Germany.” He hoped that it would cause his colleagues to be more receptive to German ideas. 3 His perspective is akin to Martin Luther King’s; in essence, he controversially proclaims, “I have a dream that one day English and German scientists will be able to sit down together at a scholarly meeting without nationalist prejudice.” In his experiment, he maintained he could show either Newton’s half deflection or Einstein’s full deflection. If the outcome was indeed half deflection, then Eddington would have a twofold problem. Newton was English, and his theory was published hundreds of years ago: Eddington and his Royal Astronomical Society hoped that “national prejudice did not prevent [them] from doing anything that [they] could to forward the progress of science” (maintaining the status quo, a twohundred-year-old English theory, is presumably not scientific progress nor seminal work for which the Royal Society presented him a Fellowship). 4 Hence, he was motivated by an ideological desire to end the boycott on German science (and even went as far as to ask for a Gold Medal of the Royal Astronomical Society for Einstein) and “was 3 John Earman and Clark Glymour, “Relativity and eclipses: The British eclipse expeditions of 1919 and their predecessors,” Historical Studies in the Physical Sciences 11 (1980): 49–85, on 81–85. 4 Royal Society, “Fellowship,” available at http://royalsociety.org/about-us/fellowship (accessed 8 Dec 2010); Earman and Glymour, “Relativity and eclipses,” 84. 3 committed to the theory before the expeditions were proposed.” 5 Both Newton’s and Einstein’s theories predicted that light from the distant stars would be deflected by the Sun’s gravitational field, but Einstein’s theory predicted a gravitational displacement (1.7 second of arcs) twice that predicted by Newton’s theory of gravitation (0.8″). 6 However, in his calculations, Einstein introduced metrics that “caused confusion among those less adept than he at getting the right answer.” 7 Eddington fared no better in his text The Mathematical Theory of Relativity, making arguments and assumptions that were accepted only because of his 1919 experiment. 8 In a way, then, Eddington’s measurements did not independently verify Einstein’s predictions. Thus his experiment could only ever prove the predicted value, not the theory, because the derivation from theory to prediction was itself problematic. Yet Eddington never mentioned this; he continually presented a trichotomy of possible results to his expeditions: no deflection, Newton’s half deflection, and Einstein’s full deflection (confirming Einstein’s theory). 9 This did not include other outcomes like too much deflection, strengthening the evidence for any one of his possibilities. Unfortunately, the Sobral and Principe expeditions’ measurements were “not sufficiently accurate” to distinguish between the possibilities—there were too many chances for human error and the instrumentation was not advanced enough to definitively support a prediction. The data could have been widely interpreted in very different ways; for example, for the 5 Earman and Glymour, “Relativity and eclipses,” 84. Harry Collins and Trevor Pinch, “Two experiments that ‘proved’ the theory of relativity,” in Collins and Pinch, The Golem: What Everyone Should Know About Science (New York: Cambridge University Press, 1993), 27–55, on 43–44. 7 Earman and Glymour, “Relativity and eclipses,” 55. 8 Earman and Glymour, “Relativity and eclipses,” 56. 9 Earman and Glymour, “Relativity and eclipses,” 79. 6 4 Sobral astrographic plates “if it is assumed that the scale has changed, then the […] deflection from the series of plates is 0.90″ [consistent with Newtonian theory]; if it is assumed that no real change of focus occurred, […] the result is 1.56″ [consistent with Einsteinian theory].” 10 Out of the three sets of plates, Eddington discarded the one which yielded a measurement that would have confirmed Newton’s prediction due to “systematic error.” 11 However, he neglected to apply this reasoning to the other two sets and gave no evidence as to why—an American contemporary fittingly wrote that “the logic of the situation does not seem entirely clear.” 12 Nevertheless, he published his study, writing that it leaves “little doubt” that the deflection confirms Einstein’s general theory of relativity. 13 Though Eddington’s study was not truly a conclusive one, why did contemporary scientists not raise their concerns in forums such as the Royal Society? They did, but Eddington savvily leveraged his reputation as Fellow of the Royal Society and the positions of his coauthors – for instance, the first author of the 1919 paper and Astronomer Royal, Sir Frank Dyson. 14 According to historian John Waller, few disputed Dyson’s specialized background, which included astrometry and even eclipse expeditions, and thus the majority was not inclined to argue with his interpretation of the 10 Kennefick, “Testing relativity,” 37, 41. F. W. Dyson, A. S. Eddington, and C. Davidson, “A determination of the deflection of light by the sun’s gravitational field, from observations made at the total eclipse of May 29, 1919,” Philosophical Transactions of the Royal Society of London 220A (1920): 291– 333, on 329. 12 John Waller, “The eclipse of Isaac Newton: Arthur Eddington’s ‘proof’ of general relativity,” in Waller, Einstein’s Luck: The Truth Behind Some of the Greatest Scientific Discoveries (New York: Oxford University Press, 2002), 49–63, on 57. 13 Dyson, Eddington, and Davidson, “Deflection of light,” 332. 14 Dyson, Eddington, and Davidson, “Deflection of light,” 291. 11 5 data (the majority could not hold their own eclipse expedition anyway). 15 In the November 6, 1919, joint meeting of the Royal Society and the Royal Astronomical Society, Hugh Newall’s suggestion that another interpretation may exist to explain the unknown relationship between gravitational and electric forces went unaddressed by the authors. Polish–American physicist Ludwik Silberstein said that Eddington’s result was an isolated one and does not prove Einstein’s general theory of relativity. Dyson essentially brushed his concerns about the current study aside and said that they would be addressed in future ones: “I think it most important that this result should be verified at the next two eclipses.” Then, Eddington told him that his experiment only confirmed the Einstein’s law of gravitation, not necessarily the theory. 16 These responses are wrong for two reasons: First, as sociologist Harry Collins and Trevor Pinch show, promising to address the problems in future studies is not the right step, because then the current study that made problematic assumptions to arrive at its result becomes a part of established scientific literature, on the basis of which other studies make their assumptions: Eddington and the Astronomer Royal did their own throwing out and ignoring of discrepancies, which in turn licensed another set of ignoring and throwing out of discrepancies, which led to conclusions […] that justified the first set of throwing out still further. 17 Second, although Eddington added that Einstein’s theory was not necessarily proved, the next day’s edition of The Times (and the general public) read “Revolution in Science / 15 Kennefick, “Testing relativity,” 38; Waller, “Eclipse of Isaac Newton,” 60. J. J. Thomson (chair), “Joint meeting of the Royal Society and the Royal Astronomical Society,” The Observatory 42 (November 1919): 389–398, on 395–398. 17 Collins and Pinch, “Two experiments,” 53. 16 6 New Theory of the Universe / Newtonian Ideas Overthrown.” 18 The joint meeting was remarkably one-sided under the leadership of Sir Joseph Thomson. Perhaps Thomson knew that the Royal Society’s name was at stake: both Dyson and Eddington were already Fellows, which meant that they had to have made seminal contributions to science and technology and been voted in by existing Fellows. 19 Since this meeting was held only to discuss the Dyson and Eddington’s eclipse experiment, Thomson made sure that the discussion was ultimately in favor of them, saying that “it is difficult for the audience to weigh fully the meaning of the figures that have been put before us, but the Astronomer Royal and Prof. Eddington have studied the material carefully.” He had already committed to their result, which he suggests explains the forces Newall raised and was “exactly where the problem [of gravitation] stands” (i.e., not an isolated one like Silberstein argued). 20 Contrast this situation with that of modern times: in societies such as the American Society for Biochemistry and Molecular Biology, the referees are anonymous to the authors and must not participate in cases of a conflict of interest or even “the appearance thereof.” 21 The American physicist Daniel Kennefick argues that “their treatment of the data appears to be vindicated by a subsequent 1979 reanalysis of their plates using modern astrometric data-reduction methods.” 22 His case can be interpreted as a ratification of either Eddington’s results or methodology; both interpretations are problematic. 18 Waller, “Eclipse of Isaac Newton,” 61. Dyson, Eddington, and Davidson, “Deflection of light,” 291; Royal Society, “Fellowship.” 20 Thomson, “Joint meeting,” 394. 21 American Society for Biochemistry and Molecular Biology, “Editorial policies, practices and guidelines,” Journal of Biological Chemistry (22 Sep 2010), available at http://jbc.org/site/misc/edpolicy.xhtml (accessed 8 Dec 2010). 22 Kennefick, “Testing relativity,” 38. 19 7 Kennefick refers to Geoffrey Harvey’s “Re-examination of the Observations of the Solar Eclipse of 1919,” published this sixty years later. 23 It has only been cited once, as of 2009 – not even sociologists John Earman and Clark Glymour references it in their 1980 (a year later) journal paper. 24 Harvey was only an assistant of Andrew Murray at the Royal Greenwich Observatory at the time, so he has no biographies written about him, to my knowledge. 25 For this reason, I extrapolate his motivations from his brief scientific background. His advisor Murray urged him to perform the study that recalculates the displacements observed during the 1919 expeditions, and according to Harvey, it marks the hundredth anniversary of Einstein’s birth. 26 It is no surprise, then, that the study’s results corroborate Eddington’s, because any other result would have besmirched the Observatory: First, Waller reports that Einstein’s theory of gravitation had already been validated “on the basis of much better results,” so Harvey had to support it while recalculating gravitational displacements from the original plates. 27 Second, it was published in The Observatory, which was founded at the Observatory and, in fact, had Eddington as one of its first editors. The magazine also regularly reports the meetings of the Royal Astronomical Society, which agreed upon Eddington’s results. 28 Third, the first author of the 1919 publication is Dyson, who was the director of the Royal Observatory. The first aforementioned interpretation, in which Harvey reproduces (and reduces the 23 G. M. Harvey, “Gravitational deflection of light: A re-examination of the observations of the solar eclipse of 1919,” The Observatory 99 (December 1979): 195–198. 24 Kennefick, “Testing relativity,” 42. 25 John D. Barrow, Cosmic Imagery: Key Images in the History of Science (New York: W. W. Norton, 2008), 562. 26 Barrow, Cosmic Imagery, 562; Harvey, “Gravitational deflection,” 196. 27 Waller, “Eclipse of Isaac Newton,” 63. 28 Editors of The Observatory, “A review of astronomy,” The Observatory, available at http://www.ulo.ucl.ac.uk/obsmag (accessed 8 Dec 2010). 8 error of) Eddington’s results, is disputable because Eddington did not have knowledge of Harvey’s modern methods and he could have committed scientific misconduct even though his study’s overall results were posthumously shown to be correct. The second one, in which Harvey verifies Eddington’s “experimental decisions,” is also suspect for three reasons. He chooses an automatic plate-measuring machine (the Zeiss Ascorecord) that allows for more operator control instead of the GALAXY, even though Murray wrote the data-reduction computer program for the GALAXY. He then selects plates for analysis without much rationale. Finally, since the astrographic plates give a “weak result,” he weights the measurements from the 4-inch plates and the ones from astrographic plates using an ad hoc reason (according to their standard errors, without providing equations) so that they come out to “1.87″ ± 0.13″, a result which is just within one standard error of [Einstein’s] predicted value.” Harvey dubs this a “significant improvement,” which brings to mind his motivations, previously discussed. 29 Those who argue that he verifies Eddington’s experimental choices would have to answer to why Harvey decided that the result from the astrographic plates (1.55″) was reportable while Eddington – who calculated 0.93″, close to the displacement predicted by Newton’s theory – chose to discard it. He did so for reasons, according to Waller, not applied to his other data. 30 Confirmation bias is “the seeking or interpreting of evidence in ways that are partial to existing beliefs [and] expectations,” and Eddington suffered from it since he conducted his study (the 1919 expeditions to Sobral and Principe) with the intention to 29 30 Harvey, “Gravitational deflection,” 196–198. Waller, “Eclipse of Isaac Newton,” 58. 9 confirm Einstein’s theory of gravitation. 31 In fact, he admitted in a book two years later that he “was not altogether unbiased.” He had personal and political reasons for advocating the general theory of relativity and that affected how he treated and presented his data. Ironically, he thought that he would be preserving “the finest traditions of science” by supporting Einstein. 32 In doing so, by selectively interpreting the data without rationale, his evaluation lacked a key feature of scientific inquiry: objectivity. 31 Earman and Glymour, “Relativity and eclipses,” 72; R. S. Nickerson, “Confirmation bias: A ubiquitous phenomenon in many guises,” Review of General Psychology 2 (1998): 175–220. 32 Earman and Glymour, “Relativity and eclipses,” 84. 10 Works Cited Barrow, John D. 2008. Cosmic Imagery: Key Images in the History of Science (New York: W. W. Norton). Collins, Harry, and Trevor Pinch. 1993. “Two experiments that ‘proved’ the theory of relativity.” In Collins and Pinch, The Golem: What Everyone Should Know About Science (New York: Cambridge University Press), 27–55. Dyson, F. W., A. S. Eddington, and C. Davidson. 1920. “A determination of the deflection of light by the sun’s gravitational field, from observations made at the total eclipse of May 29, 1919.” Philosophical Transactions of the Royal Society of London 220A: 291–333. Earman, John, and Clark Glymour. 1980. “Relativity and eclipses: The British eclipse expeditions of 1919 and their predecessors.” Historical Studies in the Physical Sciences 11: 49–85. Harvey, Geoffrey M. 1979. “Gravitational deflection of light: A re-examination of the observations of the solar eclipse of 1919.” The Observatory 99: 195–98. Kennefick, Daniel. 2009. “Testing relativity from the 1919 eclipse—A question of bias.” Physics Today 62: 37–42. Lightman, Alan. 2005. “Relativity and the cosmos.” Nova. Available at http://pbs.org/wgbh/nova/einstein/relativity. Accessed 8 Dec 2010. Nickerson, R. S. 1998. “Confirmation bias: A ubiquitous phenomenon in many guises.” Review of General Psychology 2: 175–220. Thomson, J. J. (chair). 1919. “Joint meeting of the Royal Society and the Royal Astronomical Society.” The Observatory 42: 389–398. 11 Waller, John. 2002. “The eclipse of Isaac Newton: Arthur Eddington’s ‘proof’ of general relativity.” In Waller, Einstein’s Luck: The Truth Behind Some of the Greatest Scientific Discoveries (New York: Oxford University Press), 49–63.