Uncertainty and Graphical Analysis

Uncertainty and Graphical Analysis Introduction Two measures of the quality of an experimental result are its accuracy and its precision. An accurate result is consistent with some ideal, “true” value, perhaps a commonly accepted value from the scientific literature. When a literature value is not available, we often perform an additional measurement by other methods. Different methods are usually prone to different errors. We can hope that, if two or three different methods yield consistent results, our errors are small. However, measurements made by different methods never agree exactly. If the discrepancy is small enough, we claim that the results are consistent and accurate. Most of our work with uncertainties will address the question, “How small is small enough?” Precision refers to the reproducibility of a result made using a particular experimental method. When random variations are large, the precision is low, and vise versa. While we should work hard to reduce the size of random effects, they cannot be entirely eliminated. When we claim that two measurements are consistent, we are claiming that their difference (the discrepancy) is smaller than these random variations. Since many quantities of interest are calculated from measured values, we also need to know how random variations in measured quantities affect the results of these calculations. Measurements in the presence of random deviations Mean and standard deviation of the mean1 In the presence of random variations, the best estimate of a physical quantity is generally given by the average, or mean. The average value of a set of N measurements of x, (x1 , x2 , x3 , ...xN ), is given by xavg = 1 N x1 + x2 + x3 + ... + xN = ∑ xi N N i=1 1A (13.1) good reference for much of the information in this section is John R. Taylor, An Introduction to Error Analysis—The Study of Uncertainties in Physical Measurements, 2nd Edition (University Science Books, Herndon, Virginia, USA, 1987), especially Chapter 5. 69 70 UNCERTAINTY AND GRAPHICAL ANALYSIS The individual measurements of x will generally deviate from xavg due to random errors. The standard deviation of x, denoted σ (x), indicates how far a typical measurement deviates from the mean. The value of σ (x) reflects the size of random errors. s (x1 − xavg )2 + (x2 − xavg )2 + (x3 − xavg )2 + (x4 − xavg )2 + ...(xN − xavg )2 N −1 " #1/2 N 1 =p (x1 − xavg )2 (13.2) ∑ (N − 1) i=1 σ (x) = A small standard deviation indicates that the measurements (x-values) are clustered closely around the average value, while a large standard deviation indicates that the measurements scatter widely relative to the average value. Thus a small standard deviation indicates that this particular quantity is very reproducible—that is, the measurement is very precise. Note that the units of the standard deviation are the same as the units of the individual measurements, xi . The relation between the standard deviation to the deviation of the data from its average value is illustrated in Figure 13.1. Figure 13.1 is a histogram of 100 scores, chosen from a set of over 1000 random scores with an average was 85 and a standard deviation of 7.5. Because of their random distribution, the average of the 100 scores is not exactly 85, and their standard deviation is not exactly 7.5. Because we cannot take an infinite number of measurements, Equations 13.1 and 13.2 are only approximations to the true average and standard deviation. On average, the approximations improve as the number of measurements, N, increases. Histogram of 100 Student Scores Number of Scores per Bin 30 25 20 Random Sample N = 100 xavg = 85.08 Gaussian curve N = 100 xavg = 85 x) = 7.5 15 x) = 7.454 7.5 7.5 10 5 0 60 70 80 90 100 110 Score Figure 13.1. Histogram of 100 scores with an average of 85 and a standard deviation of 7.5. The smooth curve is the Gaussian function corresponding to the same number of measurements, average, and standard deviation. The Gaussian function, G(x), corresponding to 100 scores with an average of exactly 85 and a standard deviation of exactly 7.5 is also shown in Figure 13.1. According to the Central Limit Theorem 71 of statistics, the Gaussian function represents the ideal distribution of scores for a given N, xavg , and σ (x) = σ if the scores have a finite average and the measurements are statistically independent. These conditions apply to most of the measurements made in lab. (Important exceptions are found in the stock market, among other things.)   −(x − xavg )2 N exp G(x) = 2πσ 2σ 2 (13.3) The value of the standard deviation in the context of uncertainties is that the probability of finding a score at some distance from the average falls in a predictable way as the distance increases. For an ideal Gaussian distribution, 68% of the measurements lie within one standard deviation of the mean (xavg ). In Figure 13.1, 63 scores (63% of 100) lie within 7.5 points of 85. Ideally, 95% of the scores lie within two standard deviations (here, ±15 points) of the average. Ideally, one would expect 99.7% of the points to lie within three standard deviations (here, ±22.5 points) of the average. No score in Figure 13.1, is more than three standard deviations from the average. (All of the scores lie between xavg − 3σ = 62.5 and xavg + 3σ = 107.5.) Unless the total number of scores is very high, the probability of finding a score more than 3σ from the average is quite low. Since the standard deviation characterizes random errors, we can pretty much rule out random errors as the source of any difference greater than 3σ . We will make this assumption in the physics labs, although the precise probabilities will usually differ from those given by the ideal Gaussian function. For instance, when the number of measurements is small, our estimates of xavg and σ (x) may be poor. In more advanced work, it can be important to correct for this lower precision.2 When one is attempting to show that one measurement out of a large number differs significantly from the others, a higher threshold for significance (4σ or 5σ ) may be necessary. Since the result of an experiment is generally an average value, we need a measure of the precision of the average. This is called the “standard deviation of the mean,” σ (xavg ). Although one can repeat the entire set of N measurements several times to compute σ (xavg ), statistics allows us to estimate σ (xavg ) using the original N measurements alone: 1 σ (xavg ) = p N(N − 1) " N ∑ (x1 − xavg)2 i=1 #1/2 σ (x) = √ N (13.4) The standard deviation function of most spreadsheet programs (Excel, OpenOffice), Capstone, and calculators gives σ (x), from Equation 13.2. To calculate the standard deviation of the mean from this number, you must divide by the square root of N, the number of measurements. On the other hand, spreadsheet Regression functions and Capstone’s curve fit function provide the standard deviation of the mean, σ (xavg ) from Equation 13.4. 2 Student’s t-test is used to make this adjustment in more advanced work. This is described at the end of Chapter 5 in John R. Taylor, op. cit., and in many statistics books. 72 UNCERTAINTY AND GRAPHICAL ANALYSIS Other methods for estimating the effect of random errors When several measured quantities are used in a calculation, a relatively crude measurement of one quantity may contribute little to the overall uncertainty. If so, there is little point in improving the measurement. To demonstrate that the uncertainty is small, we must provide an upper bound on the uncertainty and show that the effect of this uncertainty is indeed relatively small. Smallest division Most measuring devises have a a smallest division that can be read. In this case, one can use the size of the smallest division as an upper bound on the uncertainty. In some cases, it is appropriate to to use one-half of this smallest division. For instance the smallest division displayed on a meter stick is usually 1 mm. The distance d is read to the nearest mark. Suppose, for example, you look at the meter stick a few times and read d = 85 mm each time. Because you never measured 84 or 86 mm, you are confident that 84.5 ≤ d ≤ 85.5. That is, the magnitude of the uncertainty in d is less than 0.5 mm. This is a useful upper bound. You must use your judgement in cases where the measurement cannot be practically made with this precision. For instance, your precision can be much worse if you don’t have a clear view of the ruler. Interpolation If the uncertainty in such a measurement is not small relative to the other uncertainties in an experiment, a better estimate of the uncertainty is needed. In this case, taking the standard deviation of the mean of multiple measurements is necessary. For instance, you can estimate d to one-tenth of a mm using a meter stick. (Estimating values between the marks is called interpolation.) In this case, repeated estimates, made with care, will disagree, and you can calculate the standard deviation of their mean. Manufacturer’s specification The user manuals for many instruments (electronic ones in particular) often include the manufacturer’s specification as to the “guaranteed” reliability of the readings. For example, the last digit on the right of digital voltmeters and ammeters is notoriously inaccurate. In this case, it makes sense to use the manufacturer’s specification as a simple upper bound. Terminology—Uncertainty and significant digits Because the standard deviation is not the only measure of random variation, it helps to have another name and symbol for this quantity. We will call the the expected effect of random variation on xavg its uncertainty, and represented it by the symbol u(xavg ). If the average and standard deviation of x are available, the best estimate of x is xavg , and the best estimate of the uncertainty of xavg is the standard deviation of its mean, σ (xavg ). Then u(xavg ) = σ (xavg ). The uncertainty is often indicated by a ± sign after the average value. For instance, you might specify a length measurement as “1.05 ± 0.02 mm. Because there is more than one way to estimate the uncertainty, you must also specify how your estimate was made. For instance, the result of a length measurement may be reported as “1.05 ± 0.02 mm, where the uncertainty is the standard deviation of the mean of five length 73 readings;” or “24 ± 1 mm, where the uncertainty is the distance between marks on the meter stick.” With or without a formal uncertainty estimate, you are expected to have a general idea of the uncertainties of the numbers you use. These uncertainties are communicated by the number of significant digits you provide with the number. For instance, a length written as 3.14 mm has an implied uncertainty of less than 0.1 mm; the inclusion of a digit in the second decimal place means that you have some knowledge of it. In your lab notebook and reports, you should not use more significant digits than are justified by your knowledge. Since rounding operations slightly increase the uncertainty in the last decimal place, it is appropriate to keep one extra significant digit in each step of a calculation. However, the final result must be rounded to an appropriate number of significant digits. Most physics texts include a discussion of significant figures. Uncertainties in calculated quantities Uncertain measured values are often used to calculate other quantities. These calculated quantities will be uncertain as well, and the degree of uncertainty will depend on the uncertainty of our measurements. We will the Minimum-Maximum method to estimate uncertainties in calculated quantities. Let us start with a simple example. Assume that we have measured the quantity, x, and we need to calculate a value for the function f (x) = 1/x. Say that several measurements of x have yielded xavg = 2.0, with an uncertainty u(xavg ) = σ (xavg ) = 0.1. As long as there is no confusion, this can be reported as x = 2.0 ± 0.1. The value of f (x) when evaluated at x = 2.0 is 0.50, but how does the uncertainty in x (the ± 0.1) affect our value for f (x) (the 0.50)? For simple functions, the change in f (x) due to a change in x, ∆x, can be evaluated directly by calculating f (x + ∆x) and f (x − ∆x). Here ∆x = u(xavg ) and we have f (x + ∆x) = 1/(2.0 + 0.1) = 0.476. Similarly f (x − ∆x) = 1/(2.00.1) = 0.526. [Note that for f (x) = 1/x, f (x) increases as x decreases and vise versa.] The Minimum-Maximum method gives two uncertainties, u+ [ f (x)] =|0.526 − 0.500| = 0.026 for the upper error and u− [ f (x)] = |0.476−0.500| = 0.024 for the lower error. This can be summarized by saying that f (x) = 0.50+0.026, −0.024. Since the uncertainty is in the second place to the right of the decimal it would be legitimate to round the uncertainty in f (x) to 0.50 +0.03, –0.02. Notice that the plus and minus uncertainties are not equal even after rounding. In many cases, our goal is to use our uncertainty to compare our measured f (x) with another measurement or prediction. In this case, it is not necessary to calculate both u− [ f (x)] and u+ [ f (x)]. If the prediction is greater than f (x), then u+ [ f (x)] (the upper error) is the important quantity. Similar, if the prediction is smaller than f (x), u− [ f (x)] (the lower error) is the important quantity. Your knowledge of how f (x) varies with x will usually allow you to guess whether (x + ∆x) or (x − ∆x) is needed. If you guess wrong, you just use the other. For more complicated functions, say f (x, y), one calculates u+ [ f (x, y)] by choosing the signs of ±∆x and ±∆y that together maximize the value of the function f (x, y). For instance, if f (x, y) = x2 /y, then f (x, y) is maximized by choosing a high value of x and a low value of y. Similarly, the 74 UNCERTAINTY AND GRAPHICAL ANALYSIS function is minimized by choosing a low value of x and a high value of y. Therefore, u+ [ f (x, y)] = (xavg + ∆x)2 (y − ∆y) and u− [ f (x, y)] = (xavg − ∆x)2 (y + ∆y) (13.5) Again, you do not need to compute both u+ [ f (x)] and u− [ f (x)] if your only goal is to compare your measurement with a prediction or another measured value. The Minimum-Maximum method is relatively easy to use, but it has some drawbacks that are beyond the scope of this introduction. The problems are usually minor as long as the uncertainties are small and u+ [ f (x)] ≈ u− [ f (x)]. Using uncertainties to compare measurements or calculations Suppose you have measured a cart’s mass, mF/a , from force and acceleration measurements and Newton’s Second Law, F = ma. To check for systematic errors, you have also measured the cart’s mass using an electronic balance, with the result mbal . A straightforward way to determine whether these two measurements is to compare the discrepancy between the two measurements, say ∆ = mF/a − mbal , with the expected uncertainty of ∆, that is u(∆). As illustrated in Figure 13.1, the probability of ∆ being more than three standard deviations from the mean because of random errors alone is quite small. Therefore, if ∆ > 3u(∆) most of the discrepancy is almost definitely due to systematic problems. In this case, we say that the measurements of mF/a and mbal are not consistent. The ratio between the discrepancy and its combined standard uncertainty is a useful measure of the seriousness of a discrepancy. Because this ratio is similar to the t-statistic of classical statistics, we call it the t 0 -score.3 In this example, t0 = ∆ ∆ =q u(∆) u(mF/a )2 + u(mbal )2 (13.6) When you compare experimental results and find t 0 > 3, you should carefully review your calculations and measurement procedures for errors. If systematic errors appear to be significant, and you know what they might be, you should describe them in your lab notes. If time permits, repeating a portion of the experiment is in order. Whatever your conclusion, your lab notes must indicate how you estimated your uncertainties. In the United States, the general authority on the reporting of uncertainties is the National Institute of Standards and Technology.4 These standards have been developed in consultation with international standards bodies. When the potential consequences of a decision are critical or when the data are unusual in some way, one should consult a statistician.5 3 N. T. Holmes and D. A. Bonn, “Quantitative comparisons to promote inquiry in the introductory physics lab,” Phys. Teach. 53(7), 352 (2015). DOI: 10.1119/1.4928350 4 Ibid, Barry N. Taylor and Chris E. Kuyatt. 5 W. Edwards Deming, Out of the Crisis (MIT Press, Cambridge, Massachusetts, 1982). Some authors attribute the 75 Determining functional relationships from graphs Linear relations are simple to identify visually after graphing and are easy to analyze because straight lines are described by simple mathematical functions. It is often instructive to plot quantities with unknown relationships on a graph to determine how they relate to one another. Since data points have not only measurement uncertainties but also plotting uncertainties (especially when drawn by hand), slopes and such should not be determined by using individual data points but by using a “best-fit line” that appears to fit the data most closely as determined visually. If graphing software is used, then the slope of the line can usually be determined by a computer using a “least squares” technique. We won’t go into detail about these methods here. Linear functions (y = mx + b) If x and y are related by a simple linear function such as y = mx + b (where m and b are constants), then a graph of y (on the vertical axis) and x (on the horizontal axis) will be a straight line whose slope (“rise” over “run”) is equal to m and whose y-axis intercept is b. Both m and b can be determined once the graph is made and the “best-fit” line through the data is drawn. if x = 0 does not appear on your graph, b can be found by determining m and finding a point (x, y) lying on the “best-fit” line; then equation y = mx + b can be solved for b. Simple power functions (y = axn ) In nature we often find that quantities are related by simple power functions with n = ±0.5, ±1, ±1.5, ±2, etc., where a is a constant. Except for n = +1, making a simple graph of y (vertical axis) versus x (horizontal axis) for simple power functions will yield a curved line rather than a straight line. From the curve it is difficult to determine what the actual functional dependence is. Fortunately it is possible to plot simple power functions in such a way that they become linear. Starting with the equation y = axn , we take the natural logarithm of each side to show ln(y) = ln(axn ) = ln(a) + ln(xn ) = ln(a) + n ln(x) (13.7) If ln(y) is plotted on the vertical axis of a graph with ln(x) plotted on the horizontal axis (This is often called a doubly logarithmic, or log-log graph.), then Equation 13.7 leads us to expect that the result is a straight line with a slope equal to n and a vertical axis intercept equal to ln(a). If the relationship between y and x is a simple power law function, then a graph of ln(y) as a function of ln(x) will be linear, where the slope is n, the power of x, and the intercept is the natural logarithm of the coefficient a. This is quite useful, because it is easy to determine whether a graph is linear. If we suspect a simple power function relationship between two quantities, we can make a log-log graph. If the graph turns out to be linear, then we are correct in thinking that it should be a simple power function and can characterize the relationship by finding values for n and a. ability of Japanese automakers to break into the U.S. market to their skillful application of the principles of statistical quality control popularized by W. Edwards Deming and Joseph Juran. 76 UNCERTAINTY AND GRAPHICAL ANALYSIS Exponential functions (y = aebx ) Radioactive decay, the temperature of a hot object as it cools, and chemical reaction rates are often exponential in character. However, plotting a simple graph of y (on the vertical axis) and x (on the horizontal axis) does not generate a straight line and therefore will not be readily recognizable. A simple graphical method remedies this problem. Starting with an equation for the exponential function, (y = aebx ). We can take the natural logarithm of each side to show ln(y) = ln(aebx ) = ln(a) + ln(ebx ) = ln(a) + bx (13.8) If ln(y) is plotted on the vertical axis and x is plotted on the horizontal axis (This is called a semilog graph.), Equation 13.8 takes the form of a straight line with a slope equal to b and a vertical axis intercept equal to ln(a). Thus any relationship between two variables of this simple exponential form will appear as a straight line on a semi-log graph. We can test functions to check whether they are exponential by making a semi-log graph and seeing whether it is a straight line when plotted this way. If so, the values of a and b that characterize the relationship can be found. Using error bars to indicate uncertainties on a graph When plotting points (x, y) with known uncertainties on a graph, we plot the average, or mean, value of each point and indicate its uncertainty by means of “error bars.” If, for example, the uncertainty is primarily in the y quantity, we indicate the upper limit of expected values by drawing a bar at a position ymax above yavg , that is, at position ymax = yavg + u(yavg ). Similarly, we we indicate the lower limit of expected values by drawing a bar at position ymin = yavg − u(yavg ). Figure 13.2 shows how the upper error bar at ymax and the lower error bar at ymin are plotted. If the quantity x also has significant uncertainty, one adds horizontal error bars (a vertical error bar rotated 90°) with the rightmost error bar at position xmax and the leftmost error bar at position xmin . y (xavg, yavg) ymax xmin xmax ymin x Figure 13.2. Diagram of error bars showing uncertainties in the value of the x- and y-coordinates at point (xavg , yavg ). z1max y1max Occasionally one encounters ysystems where the upperz and lowerz error bars have different lengths. In this case, the upper uncertainty, u+ (yavg ) does notz equal the lower uncertainty, u− (yavg ). This y often happens when the Minimum-Maximum method is used to zestimate uncertainties. y 1avg 1avg 1min 1min 2max 2avg 2max The values y1avg and y2avg are not consistent, because their uncertainty ranges do not overlap. y2avg y2min z2min The values z1avg and z2avg are consistent because their uncertainty ranges overlap.