Fitting a parabola to data. Worked example recitation Solve the following problem using least squares. The data set consists of three points, the point (0, 1), the point (2, 1), and the point (3, 4). And I've drawn a rough picture of where these points are on a graph, and I'll be talking a little bit about that after you try this problem. So why don't you try to solve the problem based on the technique you learned in class? The first thing we must do is draw a line on the graph. We will then talk about what the least squares line is actually trying to do. If you remember, what you're actually trying to do is minimize a quantity, and the quantity you're trying to minimize is the difference between the actual value you get and the expected value you get. So, let's look at this pictorially. We have something that contributes to some area. And then this is another thing. It's the difference between the expected value in y and the actual value you get. And so let us try and make that squared, because we square that value. And then this last one is pretty small. To minimize the area of the sum of a series of squares, you may need to adjust the position of a line. If you tip the line down in this case, but keep it fixed at its original height, the distance between the line and each square increases, which makes the area of that square larger. You are trying to figure out how to minimize the sum of the areas of these squares by moving the line. You can move the line up and down, or you can move the line left and right. Moving the line left and right will vary the slope but fix the intercept; moving the line up and down will fix the intercept but vary the slope. In multivariable calculus, you are trying to find the best y-intercept and the best slope at the same time. You have learned this process because you are trying to optimize something that has two quantities that you can change—the y-intercept and the slope. If you write down the function, as a function of a and b, and take the derivative with respect to a, setting it equal to zero, you get the first equation. The resulting second equation is derived from a set of values and is solved for b. Next, we will fill in the details and solve for a1, which will give us the first equation. Finally, we will mention how to solve higher order, non-linear least squares problems. We'll need the sum of all the xi'a, which equals 5. Next, we'll need the sum of all the yi's, which equals 6. Finally, we'll need to find the sum of all the xi squared, which is 13. The sum of all the xiyi products is 14. Now, where are these coming from? Remember what is x1 and what is y1? If we come back over here, this is (x1, y1); (x2, y2); (x3, y3). If we switch the whole pair around, it doesn't matter. Obviously, if we switched x and y values together, it does matter. And that you see actually from the equation does matter. Right here you have a product of them. Solving a system of equations means finding values for two variables that work in both equations. In this case, we are solving the equations 13a plus 5b equals 14, and 5a plus 3b is equal to 6. To solve this equation, you must divide the numerator and denominator by the same number. If you divide both sides by 7, you get that a is equal to 6/7 and b is equal to 4/7. So what this equation tells you is that the line we actually want, the one that minimizes the sum of squared distances from each data point, is y equals 6/7x plus 4/7. That is our least-squares solution. To solve for the equation of a line that will fit these data, you can complete the following steps: For example, you can use the least squares method to fit a parabola to these data. And let me point out what the function would look like in this case. For the challenge problem, it will be a function of three variables, so it will look something like this: (a, b, c). And the point that we'll be wanting to minimize—or the function we'll be wanting to minimize again—is from 1 to 3. We have (yi minus the big quantity) times xi squared plus (axi plus bxi) times c. We square the whole thing, and so what this is giving you is, just as in the picture before, this is giving you the difference between the expected value and the actual value. So this is your actual y value, and based on the parabola you're looking for, what's in parentheses is your expected y value. When we evaluate the function at x = 0.1, for example, we will get a number. That number will be on our parabola. This will be the y-value associated with the x-value from our data. So this is the actual value of the function's output when x = 0.1. This is what we expect the function's output to be at that x-value based on the model we have constructed. We take the difference between these values and square it, because we want to minimize how we solve this problem. And that is going to involve taking the derivative of this function with respect to each of these three variables separately. So you take the derivative of f sub a with respect to a, set it equal to zero, take the derivative of f sub b with respect to b, set it equal to zero and take the derivative of f sub c with respect to c and set it equal to zero. You have a system of three equations with three variables. A strategy for solving such a system is to use matrices, for example. And so then you can actually come up with a formula for the parabola of best fit, you could think of it as. But point out that this is where you're going to be starting for the next challenge problem on finding the quadratic of best fit for these three data points.