The Geometry of Linear Approximation

The geometry of linear approximation ● A linear approximation of a function is a polynomial that passes through the sample points and has the same slope at each of them. ● The linear approximation of a function is often used in statistics; many statistical methods assume that the data are generated by some underlying model, and the linear approximation provides an alternative expression for the model's predictions. Let’s go through it. So we take the x derivative of that. If we plug in x=1 and y=1, the equation becomes negative 4. If y is the function of x and we take the derivative of y with respect to x, we obtain 2y. When we plug in 1 for x and 1 for y, we get 2. One can then write the linear approximation to this function as the value of the function times its derivative multiplying by delta x. In this box, there is a negative 4. And then, the derivative of y with respect to x multiplied by delta y, which equals 2. Now what is the other way to write it? The change in x is delta x. So let's make a little image here. So here's the point (1, comma 1), and I take another point (x, comma y) near (1, comma 1). Delta x is the change in x, or x minus 1. The change in y is delta y, which is y minus 1. Then I can substitute this expression into the equation for delta x and delta y. The function f(x, y) = -1 + 4x - 4y + 2 delta y is illustrated in the graph below. We can group the terms of this equation to make it look like a times x plus a number of times y plus a number. And the only place that x's can come from is here, so we have a negative 4x. The y's can be derived from one source, which is here, so we have plus 2y. And then there are several places from which the constant can be obtained. From here, we see a negative 1 and from there a plus 4. From there is a minus 2. If these numbers are added together, we get a total of 1. The linear approximation written the other way is negative 4x plus 2y plus 1. Here’s a visual representation of the functions that we just did, and then you’ll understand why we have two different ways to do them. Here is a picture of the level curves of this complicated function y squared minus x cubed minus x. Here is the point (1, 1). At that point, you can see that we are on the level curve of height minus 1— these matches. This is a complicated function, so we would not be able to draw it by hand, but this is what it looks like. And as you may remember, the image starts to look simpler if you zoom in on it. Now, as we see in this figure, the x and y variables are going between 0 and 2. But if we zoom in around the point (1, 1), we get a second picture. The function is the same, but we have changed the domain of the function by taking a smaller box and blowing it up. Now the level curves are much simpler to see and they look like almost parallel straight lines. This is a picture that we could hope to draw by hand by understanding linear approximation. The linear approximation tells us that the original function f and the linear approximation are very close to each other as long as we look on a little box around (1, 1). Let us make this clear by moving this slide over and over here. The function f, shown in the middle of the box, is the actual function. The level curves of the linear approximation are drawn on the right side of the box. They look almost like they match each other. Now, here is an explanation why there are these two different ways of writing it. Do not think these are completely different things. They are still the same formula. Delta x is x minus 1, and delta y is y minus 1. And then these are the same. However, why would you want to write it that way? One more example for you. Suppose you want to make a similar picture at home. First, you need to type into Mathematica the formula for the linear approximation. When given a function of x and y, Mathematica produces very nice graphs of these level curves. But if you input this into the computer, it won't know what you mean. Thus, one must write the function as a function of x and y so that the computer can draw its level curves. There is a deeper lesson to be learned from this quantization problem. The approximation of the function as a linear combination of its poles is actually a whole function. The function is its own derivative of x and y, and it's a good approximation of the original function. This is it. Next coming up is understanding functions Ax plus By plus C. And the idea is that since we understand linear approximation, we can use this knowledge to figure out stuff about f(x, y) as long as we're in a little box where the linear approximation remains valid.