So, a gradient measures how much the output of a function changes if you change the inputs a bit. So, a gradient simply measures the change in all weights with regard to the change in error. You can also think of a gradient as a slope of a function. The higher the gradient, the steeper the slope, and the faster a model can learn. But if the slope is zero, the model stops learning. In mathematical terms, a gradient is a partial derivative with respect to its inputs. So, in machine learning, a gradient is a derivative of a function that has more than one input variable, known as the slope of a function. In mathematical terms, the gradient simply measures the change in all weights with regard to the change in error. So, let's look at an example here. Imagine a blindfolded man who wants to climb to the top of a hill with the fewest steps along the way as possible. He might start climbing the hill by taking really big steps in the steepest direction, which he can do as long as he is not close to the top. As he comes closer to the top, however, his steps will get smaller and smaller to avoid overshooting it. This process can be described mathematically using the gradient. So, there's this image: the image illustrates a hill from a top-down view, and the red arrows represent the steps of a climber. Think of a gradient in this context as a vector that contains the direction of the steepest step the blindfolded man can take and also how long that step should be. So, note that the gradient ranging from X1 to X2 is much longer than the one reaching from X4 to X5. This is because the steepness, or slope, of the hill, which determines the length of the vector, is less. This perfectly represents the example of the hill because the hill is getting less steep the higher it climbs. Therefore, a reduced gradient goes along with the reduced slope and a reduced step size for the hill climber. So now, let's understand what gradient descent is. Gradient descent is the most commonly used optimization method deployed in machine learning and deep learning algorithms. It's used to train a machine learning model and is based on a convex function. It is an alternative optimization algorithm used to find the minimum value of a function. It does this to minimize a given cost function to its local minimum. So, this is the basic definition of gradient descent. Now, let's see the intuition behind it. So, consider that you are walking along the graph shown here, and you are currently at the green dot. You aim to reach the minimum, which is the red dot, but from your position, you are unable to view it. So, what might your possible actions be? You might go upward or downward, and if you decide which way to go, you might take a bigger step or a little step to reach your destination. Essentially, there are two things that you should know to reach the minimum: that is, which way to go and how big a step to take. The gradient descent algorithm can help you make these decisions efficiently and effectively with the use of derivatives. A derivative is a term that comes from calculus and is calculated as the slope of the graph at a particular point. The slope is described by drawing a tangent line to the graph at the point. So, if we can compute this tangent line, we might compute the desired direction to reach the minimum. We will talk about this in more detail later on. Now, let's understand the minimum value. In the same figure, if we draw a tangent at the green point, we know that if we are moving upward, we are moving away from the minimum. Vice versa, also, the tangent gives us a sense of the steepness of the slope. The slope at the blue point is less steep than that at the green point, which means it will take much smaller steps to reach the minimum from the blue point than from the green point. Now, let's understand how gradient descent works. If you consider the same example of the hill, think of gradient descent as hiking down to the bottom of a valley. This is a better analogy because it is a minimization algorithm that minimizes a given function. The equation, as you see here, describes what gradient descent does. So, in this equation, **B** is the next position of our climber, while **A** represents the current position. The minus sign refers to the minimization part of gradient descent. The **gamma** in the middle is a weighting factor, and the gradient term **Δf(A)** is simply the direction of the steepest descent. This formula basically tells us the next position we need to go, which is the direction of the steepest descent. Let's look at another example to really drive the concept home. Imagine you have a machine learning problem and want to train your algorithm with gradient descent to minimize your cost function, **J(W, B)**, as you see in this figure. To minimize your cost function **J(W, B)** and reach its local minimum by tweaking its parameters (that is, **W** and **B**), the image below shows the horizontal axis representing the parameters **W** and **B**, while the cost function **J(W, B)** is represented on the vertical axis. Since gradient descent is a convex function, we know we want to find the values of **W** and **B** that correspond to the minimum of the cost function. That is marked with the red arrow here. To start finding the right values, we initialize **W** and **B** with some random numbers. Gradient descent then starts at that point (somewhere around the top of our illustration), and it takes one step after another in the steepest downward direction (that is, from the top to the bottom of the illustration) until it reaches the point where the cost function is as small as possible. So, this is about the working of gradient descent.