So, first one is **Batch Gradient Descent**, which is the most straightforward type. It calculates the error for each example within the training set. After it evaluates all training examples, it updates the model parameters. This process is often referred to as a training epoch. Advantages of **Batch Gradient Descent** are that it's computationally efficient and produces a stable error gradient and a stable convergence. One disadvantage is that the stable error gradient can sometimes result in a state of convergence that isn't the best the model can achieve. It also requires the entire training set to reside in memory and be available to the algorithm. So now, the second one is **Stochastic Gradient Descent**, which updates the parameters according to the gradient of the error with respect to a single training example. This is unlike **Batch Gradient Descent**, which updates the parameters after all training examples have been evaluated. This can make **Stochastic Gradient Descent** faster than **Batch Gradient Descent**, depending on the problem. One advantage is that the frequent updates provide a daily rate of improvement. Our disadvantage is that the frequent updates are more computationally expensive than **Batch Gradient Descent**. The frequency of the updates also can result in noisy gradients and may cause the error rate to fluctuate instead of slowly decreasing. The third and the last one is **Mini-Batch Gradient Descent**, which is an often preferred method since it uses a combination of **Stochastic Gradient Descent** and **Batch Gradient Descent**. It simply separates the training set into small batches and performs an update for each of these batches. It creates a balance between the efficiency of **Batch Gradient Descent** and the robustness of **Stochastic Gradient Descent**. Common numbers of examples for **Mini-Batch** range between 30 and 500, but like for any other machine learning technique, there is no well-defined rule because the optimal number can vary for different problems. So, **Mini-Batch Gradient Descent** is commonly used for deep learning problems. So, this is the graph that shows the gradient descent variants, like all the three types of variants and their direction towards the minimum. So, as you can see in three different colors, the **Batch Gradient Descent**, the **Mini-Batch Gradient Descent**, and also the **Stochastic Gradient Descent**, what you can identify here is that the noise in **Stochastic Gradient Descent** is way more than the noise in **Mini-Batch Gradient Descent**. That is the reason, in the majority of the cases, **Mini-Batch Gradient Descent** is considered. You might be thinking, "Why is **Batch Gradient Descent** not used even though it has the least amount of noise?" So, it is because the **Batch Gradient Descent** is very slow as it goes over all the examples when we have large datasets.