Let's say we are trying to, um, optimize the number of neurons per hidden layer for one layer. That is because we can have different number of neurons in each layer in each hidden layer and the optimizer. So let's say we are trying gradient descent or miss prop momentum and item what's going to happen is that in this case we have 20 cases and that wouldn't even be very high. To try them. Right. We can just try them one by one. But if you also want to add learning rate into this, what's going to happen is all of a sudden you're going to have. 60 different possibilities because for every combination of optimizer and nber of neurons on this hidden layer, you're going to, have a different learning rate. So in total, you're going to have six different cases that you need to try six different. Let's say you also want to optimize for a bedside. So if you try three different batch sizes, let's say 30 to 6,428. All of a sudden you have hundred 80 different settings that you need to try on to understand which one works better. And as you can imagine, this can get really big, really fast because right now you're only optimizing 1, 2, 3, 4 different Piper parameters, and we are trying four to five or even three. For some of them options for each of them. Whereas what could happen in not in a normal hyper parameter tuning problem is that we are trying to optimize five, six, maybe 10 different hyper parameters. And we are also trying out maybe attends of different values for all of these different hyper-parameters. So that can get quite big, very fast. And that's why we need some smart solutions when we're doing hyper parameter tuning. When we are searching for the best type of parameter setting.For our network. So let's look into some of the approaches that people use for. The first option is to use grid search. It is one of the very commonly used approaches when it comes to machine learning techniques, but it is a very slow because what it does is just tells us to try every single option that is in our hyper parameter space. And when we're building neural networks, this will take a very long time because as we just talked about, we have a lot of different combination of sales. That's why there is a bit of a different approach grid search that we use more commonly. And that is random search. What random search does is basically choose some random settings in your hyper parameter space and try them out. You can say out of the 20 options that I have here, I want to try five and then it tries out these five. And even though this sounds quite simple, random search is used very, very often. And most likely for your personal projects or your site project. Random searches going to be the one that you need out of all of the other hyper parameter tuning techniques that are out there. So, as I said, random search works really well for many problems, but there's one thing that you need to consider, and that is the range or the scale that you set for your hyper parameters. So if you're just sitting nber of neurons in a hidden layer, you can just use normal distribution between 108 hundred. So that could be just your range and the scale could be a normal scale. But if you are trying to also optimize for learning. You shouldn't use normal scale because what's going to happen is this scale goes from 0.0001.So basically this is going to be seen as one. And while you're going at it, you're going to choose learning rates that are, that are bigger than 0.2, , bigger than 0.5, et cetera, et cetera, all the way until one, but the learning rates that are going to make the biggest difference for you and the ones that you're interested. Our old industry region. So when you are choosing them based on a normal distribution, you're going to choose the ones that are not going to make the biggest difference,because choosing one that is on 0.7 and one are not going to be. Whereas, what you can do instead of this is just use a logarithmic scale. So then when you are choosing your random values on the normal distribution, you will be able to choose the ones that are more relevant to what you're trying to achieve. So always keep in mind the range and the scale. That you're setting while you're doing random search, but random search is not perfect. Of course there are some shortcomings. So if you just do random search, once you might not be able to find a reallyoptimal value for your hyper parameters. So what people do is do manual zooming in.