Perceptron Learning - Algorithm

The first neural model, also known as perceptron, was designed in 1958. Perceptron is a linear binary classifier used for supervised learning. Linear means perceptron learning can be used for classifying data which is linearly separable. That is, if data is separable with a straight line, we can use the perceptron network. Also, if the data has only two classes, that is, if the target has two classes, then we can use perceptron learning. If you have more than two classes, we cannot go with perceptron learning. In this case, the perceptron learning model is a combination of two concepts: the first one is McCulloch-Pitts model of an artificial neuron, and Hebbian learning for adjusting the weights. Here, we need to adjust the weights so that we can classify all the examples correctly. The Hebbian learning concept is used for adjusting the weights. Perceptron learning consists of four components. The first component is input from other neurons. You can see here X1, X2, X3, and Xn are the inputs coming from other neurons. For each and every neuron, there is an associated weight here, which is W1, W2, W3, till WN, and there is an associated bias. Those are the two things. The third one is, whenever we have input and weights, we compute something known as the net sum. That is, the multiplication of the input and weight, and then we take the sum of those things: X1*W1 + X2*W2 + X3*W3 and so on. Once you calculate this net sum, what we do is, on top of this net sum, we apply something called an activation function. Based on the activation function, we will get different outputs. Let's assume that the activation function is a step function. There are multiple activation functions, but just for explaining, I am considering the step activation function. In a step activation function, if the calculated sum is greater than or equal to some threshold value, the output will be one. If it is less than the threshold value, it will be equal to zero. So, based on the activation function, we will get different outputs. Once we calculate the output, we compare this output against the actual output and then calculate the error. Once you calculate the error, we will check whether the error is acceptable or not. If it is not acceptable, we will update the weights. In this case, we'll try to understand the perceptron learning algorithm step by step. First, we need to initialize all these weights here for each and every input. As said earlier, there will be weights and the bias. So, we need to initialize these weights as well as the bias randomly, usually in the range of -5 to +5. Now, for each epoch, until we are able to classify each and every example correctly, we need to perform the following steps. In the first step, we need to compute the weighted sum. As said earlier, we need to compute this weighted sum here. Once you compute the weighted sum using the input and the weights, we need to apply the activation function. As said earlier, we need to apply the activation function so that we can calculate the output here. So, first, we will calculate the weighted sum here, and on top of that, we will apply the activation function. Again, just for explanation purposes, I have taken the step activation function here. Now, once you calculate this output, we need to calculate the error by subtracting the estimated output (estimated means the calculated output) from the desired output (that is, the target output here). So, we have some target, and the calculated output will be subtracted from this target output so that you'll be able to calculate the error here. Once you calculate this error, if we are happy with this error, there is no need to do the weight updation. If we are not happy with this error, we need to go back and update these particular weights. That is what you can see here. We have calculated the output. We will calculate the error. If the error is acceptable, we don't update the weights. If the error is not acceptable, we will go back and update these weights in this case. Now, the question is, how to update these weights? In this case, if you want to update the weights, we need to use this equation: first we need to calculate the change in weight here that is Delta wi which is equalent to the learning rate this is the learning rate multiplied by the calculated error here multiplied by the input that is nothing but if you want to modify this particular W1 what is that we need to do Delta W1 is equalent to learning rate this learning rate is common for everything the calculated error that is again common for everything XI XI is nothing but what X1 we need to consider in this case if you want to modify this W2 we need to consider X2 and so on now once you calculate this Delta W or Delta W1 we can modify the weight something like this wi is equal to Wi that's a volt weight plus change in weight