Perceptron Rule for XOR Gate Solved Example

This is detectable for XOR gate. X1 and X2 are the two inputs, and Y is the output. The output is low only when both the inputs are low or both of them are high. In this case, the same thing can be represented as something like this: if both of them are low or high, the output is zero. If either of them is high, the output is high. In this case, if you want to classify this particular data with the help of a linear line, or you can say that a straight line, it is not possible. For example, if I draw a line over here, I am not able to classify this particular data properly. Similarly, if I draw a line over here, again I am not able to classify it. So, this is one peculiar logical gate where the data is non-linearly separable. In this case, hence we cannot use a simple perceptron rule to find or can say that implement this XOR gate. Hence, we will use this particular equation that is \( y \equiv \overline{x_1} \cdot x_2 + \overline{x_1} \cdot x_2 \). So, this equation will give you this particular output over here. So, what we do here is, starting from this particular equation, we will rewrite it something like this: \( Y = Z_1 + Z_2 \), where \( Z_1 \) is this particular part (the first part) and \( Z_2 \) is equivalent to this part. Once you get \( Z_1 \) and \( Z_2 \), we will combine them, and then we will get what you can say is the final \( Y \) over here. So, first, we need to solve this particular \( Z_1 \) and then \( Z_2 \), and after that, we need to combine them to get the final \( Y \). This is how the final network will look like: \( X_1 \) and \( X_2 \) are the two inputs, \( Z_1 \) and \( Z_2 \) are the intermediary perceptrons, and \( Y \) is the output over here. These are the weights in this case. So, one time we need to solve this particular side, another time we need to solve this particular side, and then we need to combine them to get \( Y \). Here, so we'll start with the \( Z_1 \) equation. \( Z_1 \) is equivalent to \( X_1 \) negation of \( X_2 \). So, if \( X_1 = 1 \) and \( X_2 = 0 \), the negation of \( X_2 \) will become 1, so \( 1 \) and \( 1 \) is equal to 1 over here. So, only in this particular equation will the output be 1. In all other cases, it will be 0. This is the activation function: if the value of the net input is greater than some threshold value, the output is 1. Otherwise, it will be 0. So, we need to calculate the net input here. If it is greater than the threshold, it will be 1; otherwise, it will be 0. This is for \( Z_1 \), and then we need to calculate it for \( Z_2 \) at a later stage. Now, we need to initialize some of the variables, i.e., weights. \( W_{11} \) and \( W_{21} \) are initialized to 1. You can initialize them to anything; it is not hard and fast that we have to start with 1. We need to randomly initialize these particular weights here. Similarly, we need to initialize two more variables. The first one is the threshold. You have to set some threshold value because we are going to compare this net input with the help of the threshold. So we need to initialize it randomly. I have set it to 1 in this case. Similarly, we have to set the value of the learning rate. For this particular example, I have considered 1.5. You can consider anything again here. Now, I will start with the first input, that is this one, 0 0. The net input, that is \( Z_1 \), is equivalent to \( W_{ij} \cdot X_1 \). So, \( W_{ij} \) is \( W_{11} \) in the first case, and in the second case, it will be \( W_{21} \cdot X_1 \), and the second case will be \( X_2 \). So, this one is 1 here. So, \( 1 \times 0 \) is the first part, and this \( 1 \times 0 \) is the second part over here. We will get 0 as the net input. So, this net input is less than \( \theta \), which is 1 here. So, net input 0 is not greater than 1. Because it is not greater than 1, the output will be 0 here, and the expected output is also 0 here. No issues in this particular case. Now, I will start with the second input, that is 0 1. Again, I will put it in this particular equation, and I got the value as 1 as the net input. This net input is greater than or equal to 1 in this case. So, because of that, the output is equivalent to 1, but the expected answer or the target answer is 0 here. Because they are not the same, we have to update these particular weights. To update the weights, we have to use this particular equation: \[ W_{ij} = W_{ij} \text{ (old weight)} + \text{learning rate} \times (T - O) \times X_i \] So, \( W_{11} \) is equal to \( W_{11} \) (the previous weight was 1, that is what I have written here). The learning rate is 1.5, \( T - O \) is \( 0 - 1 \), and \( X_i \) is the input. In this case, \( X_1 \) is 1. So, if I solve this particular equation, I will get 0 as the answer here. Now, coming back to \( W_{21} \), that's the second weight. This one is 1 here, and previously it was 1, so I have written 1 here. 1.5 is the learning rate, \( 0 - 1 \) is the target minus the calculated output, and the input, \( X_i \), is 1 here. So, if I solve this one, \( 1 + 1.5 \times (-1) \), that will become \( 1 - 1.5 \), so it will become -0.5 over here. So, \( W_{11} \) is 1 only, but \( W_{21} \) has been changed to -0.5. Now, I will start with the first input here. This is the first input, and then I will calculate \( Z_1 \). \( Z_1 \) is equivalent to the same equation here, \( W_{ij} \times X_i \). If I solve this particular thing, I got 0 as the net input because the net input is not greater than the threshold (it should be greater than 1). Since it is not greater, we've got 0 here, and the expected output is also 0. No issues. I will start with the second input. If I put all the values in this particular equation, I got -0.5. This -0.5 again is not greater than 1. Because of that, I got output as 0 again. It is similar to the target, so there is no need to do the weight updation. I will consider the input as 1, 0, and then if I put all those things in this equation, I got 1 as the \( Z \) input or \( Z_{\text{in}} \). Now, this net input is greater than or equal to 1. So, 1 is greater than or equal to 1, so the output in this case is 1, and what is expected here? The same thing is expected. I will continue with the next input, which is 1, 1. If I put everything in this equation, I will get 0.5. So, 0.5 is not greater than 1. Because it is not greater than 1, the output is 0 here. So, you can say that all these particular examples were correctly classified with the help of these particular weights. That is, \( W_{11} \) is 1, \( W_{21} \) is -0.5, the threshold is 1, and the learning rate is 1.5 in this case. Now we will continue with the next function, that is \( Z_2 \), which is equivalent to \( X_1 \bar{X_2} \) or negation of \( X_1 \) and \( X_2 \) in this case. The truth table for this one is shown over here. If the first one, that is \( X_1 \) bar, is equivalent to 0, the negation of this will become 1, and \( X_2 \) is 1 in this particular case. \( Z_2 \) will become high. In all other cases, it will become 0 here. So, that has been shown over here. The activation function will remain the same. We need to find the optimal weights for this one, as well as the threshold and the learning rate for this particular part of the network in this case. Now I will continue with initializing the weights, that is, \( W_{12} \) and \( W_{22} \), where initialized to 1. Again, you can set any values randomly; we need to initialize here. Threshold is set to 1 here, and the learning rate is set to 1.5 over here. Same values I have taken. I will continue with the first input, that is, 0, 0. I need to calculate \( Z_2 \) in, that is, nothing but \( W_{ij} \times X_i \). If I put these particular values, that is \( W_{12} \) and \( W_{22} \), as well as this input, I will get 0 as the \( Z_2 \) in, because 0 is not greater than the threshold value, that is 1. 0 is not greater than 1, so I will get the output as 0 here. The same thing is expected. Now I'll continue with the second input. The second input is 0, 1. So, if I put all those values in this equation, I got \( Z_2 \) in as 1. And because \( Z_2 \) in is greater than 1 here, \( Z_2 \) in is greater than or equal to 1, you can say the output is 1. This output and the target are matching. Again, no issues. Now I will go with this particular third one, that is 1, 0. If you look at this particular 1, 0, we got the \( Z_2 \) in as 1. One is greater than or equal to 1, output is 1 here. But what is expected? The expected answer is 0 here. So what I need to do is, I need to use this particular weight updation equation and then we need to modify these weights here. So, \( W_{ij} = W_{ij} + \eta \times (T - O) \times X_i \), where \( W_{ij} \) is the old weight, \( \eta \) is the learning rate, \( T \) is the target, \( O \) is the actual output calculated, and \( X_i \) is the input here. Put those particular things in the equations like: - \( W_{12} = 1 \), learning rate is 1.5, target is 0, but calculated is 1 here, so \( 0 - 1 \times 1 \). If I solve it, it will become -0.5. Second time, this will become 0, and the answer will be 1 in this case. So, we got the updated weights, that is, the first one is -0.5 and the second one is 1. Here, now we will put this particular input again. If I put this particular 0, 0, I got \( Z_2 \) in as 0, which is not greater than or equal to 1 here, so the output is zero here. That's perfectly fine, and that is what is expected. When I continue with 0, 1, if I solve it, I got \( Z_2 \) in as 1. 1 is greater than or equal to 1, because of that, I got one here. One is the output, and the target is also 1 here. Similarly, next time I got -0.5, output is 0, expected is 0 here. The last one is 0.5. Again, 0.5 is not greater than 1 here, so output is 0, and expected is also 0 here. So, with respect to these weights, I am able to classify that particular data correctly in this case also. Now I will continue with the next one, that is \( y = Z_1 \, \text{or} \, Z_2 \). \( Z_1 \) is known to us, and \( Z_2 \) is known to us. If you go back and see this particular thing, \( Z_2 = 0, 1, 0, 0 \) and \( Z_1 = 0, 0, 1, 0 \); that is what we have already calculated. So, \( Z_1 \) I have written here, and then \( Z_2 \) I have written here. \( Y \) is what? \( Y \) is the OR between \( Z_1 \) and \( Z_2 \), that is, this one is 0, this one is 1, this one is 1, and this one is 0 here, because the OR will be high on, you can say that, if any one of those particular inputs is high. In this case, both are low. In this case, also both are low. So, we have to consider this particular input, and then we need to find out the value of \( V_1 \) and \( V_2 \) here. These are the inputs, and this is the output. If you look at this one, with respect to \( X_1 \) and \( X_2 \), we are expecting the same thing here. If these two things are there, we are expecting one over here. Now, again, we will initialize these two weights, that is \( V_1 \) and \( V_2 \), to 1 here. Threshold is 1, and the learning rate is equal to 1.5. I will start with the first input. I got the \( V_1 \) in as, or you can say \( Y_n \). Actually, \( Y_n \) is equivalent to 0. 0 is not greater than the threshold, that is, 1. So, the output is 0, and expected is also 0 here. Continue with the second one. I got \( Y_n \) as 1. 1 is greater than or equivalent to 1, so the output is 1, and the target is also 1 here. Coming back to the next one, that is, one zero. You can see here, this is one zero. I got the \( Y_n \) as 1, because it is greater than or equivalent to 1. The output is 1, and the expected is also 1 here. Now, again, one more input is there, that is, zero zero. Don't consider this one because we are talking about \( Z_1 \) and \( Z_2 \) here. This zero zero zero zero, because we have zero zero, \( Y_n \) is zero. Zero is not greater than or equivalent to zero, so the output is zero here, which is as expected. Now, with respect to \( V_1 \) and \( V_2 \), that is, if both the values are 1, we were able to classify this particular data successfully. So, that is the final weights for \( V_1 = 1 \), \( V_2 = 1 \), for this particular threshold and learning rate. So, finally, we got \( W_{11} \) as 1, \( W_{21} \) as -0.5, \( W_{12} \) as -0.5, \( W_{22} \) as 1.5, and \( V_2 \) as 1 in this case. So, putting all these particular values in this particular diagram, you will get the final perceptron rule for the XOR gate in this case.