Majority voting Why Majority Vote? assume n independent classifiers with a base error rate E here, independent means that the errors are uncorrelated assume a binary classification task assume the error rate is better than random guessing (i.e., lower than 0.5 for binary classification) VEE €1, €2,..., €), €₁ <0.5 Why Majority Vote? assumen independent classifiers with a base error rate C here, independent means that the errors are uncorrelated assume a binary classification task assume the error rate is better than random guessing (e lower than 0.5 for binary classification) Veel). <0.5 The probability that we make a wrong prediction via the ensemble if k classifiers predict the same class label P(k) = ()(1-(1-6)-k k> [n/2] Majority Voting Classifier Training set Classification models Predictions New data Final prediction Voting mode h(x), h(x),... h(x)) where h(x) = Why Majority Vote? The probability that we make a wrong prediction via the ensemble if ik classifiers predict the same class tabel P(k) = binomial(n,k) * c ^ k * (1 - e) ^ (n - k) k> [n/2] Ensemble error: epsilon ens = sum k ^ n binom n k e^ k (1-e)^ n-k cumulative prob, distribution) epsilon ens = sum k=6 ^ 11 binom 11 k 0.25^ k (1-0.25)^ 11-k =0.034 epsilon ens = sum k ^ n binom n k epsilon^ k (1- epsilon)^ n-k Ensemble error Base error 08 0.2 0.0 0.0 0.2 0.4 0.6 Base error 0.8 1.0 "Soft" Voting hat y = argma*x_{j} sum i=1 ^ n w i p ij predicted class membership p i, j : probability of the ith classifier for class label j w_{i} optional weighting parameter, default w_{i} = 1 / n forall w_{i} in\ w 1 ,...,w n \ "Soft" Voting hat y = argma*x_{j} sum i=1 ^ n w i p i,j Binary classification example j \in \{0, 1\} h_{i}(i \in \{1, 2, 3\}) h_{1}(x) -> [0.9, 0.1] h_{2}(x) -> [0.8, 0.2] h_{3}(x) -> [0.4, 0.6] "Soft" Voting hat y = argma*x_{j} sum i=1 ^ n w i p ij Binary classification example j \in \{0, 1\} h 1 (i in\ 1, 2, 3 |) h_{1}(x) -> [0.9, 0.1] h_{2}(x) -> [0.8, 0.2] h_{3}(x) -> [0.4, 0.6] p(j=0|x)= 0.2 * 0.9 + 0.2 * 0.8 + 0.6 * 0.4 = 0.58 p(j=1|x)= 0.2 * 0.1 + 0.2 * 0.2 + 0.6 * 0.6 = 0.42 hat y = argma*x_{j} \ p(j=0|x),p(j=1|x)\ FreeClassifier from sklearn.selection part creat sklearn immert datasets Classifier Code tresses, train, lest latelsesferaserasifier le Test Accuracy: 18.27 & ect.corel test testi