Situation or Problem From Your Job (Part 2)

Question 4.1 Question: Describe a situation or problem from your job, everyday life, current events, etc., for which a clustering model would be appropriate. List some (up to 5) predictors that you might use. Answer: One clustering problem I’ve dealt with is trying to group retail franchises together to tailor our level of support to these stores. Any store can have a number of different attributes - meaning support may be needed for different areas of business in two stores in the same location. We didn’t know beforehand the true clusters of the stores, but wanted a data-driven process to guide us. We ended up clustering on: - store square footage online presence metrics population density of the census tract they were located in type of building and store set-up cost of leasing the store lot This project allowed us to develop sets of grouped retail stores - and allowed us to tailor our investment messages to each store’s particular attributes. It also let us understand what strong performance looks like for stores with different locations and physical constraints. Eventually with more supplemental analysis - the clustering allowed us to make network decisions and invest in stores to reach the optimal ‘tier’ for their unique attributes. Question 4.2 Question: The iris data set iris.txt contains 150 data points, each with four predictor variables and one categorical response. The predictors are the width and length of the sepal and petal of flowers and the response is the type of flower. The data is available from the R library data sets and can be accessed with iris once the library is loaded. It is also available at the UCI Machine Learning Repository (https://archive.sci.uci.edu/ml/data sets/Iris ). The response values are only given to see how well a specific method performed and should not be used to build the model. Use the R function kmeans to cluster the points as well as possible. Report the best combination of predictors, your suggested value of k, and how well your best clustering predicts flower type. Answer: Here are the steps to load the data and investigate some of the natural clustering of the data. In a real life application we would not have these answers to the correct cluster. This data here will be used to validate our final model. If we find a good clustering algorithm - our clusters should group together in similar clusters to the ‘right’ answers. At first glance it might looks like setosa has significant dispersion away from versicolor and virginica, especially with the Petal Length and Petal Width measurements. Other measurements look like there is overlap between species. It will be interesting to see if we can build a flexible enough clustering algorithm to pick up on this overlap. I examine the correlation between variables to determine if there is any overlap between the different attributes. I suspect that because we are dealing with similar measures, we may have highly correlated predictors. I also plot specific variables against each other to examine their correlation. The predictors Sepal Length and Sepal Length are highly correlated with each other. Sepal width seems to be the only predictor that is not highly correlated with the other measurements. Based on this Exploratory Data Analysis - I am going to try fitting the clustering model using both Petal measurements and the Sepal Length measurement. These highly correlated variables should give our model enough information to develop accurate clusters. In the last step I select the modeling variables and scale them for consistent magnitude. data("iris") # load the iris dataset iris_df <- iris %>% as_tibble() # investigate dataset str(iris_df) ## Classes 'tbl_df', 'tbl' and 'data.frame': 150 obs. of 5 variables: ## $ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ... ## $ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ... ## $ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ... ## $ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ... ## $ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ... # correlation plot - maybe we should eliminate correlated value? corrplot(cor(iris_df[,-5]), type = 'upper' , diag = F) # plot results - EDA ggplot(data = iris_df %>% gather(key, value, -Species), aes(x = key, y = value, col or = Species)) + geom_boxplot() + geom_jitter(alpha = .3, pch = 21) + theme_few() + xlab('Measurement') + ylab('Inches') + labs(title = 'Iris Dataset Visualization', subtitle = 'Species shown in color by measurements') # quick look at correlation #1 ggplot(data = iris_df) + geom_point(aes(x = Sepal.Length, y = Sepal.Width, color = Species)) + theme_few() + xlab('Measurement') + labs(title = 'Sepal Width vs. Sepal Length', subtitle = 'Attributes grouped by Species') + geom_smooth(data = iris_df, aes(x = Sepal.Length, y = Sepal.Width), method = 'lm', se = F, color = 'dark grey') # quick look at correlation #2 ggplot(data = iris_df) + geom_point(aes(x = Petal.Length, y = Petal.Width, color = Species)) + theme_few() + xlab('Measurement') + labs(title = 'Petal Width vs. Petal Length', subtitle = 'Attributes grouped by Species') + geom_smooth(data = iris_df, aes(x = Petal.Length, y = Petal.Width), method = 'lm', se = F, color = 'dark grey') # modeling data frame - scale our inputs! iris_mod <- iris_df %>% dplyr::select(Sepal.Length, Petal.Width, Petal.Length) %>% scale(.) %>% as_tibble() Answer: To run the k-means clustering algorithm we input the chosen predictors into the kmeans function in the base R stats library. I set up a for loop to input possible values of K from 1 to 15 and will extract the Total Within Sum of Squared Distance to determine the optimal K. I then plot the Total Within Sum of Squared Distance versus the values of K. This is the ‘elbow’ plot that can determine which K is best. Ideally, we’d choose the point K that starts a chain of diminishing returns in Total Sum of Squared Distance. Interestingly - the graph below indicates the best K could be anywhere for 3-6 clusters. With a real life problem we would not know the true answers to compare to. To compare to the actuals I will use K = 3 I then re-fit the k-means model with K = 3 and observe the summary statistics. Using the cluster centers we can draw inference to try to understand why certain clusters are fit by the model. We can also see difference performance metrics given by the model. We are also provided with the predictions for each cluster in the data set. We will use these predictions to compare to the ‘true’ answers. The table shows that our K-means model accurately clustered 86% of the observations. Our model performed well for the setosa and versicolor species but struggled with the virginica species. This might have been accepted as our exploratory data analysis showed separation from setosa and the other species, but overlap between versicolor and virginica. Possibly there were quite a few ‘edge cases’ of virginica that the model could not separate with the information provided. # set up kmeans - will iterate through optimal K by using within cluster distance possible_K <- as.list(seq(from = 1, to = 15, by = 1)) # blank list to store for loop values within_dist <- list() for (i in seq_along(possible_K)) { k = possible_K[[i]] kmean_fit <- kmeans( x = iris_mod, centers = k, nstart = 20 ) within_dist[[i]] = data.frame(between = kmean_fit$betweenss, total = kmean_fit$totss, total.within = kmean_fit$tot.withinss) } (distance_df <- data.frame( ss = reduce(within_dist, rbind), possible_K = reduce(possible_K, rbind) ) %>% mutate(ratio = (ss.between / ss.total)) %>% ggplot(aes(x = possible_K, y = ss.total.within)) + geom_line(color = 'dark grey') + geom_point(color = 'black') + theme_few() + ylab('Total Sum of Squared Distances') + xlab('Possible K Values') + labs(title = 'Iris K-Means Clustering: K Optimization', subtitle = 'Total Within Sum of Squared Distance by possible K values' ) + geom_vline(xintercept = 3, linetype = 'dotted') ) # fit model with our best k (kmeans_bestfit <- kmeans( iris_mod, centers = 3, nstart = 100 )) ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## K-means clustering with 3 clusters of sizes 51, 41, 58 Cluster means: Sepal.Length Petal.Width Petal.Length 1 -1.01370137 -1.2313076 -1.280215 2 1.22010208 1.0984134 1.073846 3 0.02887215 0.3062369 0.366608 Clustering vector: [1] 1 1 1 1 1 1 1 [36] 1 1 1 1 1 1 1 [71] 3 3 3 3 3 3 2 [106] 2 3 2 2 2 2 2 [141] 2 2 3 2 2 2 2 1 1 2 2 2 1 1 3 3 2 1 1 3 3 3 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 3 1 2 3 2 1 3 2 3 1 2 3 2 1 3 3 3 1 3 3 2 1 3 3 2 1 3 3 3 1 1 3 3 1 3 3 2 1 3 3 2 1 3 3 2 1 3 3 2 1 3 3 2 1 3 3 3 1 3 3 3 1 3 2 2 1 3 3 2 1 3 2 2 1 3 2 3 1 3 2 2 1 3 3 2 1 3 3 3 1 3 3 3 1 3 2 2 1 3 3 2 1 3 2 2 1 3 2 3 1 3 2 2 Within cluster sum of squares by cluster: [1] 12.32795 23.04022 27.25275 (between_SS / total_SS = 86.0 %) Available components: [1] "cluster" "centers" [5] "tot.withinss" "betweenss" [9] "ifault" "totss" "size" "withinss" "iter" # quick look at the model's clusters kmeans_bestfit$cluster ## [1] 1 ## [36] 1 ## [71] 3 ## [106] 2 ## [141] 2 1 1 3 3 2 1 1 3 2 3 1 1 3 2 2 1 1 3 2 2 1 1 3 2 2 1 1 2 2 2 1 1 2 2 2 1 1 3 3 2 1 1 3 3 3 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 2 1 1 3 3 1 2 3 2 1 3 2 3 1 2 3 2 1 3 3 3 1 3 3 2 1 3 3 2 1 3 3 3 1 1 3 3 1 3 3 2 1 3 3 2 1 3 3 2 1 3 3 2 # quick look at the model's final centroids - we can draw inference from here but n eed to unscale kmeans_bestfit$centers %>% as_tibble() ## # A tibble: 3 x 3 ## Sepal.Length Petal.Width Petal.Length ## -1.28 -1.23 ## 1 -1.01 1.07 1.10 ## 2 1.22 0.367 0.306 ## 3 0.0289 # check accuracy against the original data table(kmeans_bestfit$cluster, iris_df$Species) ## ## ## ## ## setosa versicolor virginica 50 1 0 0 5 36 0 44 14