Statistical Thinking Applications

Statistical Thinking Applications: Statistical Thinking Definition: philosophy of learning and action based on the fundamental principles that “all work exists in a system of interconnected processes” and “variation exists in all processes” ○ All work occurs in a system of interconnected processes ○ Variation exists in all processes ■ If there were no variation: ■ Processes would run better ■ Products would have the desired quality ■ Service would be more consistent ■ Managers would manage better Statistical Thinking Strategy: ● Subject matter knowledge ○ The first step of applying statistical thinking is “identifying, documenting, and understanding the business process itself” (Hoerl et al., 2020). ○ Statistical thinking does not begin with data; it begins with subject matter knowledge ● Designing a process: “set of connected activities in which “inputs” are transformed into “outputs” for a specific purpose” (Hoerl et al., 2022) ○ Usually the problem is in the process; to improve results, the process must be improved. Applications of statistical thinking can help whatever situation there is by reducing the variation and thereby improve the process ● Reducing variation ○ The objective of statistical thinking, after attaining subject matter knowledge, is to understand and reduce the variation, which is the key to success (which thus improves the process) ● Kinds of variation: systematic, random, outlier ● Improving the process: ○ Data are the link between the process and the improvement activity, and when analyzing the data, we revisit the subject matter knowledge to understand anything that does not make sense ● Sequential approaches ○ “Each time data were gathered, something new was learned. Some questions were answered, but new ones were invariably raised. For example, the discovery that reduced advertising produces increased sales was incorporated into subsequent experiments to improve advertising effectiveness well beyond what anyone had originally thought possible” (Hoerl et al., 2020). ○ “Statistical thinking is a sequential process , and what we see the first time around may suggest additional data collection or analyses” (Hoerl et al., 2020). ● Value-added vs non value added work Problem solving versus process improvement Problem solving: addresses specific problems that are not part of the process’ normal behavior; something went wrong and needs to be fixed 1. Determine the scope of the problem 2. Identify the root causes 3. Select, implement, and standardize corrections ● The problem-solving framework incorporates the key elements of the statistical thinking strategy: ○ Improves results by improving the process ○ Uses the synergy between subject matter knowledge and data ○ Diagnoses and reduces variation ○ Uses a sequential approach Process improvement: series of activities aimed at fundamentally improving the performance of the process - flowcharting - collecting data to assess current performance - identifying areas where process could be improved - changing the process to implement improvement ideas - checking the impact of improvement efforts Process improvement is required to take a stable process (i.e., one with only random variation) to a new level. Problem-solving is required to eliminate the outliers due to special causes in an unstable process.” Understanding measures of central tendency and variation o Mean (average), median, mode, range and standard deviation o Note: You will NOT need a calculator for this quiz; any calculations required will be doable by hand ● Mean: average o Advantage of mean: easy to calculate and understand o Limitation of mean: sensitive to outliers o About 5 to 10 data points are required to get a reasonable estimate of the average ● Median: middle value or 50th percentile o average 2 middle values when n is even o numbers must be put in order from smallest to largest o Advantage: insensitive to outliers o Disadvantage: harder to interpret ▪ Sometimes, extreme values are important ● Mode: most frequently occurring number o can have no mode o o can have multiple modes Mode advantage: easy to understand ▪ ● however, can be misleading; may not be in “middle” Variation o range: max-min ▪ Advantage: easy to calculate and understand ▪ Disadvantage: dependent on n (number of data points) ▪ About 20-30 data points are required to get a reasonable estimate of the level of variation (standard deviation) ● Variance: s^2 (or sigma squared) o Measure of how much each data point varies from the mean o The “typical” deviation of individual points from the mean Random, systematic, and outlier sources of variation Random variation: unpredictable & unexplainable ● Always present ● Many causes ● Caused by within process ● Causes have minor impact ● Not identifiable nor predictable Systematic variation: predictable & explainable ● Always present ● Few number of causes ● Caused within the process ● Causes have moderate impact ● Identifiable and predictable ● Our objective is to figure out the key systematic sources of variation and manipulate them to improve process performance. Outlier variation: consists of extreme data points that don’t fit the pattern of the rest of the data ● Intermittent presence ● Tend to have the most impact so addressing them first typically will result in dramatic improvement ● Not predictable but explainable after the fact SIPOC model for identifying and interpreting processes ● SIPOC (in theory) ○ start with outputs (what is produced?) ○ then customer (who cares?) ○ then process (how specifically do you produce this output?) ○ ○ ○ then inputs (what do you need for this process?) then suppliers (where do you obtain your inputs?) OCPIS (in practice) Value-added versus non-value added work ● Non-value adding work: does not actually add value ○ would not be required in a “perfect process” ● Value adding work: “adds value” to the output ○ someone will pay for this work Data quality and pedigree; what makes sampling questionable ● ● ● ● ● ● ● ● ● ● Data quality: the degree to which data satisfies the needs of the analyst ○ context dependent ○ exists on a continuum ○ Involves the questions: ■ Are the data right? ● correct-free from error ■ Are these the right data? ● i.e. appropriate for my purposes Data pedigree: documentation of the origins and history of a data set, including its technical meaning, background on the process that produced it, the original collection of samples, measurement processes used, and the subsequent handling of data, including any modifications or deletions, up to the present. ○ Pedigree provides insight into who, when, where, and how the data were collected, thereby documenting its strengths and limitations Large sample size does not necessarily equate to high quality data Sample size can never make up for flaws in data quality Problem: Not actually random sampling and the sample is biased toward something ○ Random sampling ensures that the sample data we analyze are conceptually representative of the total set of objects, or population of interest Non-response bias (surveys) ○ Two techniques to reduce this: ■ providing some incentive to respond ■ Follow-up Leading questions, sensitive questions in surveys It is desirable to have each person have an equal chance of being included in the sample to accurately represent the population and avoid bias in the sample. Sampling error/random variation can be controlled by using an adequate sample size ○ Non-sampling error (inadequate or inappropriate sampling frames, poorly designed questions, faulty survey execution) cannot be controlled in this way Margin of error varies because of variance in sample size. As the sample size decreases, the margin of error increases. Types of Questions to Expect: ● Perform a SIPOC analysis for your process to wake up and get ready for class in the morning o Output: Time it takes for me to wake up and get ready o Customer: Myself o Process: o Inputs: phone alarm, bed, clothes o Suppliers: Samsung, dorm room, store ● If the average score on the quiz turns out to be 80, with a standard deviation of 5, explain in plain English how to interpret the standard deviation in this case. Do not define the standard deviation mathematically; rather explain how to interpret this particular value of 5. o In plain English, the standard deviation of 5 in this case tells us how much individual quiz scores typically vary from the average score of 80. A standard deviation of 5 means that most scores are within 5 points above or below the average. So, if a student scores one standard deviation above the average, their score would be around 85. Conversely, if they score one standard deviation below, their score would be around 75. The larger the standard deviation, the more spread out the scores are from the average. In this scenario, a standard deviation of 5 indicates a moderate amount of variability in quiz scores around the mean of 80. ● Suppose I wish to improve my golf game. Explain in one paragraph why subject matter knowledge might be important in my improvement efforts. Also, explain in one paragraph why a sequential approach is more likely to succeed than a “one-shot study”. o Subject matter knowledge is crucial for improving your golf game because understanding the fundamentals of golf, such as proper swing mechanics, club selection, and course strategy, provides a foundation for effective skill development. Knowing the principles behind each aspect allows you to identify specific areas for improvement and tailor your practice routines accordingly. For instance, if you understand the mechanics of a proper golf swing, you can focus on refining your technique and address any specific challenges you may encounter. Without subject matter knowledge, improvement efforts might be based on trial and error, making progress slower and less targeted. o A sequential approach is more likely to succeed in golf improvement than a "one-shot study" because mastering the game involves a combination of skills that build upon each other. Golf requires a multifaceted set of abilities, including putting, chipping, driving, and course management. Taking a sequential approach allows you to systematically work on and refine each skill, gradually progressing from basic to advanced levels. This methodical progression helps reinforce the fundamentals and ensures a solid foundation before moving on to more complex aspects of the game. In contrast, a "one-shot study" might result in a scattered and less effective learning experience, as it overlooks the step-by-step nature of skill development in golf. ● Given the following data collection plans, what data quality issues do you see? (Sampling plan follows here) ● Give examples of random, systematic, and outlier variation in the following scenario. (Scenario follows here) Process improvement is required to take a stable process (i.e., one with only random variation) to a new level. Problem-solving is required to eliminate the outliers due to special causes in an unstable process. Basic Graphical Tools: ● Run chart: plot of data over time (if it is in sequence) ○ Usually should be the first graphical tool applied ○ Time is a surrogate for any lurking variable that might have changed overtime and caused systematic variation ○ Looking for several things: ■ Stable over time? ■ Trends? ■ Outliers? ■ Anything surprising? ○ Advantage of run chart: ■ Simple ■ data over time ○ Downside of run chart: ■ subjective and up to individual interpretation ● ID: inter-relationship digraph Figure 6.30 “5 arrows coming in, not coming out” ○ effect ● “0 arrows coming in, 5 arrows coming out” ○ Cause ● ● Pareto Chart ● ● ● ● ● ● ● ● For nominal data Identify and illustrate most common “issues” Bar chart ordered from most common (mode) to least common Figure 6.17 Nominal data; they are categories Vertical: how many times it happened Ordered from largest to smallest Thing across the top: ogive: cumulative count ○ where the derivative starts to go to 0 ○ initially a steep angle and then flattens out ○ Scale for the ogive on the right ○ ogive is converted to a percentage ● If it’s not in order from largest to smallest, then it’s not a pareto chart ● JMP: measure → pareto plot Histograms ● For integer/interval/ratio data ● Show: ○ central tendency (mean, median mode) ○ degree of variation ○ shape of distribution ● Disadvantage ○ Hides time dimension ○ no discussion of time when the data was collected ● Bi-modal distribution ○ ex: heights of all Union students ● Skew: the direction of the tail (the low data) ● Basic → distribution → Y Column ○ lower triangle → histogram options → click on vertical to make it horizontal Stratification ● Breaking the data down into logical categories ● helps us to identify systematic sources of variation ● Box Plots ○ Figure 6.22 ○ A lot more meaningful if we’re trying to compare several things ○ Graph builder → nominal data on horizontal → continuous data on vertical → click on box plot graph at the top ○ central tendency, degree of variation, and shape of distribution ● Scatter Plots ○ X-Y graph ○ Advantages: ■ Simple ■ No assumptions ■ Allows us to visualize relationships and see patterns we didn’t expect ○ Both nonlinear and no relationship scatter graphs have a correlation coefficient 0 ○ graph → graph builder → ■ Multivariate gives you a scatterplot matrix (JMP) ● no nominal data Correlation Coefficient ■ ■ ■ ■ ■ Quantifies the strength of linear association between two variables ● -1 less than equal to R less than equal to 1 ● 1: perfect positive relationship ● -1: perfect negative relationship ● 0: no relationship visible Weak positive/weak negative: 0.4 Strong: 0.9 Nonlinear/no relationship: both zero Correlation does not imply causation. ● There could be a lurking variable ● Causation requires subject matter knowledge Control Charts- IR ● Run chart with statistically based limits that are intended to document the range of random variation in the data ○ Any points outside the limits represent outliers and/or systematic variation. ○ Between the two red lines: range of random variation (individual chart) ■ points bouncing around within the limits ■ green line: mean ○ We want the limits to only show random variation ○ Complicating factor: data may be contaminated with outlier and systematic variation ○ Moving graph: measure of variation from point to point ○ We generally do plus or minus 3 SD (99.7%) ■ assuming perfect bell shaped distribution ○ We get the mean and go up and down 3 standard deviations (individual control chart) ○ Range due to random variation: based on what you identified as outliers ■ using the control chart Control charts - run charts with statistical limits ● documents range of random variation ● used to quickly detect outlier & systematic variation ● mean +/- 3 st. dev makes logical sense ○ approx. 99.7% of data ○ Problem: how to estimate random variation only if the data may be contaminated by outliers or systematic variation? ● Answer: moving range ○ don’t compare to average value of y, but only to the points right before and after Naive approach ● average value of y from JMP ● standard deviation from JMP ● average +/- 3 standard deviations Data and Process Tools: ● Flowchart ○ Graphically depicts the flow of a process ○ Helps everyone grasp the big picture of the overall process ● Data pedigree ○ To document the story behind the data set; to enable those analyzing the data to understand the background and context of the data ● Checksheet ○ Allows easy, consistent, and accurate collection of data ● Survey ○ Used to gather “soft” and “hard” data Knowledge-Based Tools: ● Tools that utilize our current subject matter knowledge ● Five Whys ○ Used to probe beneath the symptoms of a problem to get down to the root causes ○ Typically used in improvement projects when attempting to identify potential root causes of issues ○ Common sense ● Cause and Effect Diagram ○ Purpose: to document the relationship between observed effects and their causes in decreasing levels of detail ○ Used in improvement projects to document relationships between input or process variables and output variables ○ To model systematic variation Text Analysis: (Analytical) ● “Text analysis is becoming a greater and greater part of statistical thinking, in large part because of organizations such as Google and Amazon, which analyze massive amounts of text every day” (Hoerl et al., 2020). ● The tools are narrowly focused on generating ideas in the form of text and then analyzing the text to find the structure within it. ● Brainstorming ● Interrelationship Digraph ○ Purpose: to evaluate the cause and effect relationships between issues to identify which are the drivers and which are the effects ● Five Why’s: Ask why? Up to 5 times to go from symptom to root cause ● Tools to document our subject-matter knowledge ○ Not based on data ● The output will only be as good as our subject matter knowledge ● Cause and effect Diagram ○ Also known as Ishikawa diagram or fishbone diagram ○ Start with an “effect,” such as poor performance and documents causes at finer and finer levels of detail ○ Figure 6.12 example ● FANG: ○ Facebook, Amazon, Apple, Netflix Google ○ Excel at data acquisition , processing and analysis ● Most data now is text versus #’s ● Text: nominal data ● Question: What specific steps could Union College administration take to enhance the overall college experience for students? ● Structured brainstorming Rules: ○ Clarify specific question to be answered (in writing): to make sure all working on same problem ○ Give people time to think: diversity in personalities ○ Write down answers: level playing field and efficiency ■ (post-it notes work well) ■ At least a noun & verb; no more than a phrase ■ One Post-It per idea ■ No debate or discussion until all ideas are captured ● Affinity diagram ○ To group text similar in nature (have affinity for one another) ○ Analogous to calculating a mean in data analysis ○ Rules for team diagrams: ○ Everyone groups ideas at the same time ○ No talking or gesturing ○ Group ideas that seem similar to you ○ People can re-use Post-it notes ○ Continue until movement stops ● Prioritization ○ Two simple approaches: ■ Multi-voting ■ useful when the ideas are mutually exclusive options, or are not related ■ Many options for multi-voting ■ ex: voting for everyone but in a specific order/rank ■ Use n/3 (3 is an arbitrary number) or other % of votes ■ “Dot Method” - similar to n/3 but place the dots in any manner Modeling & Regression: Models: a generalized, hypothetical description, often based on analogy, used in analyzing and explaining something ● hypothetical description: not reality ● even though they are hypothetical, they can be useful in analyzing and explaining ● we can only model systematic variation Many Types of Models ● Two major categories: ○ Empirical: data-based primarily ■ most statistical models are empirical ○ Theoretical (deterministic) ■ Primarily based on subject matter theory ■ Ex: models in physics, chem, bio textbooks Why Do We Need Models? ● ● ● ● Predict future performance Measure the effect of changes we might introduce Control, manage, or improve the process Deepen our understanding of the process Regression Models ● Empirical, data-based ● Helps us smooth out variation and identify predictive relationships in many dimensions ● Process variables in regression model do not have to be ratio or interval scale b1 is a measure of how much systematic variation there is Where do b0 + b1 come from? ○ Mathematical criterion for “best” line to fit the data ○ We use “least squares” ● JMP regression: ○ Fit model → Fit model → y variable (MPG) ○ larger box: horsepower ○ change emphasis to minimal report ○ R^2 value: tells us what proportion of variation in y can be explained by the model ■ In this case, 90% of the variation in y is systematic ● ● Still using horsepower example: ● With the log scale, the extreme values tend to be pulled down and pulled over ○ kind of like a “square root’ scale ○ makes fitting a regression line and interpreting random variation easier ● Errors: difference between points and regression line ● R^2 is an index number (number with no units) ○ bounded by 0 and 1 ○ proportion of variation of y that can be explained ● Root Mean Square Error (RMSE): 2.19 miles per gallon ○ This is in the units of y (miles per gallon) ○ This is our estimate of the standard deviation of the residuals, or errors that fall above and below the regression line from our model ■ (i.e.) random variation ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ○ RMSE tells us how accurately we can predict y ■ we can predict most of the data within plus or minus 2.2 ■ 2/3 of the time plus or minus 2.2 SD ■ 95% of the time within plus or minus 4.4 The variation of the data points above and below the regression line: random variation Observations (or Sum Wgts): number of data points Total variation = systematic (explained) + random (unexplained or error) + outlier (assume 0 for now) How can we quantify the variation in y outside of any model? ○ range is possible ○ standard deviation ○ variance If you have no x variables, y bar is the most predictable Regression line fits better than the y bar line Explained (systematic) variation From where the model predicts to the actual data point: that’s random/unexplainable variation Sum of squares: column of variation Total sum of squares: sum of all the variation Model with large amount of random variation can be improved: ○ lurking variable ○ add more inputs Error sum of squares: random variation Model sum of squares: systematic variation Total sum of squares: total variation Mean square error: you have to know degrees of freedom: effective sample size ○ you divide by this number based on source R^2 number comes from dividing the model sum of squares by total sum of squares Three key metrics for regression: ○ R^2: proportion of systematic (explained) variation (index #) (overall thumbs up, thumbs down) ■ If R^2 is not good, that means there are a lot of lurking variables ○ RMSE: st. deviation of random (prediction) ■ how accurately I can predict ■ units of y ■ relates to precision ■ tells me how much individual data points may vary above and below the regression line ○ t/p: tells us if we can be confident that b1 (the slope) does not equal 0 (insight) (i.e. x affect y) absolute value of t greater than 2, p<0.05 ○ If b1=0, then x has no effect on y. Even if x has no effect on y, we won’t get perfectly 0 because there’s random variation in the data. ■ ■ ● ● ● ● Figure 7.3: Regression Model Building Process If we have the correct model, y hat should account for all systematic variation. Therefore e(i) (sample residual) should only have random variation Random ○ unexplainable ○ can’t be predicted ○ “noise” ● Therefore, if we plot e(i) in any way, the plots should look random, with no trends, cycles, patterns, etc. ● Figure 7.11: Sample plots of residuals vs. predicted values