Performance Metric

Performance Metrics Module 10.5 Krishnan Raghunathan / Linsey Rubenstein Michael Miller, Amazon Presentation for: ESD.60 – Lean/Six Sigma Systems MIT Leaders for Manufacturing Program (LFM) Summer 2004 These materials were developed as part of MIT's ESD.60 course on "Lean/Six Sigma Systems." In some cases, the materials were produced by the lead instructor, Joel Cutcher-Gershenfeld, and in some cases by student teams working with LFM alumni/ae. Where the materials were developed by student teams, additional inputs from the faculty and from the technical instructor, Chris Musso, are reflected in some of the text or in an appendix 1 Overview ¾ Learning Objectives ¾ Understand how metrics drive behavior and performance ¾ Understand the importance of aligning metrics with overall strategy ¾ Understand basic differences between traditional and lean metrics ¾ Session Design (20-30 min.) ¾ Part I: Introduction and Learning Objectives (1-2 min.) ¾ Part II: Lesson 1 and Exercise (3-5 min.) ¾ Part III: Balanced Scorecard and Lesson 2 (3-5 min.) ¾ Part IV: Lesson 3. Differentiate output vs. throughput metrics (5-7 min.) ¾ Part V: How does key Lean concepts apply to devising metrics (3-5 min.) ¾ Part VI: “Disconnects,” (5-7 min.) © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 2 Part V: Conclusion 2 Measure the Right Thing Exercise ¾ After LFM you take a position as a research analyst for Cost-Cutting Air (CCA), a new airline upstart. After running a Pareto Analysis (you are psyched to have the opportunity to use the tools you learned in your LFM Lean Class) you determine that fuel consumption cost is one of your highest controllable costs. You also know that pilots can minimize fuel consumption by taking measures such as turning off the air conditioning when passengers are not on the plane. This is clearly “waste/muda” that can be eliminated… and everyone knows that waste is not an option in a business where every cent counts. You want to impress your new boss, Jimmy Lean and show him how your b-school education really differentiates you from the other young motivated chaps. What metrics might you recommend to encourage pilots to reduce costs? How might you use this metric to reward CCA’s pilots? © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 3 Part V: Conclusion 3 Metrics Drive Behavior ¾ Based off of a true story from Continental Airlines after bankruptcy in 1990’s ¾ Cost cutting became the major company strategy ¾ Airline rewarded pilots for keeping fuel consumption low ¾ Behavior - Pilots skimping on air conditioning and flying more slowly ¾ Performance - Unhappy customers and behind schedule flights ¾ Results - Valuable customers moved on to competitors Lesson 1: “What gets measured gets done.” Metrics drive behavior, both good and bad. © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 4 Part V: Conclusion 4 Devising The Right Metrics ¾A Metric system should be comprehensive and focus all business activities on customer requirements and organizational objectives ¾Establishing a Balanced Scorecard ¾Define the Goals and Strategy (Listen to the Customer) ¾Analyze strategy for “Critical Performance Variables” (Understand the Job) ¾Link outcome and performance drivers measures (Understand the interrelationships) ¾Work out how to measure its progress to the goals. (Understand the cause-and-effect linkages) ¾Applied at both Enterprise and Shop floor level © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 5 Part V: Conclusion •Balanced Scorecard is a set of financial and non - financial measures that measures performance against all key stakeholder objectives •The points in blue tries to simplify the balanced score card concept to shopfloor •Know your internal customers needs (next station in the line) •Understand what you can do in your job to serve them better •Devise metrics around it. 5 Amazon’s Corporate Score-card Key Goal: Make online shopping preferred mode for all types of goods Fast and Free shipping for all types of products ¾ Customer ¾ Failed Fast Track ¾ Order cycle time mean and standard deviation ¾ Operations ¾ Received and Shipped units and backlog ¾ Ex (S&OP adherence) ¾ Cost/Unit ¾ Throughput per labor hour ¾ Units shipped per labor hour ¾ Safety ¾ Lost Time Incidents and Rate ¾ Record-able Incidents and Rate ¾ Quality ¾ Inventory Record Defect Rate ¾ Other Financial and Vendor negotiation metrics Lesson 2: Metrics must be holistic and align with the business strategy © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 6 Part V: Conclusion Scorecards can be applied for day to day operations as well •Shop floor score cards • Example of balance scorecard at shop floor: Amazon warehouse workers has to maintain both a good pickup time (orders picked from inventory) and cycle time per order and they both have opposing needs. 6 Throughput vs. Output Metrics ¾ Throughput metrics: Measures the amount of work done per unit time. Aimed at optimizing individual performance. ¾ Output Metrics: Measures the effectiveness in delivering required demand. Aimed at optimizing the flow across stations. ¾ Example: ¾ Quality training: No. of students trained vs. Impact on job after training ¾ Vantage Point in Value Stream (One man’s throughput is another’s output) ¾ Manufacturing Examples: ¾ Availability Vs. OEE (Overall Equipment Effectiveness) ¾ Overall production rate Vs. Build To Schedule © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 7 Part V: Conclusion 7 Traditional vs. Lean Metrics Complex, low volume assembly in aerospace TRADITIONAL LEAN ¾ Jobs behind schedule metrics ¾ Focus on accountability and individual performance ¾ Assumes every job is equally important ¾ Assumes individual efficiency drives overall performance ¾ Flow metrics ¾ Focus on global rather than local optimum ¾ Assumes some jobs more critical than others ¾ Assumes team drives overall performance ¾ Behavior using traditional metrics ¾ Perform “easy” jobs first to improve metric (temporarily) ¾ Out-of-sequence work ¾ “I completed my work…why should I help someone else” ¾ Focus on every problem ¾ Behavior using lean metrics ¾ Work jobs in optimal sequence ¾ Identify gaps in skills ¾ Teamwork ¾ Focus only on problems that impact overall performance © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 8 Part V: Conclusion 8 Lean Metrics – An Example Complex, low volume assembly in aerospace Critical Chain C1 C2 C3 C4 Milestone 1 Milestone 2 Milestone 3 Milestone 4 Milestone 5 Milestone 6 Milestone 7 Milestone 8 Milestone 9 Feeder Chain F1 F2 F3 F4 F5 Unit 1 Unit 2 Unit 3 Unit 4 Reaction is necessary! Should we react here? Lean metrics help management make decisions…. © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 9 Part V: Conclusion 9 This looks easy? Why doesn’t everyone go lean? Manage People Measure individual performance Core Conflict must be broken! Build Product Manage Flow Don’t measure Individual performance Lesson 3: Lean metrics optimize global performance © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 10 • Create lean teams and devise metrics for the team as opposed to individual 10 Lean Metrics - Characteristics ¾ Apply key lean concepts to metrics ¾Customer Focused ¾ Holistic Metrics aligned with Strategy ¾Eliminate Waste to Create Value ¾ Throughput vs. Output measures ¾Done at All levels in the organization ¾ Communication and Buy-in ¾Continuous Improvement ¾ Re- Assessment and Results © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 11 Part V: Conclusion •Points three and four are equally important. Covered in the backup slides to some extent. 11 Disconnects ¾ Technical Factors ¾ Social Factors ¾ Reliable and consistent metrics ¾ Learning vs. Reporting ¾ Acceptance of measures ¾ Vertical and horizontal alignment ¾ Overcoming inertia ¾ Misuse, manipulation, and gaming ¾ Adequate resource commitment ¾ Corporate values becomes important here ¾ Culture and integrity indispensable to success. ¾ Relevancy of metrics over time ¾ Right incentives for performance © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 12 Part V: Conclusion Reliable Metrics : Accurate, Actionable and Timely Consistent Metrics Avoid confusing requirements and expectations internal and external - Balanced Score card Vertical and Horizontal alignment “Catchball” process to deploy metrics to all levels. Adequate Resource commitment Manpower, Money, Facilities and Training Evaluate relevancy over time Life Cycle management for metrics Learning vs reporting Process indicators as diagnostic data, but do not optimize the system to these measures. Acceptance of measures Don’t know why Don’t know how metrics fit into big picture (correlation to end result) Overcome Inertia Accountability issues Metrics are mis- used, manipulated, and gamed They can be manipulated and used to justify present processes Right Incentives for performance 12 Bibliography ¾ November 2003, Sisk, Michael “Are the Wrong Metrics Driving Your Strategy” Harvard Business Review ¾ November 1999, “Successful Performance Measurement: A Checklist” Harvard Management Update ¾ August 1997, Baker, Bill “Driving Strategy Objectives Through Your Performance Metrics” IQPC, Chicago ¾ June 2001, Robert, Nicole A. “Design and Analysis of an Enterprise Metric System” Sloan thesis ¾ MIT Sloan Professor, Joel Cutcher-Gershenfeld © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 13 13 Appendix: Instructor’s Comments and Class Discussion on 10.5 ¾ Metrics must be assigned a level of priority ¾ Some are necessary but insufficient, others are strictly performance-based ¾ In continuous improvement situation, metrics will get tighter over time ¾ First metrics should deal with stability, second metrics should deal with flow ¾ Metrics are only one factor in organizational behavior. Other important factors: ¾ Character of employees ¾ Leadership styles of managers ¾ Organizational rewards and incentives ¾ Organizational culture: what behavior gets people promoted © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 14 14 Linking Lean Principles and Manufacturing Measurables Lean Principles Effective Work Groups Manufacturing Measurables Safety & Health Assessment Review Process Attitude Surveys Zero Waste/ Zero Defects First-Time-Through Capability Aligning Capacity with Market Demand Total Dock-to-Dock Time Build to Schedule Optimizing Production Throughput Using Total Cost to Drive Performance Overall Equipment Effectiveness Total Cost Source: Ford Motor Company © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Source: Ford Motor Company 6/9/04 -- 15 15 Build To Schedule -- What is it? ¾ In Plain Words: ¾ A way of knowing if you built the right parts, in the right quantity, in the right order ¾ A Formal Definition: ¾ Percent of units scheduled for a given day that are built on the correct day, in the correct quantity and correct mix BTS = Volume x Mix x Sequence © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 16 16 Build To Schedule -- Why use it? ¾ Lean Principle: ¾ Aligning Capacity with Market Demand ¾ BTS can help. . . ¾Keep changes in volume or mix from swamping or starving departments upstream or downstream ¾Operate with smaller “floats” -- which increases the need to track BTS ¾Respond to customer demand for “in-line vehicle sequencing” - which requires BTS © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 17 17 Sample Lessons from Build To Schedule Data -Volume Overbuilding here -- probably done to achieve central performance goals; probably used up more float than expected from feeder departments No credit for more than 100% Volume Performance Week of: Pieces Scheduled Pieces Produced % 6/2/99 14,650 16,303 100% 6/9/99 16,990 17,317 100% 6/16/99 16,380 15,755 96.18% Source: Visteon Company © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 18 18 Sample Lessons from Build To Schedule Data -- Mix Even with overbuilding, we did not make the mix -- so we are not serving customers and using excess resources Mix Performance Week of: Pieces Scheduled Pieces Produced % 6/2/99 14,650 13,425 91.64% 6/9/99 16,990 14,798 87.10% 6/16/99 16,380 11,662 74.02% The mix is now way off -- we are feeling the effects of the overbuilding Source: Visteon Company © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 19 19 Sample Lessons from Build To Schedule Data -- Overall Performance As a components manufacturing plant, we have an exemption on measuring sequence performance The performance trend is deteriorating rapidly -- all due to the way volume and mix are managed Week of: Volume % Build To Schedule Mix % Seq % BTS % 6/2/99 100% 91.64% 100% 91.64% 6/9/99 100% 87.10% 100% 87.10% 6/16/99 96.18% 74.02% 100% 71.20% Source: Visteon Company © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 20 20 Appendix: Instructor’s Guide Slide Time Topic Additional Talking Points 1-2 2-3 min Introduction, overview and learning objectives •Talk about linking the three key lessons together. Important to measure right; how to devise right metrics; and what can we learn from lean in devising metrics 3 3-5 min Key Concepts • Three key lessons TBD 7-10 min Exercises/Activities • The CCA example; Some thinking on what does lean mean to metrics TBD 5-7 min Disconnects • Important section explain all key points in the disconnects section. TBD 2-3 min Measurables • Some examples of manufacturing metrics that are lean like First time through. TBD 1-2 min Concluding comments • Summarize the three key lessons and disconnects © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT Part I: Introduction Part II: Concepts Part III: Application Part IV: Disconnects 6/9/04 -- 21 Part V: Conclusion 21 Backup Slides © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 22 22 Communication of Metrics ¾ Align individual performance with strategy ¾Communicate & Educate ¾Set Stretch goals ¾Link rewards to performance measures ¾ Measurements are meant to change behavior ¾ Change happens thru people, so everyone needs to see a need to change ¾ To achieve maximum motivation, goals should be moderately challenging. ¾ Resolve issues to achieve goal: Lean metrics not only to evaluate but also to improve. © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 23 23 Re-Assessment ¾ Continuous Improvement ¾ Metrics Re-Assessment and Results teams at Raytheon TI : PDCA Cycle. ¾ Reassessment/Deployment Team ¾“Catchball” process ¾What is the benchmark capability? ¾Next Years Metrics should be? © Linsey Rubenstein/Krishnan Raghunathan – ESD.60 Lean/Six Sigma Systems, LFM, MIT 6/9/04 -- 24 24