From Modeling to Evaluation

From Modeling to Evaluation Computing Grade 11 What is the purpose of modeling Driving Question 3 “ PDN Discuss what is modeling and evaluation ” 4 Lesson objectives ▰ What the purpose of data modeling is. ▰ Some characteristics of the modeling process. ▰ What it means to evaluate a model. ▰ Ways in which a model is evaluated. ▰ How to apply modeling and model evaluation to any data science problem. 5 Let us get started. Data Modeling In what way can data be visualised to get the answer that is required? 7 “ Modelling is the stage in the data science methodology where the data scientist has the chance to sample the data and determine if it's bang on or in need of something else. ” 8 The Purpose of Data Modeling Descriptive Modeling Prescriptive Modeling Algorithm Selection Problem Understanding Calibration Data Advancement Of the Field Model Evaluation Feedback Loops Model Deployment 9 The Purpose of Data Modeling Explained ▰ Descriptive Modeling: Descriptive models aim to understand relationships and patterns in the data. They help describe and explain past events or phenomena, such as determining preferences based on certain actions or behaviors. ▰ Predictive Modeling: Predictive models focus on forecasting or predicting future outcomes. They aim to provide yes/no or stop/go type answers based on historical data and the chosen analytic approach. ▰ Calibration: Data modeling involves using a training set, which consists of historical data with known outcomes, to calibrate the model. The training set acts as a gauge to determine if the model requires adjustments or refinements. 10 The Purpose of Data Modeling Explained ▰ Algorithm Selection: Data scientists experiment with different algorithms during the modeling stage to identify the most appropriate ones for the problem at hand. This process ensures that the variables used in the model are necessary and relevant. ▰ Problem Understanding: Data modeling depends on a clear understanding of the problem being addressed. By comprehending the question at hand, data scientists can develop models that effectively answer the question and provide valuable insights. ▰ Model Evaluation: Once the data model is built, it needs to be evaluated to assess its performance and accuracy. Model evaluation involves testing the model against new data and analyzing its outcomes. 11 The Purpose of Data Modeling Explained ▰ Model Deployment: After the evaluation, successful data models can be deployed for practical use. They can be integrated into systems or processes to generate insights and support decision-making. ▰ Feedback Loops: Feedback loops ensure that the deployed data model remains relevant and effective. Continuous refinement, adjustments, and tweaking are necessary to improve the model's performance over time. ▰ Advancement of the Field: The overall purpose of data modeling is to contribute to the development of the data science field. The more people benefit from the outcomes of data modeling, the further the field will progress, as it offers new possibilities and insights. Evaluation Does the model used really answer the question or does it need to be adjusted? 13 “ Model evaluation is performed during model development and before the model is deployed. Evaluation allows the quality of the model to be assessed but it's also an opportunity to see if it meets the initial request. ” 14 Evaluation Phases Diagnostic Measures Phase: This phase is used to ensure that the model is working as intended. If the model is a predictive model, a decision tree can be used to evaluate if the model's output aligns with the initial design. If the model is a descriptive model, a testing set with known outcomes can be applied to assess the relationships being assessed and refine the model as needed. Statistical Significance Testing: This phase involves applying statistical significance testing to the model to ensure that the data is properly handled and interpreted. This helps avoid unnecessary second-guessing when the model's answer is revealed. Statistical significance testing helps assess the validity and reliability of the model's results. 15 Exit Ticket 321 Activity 1. List three purposes of data modeling 2. Identify two main phases of evaluation 3. Write one question that you still have 16 References