University:
Boston University
Course:
CAS MA 555 | Numerical Analysis I
Academic year:

2024
Views:

214

Pages:

24
Author:

Draco M

Direct & Indirect Measurements Overview • We have learning goals – now what? • Measurements: Direct vs. Indirect – Indirect Examples – Direct Measurements – “basic” – Direct Measurements – “difficult” • The value in both • Matching the process to your resources & needs Assessment Process Definition of student learning goals/objectives Ensure alignment of curriculum with goals Identification of assessment instruments and measures Collection, analyzing, and dissemination of assessment information • Incorporation of results into feedback loop • • • • Identifying Measurements • We develop graduates competent in a general management perspective; including understanding the global nature of business, taking into account relevant risks and constraints, and devising strategies which address ethical concerns • We develop graduates with strong communication skills • We develop graduates who view business problems through a crossfunctional lens • We develop graduates with a basic understanding of technology and its application in solving management problems Identifying Measurements - General • We develop graduates competent in a general management perspective; including understanding the global nature of business, taking into account relevant risks and constraints, and devising strategies which address ethical concerns • We develop graduates with strong communication skills – Each student can disseminate, to diverse audiences, the results of analysis in cogent and concise written format. – Each student can disseminate, to diverse audiences, the results of analysis in cogent and concise oral presentation. • We develop graduates who view business problems through a cross-functional lens • We develop graduates with a basic understanding of technology and its application in solving management problems Identifying Measurements - General • Simmons graduates can communicate effectively and professionally, choosing appropriately from a portfolio of written, oral and visual techniques and styles to best convey a desired message to a selected audience (communication). – Demonstrate effective oral communication skills – Demonstrate professional written communications skills – Develop visual aids that support written communication and oral presentations. Identifying Measurements: Mission Specific • We develop graduates competent in a general management perspective; including understanding the global nature of business, taking into account relevant risks and constraints, and devising strategies which address ethical concerns • We develop graduates with strong communication skills • We develop graduates who view business problems through a crossfunctional lens – Each student can develop solutions to functionally-based problems which consider the effects on other functions • We develop graduates with a basic understanding of technology and its application in solving management problems Identifying Measurements: Mission Specific • Simmons graduates identify and appraise the strategic and behavioral roles of gender and cultural diversity in organizations and are prepared to effectively manage their careers in that context (gender and diversity). – Identify and analyze the role of diversity and its impact on personal and organizational effectiveness – Develop personal career strategies to respond to a complex work environment Identifying Measurements: Mission Specific • Simmons graduates are prepared to provide strategic and principled leadership at the individual, team and organizational levels bringing integrity and ethics to managerial decision making (principled leadership). – Analyze and articulate the ethical implications of managerial decisions – Reflect on the on the role and impact of personal and managerial social responsibility in communities and organizations. Measurements Types • Indirect: “Perceptual” measurements; can include surveys of alumni, employers, graduates; some outcomes measures/ placement rates • Direct: “Actual” observed measurements; can utilize existing course embedded measures, grades, stand-alone testing/projects/deliverables; portfolios • Pre-Selection: competency measure through admissions selection process Indirect Measurement - SMG • SMG was seeking a baseline understanding of: – If identified program learning goals resonated with students – a “check” – If there existed programmatic strengths or weaknesses not captured by learning goals – Initial understanding of student performance • Survey was developed with two primary goals: – Inform the faculty about the extent to which young alumni perceived the preparation provided by SMG – Provide indirect assessments of student learning relative to program learning goals Indirect Measurement - SMG Open ended questions on program strengths/weaknesses Targeted questions on learning goals In comparison to my professional peers who did not attend BU SMG, the undergraduate education I received at the School of Management prepared me to effectively: (Five Point Scale) ....consider relevant legal risks when analyzing business problems. ....apply technology to solve management problems. ....consider relevant ethical issues when analyzing business problems. ....consider relevant operational risks when analyzing business problems. ....consider relevant financial risks when analyzing business problems. ....communicate my ideas in writing. ....consider associated opportunities and threats when analyzing business problems. ....consider the inter-dependencies of business functions. ....analyze business problems. ....make decisions using a cross functional lens. ....deliver an effective presentation. ....collaborate within a team Indirect Measurement - SMG Competencies Relative to Professional Peers ....collaborate within a team. ....deliver an effective presentation. ....make decisions using a cross functional lens. ....analyze business problems. ....consider the inter-dependencies of business functions. ....consider associated opportunities and threats when analyzing… ....communicate my ideas in writing. ....consider relevant financial risks when analyzing business problems. ....consider relevant operational risks when analyzing business problems. ....consider relevant ethical issues when analyzing business problems. ....apply technology to solve management problems. ....consider relevant legal risks when analyzing business problems. 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 Responses on a five point scale - lower is better 2.8 3.0 “Basic” Direct Measurement – SMG Learning Goal: Functional Competence: Each student has a basic knowledge of accounting, economics, and business, operations and technology management, finance, organizational behavior, and quantitative analysis. Direct embedded measurements, in the form of identified exam questions, were used across 12 core courses to assess fundamental course principals as defined by course faculty. Each course is followed by the identified core concepts and summary of student performance in terms of percent of students correct on the questions related to each concept. MK 323: Fall 2008 midterm Concepts Topic Brand & Product Consumer Behavior Market Research New Product Development Segmentation Strategy Overall % correct 73.84% 72.62% 71.74% 76.45% 93.48% 88.29% Q1 83.57% 81.16% 85.27% 65.70% 94.69% 85.75% Q2 62.32% 47.34% 93.00% 87.20% 92.27% 90.82% Q3 89.37% 36.96% - “Basic” Direct Measurement – Simmons • “Knowledge of the Disciplines” program goal measured through performance on selected questions on multiple choice final exams for core courses Example: Introduction to Marketing final, 17 MC questions Exemplary: (% scoring 90-100) Sufficient: (% scoring 80-89) Deficient: (% scoring below 80) 4% to 23% 74% to 70% 22% to 8% “Difficult” Measurements • Not as objective: “inherently more subjective” • Higher level learning goals may include: communication skills, critical thinking, cross-functional thinking • Faculty were engaged to create new instruments, such as rubrics • Rubrics can be applied to existing deliverables • Why Rubrics? – – – – Help to describe evaluation criteria; adds clarification Can be used as a student learning tool Improves feedback to students Informs faculty of student performance “Difficult” Direct Measurement – SMG RUBRIC FOR THE EVALUATION OF ANALYTICAL AND CRITICAL THINKING IN CASE ANALYSES I. FORMULATION: IDENTIFYING AND STRUCTURING THE PROBLEM 1 1. Identifies and summarizes key problem(s) Flawed 2 Fails to identify key alternatives and/or proposes non-actionable alternatives 1 3. Given the alternatives identified, discusses constraints 2 Fails to identify key problem or to describe it clearly 1 2. Identifies courses of action (i.e., alternatives) Flawed Flawed 2 Fails to recognize some of the important constraints 3 Acceptable 4 Identifies key and secondary problems but does not adequately discuss their relative importance or priority 3 Acceptable 4 Identifies most of the main alternatives and discusses how they can be put into action 3 Acceptable 4 Recognizes most of the important constraints 5 Exemplary 6 Clearly distinguishes key and secondary problems along with their relative importance or priority 5 Exemplary 6 Identifies all main actionable alternatives and is creative in proposing less obvious courses of action 5 Exemplary 6 Recognizes the important constraints and also that some constraints can be relaxed or worked around “Difficult” Direct Measurement – SMG “Difficult” Direct Measurement – Simmons Which do I use? • A comprehensive approach to assessment can involve both direct & indirect measurements • Direct: – Exam questions, deliverables, thesis, portfolio, etc. – Quickly identify curricular needs – Opportunity for exit assessment, pre-post test, etc. • Indirect: – Current student surveys and focus groups – Surveys of alumni – Employer/recruiter feedback Your process should match your needs and resources • BU SMG: 149 FT Faculty BSBA Program; MBA Programs; Mathematical Finance; PhD 2,400 UG students; 1,200 Graduate students Assessment driven through existing Degree Committees Target faculty support through use of deliverables & instrument design – Dean’s office support – leadership & logistics – – – – – Your process should match your needs and resources • Simmons College: 26 FT faculty MBA Program, UG Program 400 Students Dedicated all-faculty “Assessment Days” to review student work – Fosters cross-faculty collaboration & curricular understanding – Enhances engagement & ownership – – – – Closing the Loop - Simmons • Assessment sessions and discussion • AoL reports to Curriculum Committee – – – – – Percent in each category for each trait Conclusions of faculty discussion Implications for assessment process Implications for curriculum Discussions with departments, program leaders • CC reports back to AoL committee • Assessment sessions and discussion Lessons Learned • Not assessing a faculty member or student – focus on the program & averages • Validity & Sampling – can be imperfect, but justifiable • Pre & Post test; standardized tests; teaching to the test • Rubrics – grading vs. assessing; distribution to students • Closing the loop – keep measurements close to the curriculum; carefully consider new addition of new assessment “structures”

Direct & Indirect Measurements

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock