Rubric Development for Assessment

+ Rubric Development for Assessment + Timeline Spring-Fall Faculty determine which learning outcomes to assess and identify courses or experiences where students demonstrate learning Assessment committee evaluates student work based on a rubric or other clear criteria and documents findings Departments constitute an assessment committee to review student work Faculty collect samples of student work in designated courses Assessment committee reports on results of assessment to department and recommends action Department assigns responsibility for implementing changes and sets priorities for the coming academic year Program summarizes findings and actions taken in an annual assessment report due Nov. 15 Minimal Rubric for Senior History Seminar Papers Title 5 4 3 2 1 Thesis 5 4 3 2 1 Argumentation 5 4 3 2 1 Historical significance of project 5 4 3 2 1 Organization 5 4 3 2 1 Citation of Sources 5 4 3 2 1 + Why use a rubric? A rubric is a scoring tool that lists the criteria for student work and articulates levels of quality for each criterion.  Rubrics make grading consistent and fair  Rubrics help clarify expectations to both students and faculty  Rubrics can help identify areas for improvement  Rubrics save time in the grading process, once they have been developed  Rubrics help track changes in student performance over time + Types of Rubrics  Holistic vs. Analytic -Holistic rubrics describe the characteristics of a performance to give an overall judgment of its quality. -Analytic rubrics judge each individual characteristic of a performance separately.  General vs. Task Specific -General or generic rubrics can be applied to a number of different tasks. -Task-specific rubrics are used to evaluate specific tasks and contain criteria and descriptions that reflect specific features of the elicited performance.  Hybrid rubrics Combine the features of both general and task-specific rubrics to provide feedback on broad dimensions of learning as well as on the performance of a specific task + Sample Holistic Rubric: Writing Strong Performance Writing has a clear focus and engages readers in the opening lines. Information is pertinent and accurate. Transitions help the reader move smoothly from one idea to the next. Errors in spelling or grammar are minor or infrequent and do not interfere with communication. Satisfactory Performance Writing has a clear opening statement and logical sequence of ideas. Information is accurate. Errors in grammar or spelling are minimal and do not interfere with communication. Needs Improvement Writing includes a purpose for reading in the opening paragraph. The information is accurate. Supporting ideas follow the opening paragraph but may not be logically ordered. Grammar and/or spelling errors may at times distract from message. + Holistic Rubrics Advantages: Quick scoring/ time saving Written generically and can be used with many tasks Good for summative assessments as they give an overall judgment of performance Disadvantages: Do not provide specific feedback about strengths or areas to improve Not useful for formative assessment + Sample Analytic Rubric: Oral Presentation Below Expectation Satisfactory Exemplary Organization -No apparent organization -Evidence not used to support assertions -Presentation has a focus -Student provides evidence that supports conclusions -Presentation is carefully organized. -Speaker provides convincing evidence to support conclusions Content -Content is inaccurate or overly general -Listeners are unlikely to learn or may be misled -Content is generally accurate, but incomplete -Listeners may learn isolated facts but are unlikely to gain new insights about topic -Content is accurate and complete -Listeners are likely to gain new insights about the topic. Delivery -Speaker appears anxious and uncomfortable -Speaker reads notes rather than speaking -Listeners are largely ignored -Speaker is fairly relaxed and comfortable -Speaker too often relies on notes -Listeners are sometimes ignored or misunderstood. -Speaker is relaxed and comfortable -Speaker speaks without undue reliance on notes -Speaker interacts effectively with listeners. + Analytic Rubrics Advantages:  More detailed feedback along several dimensions, since each criterion is evaluated separately (i.e. you know what to work on!)  Dimensions can be weighted to reflect the relative importance of different criteria  Easy to link back to instruction  Useful for formative assessment (providing feedback to students) when the same rubric categories are used consistently.  Scoring is more consistent across students and raters Disadvantages:  More time-consuming to score and to create + General Rubrics  Can use the same rubric to evaluate many different tasks  Feedback may not be specific enough to dictate clear course of action Task-Specific Rubrics  More reliable assessment of performance  Time-consuming to construct rubrics for all specific tasks + Which type of rubric should you choose?  Use a holistic rubric if you want a quick, overall snapshot of student achievement.  Use an analytic rubric when you want more detailed feedback about relative strengths and weaknesses, when you want to assess complex skills, or when you want students to self-assess.  Use a general rubric when you want to assess reasoning, skills, and products when students are not all doing exactly the same task  Use a task-specific rubric when you want to assess knowledge and consistency of scoring is important. + Rubric components: 4 parts 1) A task description = the outcome being assessed 2) The characteristics to be rated (rows)= skills, knowledge, behavior to be demonstrated 3) Levels of mastery (columns) = advanced, intermediate high, intermediate low, novice, or other descriptions (1,2,3,4, etc.) (Aim for an even number) 4) A description of each characteristic at each level of mastery (cells) + VALUE rubrics  A set of general rubrics to assess a range of student learning outcomes, including written and oral communication, civic engagement, information literacy, global understanding, and lifelong learning.  Best used as templates to guide discussion and as a basis for locally-developed rubrics.  In many cases, it makes sense to start by considering the attributes of the best student work to determine rubric categories and descriptions + Rubric norming  To produce reliable scores, faculty members need to be interpreting the rubric in the same way.  Provide an opportunity for faculty to practice rating student work using the rubric and discussing scores and discrepancies. Allow raters to explain their judgments, and attempt to arrive at consensus about the most appropriate rating in each case.  Once the group is satisfied with how the rubric is to be applied, rating can begin and reviews can begin scoring.  Present a summary of scores to the group. Discuss results as well as the effectiveness of the process itself.