Exam Revision Process Guide

Exam Revision Process 1. Review Job Task Analysis o Exam Categories/Weighting 2. Exam Item Analysis o Existing Exams 3. Consider New Technology o Input from WUOLAC, Operators, TCEQ Staff, Technology Experts 4. Consider Rule Changes o Updates to existing exams o New exams for new licenses 5. Use Course Outlines or Curriculum Guidance 6. Create Exam Questions o Relate to critical task o Revise existing questions o Reference each question to: ▪ Required core courses of the license ▪ Applicable rules o Use existing resources o Create graphics/figures as needed 7. Review of Created Questions by: o TCEQ Technical Program o WUOLAC Sub-committee Supplement with SME’s o TCEQ training Staff 8. Reviewers responsibilities: o Exam Item Analysis Results o Technical Correctness o Clarity, Grammar, Word Choice o Match Question Difficulty with License Level o Rule Compliance 9. Database Changes 10. Exam Cover & Instructions o Graphics Work/Formula Sheets 11. Final Editing & Formatting 12. Notify Approved Manual Providers • Content/Topic of exam question missing from manual/course • Disclaimer Requirement Exam Revision Status • • September 2009 o Class D Wastewater Operator-revised October 2009 o Class D Water Operator- revised • September 2010 o Class A Water/Wastewater Operator – 100 points/No essays Exam Pass Rates WASTEWATER • • • • • • • Class A Wastewater Treatment 31% Class B Wastewater Treatment 25% Class C Wastewater Treatment 69% Class D Wastewater Treatment 57% Class III Wastewater Collection 40% Class II Wastewater Collection 33% Class I Wastewater Collection 61% WATER • • • • • • • • • Class A Water 36% Class B Surface Water 15% Class B Groundwater 22% Class B Water Distribution 15% Class C Surface Water 53% Class C Groundwater 43% Class C Water Distribution 37% Class D Water 52% Customer Service Inspector 49% Overview and Purpose • • • • TCEQ Rules & Regulations Approval of Training at Regional Schools– o Achieving Compliance o Working with TWUA Central Office Audits Training Provider Responsibilities RG-373 • • • RG stands for Regulatory Guidance and as such is typically a document that TCEQ staff prepares for directions to rule compliance. RG-373 was challenged as being guidance and not part of rules & regs. Most of the RG-373 guidance document was incorporated into the previously mentioned rule to improve enforcement. Training at Regionals: Reviews & Approvals TCEQ’s contact for reviews & approvals of all TWUA training is TWUA Central Office • 30 TAC §30.28 • o Applications for approval of Special Topic Training ▪ Conference checklist items, fee ▪ Due to TCEQ 45 days prior to event ▪ TWUA Central Office coordinates Training Coordinator-David Moore Training at Regionals: Achieving Compliance • • • • • • • • Training must address the knowledge and skills determined necessary to perform the critical tasks. o Basically, training has to be directly related to typical job duties. Presentations must include subject matter related to the particular licenses. Credit for “successfully completing training,” not for just showing up; Attendance alone does not automatically constitute submittal of training hours. o If someone is disruptive, sleeps, is reading the newspaper, or otherwise not a part of the training – their hours DO NOT have to be reported. Provide roster & hours to TCEQ within 14 business days – TAC §30.28 (i)(12) Round down to nearest half hour – o If the training ends up being 1:45 minutes, you would round the time down so for this meeting there would be 1:30 minutes worth of training credit. Must be approved prior to awarding training credit; Advertise as “approved” training only after approval notification from TCEQ; Training at Regionals: Special Topic Presentations • • • • • Must not promote or endorse the products, product lines, or services of a manufacturer, distributor, or service provider. ▪ Generic vs. proprietary presentation ▪ Samples of several manufacturers’ products or services ▪ No “Our brand X is better than their brand Y…” ▪ No price quoting or comparison ▪ Provide separate time & place to answer product-specific questions/sales Training must NOT be used as an opportunity for advertisement. Should not sound like a sales presentation or sales pitch. Vendors are a tremendous asset they just need to know what is considered acceptable. How? o Look at presentations. o Check with TWUA Central for guidance & acceptable topics/speakers. o Feedback from participants. Training should – • • • provide the means to accomplish the learning objectives identified for the training; include visual aids, graphics-equipment samples, photos, video clips; interactivity-question and answer time; • • • • • • • • • include regular monitoring of comprehension throughout the training with feedback from the training provider or presenter. Presenters must be Subject Matter Experts. Who is an SME? A person having a minimum of three years of work-related experience or expert knowledge in a particular content area related to the training topic. SME – Example: o If you wanted to have a presentation of Self Contained Breathing Apparatus – o A Fireman would be a SME o A Volunteer Fireman who is also an Operator would be a SME o A Manufacturing Representative would be a SME o Joe the local plumber would not be a SME on this subject TWUA Central maintains TCEQ approval for core courses. o Examples: Basic Water, Pump & Motor Maintenance Some courses are presented only at a particular regional school-approved only for this school ex. Beaumont’s Utility Maintenance Instructors are the key! Qualified instructors must be selected. o Appropriate license related to the training, ex. Usually class B/II min. o 3 years (minimum) of current hands-on experience related to training topics. o 60 hours classroom experience or instructor training. Training must be presented as approved: • • • • Plant tours, added related activities, etc. okay if approved material covered. Use approved training materials, including manual. Communicate with TWUA Central Office! o TWUA Central Office is accountable. TWUA Regional & Central Office auditing: o Walk around, visit classrooms; o Be sure instructors have what they need to get started & be effective; o Help instructor, if needed, ex. Illness. Why TWUA? • • • • • Variety of training courses offered; Regional training effects a large segment; Training review staff available to assist TCEQ licensing staff; Available to answer questions & make presentations related to operator training & exams. Supporting TWUA’s desire to improve. What is TCEQ looking for? • • • • Rule compliance-training, licensing, & technical rules Instructor effectiveness******** Environment conducive to learning? Adequate exam prep without “teaching the test” Final Comment: What you do, as the training provider, may result in rescinding, denying, or suspending training approval.