Reliability and Validity in Measurement Procedures

RELIABILITY & VALIDITY ▪1) Does the instrument or the test measure a variable consistently? and ▪2) Is the instrument or the test a true measure of the variable? The first question is concerned with reliability while the second one is concerned with validity. ▪ Definition Reliability A measurement procedure is said to be reliable if repeated measurements of the same individual under the same conditions produce identical (or nearly identical) values. Reliability is the stability or the consistency of measurement (ability of a measure to remain the same over time). It is the degree to which an assessment tool produces stable and consistent results. ▪ Reliability coefficient is a way of Measurement of Reliability confirming how accurate a test or measure. It is a measure of the accuracy of a test or measuring instrument obtained by measuring the same individuals twice and computing the correlation of the two sets of measures. It ranges between 0.00 to 1.00; where 0.00 means no reliability and 1.00 means perfect reliability. ▪ A. Internal Reliability 1. Parallel-forms Reliability (equivalent forms reliability) is used to assess the consistency of the results of two tests constructed in the same way from the same content domain. Types of Reliability 2. Split Half Reliability ▪ The split-half method assesses the internal consistency of a test, such as psychometric tests and questionnaires. There, it measures the extent to which all parts of the test contribute equally to what is being measured. This is done by comparing the results of one half of a test with the results from the other half. A test can be split in half in several ways, e.g. first half and second half, or by odd and even numbers. If the two halves of the test provide similar results this would suggest that the test has internal reliability B. External Reliability 1. Test-retest (Stability) Reliability ▪ It is used to assess the consistency of a Types of Reliability measure from one time to another. It examines the consistency of the results of repeated test administration. It is a measure of reliability obtained by administering the same test twice over a period of time to a group of individuals (a measure or test is repeated on the same subjects at a future date). The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for stability over time. 2. Rater Reliability It includes ▪ inter-rater reliability (reliability of data collection between two or more raters) and ▪ intra-rater reliability (reliability of data collection from one individual over two or more trials). a ) Inter-rater Reliability ▪ It is the extent to which two or more inter-rater reliability individuals (raters) agree. It is used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon. It assesses the consistency of how a measuring system is implemented. The reliability depends upon the raters to be consistent in their evaluation of behaviors or skills (ability of two or more individuals to be consistent). Training, education and monitoring skills can enhance inter-rater reliability) Inter-rater Reliability ▪. ▪ when two or more physical therapists Example: apply muscle testing by the use of grading muscle testing. If one researcher gives a grade one to a patient, while another researcher gives grade three to the same patient, obviously the inter-rater reliability would be inconsistent intra-rater reliability It refers to the degree of agreement among multiple repetitions of a diagnostic test performed by a single rater. It is the consistency of grading by a single rater. ▪ Definition ▪ Validity is the measuring of what the subgect claimed to measure. ▪ Example: Palpation Technique. Validity Assuming that the cardiac catheterization measure is the more accurate method to measure heart rate, the amount of agreement between it and palpation technique constitutes the degree of validity of the palpation technique. Types of Validity 1. whole studies (often called inference validity) and 2. the measurement of individual variables (often called construct validity) Types of Error in Measurement ▪ All experimental uncertainty is due to either random errors or systematic errors. ▪ They are statistical fluctuations (in either 1. Random Errors direction) in the measured data due to the precision limitations of the measurement device. Random errors usually result from the experimenter's inability to take the same measurement in exactly the same way to get exact the same number which may be due to fatigue or inattention. ▪ It affect reliability Examples of causes of random errors are: ▪ electronic noise in the circuit of an electrical instrument, ▪ irregular changes in the heat loss rate from a solar collector due to changes in the wind. ▪ They are reproducible inaccuracies that 2. Systematic Errors are consistently in the same direction. Systematic errors are often due to a problem which persists throughout the entire experiment. Systematic errors are constant and, therefore, do not present a problem for reliability. Systematic errors are primarily a concern of validity, because, although they are consistent, test values are not true representations of the quantity being measured.