Lecture 28 - The logic Behind Hypothesis Testing

LECTURE 28 The logic behind hypothesis testing We play a game in which we win 1 if a coin tosses head, we lose 1 if a tail is thrown. Initially, we played because we believe that the coin was fair. At the first trial, we had no reason to think that the coin is not fair since we assumed a success and a failure to be equally likely. As we go on, we suspect that there is something wrong, the sequence we observe is highly unlikely to be observed for a fair coin. We started playing under the hypothesis that the "coin is fair," but after several rounds, we reject the hypothesis. Null hypothesis Assume we have a population with an associated parameter: θ. We make a hypothesis about the parameter value, that is called the null hypothesis H₀ and that can be of different types: - H₀: θ = θ₀ (simple hypothesis) - H₀: θ ≥ θ₀ - H₀: θ ≤ θ₀ - H₀: θ ≠ θ₀ We observe a sample and our goal is to test whether the null hypothesis H₀ is compatible with the observed sample or not. The sample observation may not support our hypothesis. Example: new Significance test for a proportion under H₀: p = p₀ In general terms, for a null hypothesis H₀: p = p₀, under the assumption that the H₀ is true, the sample proportion computed on a sample of size n is approximately Normal: p̂ ~ N(p₀, p₀(1-p₀)/n) We observe p̂ and we evaluate how compatible this is with the null hypothesis by computing the probability of observing an equally large deviation from p₀ if the null hypothesis is true, called p-value. p(|d| - |d₀|) = P(|z| - |z₀|) = 2(1-Φ(|z₀|)) = 2(1-Φ(0.9)) = 0.332 If this probability is small, we conclude that we don't have evidence from the sample in favor of the null hypothesis. D₀ = 0.93 p(|d| - |d₀|) = P(|z| - |z₀|) = 2(1-Φ(|z₀|)) = 2(1-Φ(0.9)) = 0.332 Significance test for a proportion composite hypothesis: H₀: π = π₀ If the null hypothesis is H₀: π = π₀, and we observe p̂₀, we may reject H₀ if p̂₀ is greater than π₀. We measure how compatible this is with the hypothesis on the basis of the probability of observing a larger deviation from π₀ under the null hypothesis. P(D₀ > d₀) = P(z > z₀) = 1 - Φ(z₀) = 1 - Φ(0.9) = 0.166 Significance test for the Normal variance known If the null hypothesis is H₀: μ = μ₀, the sample mean distribution follows a Normal, then: The measure of the deviation between the null hypothesis and the observation is: D₀ = (x̄ - μ₀) / (σ / √n) ~ N(0, 1) Significance test for the Normal variance unknown If the null hypothesis is H₀: μ = μ₀, the quantity to evaluate is: D₀ = (x̄ - μ₀) / (s / √n) ~ t(n-1) Let t be the estimate, the deviation measure between null hypothesis and estimate is: d₀ = |t| and compute it using the cumulative distribution function of the Student t distribution with n-1 degrees of freedom. Summary significance test Summary p-value With Fₙ is the cumulative distribution function of Student t distribution with n-1 degrees of freedom Example: Body temperature Normal, d' unknown Student t distribution->30×130 Normal Proportion, H₀: π₀ 36.831 Neyman-Pearson approach The Neyman-Pearson approach refers to a system of hypothesis -Null hypothesis, H₀ -Alternative hypothesis, H₁ Before observing the sample, we want to formulate a rule according to which we accept/reject the null hypothesis once the sample is extracted. The rule is a set of sample values called the rejection region: -if x ∈ R, the null hypothesis is REJECTED -if x ∉ R, the null hypothesis is ACCEPTED The complementary of the rejection region is called the acceptance region. Rejection region, mean with known variance To choose α, we need to find the value of k such that: P(D₀ > k) = α and construct the rejection region: R = {x̄: (x̄ - μ₀) / σ > k} To determine the power of the test, we need to calculate the probability of rejecting the null hypothesis when the alternative hypothesis is true.   This probability is called the power   function and is denoted by β(μ₁). β(μ₁) = 1 - Φ(k - (μ₁ - μ₀) / σ) Rejection regions: two-sided case Assume the following two-sided alternative hypothesis: H₁: μ ≠ μ₀ It is easy if we refer to the quantity D₀ that, under the null hypothesis, follows a standard normal distribution. Consider the rejection region: R = {x̄: |(x̄ - μ₀) / σ| > k} The Type I error probability is 2(α - Φ(k)), then: α = 2(α - Φ(k)) k = Φ⁻¹(1 - α/2) Rejection regions: one-sided case Assume the following one-sided alternative hypothesis: \[\begin{cases}H_{0}: μ ≤ μ₀\\ H_{1}: μ > μ₀\end{cases}\] In this case, with a composite null hypothesis, we don't have the distribution of x̄ under the null hypothesis but again we refer to the quantity D₀: D₀ = (x̄ - μ₀) / σ