Lecture Note
Basic Analysis of Variance and the General Linear Model
-
University:
California State University, Northridge -
Course:
PSY 534/S | Latent Variable Analysis and Seminar Academic year:
2007
-
Views:
125
Pages:
40
Author:
Daneshfarnjp
Basic Analysis of Variance and
the General Linear Model Why is it called analysis of variance
anyway?
If we are interested in group mean differences, why
are we looking at variance?
t-test only one place to look for variability
More groups, more places to look
Variance of group means around a central tendency
(grand mean – ignoring group membership) really
tells us, on average, how much each group is
different from the central tendency (and each other) Why is it called analysis of
variance anyway?
Average mean variability around GM needs to
be compared to average variability of scores
around each group mean
Variability in any distribution can be broken
down into conceptual parts:
total variability = (variability of each group
mean around the grand mean) + (variability of
each person’s score around their group mean) General Linear Model (GLM)
The basis for most inferential statistics
(e.g. 420, 520, 524, etc.)
Simple form of the GLM
score=grand mean + independent variable + error
Y = µ +α + ε General Linear Model (GLM)
The basic idea is that everyone in the
population has the same score (the grand
mean) that is changed by the effects of an
independent variable (A) plus just random
noise (error)
Some levels of A raise scores from the
GM, other levels lower scores from the
GM and yet others have no effect. General Linear Model (GLM)
Error is the “noise” caused by other variables you aren’t
measuring, haven’t controlled for or are unaware of.
Error like A will have different effects on scores but this
happens independently of A.
If error gets too large it will mask the effects of A and make it
impossible to analyze the effects of A
Most of the effort in research designs is done to try and
minimize error to make sure the effect of A is not “buried” in
the noise.
The error term is important because it gives us a “yard stick”
with which to measure the variability cause by the A effect.
We want to make sure that the variability attributable to A is
greater than the naturally occurring variability (error) GLM
Example of GLM – ANOVA backwards
We can generate a data set using the GLM formula
We start off with every subject at the GM (e.g. µ=5)
a1
Case
s1
s2
s3
s4
s5
a2
Score
5
5
5
5
5
Case
s6
s7
s8
s9
s10
Score
5
5
5
5
5 GLM
Then we add in the effect of A (a1 adds 2 points and
a2 subtracts 2 points)
a1
Case
s1
s2
s3
s4
s5
Score
5+2=7
5+2=7
5+2=7
5+2=7
5+2=7
∑ Ya1 = 35
a2
Case
s6
s7
s8
s9
s10
Score
5–2=3
5–2=3
5–2=3
5–2=3
5–2=3
∑ Ya2 = 15
2
Y
∑ a1 = 245
2
Y
∑ a2 = 45
Ya1 = 7
Ya3 = 3 GLM
Changes produced by the treatment represent
deviations around the GM
n∑ (Yj − GM ) = n[(7 − 5) + (3 − 5) ] =
2
2
2
2
2
2
2
5(2) + 5(−2) or 5[(2) + (−2) ] = 40 GLM
Now if we add in some random variation (error)
a1
a2
SUM
Case
Score
Case
Score
s1
5+2+2=9
s6
5–2+0=3
s2
5+2+0=7
s7
5–2–2=1
s3
5+2–1=6
s8
5–2+0=3
s4
5+2+0=7
s9
5–2+1=4
s5
5+2–1=6
s10
5–2+1=4
∑ Ya1 = 35
∑ Ya2 = 15
∑ Y = 50
2
Y
∑ a1 = 251
2
Y
∑ a2 = 51
2
Y
∑ = 302
Ya1 = 7
Ya3 = 3
Y =5 GLM
Now if we calculate the variance for each group:
2
2
(
Y
)
35
∑
2
Ya1 −
251 −
∑
N
5 = 1.5
sN2 −1 =
=
N −1
4
2
2
(
Y
)
15
∑
2
−
Y
51 −
∑
a2
N
5 = 1.5
=
sN2 −1 =
4
N −1
The average variance in this case is also going to
be 1.5 (1.5 + 1.5 / 2) GLM
We can also calculate the total variability in
the data regardless of treatment group
2
2
(
Y
)
50
∑
2
Y
−
302 −
∑
N
10 = 5.78
sN2 −1 =
=
N −1
9
The average variability of the two groups is
smaller than the total variability. Analysis – deviation approach
The total variability can be partitioned
into between group variability and
error.
(Y
ij
− GM ) = (Yij − Y j ) + (Y j − GM ) Analysis – deviation approach
If you ignore group membership and
calculate the mean of all subjects this
is the grand mean and total variability
is the deviation of all subjects around
this grand mean
Remember that if you just looked at
deviations it would most likely sum to
zero so… Analysis – deviation approach
∑∑(Y − GM )
ij
i
2
= n∑(Yj − GM ) + ∑∑(Yij −Yj )
j
SStotal = SSbg + SSwg
SStotal = SSA + SSS / A
2
j
i
j
2 Analysis – deviation approach
A
a1
a2
Score
9
7
6
7
6
3
1
3
4
4
∑ Y = 50
∑Y
2
= 302
Y =5
(Y
ij
− GM )
16
4
1
4
1
4
16
4
1
1
∑ =52
2
(Y
j
− GM )
2
(7 – 5)2 = 4
(3 – 5)2 = 4
∑=8
n∑ = 5(8) = 40
52 = 40 + 12
(Y
ij
− Yj )
4
0
1
0
1
0
4
0
1
1
∑ = 12
2 Analysis – deviation approach
degrees of freedom
DFtotal = N – 1 = 10 -1 = 9
DFA = a – 1 = 2 – 1 = 1
DFS/A = a(S – 1) = a(n – 1) = an – a =
N – a = 2(5) – 2 = 8 Analysis – deviation approach
Variance or Mean square
MStotal = 52/9 = 5.78
MSA = 40/1 = 40
MSS/A = 12/8 = 1.5
Test statistic
F = MSA/MSS/A = 40/1.5 = 26.67
Critical value is looked up with dfA, dfS/A
and alpha. The test is always nondirectional. Analysis – deviation approach
ANOVA summary table
Source
A
S/A
Total
SS
40
12
52
df
1
8
9
MS
40
1.5
F
26.67 Analysis – computational
approach
Equations
SSY = SST = ∑ Y
SS A
a )
(
∑
∑
=
j
n
SS S / A = ∑ Y −
2
2
Y)
(
∑
−
N
2
2
2
T
= ∑Y −
an
2
2
T
−
an
2
∑ (∑ a j )
n
Under each part of the equations, you divide by the
number of scores it took to get the number in the
numerator Analysis – computational
approach
Analysis of sample problem
2
50
SST = 302 −
= 52
10
2
2
2
35 + 15 50
SS A =
−
= 40
5
10
2
2
35 + 15
SS S / A = 302 −
= 12
5 Analysis – regression
approach
Levels of A
a1
a2
Sum
Squares Summed
N
Mean
Cases
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
Y
9
7
6
7
6
3
1
3
4
4
50
302
10
5
X
1
1
1
1
1
-1
-1
-1
-1
-1
0
10
YX
9
7
6
7
6
-3
-1
-3
-4
-4
20 Analysis – regression
approach
Y = a + bX + e
e = Y – Y’ Analysis – regression
approach
Sums of squares
SS (Y ) = ∑ Y 2
Y)
(
∑
−
N
SS ( X ) = ∑ X 2
SP (YX ) = ∑ YX −
2
502
= 302 −
= 52
10
X)
(
∑
−
N
2
02
= 10 − = 10
10
(∑ Y )(∑ X )
N
(50)(0)
= 20 −
= 20
10 Analysis – regression
approach
SS(Total ) = SS (Y ) = 52
2
SS ( regression ) =
[ SP (YX ) ]
SS ( X )
2
20
=
= 40
10
SS( residual ) = SS(total ) − SS( regression) = 52 − 40 = 12
Slope
(∑ Y )(∑ X )
∑ YX −
SP(YX ) 20
N
b=
=
=
=2
2
SS
(
X
)
10
X
(
)
∑
2
∑X − N
Intercept
a = Y − bX = 5 − 2(0) = 5 Analysis – regression
approach
Y ' = a + bX
For a1 :
Y ' = 5 + 2(1) = 7
For a 2 :
Y ' = 5 + 2(−1) = 3 Analysis – regression
approach
Degrees of freedom
df(reg.) = # of predictors
df(total) = number of cases – 1
df(resid.) = df(total) – df(reg.) =
9–1=8 Statistical Inference
and the F-test
Any type of measurement will include a
certain amount of random variability.
In the F-test this random variability is seen
in two places, random variation of each
person around their group mean and each
group mean around the grand mean.
The effect of the IV is seen as adding further
variation of the group means around their
grand mean so that the F-test is really: Statistical Inference
and the F-test
effect + errorBG
F=
errorWG
If there is no effect of the IV than the
equation breaks down to just:
errorBG
F=
≈1
errorWG
which means that any differences between the
groups is due to chance alone. Statistical Inference
and the F-test
The F-distribution is based on having
a between groups variation due to the
effect that causes the F-ratio to be
larger than 1.
Like the t-distribution, there is not a
single F-distribution, but a family of
distributions. The F distribution is
determined by both the degrees of
freedom due to the effect and the
degrees of freedom due to the error. Statistical Inference
and the F-test Assumptions of the analysis
Robust – a robust test is one that is
said to be fairly accurate even if the
assumptions of the analysis are not
met. ANOVA is said to be a fairly
robust analysis. With that said… Assumptions of the analysis
Normality of the sampling distribution of
means
This assumes that the sampling distribution of
each level of the IV is relatively normal.
The assumption is of the sampling distribution
not the scores themselves
This assumption is said to be met when there
is relatively equal samples in each cell and the
degrees of freedom for error is 20 or more. Assumptions of the analysis
Normality of the sampling distribution
of means
If the degrees of freedom for error are small
than:
The individual distributions should be
checked for skewness and kurtosis (see
chapter 2) and the presence of outliers.
If the data does not meet the distributional
assumption than transformations will need to
be done. Assumptions of the analysis
Independence of errors – the size of the error for
one case is not related to the size of the error in
another case.
This is violated if a subject is used more than once
(repeated measures case) and is still analyzed with
between subjects ANOVA
This is also violated if subjects are ran in groups. This
is especially the case if the groups are pre-existing
This can also be the case if similar people exist within
a randomized experiment (e.g. age groups) and can
be controlled by using this variable as a blocking
variable. Assumptions of the analysis
Homogeneity of Variance – since we are
assuming that each sample comes from
the same population and is only affected
(or not) by the IV, we assume that each
groups has roughly the same variance
Each sample variance should reflect the
population variance, they should be equal to
each other
Since we use each sample variance to estimate
an average within cell variance, they need to be
roughly equal Assumptions of the analysis
Homogeneity of Variance
Fmax =
An easy test to assess this assumption is:
2
largest
2
smallest
S
S
Fmax ≤10, than the variances are roughly homogenous Assumptions of the analysis
Absence of outliers
Outliers – a data point that doesn’t really
belong with the others
Either conceptually, you wanted to study
only women and you have data from a man
Or statistically, a data point does not cluster
with other data points and has undue
influence on the distribution
This relates back to normality Assumptions of the analysis
Absence of outliers
Related Documents
Basic Analysis of Variance and the General Linear Model
of 40
Recommended Documents
Report
Tell us what’s wrong with it:
Thanks, got it!
We will moderate it soon!
Free up your schedule!
Our EduBirdie Experts Are Here for You 24/7! Just fill out a form and let us know how we can assist you.
Take 5 seconds to unlock
Enter your email below and get instant access to your document