Data Screening Check List

Data Screening Check List Name of Data file ________________________________________________________ 1. Accuracy of the Data file – Was the data entered correctly? If possible the data should be proofread against the original data (on the questionnaires, etc) to check that item has been entered correctly. Preferably someone other than the person who entered the data should do this. Date Verified ___________________ Verifier Name (please print)______________________ 2. Missing Data – the important thing in dealing with missing data is to figure out if the data is missing randomly or if there is some pattern (reason) to why the data points are missing. a. If no pattern can be found by looking at the data run this test. i. Step 1: Dummy code a variable that puts those missing in one group and those remaining in another ii. Step 2: Do a T-test using other related variables as a DV (i.e. if there is a missing value on ethnicity test the groups on attitudinal scales). iii. Step 3: If there is no difference than you can feel safe deleting subjects (or selecting subjects out of the analysis) with missing values (given that there are no too many) or if the missing values are concentrated to one variable consider deleting the variable instead. If there is a significant difference then other steps need to be taken to replace the missing values (see below). Test Ran: yes no Date: ______________ Significant: yes no Missing: left deleted selected out b. If there is a pattern to the missing data or there are too many missing (there is no strict guideline to how many is too many) choose one of the following options: i. Option 1: Replace values with numbers that are known from prior knowledge or from an educated guess. Easily done but can lead to researcher bias if your not careful. ii. Option 2: Replace missing values with variable mean. The simplest option but it does lower variability and in turn can bias results. iii. Option 3: Replace missing values with a group mean (i.e. the mean for prejudice grouped by ethnicity). The missing value is replaced with the mean of the group that the subject belongs to. A 1 little more complicated but there is not as much of a reduction in the variability. iv. Option 4: Using regression to predict the missing values. Other variables act as IVs predicting the variable with the missing values (which acts as the DV). This method is only works if there is significant prediction of the variable with the missing values. Variability is still reduced but it is more objective than a guess and not as blind as inserting a mean. Option used: 1 2 3 4 Date values imputed: __________________ Name of imputer: _____________________ Note: You may want to create a dummy variable that keeps track of the missing values so it can be used as a variable later (Differences between complete and incomplete subjects, etc). It is also wise to run the analysis with and without missing values. If there is a difference in the results then you may want to investigate. 3. Outliers - These are case scores that are extreme and therefore have a much higher impact on the outcome of any statistical analysis. In order to avoid biased results the data set must be checked for both univariate (outliers on one variable alone) and multivariate (outliers on a combination of variables) outliers. a. Four basic reasons you’d get an outlier i. There was a mistake in data entry (a 6 was entered as 66, etc.), hopefully step one above would have caught all of these. ii. The missing values code was not specified and missing values are being read as case entries (99 is typically the missing value code but it will be read as an entry and possibly an outlier if the computer is not told that these are missing values, in SPSS 10 variable view click on the missing values box and select discrete missing values, in the first box enter 99, click on OK). iii. The outlier is not part of the population from which you intended to sample (you wanted a sample of 10 year olds and the outlier is a 12 year old). In this case the outliers should be removed from the sample. iv. The outlier is part of the population you wanted but in the distribution it is seen as an extreme case. In this case you have three choices 1) delete the extreme cases or 2) change the outliers’ scores so that they are still extreme but they fit within a normal distribution (for example: make it a unit larger or smaller than last case that fits in the distribution) 3) if the outliers seem to part of an overall non-normal distribution than a transformation can be done but first check for normality (see below). 2 b. Detecting Outliers i. Among dichotomous variables – If you have a dichotomous variable with an extremely uneven split (i.e. 90 – 10 split, 90% say yes and 10% say no) this will produce an outlier. The only fix for this is to delete the variable. This is easily identified by SPSS FREQUENCIES. List any dichotomous variables with uneven splits (delete any in excess of a 90 – 10 split). If you need more room attach a sheet to the back of this worksheet. Name ______________________ ii. Among continuous variables – whether searching for univariate or multivariate outliers the method depends on whether the data is grouped or not. If you are performing analyses with ungrouped data (i.e. regression, canonical correlation, factor analysis, or structural equations modeling) univariate and multivariate outliers are sought among all cases at once. If you are going to perform on of the analyses with grouped data (ANOVA, ANCOVA, MANOVA, MANCOVA, profile analysis, discriminant function analysis, or logistic regression) both univariate and multivariate outliers are sought within each group separately. 1. Univariate outliers are those with very large standardized scores (z scores greater than 3.3) and that are disconnected from the distribution. SPSS DESCRIPTIVES will give you the z scores for every case if you select save standardized values as variables and SPSS FREQUENCIES will give you histograms (use SPLIT FILE/ Compare Groups under DATA for grouped data). List all univariate outliers here and how they were handled. Attach a sheet if you need more room. (Note: it is wise to check if a univariate outlier is also a multivariate outlier before making any decisions about what to do with it.) Describe where they were found (what type of analysis, grouped, ungrouped, “A Case grouped by B”, etc.) Number Z score *Only after a check for normality is performed. Reason for being an outlier (refer to 1-4 above). How handled (fixed, changed, deleted, *transformed) Name _________________________ 3 2. Multivariate Outliers are found by first computing a Mahalanobis Distance for each case and once that is done the Mahalanobis scores are screened in the same manner that univariate outliers are screened. a. To compute Mahalanobis distance in SPSS you must use REGRESSION/ LINEAR under ANALYZE. Use a dummy variable as the DV and all variables that need to be screened as IVs, under SAVE/ Distances check the Mahalanobis box. There should be a new variable saved in you data set. For grouped data the Mahalanobis distances must be computed separately for each group. List all Multivariate outliers here and how they were handled. If you need more room attach a sheet. Describe where they were found (what type of analysis, grouped, ungrouped, “A grouped by B”, etc.) Case Number Mahalanobis Score *Only after a check for normality is performed. 4. Z score How handled (fixed, changed, deleted, *transformed) Name __________________________ Normality – The data needs to follow a normal distribution in order for most analyses to work properly. Even in situations where normality is not required if normality exists it will make for a stronger assessment. There are two aspects to normality of a distribution, skewness and kurtosis, and both must be tested before normality can be established. a. Skewness – this describes how unevenly the data is distributed with a majority of scores piled up on one side of the distribution and a few stragglers off in one tail of the distribution. Skewness is often but not always caused by outliers, which hopefully were taken care of in step 3. i. Skewness test – in SPSS DESCRIPTIVES/ EXPLORE skewness and the standard error for skewness are given as a default. By hand divide the skewness value by the standard error for skewness and what you get is a z score for skewness. If the number is greater than 3.3 then there is a problem. Note: skewness tends to have more influence on analyses then kurtosis. b. Kurtosis – this describes how “peaked” or “flat” a distribution is. If too many or all of the scores are piled up on the or around the mean then the distribution is too peaked and it is not normal, vice versa for when a distribution is too flat. 4 i. Kurtosis test - in SPSS DESCRIPTIVES/ EXPLORE kurtosis and the standard error for kurtosis are given as a default. By hand divide the kurtosis value by the standard error for kurtosis and what you get is a z score for kurtosis. If the number is greater than 3.3 then there is a problem. Note: the larger the sample size you are using the more likely you will get violations of skewness and/or kurtosis with just small deviations. With larger sample sizes you may want to use less conservative number than a z-score of 3.3. c. How to Fix Non-Normality – If a variable is not distributed normally than a transformation can be used as a correction. Transformations will be discussed below. 5. Homoscedasticity, Homogeneity of Variance and Homogeneity of VarianceCovariance Matrices – If you can say the first word in the title of this section you have a head start. All three of these are related in that they are similar types of assumptions that need to be met before an analysis can be interpreted correctly. The question is which assumption goes with what analysis? a. Homoscedasticity – this is an assumption for analyses using ungrouped univariate data. Variables are said to be homoscedastic when “the variability in scores for one continuous variable is roughly the same at all values of another continuous variable”. This is related to the assumption of normality because if both variables are normally distributed than you should have homoscedasticity. There is no formal test for this, but is can be seen graphically. “The bivariate scatterplots between two variables are roughly the same width all over with some bulging toward the middle” (see page 81). In SPSS GRAPHS choose SCATTERPLOT/ SIMPLE and enter two variables in the X and Y axes. Heteroscedasticity is corrected by transformations, which will be discussed below. Pair of variables to be checked. Pass / Fail Researcher Name b. Homogeneity of Variance – This assumption is important when you have grouped data. The assumptions states that the “variability in the DV is expected to be about the same at all levels of the grouping variable (IV)”. In most analyses that require this SPSS gives the Levene’s test as a measure of homogeneity of variance. Reaching a significant value (p < .05) on the Levene’s test means that you have heterogeneity of variance, 5 6. 7. but the test is very conservative. So you either should increase the alpha level to .001 for the test or better yet apply the F-Max test. If the sample sizes in the cells are roughly equal (4 to 1 or less) then take the largest cell variance and divide it by the smallest cell variance and if that number is less than 10 the assumption has been met. Note: If Levene’s test is passed there is no need to run F-Max test. Failure to achieve homogeneity of variance can be corrected by 1) increasing the alpha level of you test (.25, .001, etc) or 2) through transformation (which is discussed below). c. Homogeneity of Variance-Covariance Matrices – this is a multivariate assumption that is similar to homogeneity of variance. It roughly states that an entry in a variance-covariance matrix using one DV should be similar to the same entry in a matrix using another DV. The formal test for this in SPSS is Box’s M, it is given automatically any time the assumption is needed. It is a highly conservative test that is far too strict with large samples of data. It is better to increase the alpha level to .01 or .001. Multicollinearity and Singularity – Both of these have to do with correlations among variables. If you have a correlation between two variables that is .90 or greater they are multicollinear, if two variables are identical or one is a subscale of another they are singular. Either way you cannot have variables that are multicolliear or singular in the same analysis because the analysis will not work (I will spare you the explanation). There are two things you can do to find out 1) run bivariate correlations between all of your variables and make sure they are neither subscales of another or above .90 or 2) run your analysis and wait and see if you get and error/warning message telling you that you have multicollinearity/singularity. The only way to fix this is to drop one or the variables from the analysis. Common Data Transformations – transformations are recommended as a last resort only because of the added difficulty of interpreting a transformed variable. There are three common types of transformations 1) square root, used when there is moderate skewness/ deviation, 2) logarithm, used when there substantial skewness/ deviation and 3) inverse, used when there is extreme skewness/ deviation. The best approach is to try square root first to see if that fixes your problem, if not then move to logarithm and then to inverse until you have satisfactory normality/ homoscedasticity. 6 Common Transformations in SPSS Condition of Variable SPSS Compute Language Moderate Positive Skewness NEWX=SQRT(X) Substantial Positive Skewness NEWX=LG10(X) Variable includes 0 NEWX=LG10(X + C) Severe Pos Skewness, L Shaped NEWX=1/(X) Variable includes 0 NEWX=1/(X + C) Moderate Negative Skewness NEWX=SQRT(K - X) Substantial Negative Skewness NEWX=LG10(K - X) Severe Neg Skewness, J Shaped NEWX=1/(K - X) List variables that apply C = constant added to each score so that the smallest score is 1. K = constant from which each score is subtracted so that the smallest score is 1; usually equal to the largest score + 1. 7