SPSS混合线性模型课件.ppt

1、1Mixed Analysis of Variance Models with SPSSRobert A.Yaffee,Ph.D.Statistics,Social Science,and Mapping GroupInformation Technology Services/Academic Computing ServicesOffice location:75 Third Avenue,Level C-3Phone:212-998-34022Outline1.Classification of Effects2.Random Effects1.Two-Way Random Layout

2、2.Solutions and estimates3.General linear model1.Fixed Effects Models1.The one-way layout4.Mixed Model theory1.Proper error terms5.Two-way layout6.Full-factorial model1.Contrasts with interaction terms2.Graphing Interactions3Outline-Contd Repeated Measures ANOVA Advantages of Mixed Models over GLM.4

3、Definition of Mixed Models by their component effects1.Mixed Models contain both fixed and random effects2.Fixed Effects:factors for which the only levels under consideration are contained in the coding of those effects3.Random Effects:Factors for which the levels contained in the coding of those fa

4、ctors are a random sample of the total number of levels in the population for that factor.5Examples of Fixed and Random Effects1.Fixed effect:2.Sex where both male and female genders are included in the factor,sex.3.Agegroup:Minor and Adult are both included in the factor of agegroup4.Random effect:

5、1.Subject:the sample is a random sample of the target population6Classification of effects1.There are main effects:Linear Explanatory Factors 2.There are interaction effects:Joint effects over and above the component main effects.78Classification of Effects-contdHierarchical designs have nested effe

6、cts.Nested effects are those with subjects within groups.An example would be patients nested within doctors and doctors nested within hospitalsThis could be expressed bypatients(doctors)doctors(hospitals)910Between and Within-Subject effectsSuch effects may sometimes be fixed or random.Their classif

7、ication depends on the experimental designBetween-subjects effects are those who are in one group or another but not in both.Experimental group is a fixed effect because the manager is considering only those groups in his experiment.One group is the experimental group and the other is the control gr

8、oup.Therefore,this grouping factor is a between-subject effect.Within-subject effects are experienced by subjects repeatedly over time.Trial is a random effect when there are several trials in the repeated measures design;all subjects experience all of the trials.Trial is therefore a within-subject

9、effect.Operator may be a fixed or random effect,depending upon whether one is generalizing beyond the sampleIf operator is a random effect,then the machine*operator interaction is a random effect.There are contrasts:These contrast the values of one level with those of other levels of the same effect

10、.11Between Subject effects Gender:One is either male or female,but not both.Group:One is either in the control,experimental,or the comparison group but not more than one.12Within-Subjects Effects These are repeated effects.Observation 1,2,and 3 might be the pre,post,and follow-up observations on eac

11、h person.Each person experiences all of these levels or categories.These are found in repeated measures analysis of variance.13Repeated Observations are Within-Subjects effects Trial 1 Trial 2 Trial 3 GroupGroup is a between subjects effect,whereas Trial is a within subjects effect.14The General Lin

12、ear Model1.The main effects general linear model can be parameterized as()()()exp(,)ijijijijiijjijYbwhereYobservation for ithgrand mean an unknown fixed parmeffect of ith value ofabeffect of jth value of b berimental errorN2015A factorial modelIf an interaction term were included,the formula would b

13、eijiiijijyeThe interaction or crossed effect is the joint effect,over and above the individual main effects.Therefore,the main effects must be in the model for the interaction to be properly specified.()()i jijijyy16Higher-Order InteractionsIf 3-way interactions are in the model,then the main effect

14、s and all lower order interactions must be in the model for the 3-way interaction to be properly specified.For example,a 3-way interaction model would be:ijkijkijikjkijkijkyabcabacbcabce17The General Linear Model In matrix terminology,the general linear model may be expressed asYXwhereYtheobserved d

15、atavectorXthedesignmatrixthevectorof unknown fixed effect parametersthevectorof errors18AssumptionsOf the general linear model()var()var()()EIYIE YX22019General Linear Model Assumptions-contd1.Residual Normality.2.Homogeneity of error variance3.Functional form of Model:Linearity of Model4.No Multico

16、llinearity5.Independence of observations6.No autocorrelation of errors 7.No influential outliersWe have to test for these to be sure that the model is valid.We will discuss the robustness of the model in face of violations of these assumptions.We will discuss recourses when these assumptions are vio

17、lated.20Explanation of these assumptions1.Functional form of Model:Linearity of Model:These models only analyze the linear relationship.2.Independence of observations3.Representativeness of sample4.Residual Normality:So the alpha regions of the significance tests are properly defined.5.Homogeneity o

18、f error variance:So the confidence limits may be easily found.6.No Multicollinearity:Prevents efficient estimation of the parameters.7.No autocorrelation of errors:Autocorrelation inflates the R2,F and t tests.8.No influential outliers:They bias the parameter estimation.21Diagnostic tests for these

19、assumptions1.Functional form of Model:Linearity of Model:Pair plot2.Independence of observations:Runs test3.Representativeness of sample:Inquire about sample design4.Residual Normality:SK or SW test5.Homogeneity of error variance Graph of Zresid*Zpred6.No Multicollinearity:Corr of X7.No autocorrelat

20、ion of errors:ACF8.No influential outliers:Leverage and Cooks D.22Testing for outliersFrequencies analysis of stdres cksd.Look for standardized residuals greater than 3.5 or less than 3.5 And look for Cooks D.23Studentized Residuals()()()isiiisiiieeshwhereestudentized residualsstandard deviationwher

21、eithobsisdeletedhleverage statistic21Belsley et al(1980)recommend the use of studentizedResiduals to determine whether there is an outlier.24Influence of Outliers1.Leverage is measured by the diagonal components of the hat matrix.2.The hat matrix comes from the formula for the regression of Y.()(),Y

22、XXX XX Ywhere XX XXthe hatmatrix HThereforeYHY1125Leverage and the Hat matrix1.The hat matrix transforms Y into the predicted scores.2.The diagonals of the hat matrix indicate which values will be outliers or not.3.The diagonals are therefore measures of leverage.4.Leverage is bounded by two limits:

23、1/n and 1.The closer the leverage is to unity,the more leverage the value has.5.The trace of the hat matrix=the number of variables in the model.6.When the leverage 2p/n then there is high leverage according to Belsley et al.(1980)cited in Long,J.F.Modern Methods of Data Analysis(p.262).For smaller

24、samples,Vellman and Welsch(1981)suggested that 3p/n is the criterion.26Cooks D1.Another measure of influence.2.This is a popular one.The formula for it is:()iiiiiheCook s Dphsh22111Cook and Weisberg(1982)suggested that values of D that exceeded 50%of the F distribution(df=p,n-p)are large.27Cooks D i

25、n SPSSFinding the influential outliersSelect those observations for which cksd (4*p)/n Belsley suggests 4/(n-p-1)as a cutoffIf cksd (4*p)/(n-p-1);28What to do with outliers1.Check coding to spot typos2.Correct typos3.If observational outlier is correct,examine the dffits option to see the influence

26、on the fitting statistics.4.This will show the standardized influence of the observation on the fit.If the influence of the outlier is bad,then consider removal or replacement of it with imputation.29Decomposition of the Sums of Squares1.Mean deviations are computed when means are subtracted from in

27、dividual scores.1.This is done for the total,the group mean,and the error terms.2.Mean deviations are squared and these are called sums of squares3.Variances are computed by dividing the Sums of Squares by their degrees of freedom.4.The total Variance=Model Variance+error variance30Formula for Decom

28、position of Sums of SquaresSS total =SS error +SSmodel.()(.)()()(.)()()(.)i jijjji jijjji jijjjyyyyyytotaleffecterror withinmodel effectwe square the termsyyyyyyand sum them over the data setyyyyyySStotalSSerrorGroupSSwhere SSSumsof222222Squares31Variance DecompositionDividing each of the sums of sq

29、uares by their respective degrees of freedom yields the variances.Total variance=error variance+model variance.in fixed effects modelsmodelvarianceFerrorvarianceSStotalSSerrorSSmodelnnkk1132Proportion of Variance ExplainedR2 =proportion of variance explained.SStotal=SSmodel+SSerrrorDivide all sides

30、by SStotalSSmodel/SStotal =1-SSError/SStotalR2=1-SSError/SStotal33The Omnibus F testThe omnibus F test is a test that all of the means of the levels of the main effects and as well as any interactions specified are not significantly different from one another.Suppose the model is a one way anova on

31、breakingpressure of bonds of different metals.Suppose there are three metals:nickel,iron,andCopper.H0:Mean(Nickel)=mean(Iron)=mean(Copper)Ha:Mean(Nickel)ne Mean(Iron)or Mean(Nickel)ne Mean(Copper)or Mean(Iron)ne Mean(Copper)34Testing different Levels of a Factor against one another Contrast are test

32、s of the mean of one level of a factor against other levels.:aHH012312231335Contrasts-contd A contrast statement computes()()L L V LLZZFrank L1 The estimated V-is the generalized inverse of the coefficient matrix of the mixed model.The L vector is the kb vector.The numerator df is the rank(L)and the

33、 denominatordf is taken from the fixed effects table unless otherwisespecified.36Construction of the F tests in different modelsThe F test is a ratio of two variances(Mean Squares).It is constructed by dividing the MS of the effect to betested by a MS of the denominator term.The divisionshould leave

34、 only the effect to be tested left over as a remainder.A Fixed Effects model F test for a=MSa/MSerror.A Random Effects model F test for a=MSa/MSabA Mixed Effects model F test for b=MSa/MSabA Mixed Effects model F test for ab=MSab/MSerror37Data format The data format for a GLM is that of wide data.38

35、Data Format for Mixed Models is Long39Conversion of Wide to Long Data Format Click on Data in the header bar Then click on Restructure in the pop-down menu40A restructure wizard appearsSelect restructure selected variables into cases and click on Next41A Variables to Cases:Number of Variable Groups

36、dialog box appears.We select one and click on next.42We select the repeated variables and move them to the target variable box43After moving the repeated variables into the target variable box,we move the fixed variables into the Fixed variable box,and select a variable for case idin this case,subje

37、ct.Then we click on Next44A create index variables dialog box appears.We leave the number of index variables to be created at one and click on next at the bottom of the box45When the following box appears we just type in time and select Next.46When the options dialog box appears,we select the option

38、 for dropping variables not selected.We then click on Finish.47We thus obtain our data in long format48The Mixed Model The Mixed Model uses long data format.It includes fixed and random effects.It can be used to model merely fixed or random effects,by zeroing out the other parameter vector.The F tes

39、ts for the fixed,random,and mixed models differ.Because the Mixed Model has the parameter vector for both of these and can estimate the error covariance matrix for each,it can provide the correct standard errors for either the fixed or random effects.49The Mixed ModelyXZwherefixed effects parameter

40、estimatesXfixed effectsZRandom effects parameter estimatesrandom effectserrorsVariance of yVZGZRG and R require covariancestructurefitting50Mixed Model Theory-contdLittle et al.(p.139)note that u and e are uncorrelated random variables with 0 means and covariances,G and R,respectively.,()()Because t

41、hecovariance matrixVZGZRthe solution forX VXX VyuGZ VyX11V-is a generalized inverse.Because V is usually singular and noninvertible AVA=V-is an augmented matrix that is invertible.It can later be transformed back to V.The G and R matrices must be positive definite.In the Mixed procedure,the covarian

42、ce type of the random(generalized)effects defines the structure of G and a repeated covariance type defines structure of R.51Mixed Model Assumptions0uE 00uGVarianceR A linear relationship between dependent and independent variables52Random Effects Covariance Structure This defines the structure of t

43、he G matrix,the random effects,in the mixed model.Possible structures permitted by current version of SPSS:Scaled Identity Compound Symmetry AR(1)Huynh-Feldt53Structures of Repeated effects(R matrix)-contdVariance Components2122232400 000 0000000Compound Symmetry222221111222221211222221131222221114(

44、)AR 2322321111154Structures of Repeated Effects(R matrix)HuynhFeldt22222131212222223212222223132322222255Structures of Repeated effects(R matrix)contdunstructured 21121213132212122323231313232356R matrix,defines the correlation among repeated random effects.R 211121112111211121112111One can specify

45、the nature of the correlation among therepeated random effects.57GLM Mixed ModelThe General Linear Model is a special case of theMixed Model with Z=0(which means thatZu disappears from the model)and 2RI58Mixed Analysis of a Fixed Effects modelSPSS tests these fixed effects just as it does with the G

46、LMProcedure with type III sums of squares.We analyze the breaking pressure of bonds made from three metals.We assume that we do not generalize beyond our sample and that our effects are all fixed.Tests of Fixed Effects is performed with the help of the L matrix by constructing the following F test:(

47、)()L L X VXLLFrank L1Numerator df=rank(L)Denominator df=RESID(n-rank(X)df=Satherth 59Estimation:Newton ScoringiisHgwhereggradientmatrixofst derivativesHHessian matrixofnd derivativessincrementof step parameter111260Estimation:Minimization of the objective functions11111111(,):log|log(1 log(2/)2221(,

48、):log|log|22log 1 log|2/()|22()()(nnML G RVr V rnnREML G RVX VXnpnpr V rnpwhere ryX X VXX Vyprank of Xso that the probabilities ofX VXX Vy andGZ V1().yXare maximizedUsing Newton Scoring,the following functions are minimized61Significance of Parameters11111:0Lis a linear combinationHotLCLwhereX R XX

49、R ZCZ R XZR ZG 62Test one covariance structure against the other with the IC The rule of thumb is smaller is better-2LL AIC Akaike AICC Hurvich and Tsay BIC Bayesian Info Criterion Bozdogans CAIC63Measures of Lack of fit:The information Criteria-2LL is called the deviance.It is a measure of sum of s

50、quared errors.AIC=-2LL+2p(p=#parms)BIC=Schwartz Bayesian Info criterion=2LL+plog(n)AICC=Hurvich and Tsays small sample correction on AIC:-2LL+2p(n/(n-p-1)CAIC=-2LL+p(log(n)+1)64Procedures for Fitting the Mixed Model One can use the LR test or the lesser of the information criteria.The smaller the in