home > supplemental chapters > estimation of nonlinear effects

Supplemental Chapter A:
Nonlinear Effects of Observed and Latent Variables

Kline, R. B. (1998). Principles and practice of structural equation modeling.
New York:
Guilford Press.

Copyright © 2000 Rex B. Kline. All rights reserved.

Beauty is a function of truth, truth a function of beauty. They can be separated by analysis
 but in the lived experience of the creative act—and of its re-creative echo in the beholder
—they are as inseparable as thought is inseparable from emotion.

Arthur Koestler (1989/1964, p. 331)

Comment to author

Note About SEM Programs

Since these supplemental chapters were written in 1998, new versions of all SEM programs mentioned here, including Amos, EQS, LISREL, and Mx (now called Mplus), have been released. All of these programs have capabilities beyond those described here. For more information, follow the links below.


Contents


A.1 Foreword

All of the effects in models considered up to this point have been assumed to be linear. This chapter covers the analysis of nonlinear effects in SEM. Recall from discussion in Chapter 2 (section 2.6) that there are two kinds of nonlinear effects, curvilinear relations that involve only two variables and interactive effects that involve at least three. Interaction effects are also called moderator effects that occur when, in their most basic form, the relation of one variable to another changes across the levels of a third. Although any multi-sample SEM can be viewed as a test of group membership as a (categorical) moderator variable, the interaction effects considered in this chapter are between variables represented within models that can be estimated within a single sample.

The first part of this chapter concerns the estimation of nonlinear effects in structural models of observed variables (i.e., path models). The second part extends this basic rationale to the representation and analysis of nonlinear effects of latent variables in hybrid models. Just as the estimation of linear effects of latent variables in standard hybrid models takes measurement error into account, the same is also true of the analysis of the nonlinear effects with the hybrid models considered in this chapter. The capabilities just mentioned also distinguish the SEM approach to the study of nonlinear relations. For example, although both multiple regression and ANOVA can analyze nonlinear effects of observed variables, neither can readily do so with a multiple indicator approach that controls for measurement error.

A few words before we begin. The topic of nonlinear effects is a broad one, and it is not possible to cover all aspects of it in a single chapter. Instead, the goal here is more modest, to introduce the estimation of nonlinear effects from the perspective of SEM. Also, although methods to estimate nonlinear effects of observed variables are fairly well established, this is not the case for latent variables. Some methods are discussed and examples of their application are reviewed, but there are relatively few examples of this type of analysis in the SEM literature. Thus, a sense of pioneer spirit would be a healthy perspective to bring to this particular subject. The payoff, though, may be an increased awareness of possibilities to test hypotheses about latent variables.


A.2 Nonlinear Effects of Observed Variables

Introduction

Multiple regression offers a straightforward way to represent curvilinear and interactive effects of observed variables. This method not only underlies the evaluation of trend and interaction effects in ANOVA (which is a special case of multiple regression), it can also be used in SEM. In the examples that follow, it is assumed that all variables are continuous. (The case of categorical exogenous variables is considered later.) Let's start with curvilinear relations.

Suppose that variable X is children's age and variable Y is the size of their vocabularies. If vocabulary size increased uniformly with age, the relation between X and Y would be solely linear. In reality, though, the rate of increase in vocabulary breadth is greater for younger children, which adds a curvilinear component to the relation between X and Y, specifically a quadratic one. Now suppose that X is the only predictor of Y in a regression equation. The regression coefficient for X reflects just the linear aspect of its relation to Y. To also represent the quadratic relation of X to Y, all that is necessary is to create the product variable X2 (literally, square each child's age) and enter it along with X as a predictor of Y in a multiple regression equation. The presence of X2 in the equation adds one "bend" to the regression line, and its regression coefficient estimates the magnitude of the quadratic aspect of the relation between X and Y, in this case the relation between age and shoe size.

As an example, consider the raw scores presented at the top of Table A.1. The relation between X and Y is obviously quadratic: Scores on Y decline and then rise as scores on X increase. Regressing Y on X yields a Pearson correlation of only -.10 and a non-significant unstandardized regression coefficient for X of -.04. However, if Y is regressed on both X and X2, the (multiple) correlation increases to .93 and both regression coefficients are significant. The results with X2 in the equation are so different because this product variable represents the substantial quadratic relation between X and Y, and the regression coefficient for X2 (controlling for X) estimates the magnitude of this effect. With just X as a predictor, only its (weak) linear association with Y is estimated.

Table A.1 Quadratic and interactive effects between observed variables
represented with product variables.
  Quadratic Effect  
  Predictor Product Criterion  
Subject X X2 Y  
A 5 25 15  
B 7 49 14  
C 10 100 11  
D 13 169 9  
E 15 225 9  
F 17 289 5  
G 19 361 8  
H 23 529 11  
I 25 625 14  
J 26 676 14  
Unstandardized
regression coefficients:
.04 rXY = -.10  
  2.20** .07** RY•X, X2 = .93**  
  Interaction Effect
  Predictors Product Criterion
Subject X W XW Y
A 2 10 20 5
B 4 12 48 7
C 6 12 72 9
D 8 13 104 11
E 11 10 110 10
F 2 22 44 10
G 4 24 96 11
H 7 19 133 9
I 8 18 144 7
J 11 25 275 5
Unstandardized
regression coefficients:
-.01 .02 RY•X, W = .05
  1.54** .62** -.09** RY•X, W, XW = .89*
*p<.05 **p<.01        
Table A.1 Quadratic and interactive effects between observed variables
represented with product variables.

The same approach can be used to represent interactive effects of observed variables. As an applied example, consider a study by Frederiksen and Melville (1954). These authors used scores on a measure of interest in engineering to predict grade-point average after one year of engineering school. The overall correlation between interest and achievement in their sample was only .10. However, Frederiksen and Melville speculated that the relation between interest and achievement varies as a function of a third variable, compulsiveness. That is, very compulsive students may invest a lot of energy in non-productive, ritualistic behavior that may interfere with the expression of their interest through their school work. Using a measure of compulsiveness administered at the same time as the interest measure, Frederiksen and Melville divided their sample into two groups, high and low compulsive. For the high-compulsive students, the correlation between interest and achievement was essentially zero (-.01), but for the low-compulsive group the correlation was greater, .25. Thus, interest interacts with compulsiveness in the prediction of achievement. Specifically, the magnitude of the association between interest and achievement decreases as compulsiveness increases.

Partitioning subjects into two groups based on their scores on a continuous variable W (e.g., compulsiveness in the above example) to evaluate whether the relation between X and Y is different within each group is potentially problematic. Dichotomizing a continuous moderator variable W not only results in the loss of numerical information, it may also lead to an inaccurate reflection of the interaction (Cohen & Cohen, 1983). For example, suppose that the relation between X and Y gradually changes across the levels of W or changes abruptly (e.g., from positive to negative) at a point on W that is not where the researcher dichotomized the scores. In either case, the evaluation of whether dichotomized variable W moderates the relation between X and Y may be biased. The same problems may also arise if the subjects are partitioned into three or more groups on a continuous variable W.

It is possible to analyze an interactive effect of continuous variables without having to categorize either one. This approach involves the creation of a product variable that represents the interactive effect. Suppose that X and W are two continuous predictors of Y. When Y is regressed on both X and W, the regression coefficients for the predictors are estimates of their linear relations to Y adjusted for their correlation with each other. Let XW represent a product variable created by multiplying each subject's scores on X and W. The product term XW represents the interactive effect of its component variables. Specifically, if Y is regressed on X, W, and XW, then (1) the regression coefficient for XW estimates the magnitude of the interaction between X and W; and (2) the regression coefficients for X and W estimate their linear relations to Y adjusted for their correlation and interaction. Likewise, the regression coefficient for the product term XW controls for the linear effects of both X and W. A regression coefficient for XW that is significant indicates that the relation of X to Y changes as a function of W. It could, as in the example described above, change from positive to zero across the levels of W. Other patterns are possible (e.g., the relation of X to Y changes from negative to positive across W), but the key point here is that relation between X and Y is moderated by W. Because an interaction effect is a joint one, however, X also moderates the relation of W to Y. For this reason, the use of multiple regression with product variables like XW to estimate interactions is sometimes called moderated multiple regression.

As an example of the use of a product variable to represent an interaction effect with moderated multiple regression, consider the raw data presented in the lower part of Table A.1. If X and W are the only predictors of Y, then neither has a significant regression coefficient and the multiple correlation is only .05. Inspection of the raw scores in the table indicates that the relation of X to Y is positive for subjects with low scores on W ( <12) but is negative at higher levels of W. Although not as apparent, there is a similar change in the direction of the linear relation between W and Y: positive at higher levels of X, negative at lower. When X, W, and XW are all in the equation as predictors of Y, the multiple correlation increases to .89 (from just .05 without XW) and the regression coefficients of all three predictors are significant.

Three additional points about the representation of nonlinear effects of observed variables with product variables. First, its is possible to estimate higher-order curvilinear or interaction effects than those considered in the previous examples. For example, the product variable X3 represents the cubic relation of X to Y; X3 a quartic relation; and so on. Product variables that represent quadratic or higher-order curvilinear effects are sometimes called power polynomials. To test a higher-order power polynomial (e.g., X3), all lower-order power terms should be in the equation (e.g., X and X2). In the social sciences, however, it is relatively rare that power terms beyond quadratic ones are significant.

The same principle just described above also applies to higher-order interactions. For instance, the product variable i represents the linear×linear interaction of X and W. If significant, it means that the linear relation between X and Y changes uniformly (i.e., in a linear way) across the levels of W. In contrast, the product term XW2 represents a linear×quadratic interaction, which means that the linear relation of X to Y changes faster at higher (or lower) levels of W. Estimation of the interaction effect represented by the term XW2 requires that X, W, XW, and W2 (i.e., all lower-order effects) are all in the equation. It is also possible to estimate three-way interactive effects with product variables like XWU, which represents a linear×linear interaction between X and W that changes in a uniform way across the levels of U. Entry of XWU into a regression equation requires the presence of product variables that represent all lower linear and two-way interactions (six in total). This approach can be extended even further (e.g., four-way interactions), but it should be clear that the representation of higher-order interactions can require the addition of numerous product variables to the model, which can greatly increase its complexity. (More about this point later). Also, it is relatively infrequent in social science research that such higher-order, multi-variable interaction effects are large in magnitude or statistically significant.

Second, product terms like X2 and XW can be so highly correlated with their component variables as to cause problems due to multicollinearity. For example, the correlation between X and X2 for the first data set of Table A.1 is .98, which is extremely high. There are ways, however, to create product terms that reduce or eliminate correlations with their component variables. For example, Lance (1988) and Aiken and West (1991) demonstrate how to create a product term like XW that is uncorrelated with X and W but nevertheless represents an interaction effect. This method is described later. For product variables like X2 that reflect curvilinear relations, Cohen and Cohen (1983, pp. 237-238, 242-249) describe additional methods to reduce multicollinearity, such as re-scaling X so that its mean is zero before computing X2.

Finally, product variables can also represent interactive effects that involve categorical variables. For example, if X and W are both dichotomous variables, then the analysis of X, W, and XW in the same regression equation is essentially the same thing as a 2×2 ANOVA. The dichotomy represented by either X or W could reflect an experimental manipulation (e.g., random assignment to treatment versus control) or a non-experimental variable (e.g., gender). The same concept applies to the case where, say, X is continuous and W is a categorical variable. The analysis of continuous X, categorical W, and the XW product term with multiple regression is thus identical to an analysis of covariance (ANCOVA) with a test of its assumption of homogeneity of regression. (See Cohen & Cohen, 1983, and Keppel & Zedeck, 1989, for more information about how any type of ANOVA or ANOCOVA is a special case of multiple regression.) Thus, product terms can represent the whole range of interactive effects of observed variables.

Representation in Path Models

Nonlinear effects of observed variables are represented in path models with the appropriate product terms of the types described above. Consider the two path models presented in Figure A.1. Path model (a) represents X and X2 as exogenous and Y as endogenous. Note that the unanalyzed association is represented in this model with a dashed line instead of solid one. This represents the possibility mentioned earlier that a power term can be created so that it has reduced or even zero correlations with its component variables. Path coefficients for the direct effects of X on Y and of X2 on Y of model (a) of the figure respectively estimate the linear and quadratic effects of X on Y. Path model (b) of the figure depicts X, W, and the product variable XW as causes of Y. Observed exogenous variables X and W are assumed to covary. The curved lines in this model that connect the product variable to its component variables have the same meaning as for model (a). Estimates of the direct effects of model (b) reflect the linear effects of X and of W on Y and of their interactive effect on Y. Models (a) and (b) of the figure can be extended to represent all of the variations mentioned above. For example, X or W could be either dichotomous or continuous variables. Higher-order effects such as a cubic effect of X on Y (model (a)) or a linear×quadratic interaction of X and W (model (b)) could be added to these models with the appropriate product terms. Before some of these more complicated variations are considered, let's go through an example with actual data.

Figure A.1 Two path models with product variables that
represent nonlinear effects of observed variables.
Figure A.1 Two path models with product variables that
represent nonlinear effects of observed variables.

Example

At the beginning of the semester, a total of 104 students enrolled in introductory statistics courses were administered a test of their algebraic skills (Kline, 1994) and a questionnaire called the Math Anxiety Rating Scale by Suinn (1988). Higher scores on the first measure indicate better basic math skills and on the second greater anxiety about math-related content. The letter grades these students earned in statistics at the end of the semester were also recorded. Letter grades were converted to numerical values on a standard four-point scale (A=4.0, B=3.0, C=2.0, D=1.0, fail=0). Because some students withdrew from the course or enrolled in the course late in the semester after the math skill and anxiety questionnaires were administered, there are missing observations on these variables. (The application of a special estimation procedure that fits a model to these incomplete raw data is demonstrated in a later chapter.) The analysis described here concerns only the 75 cases (of 104) with complete data on all three variables.

The data for this example are summarized in Table A.2. The two predictor variables are the math skill and the math anxiety measures. The magnitudes of the absolute values of their correlations with grades (.23 and -.12, respectively) are not very large, but these correlations reflect only linear relations. To evaluate whether the relation of basic math skills to grades in introductory statistics is moderated by math-related anxiety (and vice-versa), the product variable skill×anxiety was created to represent the interaction effect. The correlation of this product variable with one of its component variables, math skill, is fairly substantial, .75. Also, the multiple correlation between the skill×anxiety product variable and both of its component variables is so high (.98) that multicollinearity may be a problem.

Table A.2 Evaluation of a path model with an interactive effect of math skill and math-related anxiety on statistics grades.
Correlations, Means, and Standard Deviations: N = 75 university students in introductory statistics courses.
             
Variable 1 2 3 4 5  
1 Math Skill          
2 Math Anxiety -.36        
3 Skill × Anxiety .75 .33      
4 Residualized Skill × Anxiety .00 .00 .20    
5 Grade .23 -.12 .15 .02  
M 10.88 212.11 2257.37 .00 2.52  
SD 3.17 45.44 679.97 136.09 1.27  
               
Example of Creating a Residualized Product Term
Case Skill Anxiety Skill×Anxiety Predicted
Skill×Anxiety
Residualized
Skill×Anxiety
Grade
1 7 188 1316 1282.358 33.642 3.7
2 13 224 2912 2743.106 168.894 3.3
3 16 146 2336 2498.600 -162.600 4.0
7 10 216 2160 2114.282 45.718 1.7
8 8 279 2232 2388.991 -156.991 3.7
9 13 186 2418 2357.216 60.784 1.0
11 11 123 1353 1352.395 .605 2.0
13 15 173 2595 2590.257 4.743 1.0  
14 9 211 1899 1880.979 18.021 3.0  
15 5 195 975 988.387 -13.387 1.3  
               
Steps
1. Regress Skill × Anxiety on Skill and Anxiety, record unstandardized regression weights.
2. Compute Predicted Skill × Anxiety = 182.528 (Skill) + 10.155 (Anxiety) - 1904.478.
3. Compute Residualized Skill × Anxiety = (Skill × Anxiety) - (Predicted Skill × Anxiety)
Table A.2 Evaluation of a path model with an interactive effect of math skill and math-related anxiety on statistics grades.

A procedure described by Lance (1988) was used to create a residualized product term that represents the interaction of math skill and math anxiety but is uncorrelated with each component variable. This method is demonstrated in Table A.2 with the first 15 cases of this data set with non-missing observations on all variables. The first step of this procedure is to create the product variable skill×anxiety. In the second step, this observed product variable is regressed on its two component variables, math skill and math anxiety. One records the unstandardized regression weights from this analysis and then derives predicted skill×anxiety scores. These predicted product scores are then subtracted from the observed product scores. The results are residualized product scores that are uncorrelated with math skill and math anxiety, which takes care of the multicollinearity problem. (See also the correlation matrix at the top of Table A.1 for the whole sample of N = 75.). The residualized product variable thus reflects the interaction of math skill and math anxiety controlling for their separate linear relations to course grades.

Presented in Figure A.2 is the path model analyzed for these data. This recursive model has three exogenous variables, math skill, math anxiety, and the residualized product variable that represents their interaction. No paths are included in the model to represent unanalyzed associations between the residualized product variable and the other two exogenous variables because these correlations are zero. (Compare Figure A.2 to model (b) of Figure A.1.)

Figure A.2 A path model with a residualized product term that represents
an interaction effect.
Figure A.2 A path model with a residualized product term that represents
an interaction effect.

A covariance matrix of these three variables and grades was submitted to a model-fitting for maximum likelihood (ML) estimation. Summarized below are the unstandardized estimates of the three direct effects and the disturbance variance for grades; the standardized estimates are reported in parentheses:

Predictor Direct Effect
on Grade
Disturbance
Variance
Math Skill .087* (.22) 1.521** (.94)a
Math Anxiety -.001 (-.04)  
Residualized Skill×Anxiety .001 (.02)  
*p < .05 **p < .01    
aThe value in parentheses is the proportion of unexplained variance.

The results reported above indicate that the only significant direct effect is the one for the positive linear direct effect of math skill on course grades. The linear effect of math-related anxiety on grades is negative but non-significant and small in absolute magnitude. The non-significant path coefficient for the interaction effect means that the relation of math skills to grades in introductory statistics is unaffected by the level of math-related anxiety. The same result also means that math anxiety is unessentially unrelated to course performance regardless of students' basic algebraic skills. Although it is not necessary for this example, there is a method for interpreting a significant interaction between two continuous variables that has been suggested by Cohen and Cohen (1983, pp. 320-324) and others (e.g., Aiken & West, 1991). Because this method is applied in an upcoming example, it is not described here.

Additional examples of the estimation of nonlinear effects of observed variables can be found in McArdle (1994), who studied the linear, quadratic, and cubic effects of age to latent cognitive ability factors, and Loehlin, Horn, and Willerman (1990), who included quadratic effects of age in models of the stability of extroversion, socialization, and emotional lability among adopted and biological children of adoptive parents. Cohen and Cohen (1983, pp. 369-371) describe the analysis of a path model of university productivity and salary that includes an interaction effect. Also, a recent chapter by Wood (1994) deals with the effects of omitted variables and their interactions on the estimation of structural models with observed variables.

Mediators and Moderators Together

Baron and Kenny (1986) and James and Brett (1984) noted that it is possible to represent in the same path model interactive (moderator) effects that are mediated by another variable (e.g., XW=>U=>Y) or interaction effects that mediate causal effects of prior variables (e.g., U=>XW=>Y). The first combination of moderators and mediators together in the same model is called mediated moderation, and the second is known as moderated mediation. Lance (1988) evaluated a path model with an interaction effect that was specified to be mediated by another variable (mediated moderation). Specifically, Lance studied the relation of memory demand (immediate versus delayed recall), complexity of social perception, and their interaction on the overall accuracy of recall of the script of a lecture. The path model tested by Lance included a fourth variable, recollection of specific behaviors mentioned in the script, through which the interaction effect was specified to affect overall accuracy of recall. Lance's results suggested that (1) complexity of social perception affected the recall of specific behaviors only in the delayed recall condition; (2) the indirect effect of the complexity×memory demand interaction was significant; but (3) the direct effects of the individual component variables on overall accuracy were not significant. For more information about path models that represent moderator and indirect effects in the same model, readers are referred to the works just cited.

It is probably apparent by now that is possible to specify and test very complex models that may have higher-order product terms or a mix of indirect (mediator) and interactive (moderator) effects. The costs of increased complexity, through, are the same as for any type of structural equation model (e.g., the need for ever larger samples, programming complexity, more potential problems with starting values). The researcher should thus have solid theoretical reasons to guide the addition of terms that reflect curvilinear or interactive term to a structural equation model.


A.3 Nonlinear Effects of Latent Variables

Introduction

The estimation of curvilinear or interactive effects of observed variables in path models is limited by the use of a single indicator of each theoretical variable and the resulting inability to take account of measurement error. The second limitation may be especially critical for product variables because they reflect the combined measurement errors of their component variables. For example, if either X or W is not very reliable, then the reliability of XW must also be poor. Although there are ways to adjust path coefficients for product terms of observed variables for measurement error (Aiken & West, 1991; Jaccard & Wan, 1995; MacCallum & Mar, 1995; McClelland & Judd, 1993, Ping, 1996b), the only way to overcome the first limitation just mentioned is to use multiple indicators of each construct. The estimation of nonlinear effects of latent variables within a multiple indicator approach to measurement is demonstrated below.

Consider model (a) of Figure A.3. In a departure from notation used in previous chapters, latent variables are represented with upper-case letters (e.g., X) and observed variables are depicted with lower-case letters (e.g., x1). Model (a) is a partially latent hybrid model that represents the linear effect of factor X on the observed exogenous variable y. (The rationale outlined here also applies to the use of multiple indicators to measure an endogenous construct.) The exogenous factor X has two indicators, x1 and x2. The measurement model for indicators x1 and x2 can also be represented with the following structural equations:

Equations [A.1]
x1 = X + E1
x2 = L2 X + E2

In Equations [A.1] above, the loading of x1 is fixed to 1.0 to scale X and the loading of x2 is a free parameter represented by L2. The other parameters of this measurement model are the variances of X and of the two measurement error terms, E1 and E2.

Suppose that a researcher wished to estimate the quadratic effect of latent variable X on the endogenous variable. This calls for adding to model (a) of the figure the latent product variable X2 that represents this quadratic effect, which is estimated by the path coefficient for the direct effect X2=>y. The indicators of latent product variables are products of observed variables. Accordingly, the indicators of X2 are all products of X's indicators that represent a quadratic effect, x12, x22, and x1x2. Note that the third product indicator just listed, x1x2, does not represent an interaction because its components (x1 and x2) are specified to measure the same factor. By squaring or taking the product of the relevant equations in [A.1] above, the measurement model for the product indicators x12, x22, and x1x2 is:

Equations [A.2]
x12 = X2 + 2XE1 + E12
x22 = L22X2 + 2L2XE2 + E22
x1x2 = L22X2 + L2XE1 + XE2 + E1E2

Equations [A.2] above show that the measurement model for the three product indicators involves not only the latent quadratic term X2 but also five additional latent product variables, XE1, XE2, E12, E22, and E1E2. (The latter three product variables are error terms.) The parameters of this measurement model are the loadings of the product indicators on these latent product variables and their variances. Note that the factor loadings in Equations [A.2] are either constants (e.g., 1.0 for X2=>x12, 2.0 for XE1=>x12) or functions of L2, which is the loading of the observed variable x2 on X from Equations [A.1]. For example, the loading of the product indicator x22 on the product factor XE2, is two times L2. Thus, no new factor loadings are needed. Also, Kenny and Judd (1984) demonstrated that the latent variable X and the six latent product variables of Equations [A.2] are uncorrelated if X and the measurement errors of its indicators (E1 and E2) are normally distributed and have means of zero. Using the same assumption, Kenny and Judd (1984) also showed that the variances of the latent product variables of Equations [A.2] are related to the variances of X and the measurement errors of its indicators as follows; note that Var below indicates variance:

Equations [A.3]
Var X2 = 2(Var X)2
Var XE1 = (Var X)(Var E1)
Var XE2 = (Var X)(Var E2)
Var E12 = 2(Var E1)2
Var E22 = 2(Var E2)2
Var E1E2 = (Var E1)(Var E2)

For example, the variance of the latent product variable X2 is two times the squared variance of X, and the variance of the latent product term E1E2 is the product of the variances of the measurement errors of the indicators of X, E1, and E2.

Figure A.3 Hybrid models with product indicators and product latent
variables that represent linear or curvilinear effects of a latent variable.
Note: Latent variables are designated by upper-case letters and
indicators by lower-case letters.
Figure A.3 Hybrid models with product indicators and product latent
variables that represent linear or curvilinear effects of a latent variable.
Note: Latent variables are designated by upper-case letters and
indicators by lower-case letters.

Model (b) in Figure A.3 includes the entire measurement model of the product indicators. Although the whole model looks complicated, there are only seven parameters in total that require estimates under the assumption of normal distributions and means of zero. These seven free parameters include the two path coefficients for the linear and quadratic direct effects of factor X on the endogenous variable y; the variance of the disturbance term for y; the variance of the latent variable X; the loading of indicator x2 on X, L2; and the variances of the measurement errors for x1 and x2, E1 and E2, respectively. That's all there is to it. As per Equations [A.2] and Equations [A.3], all of the parameters of the measurement model of the product indicators are known once you have estimates for the parameters of the measurement model of their component variables, x1 and x2. Specific estimation methods are considered after interactive effects of latent variables are introduced.

Suppose that there are two indicators each of latent exogenous variables X and W. The measurement model for these four observed variables is as follows:

Equations [A.4]
x1 = X + Ex1
x2 = Lx2X + Ex2
w1 = W + Ew1
w2 = Lw2W + Ew2

where the loadings of x1 and w1 are each fixed to 1.0 to scale their respective factors. The parameters of the above measurement model are the loadings of x2 and w2 on their respective factors (designated Lx2 and Lw2); the variances of latent variables; X, W; and the variances of the four measurement errors.

The product latent variable XW represents the linear×linear interactive effect of factors X and W. Its indicators are the products of the indicators of X and W that represent linear×linear interactions, x1w1, x1w2, x2w1, and x2w2. The equations for the measurement model of these four product indicators are the products of Equations [A.4], or:

Equations [A.5]
x1w1 = XW + XEw1 + WEx1 + Ex1Ew1
x1w2 = Lw2XW + XEw2 + Lw2WEx1 + Ex1Ew2
x2w1 = Lx2XW + Lx2XEw1+ WEx2+ Ex2Ew1
x2w2 = Lx2Lw2XW + Lx2XEw2+ Lw2WEx2 + Ex2Ew2

Equations [A.5] show that the four product indicators load not just on XW but also on eight other latent constructs or error terms, XEw1, XEw2, WEx1, WEx2, Ex1Ew1, Ex1Ew2, Ex2Ew1, and Ex2Ew2. The parameters of the model are the loadings of the four product indicators on these eight latent variables and the variances of the latter. However, all of these parameters are determined by those of the measurement model for the observed variables (Equations [A.4]). For example, all of the factor loadings in Equations [A.5] equal either 1.0, the loadings of x2 or w2 (Lx2, Lw2), or the product of their loadings (Lx2Lw2). Assuming that X, W, and the measurement errors of their indicators are normally distributed with means of zero, then X and W are uncorrelated with all of the latent variables in the measurement model of the four product indicators. Assuming normality and that the means of all the latent variables equal zero, the variances of the latent variables in Equations [A.5] are all functions of the variances and covariances of X, W, and the measurement errors of their indicators (Kenny & Judd, 1984), as follows; note that Cov below indicates covariance:

Equations [A.6]
Var XW = (Var X)(Var W) + (Cov X, W)2
Var XEw1 = (Var X)(Var Ew1)
Var XEw2 = (Var X)(Var Ew2)
Var WEx1 = (Var W)(Var Ex1)
Var WEx2 = (Var W)(Var Ex2)
Var Ex1Ew1 = (Var Ex1)(Var Ew1)
Var Ex1Ew2 = (Var Ex1)(Var Ew2)
Var Ex2Ew1 = (Var Ex2)(Var Ew1)
Var Ex2Ew2 = (Var Ex2)(Var Ew2)

Just as for a latent quadratic effect, the parameters of the measurement model of product indicators of a latent interaction effect are all determined by the parameters of their component observed variables. Fortunately for both types of nonlinear effects of latent variables, there is less complexity than is at first apparent because relatively small numbers of parameters need to be estimated, which is the topic covered next.

Estimation

Kenny and Judd (1984) described a method for estimating nonlinear effects of latent variables. Their method involves the specification of all the latent variables in the measurement model of the product indicators and the imposition of constraints on estimates of its parameters. For example, model (b) of Figure A.3 with a quadratic effect of a latent variable is specified as it appears in the figure and is estimated with the constraints indicated in Equations [A.2] for the factor loadings and in Equations [A.3] for the variances. Some of these constraints are nonlinear, which means that a parameter estimate is forced to equal an arithmetic product of other parameters. For example, the loading of x1x2 on X2 in this model is forced to equal the squared loading of x2 on X, and the variance of X2 is set to equal two times the squared variance of X. The model described earlier with two indicators each of exogenous factors X and W and product indicators of an interaction effect between them is analyzed in an analogous way: The measurement portions of the model are specified as per Equations [A.4] and Equations [A.5], and the whole model is estimated with constraints that match the dependencies among the parameters implied by Equations [A.6] given the assumptions of normality and that the means of the non-product latent variables are zero. See Kenny and Judd (1984) and Jöreskog and Yang (1996) for examples of this method.

A potential problem with the Kenny-Judd method is that not all model-fitting programs allow the imposition of nonlinear constraints. The COSAN subroutine of the CALIS program (Hartmann, 1992) and the latest version of LISREL (8; reviewed in Chapter 11) have this capability, but other programs may not. Hayduk (1987 pp. 232-242) devised a way to implement the Kenny-Judd method with model-fitting programs that do not allow nonlinear constraints, but this procedure is quite complicated. Even with a model-fitting program that is capable of imposing nonlinear constraints, the programming required to apply the Kenny-Judd method may still be fairly complex (e.g., Jöreskog & Yang, 1996).

Ping (1996a) described a simpler procedure for estimating nonlinear effects of latent variables. This method is based on the same assumptions and fundamental formulae as the Kenny-Judd method but differs from the latter in two ways. First, Ping's method features two-step estimation instead of the imposition of nonlinear constraints on the parameters of product variables. Second, Ping's method does not require the specification of all the latent variables that underlie the product indicators (e.g., Equations [A.2] and Equations [A.5], model (b) of Figure A.3). In this sense, Ping's method can be viewed as an approximation of the full Kenny-Judd model for nonlinear effects of latent variables.

In the first step of Ping's method, the model without the product indicators is analyzed. Results from this analysis are used to calculate parameters of the measurement model of the product indicators. These parameters include the factor loadings of the product indicators, their measurement error variances, and the variances of latent product variables that represents a nonlinear effect (e.g., X2, XW). These calculated values are then specified as fixed parameters in the analysis of the original model with the product indicators in the second step of Ping's method. From this second analysis, one obtains the path coefficients that estimate the linear and nonlinear effects of the latent variables.

Ping (1996a) created artificial data that are summarized with the covariance matrix presented in Table A.3. The variables include two indicators of a latent exogenous factor, X; three product indicators of a latent quadratic factor, X2; and an observed endogenous variable y. The first step in Ping's method is analyze a standard hybrid model that specifies factor X as a cause of y and that x1 and x2 are the indicators of X (i.e., model (a) of Figure A.3). One records from this analysis the unstandardized estimates of the parameters of the measurement model for x1 and x2. The analysis of the covariance matrix of x1, x2, and y in Table A.3 with a model-fitting program yielded the unstandardized ML estimates that are reported in the middle of the table.

Table A.3 Estimation of a quadratic effect of a latent variable
Covariance Matrix: Ping (1996a), computer-generated data.
Variable x1 x2 x12 x22 x1x2 y
x1 1.272          
x2 .605 .826        
x12 .202 .105 3.614      
x22 .208 .131 1.687 1.553    
x1x2 .162 .141 .860 1.175 1.731  
y .186 .112 -1.317 -.729 -.443 .898
Unstandardized Estimates from Model without Product Indicators
Variable Parameter Unstandardized Estimate
X Variance of exogenous factor, Var X 1.009
x1 Loading of indicator x1 on factor X (fixed to 1.0), Lx1 1.000
  Measurement error variance of indicator x1, Var Ex1 .262
x2 Loading of indicator x1 on factor X on X, , Lx2 .599
  Measurement error variance of indicator x2, Var Ex2 .463
Formulae for Calculating Parameters of Measurement Model of Product Indicators
Variance X2 = 2(Var X)2
Covariance (X, X2 ) = 0
Factor loading Lxi xj = Lxi Lxj
Variance of measurement error Exi xj :
  If x i = x j: 4Lx2 Var X Var Ex + 2(Var Ex)2
  If x i != x j (!= means '"not ="): Lxi2 Var X Var Exj + Lxj 2 Var X Var Exi + Var Exi Var Exj  
Calculated Unstandardized Parameter Estimates for Measurement Model of Product Variables
Variable(s) Parameter Value and Derivation
X2 Variance of product factor (X2) 2.036=2(1.009)2
X2, X Covariance of product factor (X2) with factor X 0
x12 Loading of indicator x12 on product factor (X2) 1.000=1.0002
  Measurement error variance of indicator x12 1.195=4(1.000)2(1.009)(.262)+2(.262)2
x22 Loading of indicator x22 on product factor (X2) .359=.5992
  Measurement error variance of indicator x22 1.099=4(.599)2(1.009)(.463)+2(.463)2
x1x2 Loading of indicator x1x2 on product factor (X2) .599=(1.000)(.559)
  Measurement error variance of indicator x1x2 .683 = 1.0002(1.009)(.463)+.5992(1.009)(.262)+(.463)(.262)
Table A.3 Estimation of a quadratic effect of a latent variable

With estimates of the measurement model of the non-product indicators in hand, one then calculates the parameters of the measurement model for the three product indicators, x12, x22, and x1x2 using the formulae that are also presented in Table A.3. These formulae basically integrate Equations [A.2] and Equations [A.3] such that one works with a simplified measurement model for the product indicators. The parameters of this simplified measurement model include only the loadings of the product indicators on X2 and the variances of X2 and the measurement error terms. The calculated values of the parameters of this simplified measurement model are presented in the lower part of the table. For example, the variance of the latent quadratic factor X2 is calculated as two times the squared variance of the latent linear factor X, or 2(1.009)2 = 2.036. This calculated value simulates the nonlinear constraint that restricts the variance of X2 to 2(Var X)2 as per Equations [A.3]. In the upcoming analysis of the whole model with the product indicators, the variance of X2 is fixed to equal 2.036. Likewise, the loadings of the product indicators on X2 are the respective products of the loadings of the non-product indicators , x1 and x2 on X. The derivations for the measurement error variances of the product indicators are more complex, but they eliminate the need to represent all of the latent variables that underlie the product indicators implied by Equations [A.2].

The second step is to take the calculated estimates for the measurement model of the product indicators and specify them as fixed parameters in the analysis of the whole model. All of the other parameters in the whole model are freely estimated. Given an acceptable measurement model of the non-product indicators, one should see only trivial changes in the values of these parameters in the analysis of the whole model. If not, then interpretational confounding is suggested (Chapter 8, section 8.3). The other freely estimated parameters from this analysis include the direct effects of the latent linear and quadratic effects of X on y and the disturbance variance of the latter. The whole covariance matrix at the top of Table A.3 was submitted to a model-fitting program to analyze both the product and non-product indicators with the parameters of the product indicators fixed to equal the calculated values in the table. The results of this analysis are presented in Figure A.4. Note that the estimates for the measurement model of the product indicators in the figure match the calculated values in Table A.3. Also, estimates for the measurement model of the indicators of X from this analysis are similar to those from the first step of Ping's method (Table A.3), which is a desired result. The unstandardized (and standardized) estimates of the linear and quadratic effects of X on y are respectively .296 (.31) and -.521 (-.79). These results indicate that the although there is a positive linear effect of X on y, there is also a negative quadratic aspect to their relation that is greater in absolute magnitude than the linear trend. A negative quadratic relation is shaped like a rainbow, in this case one that is inverted. Although X alone accounts for some portion of y's variance, prediction is better when X2 is also included in the model.

Figure A.4 A model with a quadratic effect of latent variable
X estimated with Ping's (1996a) method. Note: Latent variables
are represented by upper-case letters and indicators by lower-
case letters. The values in parentheses are standardized estimates.
The standardized value for the disturbance is the proportion of
unexplained variance.
Figure A.4 A model with a quadratic effect of latent variable
X estimated with Ping's (1996a) method. Note: Latent variables
are represented by upper-case letters and indicators by lower-
case letters. The values in parentheses are standardized estimates.
The standardized value for the disturbance is the proportion of
unexplained variance.

The estimation of an interactive effect of latent variables X and W using Ping's method is demonstrated in Table A.4. The covariance matrix of five observed variables and four product indicators presented at the top of the table was created by Ping (1996a). In the first step of Ping's method, a standard hybrid model was tested. This model featured x1 and x2 as indicators of X; w1 and w2 are indicators of W; an unanalyzed association between X and W; and direct effects of X and W on y. The covariance matrix for these five observed variables was submitted to a model-fitting program for ML estimation. The unstandardized estimates for the measurement model of the non-product indicators are reported in the middle of Table A.4. These values were used to calculate the parameters of the measurement of the product indicators x1w1, x1w2, x2w1, and x2w2 according to the formulae listed in the table. These calculated values were then specified as fixed parameters in the analysis of the whole covariance matrix.

Table A.4 Estimation of an interactive effect of latent variables
Covariance Matrix: Ping (1996a), compter-generated data.
Variable x1 x2 w1 w2 x1w1 x1w2 x2w1 x2w2 y
x1 2.376                
x2 1.234 1.492              
w1 .319 .139 2.129            
w2 .206 .068 1.174 1.497          
x1w1 .106 .004 .179 .199 5.130        
x1w2 -.244 -.121 .174 .187 2.830 3.927      
x2w1 -.003 .074 .166 .095 2.516 1.326 2.885    
x2w2 -.125 -.161 .085 .096 1.333 1.908 1.572 2.153  
y -.232 -.188 .595 .453 2.688 2.009 1.396 1.040 2.324
Unstandardized Estimates from Model without Product Indicators
Variable(s) Parameter Unstandardized Estimate
X Variance of exogenous factor, Var X 2.223
W Variance of exogenous factor, Var W 1.604
X, W Covariance of exogenous factors, Cov (X, W) .237
x1 Loading of indicator x1 on X (fixed to 1.0), Lx1 1.000
  Measurement error variance of indicator of x1, Var Ex1 .152
x2 Loading of indicator of x2 on X, Lx2 .554
  Measurement error variance of indicator of x2, Var Ex2 .807
w1 Loading of indicator w1 on W (fixed to 1.0), Lw1 1.000
  Measurement error variance of indicator w1, Var Ew1 .525
w2 Loading of indicator w2 on W, Lw2 .731
  Measurement error variance of indicator w2 , Var Ew2 .637
Formulae for Calculating Parameters of Measurement Model of Product Indicators
Var XW = Var X Var W + (Cov X, W)2
Cov (X, XW) = Cov (W, XW) = 0
Factor loading Lxw = Lx Lw
Variance of measurement error Exw = Lx2 Var X Var Ew + Lw2 Var W Var Ex + Var Ex Var Ew
Calculated Unstandardized Parameter Estimates of Measurement Model of Product Variables
Variable Parameter Value and Derivation
XW Variance of product factor (XW) 3.622=(2.223)(1.604)+(.237)2
X, XW Covariance of exogenous factors
(X, XW)
0
W, XW Covariance of exogenous factors
(W, XW)
0
x1w1 Loading of product indicator x1w1
on product factor (XW)
1.000=(1.000)(1.000)
  Measurement error variance of
product indicator x1w1
1.491=1.0002(2.223)(.525)+1.0002 (1.604)(.152)+(.525)(.152)
x1w2 Loading of product indicator x1w2
on product factor (XW)
.731=(1.000)(.731)
  Measurement error variance of
product indicator x1w2
1.643 = 1.0002(2.223)(.637)+.7312(1.604)(.152)+(.152)(.637)
x2w1 Loading of product indicator x2w1
on product factor (XW)
.554=(.554)(1.000)
  Measurement error variance of product indicator x2w1 2.076=.5542(2.223)(.525)+1.0002(1.604)(.807)+(.807)(.525)
x2w2 Loading of product indicator x2w2
on product factor (XW)
.405=(.554)(.731)
  Measurement error variance of product indicator x2w2 1.640=.5542(2.223)(.637)+.7312(1.604)(.807)+(.807)(.637)
Table A.4 Estimation of an interactive effect of latent variables

The unstandardized estimates from the analysis of the non-product and product indicators is presented in Figure A.5. Note that the estimates from the measurement model of the product indicators are the same as the calculated values in Table A.4. The only really new results are the path coefficients for the linear and interactive effects of X and W on the endogenous variable and the disturbance variance. The standardized estimates of the interaction effect indicate that its absolute value (.87) is greater than either linear effect of X or W on y (-.14, .28, respectively). Thus, the effect of X seems to depend upon the level of W and vice-versa. How to more precisely interpret an interaction effect is demonstrated later.

Figure A.5 A model with an interactive effect of latent variables X and W
estimated with ping's method. Note: Latent variables are represented by
upper-case letters and indicators by lower-case letters. The unstandardized
solution is shown. The values in parentheses are standardized estimates.
The standardized value for the disturbance is the proportion of unexplained
variance.
Figure A.5 A model with an interactive effect of latent variables X and W
estimated with ping's method. Note: Latent variables are represented by
upper-case letters and indicators by lower-case letters. The unstandardized
solution is shown. The values in parentheses are standardized estimates.
The standardized value for the disturbance is the proportion of unexplained
variance.

Three other points about estimation of nonlinear effects of latent variables before we consider an example with actual data. First, the models in Figure A.4 and Figure A.5 (and later in Figure A.6, too) cannot be estimated in a single step. That is, one could not specify these models as they appear in the figures and analyze them with covariance matrices of product and non-product indicators without going through both steps of Ping's (1996a) method. The models in these figures only approximate the full models implied by Equations [A.1] through Equations [A.6].

Second, the assumption of the Kenny-Judd and Ping methods that the non-product factors and measurement errors are normally distributed is important. If it is violated, then the constraints imposed in each method may be incorrect. Normality of the non-product factors and error terms can be evaluated by examining the distributions of the non-product observed variables. If these distributions are obviously non-normal, then something should be done such as the use of a transformation (Chapter 4, section 4.6). Even if the distributions of the non-product variables are normal, those of the product variables are not. Thus, the multivariate normality assumption of ML estimation may be violated. Although estimates generated by ML are reasonably accurate when the data are non-normal, recall that tests of significance may be positive too often. The use of corrected test statistics such as the Satorra-Bentler statistic and robust standard errors may reduce some of this bias (Chapter 7, section 7.6). The use of estimation procedures with alternative distributional assumptions is also possible (Chapter 5, section 5.11).

Finally, the assumption of the Kenny-Judd and Ping methods that the means of all non-product latent variables are zero may be inappropriate in some cases. These include longitudinal studies in which means are expected to change over time and multi-sample analyses in which group means may be different. See Jöreskog and Yang (1996) for additional discussion of this issue.

Example

This example involves the application of Ping’s (1996a) method to estimate a model with an interactive effect of two latent variables. The data for this problem are from Kenny and Judd (1984), who used them to demonstrate their analytical method for models with product variables. The model they analyzed was quite complex in that it contained a quadratic effect and two interaction effects, one linear×linear and the other linear×quadratic. Of these three nonlinear effects, however, the one for the linear×linear interaction effect was clearly the largest. For this reason and for clarity’s sake, the model analyzed here contains only the linear×linear interaction. This simplified model is thus nested under the one evaluated by Kenny and Judd (1984).

The main question of this analysis is whether the relation of voters’ views on issues to their perceptions of a candidate’s positions on the same issues varies as a function of voters’ sentiment toward the candidate. For example, voters who dislike a candidate may overestimate the discrepancies between their views and those of the candidate. Likewise, voters may tend to overestimate their agreement with a candidate they personally like. The data analyzed here are from a survey conducted in 1968 of 1,160 voters in the United States. Note that this sample size is not extraordinarily large for the analysis of an interaction effect of latent variables. Large samples (e.g., N > 500) may be needed in such analyses for the results to be statistically stable. The voters in this study were asked their views on two issues, the Vietnam War and crime. The same subjects were asked to rate the views of two presidential candidates, Hubert Humphrey and Richard Nixon, on the same issues and to indicate their sentiment toward each candidate. Kenny and Judd (1984) conceptualized ratings of each issue as the indicators of two latent variables, one that represents voters’ views (V) and the other their perceptions of a candidate’s views (C). There was only a single measure of sentiment. Accordingly, this observed variable is represented with a lower-case s.

The model presented in Figure A.6 represents C (perception of candidates’ views) as affected by V (voters’ views) and s (sentiment). The model also contains unanalyzed associations between the exogenous variables V and s and between the measurement errors of voter and candidate views of the same issue. Also note that the loadings of the indicators of V and of C are all constrained to equal one. Because the V and C factors each have only two indicators, this constraint is necessary to identify the model given the measurement error correlations. However, these particular equality constraints are unrelated to Ping’s estimation method. The exogenous product factor Vs with its product indicators (v1s and v1s) represents the interactive effect of voters’ views and sentiment. If the path coefficient for the direct effect of Vs on C is substantial, then the relation between voters’ views and their judgements of candidates’ positions is moderated by sentiment. (By the same token, the effect of s on C would also vary with V.) The data are summarized in Table A.5 are for one presidential candidate (Nixon) and they include the two product variables that are the indicators of the latent interaction factor. Consistent with the assumption of their method that the means of the non-product latent variables are all are zero, Kenny and Judd (1984) re-scaled the observed variables so that their means were zero before deriving the product variables.

Figure A.6 A model with a latent interactive effect of voters' positions
(V) and their sentiment toward a candidate (s) on the perception of a
candidate's positions (C) estimated with Ping's method.
Figure A.6 A model with a latent interactive effect of voters' positions
(V) and their sentiment toward a candidate (s) on the perception of a
candidate's positions (C) estimated with Ping's method.

The first step of Ping’s method is to analyze the model in Figure A.6 without the product variables. From the results of this analysis, one records the unstandardized parameter estimates for the measurement model of the observed variables, v1, v2, and s. Because s is the sole indicator of a sentiment factor, its factor loading is assumed to be 1.0 and its measurement error variance zero. Also recall that the loadings of v1 and v2 are both fixed to 1.0. With these fixed parameters, the other pertinent estimates of the measurement model for v1, v2, and s include the variances of s and the latent voter factor V, the covariance of these two exogenous variables, and the measurement error variances of the indicators of V.

A covariance matrix assembled from the data reported in Table A.5 for the five non-product observed variables was analyzed with a model-fitting program to derive the aforedescribed estimates with ML. These estimates are reported in the middle of Table A.5. Estimates for the parameters of the measurement model of the product indicators were then calculated using the formulae summarized in Table A.4. These calculated values include the variance of the latent product variable Vs; the loadings of the product indicators v1s and v2s on Vs; and the variances of their measurement errors. These calculated values are reported in the bottom part of Table A.5. For example, the calculated variance of Vs is 9.08, which equals Var V × Var s + (Cov V, s)2, which is 1.80(4.96) + (.39)2. The derivations of the other parameters of the measurement model of the product indicators are also listed in the table.

Table A.5 Data and basic estimates for an intertactive model of voters’ perceptions
Correlations, Means, and Standard Deviations: Kenny and Judd (1984), N = 1,160 voters.
  Candidate Voter Sentiment Product Variables
  c1 c2 v1 v2 s v1s v2s
Indicators of Candidate Perceived Position Factor (C)
c1 Vietnam War            
c2 Crime .24          
Indicators of Voter Position Factor (V)
v1 Vietnam War .20 .07        
v2 Crime -.03 .25 .29      
Single Indicator of Sentiment Toward Candidate (s)
s -.20 -.02 .10 .08    
Product Indicators of Product Factor (Vs)
v1s .31 .10 -.07 -.14 -.06  
v2s .17 .32 -.13 -.08 .02 .34
M 0 0 0 0 0
SD 1.62 1.49 1.93 1.96 2.23 4.59 4.85
Unstandardized Estimates from Model without Product Indicators
Variable(s) Parameter   Unstandardized Estimate    
V Variance of voter factor (V)   1.80    
s Variance of observed sentiment variable (s)   4.96    
V, s Covariance of voter factor (V) and sentiment (s)   .39    
v1 Loading of Vietnam War indicator on voter factor (V)   1.00    
  Measurement error variance of Vietnam War indicator   2.65    
v2 Loading of crime indicator on voter factor (V)   1.00    
  Measurement error variance of crime indicator   2.74    
Calculated Unstandardized Parameter Estimates of Measurement Model of Product Variables
Variable(s) Parameter Value and Derivation
Vs Variance of product factor (Vs) 9.08=(1.80)(4.96)+(.39)2
V, Vs Covariance of product factor (Vs)
with voter factor (V)
0
s, Vs Covariance of product factor (Vs)
with sentiment (s)
0
v1s Loading of product indicator on
product factor (Vs)
1.00
  Measurement error variance of
product indicator
13.14=1.002(1.80)(0)+1.002(4.96)(2.65)+(2.65)(0)
v2s Loading of product indicator on
product factor (Vs)
1.00
  Measurement error variance 13.59=1.002(1.80)(0)+1.002(4.96)(2.74)+(2.74)(0)
Table A.5 Data and basic estimates for an intertactive model of voters’ perceptions

The next step in Ping’s method is to treat the values calculated for the product variables as fixed parameters in a second analysis, this one of the whole model. The whole covariance matrix for all five non-product variables and two product variables assembled from the correlations and standard deviations reported in Table A.5 was submitted to a model-fitting program. The overall chi-square (15) statistic for the whole model is 164.39 and the chi-square/df ratio is 10.96. Values so high for a standard hybrid model would suggest poor fit, but these values seem fairly typical for models with product variables estimated in two-steps. Values of other fit indexes seem more consistent with expectations. For example, the Jöreskog-Sörbom Goodness of Fit Index is .96 and the standardized root mean squared residual is .01.

Reported in Table A.6 are the ML estimates for the whole model. Estimates that are fixed due to the application of Ping’s method are reported in the top part of the table followed by estimates that are fixed due to general requirements for the identification of this particular example. Reported next in the table are the estimates of the free parameters. All of these unstandardized estimates are significant at the .05 level. The main estimates of interest here are those of the linear and interactive effects of sentiment (s) and voters’ views (V) on judgement of the candidate’s positions (C). Kenny and Judd’s (1984) unstandardized estimates of V=>C, s=>C, and Vs=>C were, respectively, .18, -.11, and .21. These values are very close to those derived here using Ping’s method (.20, -.08, and .21; see Table A.6), which is encouraging in that Ping’s method is simpler. Overall, the magnitude of the interaction effect is sizable (standardized path coefficient = .81). A method to interpret the nature of this interaction effect is outlined below.

Table A.6 Parameter estimates for the full interactive model of voters’ perception
Variable(s) Parameter Estimatea
Parameters Fixed as Part of Ping’s Method
Vs Variance of product factor 9.08
V, Vs Covariance of product factor (Vs) with voter (V) factor 0
s, Vs Covariance of product factor (Vs) with Sentiment (s) 0
v1s Loading of product indicator on product factor (Vs) 1.00 (.64)
  Measurement error variance of product indicator 13.17 (.59)
v2s Loading of product indicator on prroduct factor (Vs) 1.00 (.63)
  Measurement error variance of product indicator 13.61 (.60)
Parameters Fixed Due to General Requirements for Identification
v1 Loading of Vietnam War indicator on voter factor (V) 1.00 (.54)
v2 Loading of crime indicator on voter factor (V) 1.00 (.53)
c1 Loading of Vietnam War indicator on candidate factor (C) 1.00 (.49)
c2 Loading of crime indicator on candidate factor (C) 1.00 (.52)
Free Parameters
V Variance of voter factor (V) 1.08 (1.00)
s Variance of observed sentiment variable (s) 4.96 (1.00)
V, s Covariance of voter factor (V) and sentiment (s) .39 (.16)
v1 Measurement error variance of indicator of voter factor (V) 2.64 (.71)
v2 Measurement error variance of indicator of voter factor (V) 2.75 (.72)
c1 Measurement error variance of indicator of candidate factor (C) 1.97 (.76)
c2 Measurement error variance of indicator of candidate factor (C) 1.72 (.73)
Ev1, Ec1 Measurement error covariance .60 (.26)
Ev2, Ec2 Measurement error covariance .62 (.28)
C Direct effect of voter factor (V) on candidate factor (C) .20 (.26)
  Direct effect of sentiment (s) on on candidate factor (C) -.08 (-.23)
  Direct effect of product factor (Vs) on candidate factor (C) .21 (.81)
  Disturbance variance .15 (.24)
Note. All unstandardized estimates are significant at the .05 level.
aUnstandardized (standardized). The standardized values for the disturbance
and measurement variances are proportions of unexplained variance.
Table A.6 Parameter estimates for the full interactive model of voters’ perception

Interpretation of an Interactive Effect

The method described here is an extension of a strategy for interpreting an interaction effect of observed variables (e.g., Aiken & West, 1991; Cohen & Cohen, 1983). The starting point for the present example is to write out the structural equation for the endogenous variable C (perception of candidate’s positions), which has the following form:

Equation [A.7]
C = PV (V) + Ps (s) + PVs (Vs) + D,

where P represents unstandardized path coefficients; V stands for the voter’s views latent variable; s is the single indicator of sentiment; Vs is the voter view × sentiment interaction factor; and D is the disturbance term. Dropping the disturbance (which represents unexplained variance) transforms Equation [A.7] to an equation that expresses predicted perception of a candidate’s position (Cpred) as a function of the other three product or non-product factors (V, s, Vs). Writing this prediction formula with the actual values of the path coefficients in it yields:

Equation [A.8]
Cpred =.20 V - .08 s + .21 Vs.

Equation [A.8] above is similar to a regression equation but one that generates predicted scores on a dependent latent variable. The above prediction equation can be re-arranged so that the Vs interaction term is eliminated. For example, the following version is algebraically identical to Equation [A.8]:

Equation [A.9]
Cpred = (.20 + .21 s) V - 08 s.

Note that multiplying V by s in Equation [A.9] above yields the product variable Vs. The term (.20 + .21s) in the above equation can be viewed as regression coefficient for V in the prediction of C but one that varies with the level of s, which also describes an interactive effect. The second term of this equation, (-.08 s), is analogous to an intercept that, like V’s coefficient, also takes s into account. This re-arranged prediction equation thus captures the essence of the interactive effect between V (voter’s view) and s (sentiment) in their joint effect on C (perception of candidate view).

Using Equation [A.9], one substitutes various values of s and inspects the effect on the coefficients for V. For instance, let’s plug into this prediction equation five values of s, ones that correspond to +2, +1, 0, -1, and -2 standard deviations away from its mean. Because the mean of s is already zero and its standard deviation is 2.23 (Table A.5), these five values are 4.46 (i.e., 2×2.23), 2.23, 0, -2.23, and -4.46. Substituting these values for s into Equation [A.9] yields the following results:

Value of s Meaning Prediction Formula Cpred = (.20 + .21s) V - 08 s
4.46 +2 SDs   (.20+.21(4.46)) V - .08 (4.46) = 1.14 V - .36
2.23 +1 SD   (.20+.21(2.23)) V - .08 (2.23) = .69 V - .18
0 mean   (.20+.21(0)) V - .08 (0) = .20 V
-2.23 -1 SD   (.20+.21(-2.23)) V - .08 (-2.23) = -.27 V + .18
-4.46 -2 SDs   (.20+.21(-4.46)) V - .08 (-4.46) = -.74 V + .36

Note that the coefficients for V in the prediction formulae in the right side of the above table are positive at high levels of s but negative at lower levels. Also note that predicted values of C are adjusted downward at higher levels of s and upward at lower levels, which reflects the overall negative relation between s and C. Expressed more directly: As sentiment toward a candidate declines, the relation between voter’s views and his or her judgement about the candidate’s positions on the same issues becomes increasingly negative. That is, voter’s perceive dissimilarity between their views and those of a disliked candidate. Voters who like the candidate, however, tend to see their views as being similar to those of the candidate, which yields a positive relation between the two.


A.4 Summary

A popular stereotype of social scientists is that in response to the question, “How is X related to Y?” (insert your favorite pair of constructs: mores and conformity, handedness and neural organization, etc.), they respond, “Well, it depends.” The representation of curvilinear or interactive effects in structural models provides a way to evaluate “it depends” relations. Both types of nonlinear effects involve statistical effects that are not uniform (linear). A curvilinear relation between two variables means that the slope of the line that describes their scatterplot changes as scores increase. An interaction effect involves, at minimum, a relation of one variable to another that is not uniform across the levels of a third. The relation may change in direction or magnitude, but it varies with the other variable.

When a structural model includes only observed variables (i.e., a path model), curvilinear or interactive effects can be represented directly by product variables. For example, the product term X2 represents the quadratic effect of X. When both X and X2 are specified as causes of Y, the path coefficients of these direct effects respectively estimate the linear and quadratic effects of X on Y. Interactive effects of observed variables are likewise represented with product variables. For instance, the product term XW represents the interactive effect of X and W. When represented in a path model along with its component variables (X and W) as causes of Y, the path coefficient for the direct effect of XW on Y estimates the interactive (joint) effect of X and W on Y. Adding product variables to a path model makes it more complicated. This is especially true if higher-order interactive effects are estimated, which may require the presence of numerous product variables that represent all lower-order effects. Product variables should thus not be added to path models willy-nilly.

The analysis of nonlinear effects of latent variables in hybrid models is based on a similar logic. The indicators of latent product variables that represent curvilinear or interaction effects are the products of the corresponding indicators of the latent linear variables. For example, if X is a latent factor with two indicators x1 and x2, then the product variables x12, x22, and x1x2 are the indicators of X2, the latent product variable that represents the quadratic effect of X. Along the same lines, if w1 and w2 are the indicators of W, then the four product variables x1w1, x1w2, x2w1, and x2w2 are the indicators of the latent interaction factor XW.

The measurement models of product indicators actually include more latent product variables than just, say, a single quadratic or interaction factor. Also, the parameters of these measurement models are related to the parameters of the measurement models of the non-product indicators. Some of the relations between these two sets of parameters are nonlinear, which means that one parameter is a multiplicative product of other parameters. There are two estimation methods that deal with the nonlinear constraint problem. Both methods assume that the non-product latent variables and the measurement errors of their indicators are normally distributed and that the means of these variables are all zero. The Kenny-Judd method features specification of all the latent variables that underlie the product indicators and the imposition of all nonlinear constraints. Not all model-fitting programs allow nonlinear constraints, however.

In contrast, Ping’s method can be used with virtually any model-fitting program. Ping’s method uses two-step estimation to simulate the imposition of nonlinear constraints. In the first step, the model is analyzed without the product variables. Estimates of the measurement model for indicators that are components of product indicators are recorded. These results are then used to calculate parameter estimates for the measurement model of the product indicators. These calculated values are specified as fixed parameters in the second step, the analysis of all the variables, product and non-product. The path coefficients for the latent product variables that represent quadratic or interactive effects on endogenous variables are obtained from this second analysis.

If the non-product observed variables are severely non-normal, then the constraints implied by the Kenny-Judd and Ping methods may lead to incorrect results. It is thus important to carefully screen the data when nonlinear effects of latent variables are estimated with either method. Large samples (e.g., N > 500) may also be required in order for the results to be reasonably stable. Finally, neither the Kenny-Judd or Ping methods may be suitable if the assumption that the means of the latent non-product variables are zero is not tenable. For example, if latent variables A1 and A2 represent the same construct measured at different times, then it may not be reasonable to assume that both of their means are zero if the average level of subjects’ standing on this construct is expected to increase over time.

The analysis of nonlinear effects is one way to expand the usefulness of SEM. Another is to analyze means along with covariances, which is introduced in the next supplemental chapter (B).


A.5 Recommended Readings

Baron and Kenny (1986) and James and Brett (1984) provide helpful starting points for understanding the conceptual implications of the difference between interaction and indirect effects in structural models.


A.6 References

      Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.

      Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.

      Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

      Frederiksen, N., & and Melville, S. D. (1954). Differential predictability in the use of test scores. Educational and Psychological Measurement, 14, 647-656.

      Hartmann, W. M. (1992). The CALIS procedure: Extended user's guide. Cary, NC: SAS Institute.

      Hayduk, L. A. (1987). Structural equation modeling with LISREL. Baltimore, MD: John Hopkins University Press.

      Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348-357.

      James, L. R., & Brett, J. M. (1984). Mediators, moderators and tests for mediation. Journal of Applied Psychology, 69, 307-321.

      Jöreskog, K. G., and Yang, F. (1996). Nonlinear structural equation models: The Kenny-Judd model with interaction effects. In G. A. Marcoulides and R. E. Schumacker (Eds.), Advanced structural equation modeling (pp. 57-88). Mahwah, NJ: Lawrence Erlbaum.

      Kenny, D. A., & Judd, C. M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 96, 201-210.

      Keppel, G., & Zedeck, S. (1989). Data analysis for research designs. New York: W. H. Freeman.

      Kline, R. B. (1994, August). Validity of a math skills pretest for introductory statistics. Presented at the 102nd Annual Meeting of the American Psychological Association, Los Angeles.

      Koestler, A. (1989). The act of creation. New York: Penguin Books. (Original work published 1964)

      Lance, C. E. (1988). Residual centering, exploratory and confirmatory moderator analysis, and decomposition of effects in path models containing interaction effects. Applied Psychological Measurement, 12, 163-175.

      Loehlin, J. C., Horn, J. M., & Willerman, L. (1990). Heredity, environment, and personality change: Evidence from the Texas Adoption Project. Journal of Personality, 58, 221-243.

      MacCallum, R. C., & Mar, C. M. (1995). Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin, 118, 405-421.

      McArdle, J. J. (1994). Structural factor analysis experiments with incomplete data. Multivariate Behavioral Research, 29, 409-454.

      McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interaction and moderator effects. Psychological Bulletin, 114, 376-390.

      Ping, R. A. (1996a). Interaction and quadratic effect estimation: A two-step technique using structural equation analysis. Psychological Bulletin, 119, 166-175.

      Ping, R. A. (1996b). Latent variable regression: A technique for estimating interaction and quadratic coefficients. Multivariate Behavioral Research, 31, 95-120.

      Suinn, R. M. (1988). Mathematics Anxiety rating Scale (MARS). Fort Collins, CO: RMBSI.

      Wood, P. K. (1994). The effect of unmeasured variables and their interactions on structural models. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis (pp. 109-130). Thousand Oaks, CA: Sage Publications.