home > supplemental chapters > estimation of nonlinear effects 

Supplemental Chapter
A: Nonlinear Effects of Observed and Latent Variables Kline, R. B. (1998).
Principles and
practice of structural equation modeling. Copyright © 2000 Rex B. Kline. All rights reserved. Beauty is a function of truth,
truth a function of beauty. They can be separated by analysis Arthur Koestler (1989/1964, p. 331)  
Comment to author 
Note About SEM Programs
Since these supplemental chapters were written in 1998, new versions of all SEM programs mentioned here, including Amos, EQS, LISREL, and Mx (now called Mplus), have been released. All of these programs have capabilities beyond those described here. For more information, follow the links below.
Contents
All of the effects in models considered up to this point have been assumed to be linear. This chapter covers the analysis of nonlinear effects in SEM. Recall from discussion in Chapter 2 (section 2.6) that there are two kinds of nonlinear effects, curvilinear relations that involve only two variables and interactive effects that involve at least three. Interaction effects are also called moderator effects that occur when, in their most basic form, the relation of one variable to another changes across the levels of a third. Although any multisample SEM can be viewed as a test of group membership as a (categorical) moderator variable, the interaction effects considered in this chapter are between variables represented within models that can be estimated within a single sample.
The first part of this chapter concerns the estimation of nonlinear effects in structural models of observed variables (i.e., path models). The second part extends this basic rationale to the representation and analysis of nonlinear effects of latent variables in hybrid models. Just as the estimation of linear effects of latent variables in standard hybrid models takes measurement error into account, the same is also true of the analysis of the nonlinear effects with the hybrid models considered in this chapter. The capabilities just mentioned also distinguish the SEM approach to the study of nonlinear relations. For example, although both multiple regression and ANOVA can analyze nonlinear effects of observed variables, neither can readily do so with a multiple indicator approach that controls for measurement error.
A few words before we begin. The topic of nonlinear effects is a broad one, and it is not possible to cover all aspects of it in a single chapter. Instead, the goal here is more modest, to introduce the estimation of nonlinear effects from the perspective of SEM. Also, although methods to estimate nonlinear effects of observed variables are fairly well established, this is not the case for latent variables. Some methods are discussed and examples of their application are reviewed, but there are relatively few examples of this type of analysis in the SEM literature. Thus, a sense of pioneer spirit would be a healthy perspective to bring to this particular subject. The payoff, though, may be an increased awareness of possibilities to test hypotheses about latent variables.
A.2 Nonlinear Effects of Observed Variables
Introduction
Multiple regression offers a straightforward way to represent curvilinear and interactive effects of observed variables. This method not only underlies the evaluation of trend and interaction effects in ANOVA (which is a special case of multiple regression), it can also be used in SEM. In the examples that follow, it is assumed that all variables are continuous. (The case of categorical exogenous variables is considered later.) Let's start with curvilinear relations.
Suppose that variable X is children's age and variable Y is the size of their vocabularies. If vocabulary size increased uniformly with age, the relation between X and Y would be solely linear. In reality, though, the rate of increase in vocabulary breadth is greater for younger children, which adds a curvilinear component to the relation between X and Y, specifically a quadratic one. Now suppose that X is the only predictor of Y in a regression equation. The regression coefficient for X reflects just the linear aspect of its relation to Y. To also represent the quadratic relation of X to Y, all that is necessary is to create the product variable X^{2} (literally, square each child's age) and enter it along with X as a predictor of Y in a multiple regression equation. The presence of X^{2} in the equation adds one "bend" to the regression line, and its regression coefficient estimates the magnitude of the quadratic aspect of the relation between X and Y, in this case the relation between age and shoe size.
As an example, consider the raw scores presented at the top of Table A.1. The relation between X and Y is obviously quadratic: Scores on Y decline and then rise as scores on X increase. Regressing Y on X yields a Pearson correlation of only .10 and a nonsignificant unstandardized regression coefficient for X of .04. However, if Y is regressed on both X and X^{2}, the (multiple) correlation increases to .93 and both regression coefficients are significant. The results with X^{2} in the equation are so different because this product variable represents the substantial quadratic relation between X and Y, and the regression coefficient for X^{2} (controlling for X) estimates the magnitude of this effect. With just X as a predictor, only its (weak) linear association with Y is estimated.
Table A.1
Quadratic and interactive effects between observed variables
represented with product variables.  
Quadratic Effect  
Predictor  Product  Criterion  
Subject  X  X^{2}  Y  
A  5  25  15  
B  7  49  14  
C  10  100  11  
D  13  169  9  
E  15  225  9  
F  17  289  5  
G  19  361  8  
H  23  529  11  
I  25  625  14  
J  26  676  14  
Unstandardized regression coefficients: 
.04  —  r_{XY} = .10  
2.20**  .07**  R_{Y•X, X}_{2} = .93**  
Interaction Effect  
Predictors  Product  Criterion  
Subject  X  W  XW  Y 
A  2  10  20  5 
B  4  12  48  7 
C  6  12  72  9 
D  8  13  104  11 
E  11  10  110  10 
F  2  22  44  10 
G  4  24  96  11 
H  7  19  133  9 
I  8  18  144  7 
J  11  25  275  5 
Unstandardized regression coefficients: 
.01  .02  —  R_{Y•X, W} = .05 
1.54**  .62**  .09**  R_{Y•X, W, XW} = .89*  
*p<.05 **p<.01  
Table A.1 Quadratic
and interactive effects between observed variables represented with product variables. 
The same approach can be used to represent interactive effects of observed variables. As an applied example, consider a study by Frederiksen and Melville (1954). These authors used scores on a measure of interest in engineering to predict gradepoint average after one year of engineering school. The overall correlation between interest and achievement in their sample was only .10. However, Frederiksen and Melville speculated that the relation between interest and achievement varies as a function of a third variable, compulsiveness. That is, very compulsive students may invest a lot of energy in nonproductive, ritualistic behavior that may interfere with the expression of their interest through their school work. Using a measure of compulsiveness administered at the same time as the interest measure, Frederiksen and Melville divided their sample into two groups, high and low compulsive. For the highcompulsive students, the correlation between interest and achievement was essentially zero (.01), but for the lowcompulsive group the correlation was greater, .25. Thus, interest interacts with compulsiveness in the prediction of achievement. Specifically, the magnitude of the association between interest and achievement decreases as compulsiveness increases.
Partitioning subjects into two groups based on their scores on a continuous variable W (e.g., compulsiveness in the above example) to evaluate whether the relation between X and Y is different within each group is potentially problematic. Dichotomizing a continuous moderator variable W not only results in the loss of numerical information, it may also lead to an inaccurate reflection of the interaction (Cohen & Cohen, 1983). For example, suppose that the relation between X and Y gradually changes across the levels of W or changes abruptly (e.g., from positive to negative) at a point on W that is not where the researcher dichotomized the scores. In either case, the evaluation of whether dichotomized variable W moderates the relation between X and Y may be biased. The same problems may also arise if the subjects are partitioned into three or more groups on a continuous variable W.
It is possible to analyze an interactive effect of continuous variables without having to categorize either one. This approach involves the creation of a product variable that represents the interactive effect. Suppose that X and W are two continuous predictors of Y. When Y is regressed on both X and W, the regression coefficients for the predictors are estimates of their linear relations to Y adjusted for their correlation with each other. Let XW represent a product variable created by multiplying each subject's scores on X and W. The product term XW represents the interactive effect of its component variables. Specifically, if Y is regressed on X, W, and XW, then (1) the regression coefficient for XW estimates the magnitude of the interaction between X and W; and (2) the regression coefficients for X and W estimate their linear relations to Y adjusted for their correlation and interaction. Likewise, the regression coefficient for the product term XW controls for the linear effects of both X and W. A regression coefficient for XW that is significant indicates that the relation of X to Y changes as a function of W. It could, as in the example described above, change from positive to zero across the levels of W. Other patterns are possible (e.g., the relation of X to Y changes from negative to positive across W), but the key point here is that relation between X and Y is moderated by W. Because an interaction effect is a joint one, however, X also moderates the relation of W to Y. For this reason, the use of multiple regression with product variables like XW to estimate interactions is sometimes called moderated multiple regression.
As an example of the use of a product variable to represent an interaction effect with moderated multiple regression, consider the raw data presented in the lower part of Table A.1. If X and W are the only predictors of Y, then neither has a significant regression coefficient and the multiple correlation is only .05. Inspection of the raw scores in the table indicates that the relation of X to Y is positive for subjects with low scores on W ( <12) but is negative at higher levels of W. Although not as apparent, there is a similar change in the direction of the linear relation between W and Y: positive at higher levels of X, negative at lower. When X, W, and XW are all in the equation as predictors of Y, the multiple correlation increases to .89 (from just .05 without XW) and the regression coefficients of all three predictors are significant.
Three additional points about the representation of nonlinear effects of observed variables with product variables. First, its is possible to estimate higherorder curvilinear or interaction effects than those considered in the previous examples. For example, the product variable X^{3} represents the cubic relation of X to Y; X^{3} a quartic relation; and so on. Product variables that represent quadratic or higherorder curvilinear effects are sometimes called power polynomials. To test a higherorder power polynomial (e.g., X^{3}), all lowerorder power terms should be in the equation (e.g., X and X^{2}). In the social sciences, however, it is relatively rare that power terms beyond quadratic ones are significant.
The same principle just described above also applies to higherorder interactions. For instance, the product variable i represents the linear×linear interaction of X and W. If significant, it means that the linear relation between X and Y changes uniformly (i.e., in a linear way) across the levels of W. In contrast, the product term XW^{2} represents a linear×quadratic interaction, which means that the linear relation of X to Y changes faster at higher (or lower) levels of W. Estimation of the interaction effect represented by the term XW^{2} requires that X, W, XW, and W^{2} (i.e., all lowerorder effects) are all in the equation. It is also possible to estimate threeway interactive effects with product variables like XWU, which represents a linear×linear interaction between X and W that changes in a uniform way across the levels of U. Entry of XWU into a regression equation requires the presence of product variables that represent all lower linear and twoway interactions (six in total). This approach can be extended even further (e.g., fourway interactions), but it should be clear that the representation of higherorder interactions can require the addition of numerous product variables to the model, which can greatly increase its complexity. (More about this point later). Also, it is relatively infrequent in social science research that such higherorder, multivariable interaction effects are large in magnitude or statistically significant.
Second, product terms like X^{2} and XW can be so highly correlated with their component variables as to cause problems due to multicollinearity. For example, the correlation between X and X^{2} for the first data set of Table A.1 is .98, which is extremely high. There are ways, however, to create product terms that reduce or eliminate correlations with their component variables. For example, Lance (1988) and Aiken and West (1991) demonstrate how to create a product term like XW that is uncorrelated with X and W but nevertheless represents an interaction effect. This method is described later. For product variables like X^{2} that reflect curvilinear relations, Cohen and Cohen (1983, pp. 237238, 242249) describe additional methods to reduce multicollinearity, such as rescaling X so that its mean is zero before computing X^{2}.
Finally, product variables can also represent interactive effects that involve categorical variables. For example, if X and W are both dichotomous variables, then the analysis of X, W, and XW in the same regression equation is essentially the same thing as a 2×2 ANOVA. The dichotomy represented by either X or W could reflect an experimental manipulation (e.g., random assignment to treatment versus control) or a nonexperimental variable (e.g., gender). The same concept applies to the case where, say, X is continuous and W is a categorical variable. The analysis of continuous X, categorical W, and the XW product term with multiple regression is thus identical to an analysis of covariance (ANCOVA) with a test of its assumption of homogeneity of regression. (See Cohen & Cohen, 1983, and Keppel & Zedeck, 1989, for more information about how any type of ANOVA or ANOCOVA is a special case of multiple regression.) Thus, product terms can represent the whole range of interactive effects of observed variables.
Representation in Path Models
Nonlinear effects of observed variables are represented in path models with the appropriate product terms of the types described above. Consider the two path models presented in Figure A.1. Path model (a) represents X and X^{2} as exogenous and Y as endogenous. Note that the unanalyzed association is represented in this model with a dashed line instead of solid one. This represents the possibility mentioned earlier that a power term can be created so that it has reduced or even zero correlations with its component variables. Path coefficients for the direct effects of X on Y and of X^{2} on Y of model (a) of the figure respectively estimate the linear and quadratic effects of X on Y. Path model (b) of the figure depicts X, W, and the product variable XW as causes of Y. Observed exogenous variables X and W are assumed to covary. The curved lines in this model that connect the product variable to its component variables have the same meaning as for model (a). Estimates of the direct effects of model (b) reflect the linear effects of X and of W on Y and of their interactive effect on Y. Models (a) and (b) of the figure can be extended to represent all of the variations mentioned above. For example, X or W could be either dichotomous or continuous variables. Higherorder effects such as a cubic effect of X on Y (model (a)) or a linear×quadratic interaction of X and W (model (b)) could be added to these models with the appropriate product terms. Before some of these more complicated variations are considered, let's go through an example with actual data.
Figure A.1 Two
path models with product variables that represent nonlinear effects of observed variables. 
Figure A.1 Two path
models with product variables that represent nonlinear effects of observed variables. 
Example
At the beginning of the semester, a total of 104 students enrolled in introductory statistics courses were administered a test of their algebraic skills (Kline, 1994) and a questionnaire called the Math Anxiety Rating Scale by Suinn (1988). Higher scores on the first measure indicate better basic math skills and on the second greater anxiety about mathrelated content. The letter grades these students earned in statistics at the end of the semester were also recorded. Letter grades were converted to numerical values on a standard fourpoint scale (A=4.0, B=3.0, C=2.0, D=1.0, fail=0). Because some students withdrew from the course or enrolled in the course late in the semester after the math skill and anxiety questionnaires were administered, there are missing observations on these variables. (The application of a special estimation procedure that fits a model to these incomplete raw data is demonstrated in a later chapter.) The analysis described here concerns only the 75 cases (of 104) with complete data on all three variables.
The data for this example are summarized in Table A.2. The two predictor variables are the math skill and the math anxiety measures. The magnitudes of the absolute values of their correlations with grades (.23 and .12, respectively) are not very large, but these correlations reflect only linear relations. To evaluate whether the relation of basic math skills to grades in introductory statistics is moderated by mathrelated anxiety (and viceversa), the product variable skill×anxiety was created to represent the interaction effect. The correlation of this product variable with one of its component variables, math skill, is fairly substantial, .75. Also, the multiple correlation between the skill×anxiety product variable and both of its component variables is so high (.98) that multicollinearity may be a problem.
Table A.2 Evaluation of a path model with an interactive effect of math skill and mathrelated anxiety on statistics grades.  
Correlations, Means, and Standard Deviations: N = 75 university students in introductory statistics courses.  
Variable  1  2  3  4  5  
1 Math Skill  —  
2 Math Anxiety  .36  —  
3 Skill × Anxiety  .75  .33  —  
4 Residualized Skill × Anxiety  .00  .00  .20  —  
5 Grade  .23  .12  .15  .02  —  
M  10.88  212.11  2257.37  .00  2.52  
SD  3.17  45.44  679.97  136.09  1.27  
Example of Creating a Residualized Product Term  
Case  Skill  Anxiety  Skill×Anxiety  Predicted Skill×Anxiety 
Residualized Skill×Anxiety 
Grade  
1  7  188  1316  1282.358  33.642  3.7  
2  13  224  2912  2743.106  168.894  3.3  
3  16  146  2336  2498.600  162.600  4.0  
7  10  216  2160  2114.282  45.718  1.7  
8  8  279  2232  2388.991  156.991  3.7  
9  13  186  2418  2357.216  60.784  1.0  
11  11  123  1353  1352.395  .605  2.0  
13  15  173  2595  2590.257  4.743  1.0  
14  9  211  1899  1880.979  18.021  3.0  
15  5  195  975  988.387  13.387  1.3  
Steps  
1. Regress Skill × Anxiety on Skill and Anxiety, record unstandardized regression weights.  
2. Compute Predicted Skill × Anxiety = 182.528 (Skill) + 10.155 (Anxiety)  1904.478.  
3. Compute Residualized Skill × Anxiety = (Skill × Anxiety)  (Predicted Skill × Anxiety)  
Table A.2 Evaluation of a path model with an interactive effect of math skill and mathrelated anxiety on statistics grades. 
A procedure described by Lance (1988) was used to create a residualized product term that represents the interaction of math skill and math anxiety but is uncorrelated with each component variable. This method is demonstrated in Table A.2 with the first 15 cases of this data set with nonmissing observations on all variables. The first step of this procedure is to create the product variable skill×anxiety. In the second step, this observed product variable is regressed on its two component variables, math skill and math anxiety. One records the unstandardized regression weights from this analysis and then derives predicted skill×anxiety scores. These predicted product scores are then subtracted from the observed product scores. The results are residualized product scores that are uncorrelated with math skill and math anxiety, which takes care of the multicollinearity problem. (See also the correlation matrix at the top of Table A.1 for the whole sample of N = 75.). The residualized product variable thus reflects the interaction of math skill and math anxiety controlling for their separate linear relations to course grades.
Presented in Figure A.2 is the path model analyzed for these data. This recursive model has three exogenous variables, math skill, math anxiety, and the residualized product variable that represents their interaction. No paths are included in the model to represent unanalyzed associations between the residualized product variable and the other two exogenous variables because these correlations are zero. (Compare Figure A.2 to model (b) of Figure A.1.)
Figure A.2 A path
model with a residualized product term that represents an interaction effect. 
Figure A.2 A path model
with a residualized product term that represents an interaction effect. 
A covariance matrix of these three variables and grades was submitted to a modelfitting for maximum likelihood (ML) estimation. Summarized below are the unstandardized estimates of the three direct effects and the disturbance variance for grades; the standardized estimates are reported in parentheses:
Predictor  Direct
Effect on Grade 
Disturbance Variance 
Math Skill  .087* (.22)  1.521** (.94)^{a} 
Math Anxiety  .001 (.04)  
Residualized Skill×Anxiety  .001 (.02)  
*p < .05 **p < .01  
^{a}The value in parentheses is the proportion of unexplained variance. 
The results reported above indicate that the only significant direct effect is the one for the positive linear direct effect of math skill on course grades. The linear effect of mathrelated anxiety on grades is negative but nonsignificant and small in absolute magnitude. The nonsignificant path coefficient for the interaction effect means that the relation of math skills to grades in introductory statistics is unaffected by the level of mathrelated anxiety. The same result also means that math anxiety is unessentially unrelated to course performance regardless of students' basic algebraic skills. Although it is not necessary for this example, there is a method for interpreting a significant interaction between two continuous variables that has been suggested by Cohen and Cohen (1983, pp. 320324) and others (e.g., Aiken & West, 1991). Because this method is applied in an upcoming example, it is not described here.
Additional examples of the estimation of nonlinear effects of observed variables can be found in McArdle (1994), who studied the linear, quadratic, and cubic effects of age to latent cognitive ability factors, and Loehlin, Horn, and Willerman (1990), who included quadratic effects of age in models of the stability of extroversion, socialization, and emotional lability among adopted and biological children of adoptive parents. Cohen and Cohen (1983, pp. 369371) describe the analysis of a path model of university productivity and salary that includes an interaction effect. Also, a recent chapter by Wood (1994) deals with the effects of omitted variables and their interactions on the estimation of structural models with observed variables.
Mediators and Moderators Together
Baron and Kenny (1986) and James and Brett (1984) noted that it is possible to represent in the same path model interactive (moderator) effects that are mediated by another variable (e.g., XW=>U=>Y) or interaction effects that mediate causal effects of prior variables (e.g., U=>XW=>Y). The first combination of moderators and mediators together in the same model is called mediated moderation, and the second is known as moderated mediation. Lance (1988) evaluated a path model with an interaction effect that was specified to be mediated by another variable (mediated moderation). Specifically, Lance studied the relation of memory demand (immediate versus delayed recall), complexity of social perception, and their interaction on the overall accuracy of recall of the script of a lecture. The path model tested by Lance included a fourth variable, recollection of specific behaviors mentioned in the script, through which the interaction effect was specified to affect overall accuracy of recall. Lance's results suggested that (1) complexity of social perception affected the recall of specific behaviors only in the delayed recall condition; (2) the indirect effect of the complexity×memory demand interaction was significant; but (3) the direct effects of the individual component variables on overall accuracy were not significant. For more information about path models that represent moderator and indirect effects in the same model, readers are referred to the works just cited.
It is probably apparent by now that is possible to specify and test very complex models that may have higherorder product terms or a mix of indirect (mediator) and interactive (moderator) effects. The costs of increased complexity, through, are the same as for any type of structural equation model (e.g., the need for ever larger samples, programming complexity, more potential problems with starting values). The researcher should thus have solid theoretical reasons to guide the addition of terms that reflect curvilinear or interactive term to a structural equation model.
A.3 Nonlinear Effects of Latent Variables
Introduction
The estimation of curvilinear or interactive effects of observed variables in path models is limited by the use of a single indicator of each theoretical variable and the resulting inability to take account of measurement error. The second limitation may be especially critical for product variables because they reflect the combined measurement errors of their component variables. For example, if either X or W is not very reliable, then the reliability of XW must also be poor. Although there are ways to adjust path coefficients for product terms of observed variables for measurement error (Aiken & West, 1991; Jaccard & Wan, 1995; MacCallum & Mar, 1995; McClelland & Judd, 1993, Ping, 1996b), the only way to overcome the first limitation just mentioned is to use multiple indicators of each construct. The estimation of nonlinear effects of latent variables within a multiple indicator approach to measurement is demonstrated below.
Consider model (a) of Figure A.3. In a departure from notation used in previous chapters, latent variables are represented with uppercase letters (e.g., X) and observed variables are depicted with lowercase letters (e.g., x_{1}). Model (a) is a partially latent hybrid model that represents the linear effect of factor X on the observed exogenous variable y. (The rationale outlined here also applies to the use of multiple indicators to measure an endogenous construct.) The exogenous factor X has two indicators, x_{1} and x_{2}. The measurement model for indicators x_{1} and x_{2} can also be represented with the following structural equations:
Equations [A.1] 
x_{1} =
X + E_{1} x_{2} = L_{2} X + E_{2} 
In Equations [A.1] above, the loading of x_{1} is fixed to 1.0 to scale X and the loading of x_{2} is a free parameter represented by L_{2}. The other parameters of this measurement model are the variances of X and of the two measurement error terms, E_{1} and E_{2}.
Suppose that a researcher wished to estimate the quadratic effect of latent variable X on the endogenous variable. This calls for adding to model (a) of the figure the latent product variable X^{2} that represents this quadratic effect, which is estimated by the path coefficient for the direct effect X^{2}=>y. The indicators of latent product variables are products of observed variables. Accordingly, the indicators of X^{2} are all products of X's indicators that represent a quadratic effect, x_{1}^{2}, x_{2}^{2}, and x_{1}x_{2}. Note that the third product indicator just listed, x_{1}x_{2}, does not represent an interaction because its components (x_{1} and x_{2}) are specified to measure the same factor. By squaring or taking the product of the relevant equations in [A.1] above, the measurement model for the product indicators x_{1}^{2}, x_{2}^{2}, and x_{1}x_{2} is:
Equations [A.2] 
x_{1}^{2} = X^{2} +
2XE_{1} +
E_{1}^{2} x_{2}^{2} = L_{2}^{2}X^{2} + 2L_{2}XE_{2} + E_{2}^{2} x_{1}x_{2} = L_{2}^{2}X^{2} + L_{2}XE_{1} + XE_{2} + E_{1}E_{2} 
Equations [A.2] above show that the measurement model for the three product indicators involves not only the latent quadratic term X^{2} but also five additional latent product variables, XE_{1}, XE_{2}, E_{1}^{2}, E_{2}^{2}, and E_{1}E_{2}. (The latter three product variables are error terms.) The parameters of this measurement model are the loadings of the product indicators on these latent product variables and their variances. Note that the factor loadings in Equations [A.2] are either constants (e.g., 1.0 for X^{2}=>x_{1}^{2}, 2.0 for XE_{1}=>x_{1}^{2}) or functions of L_{2}, which is the loading of the observed variable x_{2} on X from Equations [A.1]. For example, the loading of the product indicator x_{2}^{2} on the product factor XE_{2}, is two times L_{2}. Thus, no new factor loadings are needed. Also, Kenny and Judd (1984) demonstrated that the latent variable X and the six latent product variables of Equations [A.2] are uncorrelated if X and the measurement errors of its indicators (E_{1} and E_{2}) are normally distributed and have means of zero. Using the same assumption, Kenny and Judd (1984) also showed that the variances of the latent product variables of Equations [A.2] are related to the variances of X and the measurement errors of its indicators as follows; note that Var below indicates variance:
Equations [A.3] 
Var
X^{2} = 2(Var
X)^{2}Var XE_{1} =
(Var X)(Var
E_{1}) Var XE_{2} = (Var X)(Var E_{2}) Var E_{1}^{2}_{ }= 2(Var E_{1})^{2}Var E_{2}^{2}_{ }= 2(Var E_{2})^{2}Var E_{1}E_{2} = (Var E_{1})(Var E_{2}) 
For example, the variance of the latent product variable X^{2} is two times the squared variance of X, and the variance of the latent product term E_{1}E_{2} is the product of the variances of the measurement errors of the indicators of X, E_{1}, and E_{2}.
Figure
A.3 Hybrid models with product
indicators and product latent variables that represent linear or curvilinear effects of a latent variable. Note: Latent variables are designated by uppercase letters and indicators by lowercase letters. 
Figure A.3 Hybrid models with product
indicators and product latent variables that represent linear or curvilinear effects of a latent variable. Note: Latent variables are designated by uppercase letters and indicators by lowercase letters. 
Model (b) in Figure A.3 includes the entire measurement model of the product indicators. Although the whole model looks complicated, there are only seven parameters in total that require estimates under the assumption of normal distributions and means of zero. These seven free parameters include the two path coefficients for the linear and quadratic direct effects of factor X on the endogenous variable y; the variance of the disturbance term for y; the variance of the latent variable X; the loading of indicator x_{2} on X, L_{2}; and the variances of the measurement errors for x_{1} and x_{2}, E_{1} and E_{2}, respectively. That's all there is to it. As per Equations [A.2] and Equations [A.3], all of the parameters of the measurement model of the product indicators are known once you have estimates for the parameters of the measurement model of their component variables, x_{1} and x_{2}. Specific estimation methods are considered after interactive effects of latent variables are introduced.
Suppose that there are two indicators each of latent exogenous variables X and W. The measurement model for these four observed variables is as follows:
Equations [A.4] 
x_{1} =
X +
E_{x}_{1} x_{2} = L_{x}_{2}X + E_{x}_{2} w_{1} = W + E_{w}_{1} w_{2} = L_{w}_{2}W + E_{w}_{2} 
where the loadings of x_{1} and w_{1} are each fixed to 1.0 to scale their respective factors. The parameters of the above measurement model are the loadings of x_{2} and w_{2} on their respective factors (designated L_{x}_{2} and L_{w}_{2}); the variances of latent variables; X, W; and the variances of the four measurement errors.
The product latent variable XW represents the linear×linear interactive effect of factors X and W. Its indicators are the products of the indicators of X and W that represent linear×linear interactions, x_{1}w_{1}, x_{1}w_{2}, x_{2}w_{1}, and x_{2}w_{2}. The equations for the measurement model of these four product indicators are the products of Equations [A.4], or:
Equations [A.5] 
x_{1}w_{1} = XW +
XE_{w}_{1} +
WE_{x}_{1} +
E_{x}_{1}E_{w}_{1} x_{1}w_{2} = L_{w}_{2}XW + XE_{w}_{2} + L_{w}_{2}WE_{x}_{1} + E_{x}_{1}E_{w}_{2} x_{2}w_{1} = L_{x}_{2}XW + L_{x}_{2}XE_{w}_{1}+ WE_{x}_{2}+ E_{x}_{2}E_{w}_{1} x_{2}w_{2} = L_{x}_{2}L_{w}_{2}XW + L_{x}_{2}XE_{w2}+ L_{w}_{2}WE_{x}_{2} + E_{x}_{2}E_{w}_{2} 
Equations [A.5] show that the four product indicators load not just on XW but also on eight other latent constructs or error terms, XE_{w}_{1}, XE_{w}_{2}, WE_{x}_{1}, WE_{x}_{2}, E_{x}_{1}E_{w}_{1}, E_{x}_{1}E_{w}_{2}, E_{x}_{2}E_{w}_{1}, and E_{x}_{2}E_{w}_{2}. The parameters of the model are the loadings of the four product indicators on these eight latent variables and the variances of the latter. However, all of these parameters are determined by those of the measurement model for the observed variables (Equations [A.4]). For example, all of the factor loadings in Equations [A.5] equal either 1.0, the loadings of x_{2} or w_{2} (L_{x}_{2}, L_{w}_{2}), or the product of their loadings (L_{x}_{2}L_{w}_{2}). Assuming that X, W, and the measurement errors of their indicators are normally distributed with means of zero, then X and W are uncorrelated with all of the latent variables in the measurement model of the four product indicators. Assuming normality and that the means of all the latent variables equal zero, the variances of the latent variables in Equations [A.5] are all functions of the variances and covariances of X, W, and the measurement errors of their indicators (Kenny & Judd, 1984), as follows; note that Cov below indicates covariance:
Equations [A.6] 
Var XW = (Var
X)(Var W) + (Cov X,
W)^{2} Var XE_{w}_{1} = (Var X)(Var E_{w}_{1}) Var XE_{w}_{2} = (Var X)(Var E_{w}_{2}) Var WE_{x}_{1} = (Var W)(Var E_{x}_{1}) Var WE_{x}_{2} = (Var W)(Var E_{x2}) Var E_{x}_{1}E_{w}_{1} = (Var E_{x}_{1})(Var E_{w}_{1}) Var E_{x}_{1}E_{w}_{2} = (Var E_{x}_{1})(Var E_{w}_{2}) Var E_{x}_{2}E_{w}_{1} = (Var E_{x}_{2})(Var E_{w}_{1}) Var E_{x}_{2}E_{w}_{2} = (Var E_{x}_{2})(Var E_{w}_{2}) 
Just as for a latent quadratic effect, the parameters of the measurement model of product indicators of a latent interaction effect are all determined by the parameters of their component observed variables. Fortunately for both types of nonlinear effects of latent variables, there is less complexity than is at first apparent because relatively small numbers of parameters need to be estimated, which is the topic covered next.
Estimation
Kenny and Judd (1984) described a method for estimating nonlinear effects of latent variables. Their method involves the specification of all the latent variables in the measurement model of the product indicators and the imposition of constraints on estimates of its parameters. For example, model (b) of Figure A.3 with a quadratic effect of a latent variable is specified as it appears in the figure and is estimated with the constraints indicated in Equations [A.2] for the factor loadings and in Equations [A.3] for the variances. Some of these constraints are nonlinear, which means that a parameter estimate is forced to equal an arithmetic product of other parameters. For example, the loading of x_{1}x_{2} on X^{2} in this model is forced to equal the squared loading of x_{2} on X, and the variance of X^{2} is set to equal two times the squared variance of X. The model described earlier with two indicators each of exogenous factors X and W and product indicators of an interaction effect between them is analyzed in an analogous way: The measurement portions of the model are specified as per Equations [A.4] and Equations [A.5], and the whole model is estimated with constraints that match the dependencies among the parameters implied by Equations [A.6] given the assumptions of normality and that the means of the nonproduct latent variables are zero. See Kenny and Judd (1984) and Jöreskog and Yang (1996) for examples of this method.
A potential problem with the KennyJudd method is that not all modelfitting programs allow the imposition of nonlinear constraints. The COSAN subroutine of the CALIS program (Hartmann, 1992) and the latest version of LISREL (8; reviewed in Chapter 11) have this capability, but other programs may not. Hayduk (1987 pp. 232242) devised a way to implement the KennyJudd method with modelfitting programs that do not allow nonlinear constraints, but this procedure is quite complicated. Even with a modelfitting program that is capable of imposing nonlinear constraints, the programming required to apply the KennyJudd method may still be fairly complex (e.g., Jöreskog & Yang, 1996).
Ping (1996a) described a simpler procedure for estimating nonlinear effects of latent variables. This method is based on the same assumptions and fundamental formulae as the KennyJudd method but differs from the latter in two ways. First, Ping's method features twostep estimation instead of the imposition of nonlinear constraints on the parameters of product variables. Second, Ping's method does not require the specification of all the latent variables that underlie the product indicators (e.g., Equations [A.2] and Equations [A.5], model (b) of Figure A.3). In this sense, Ping's method can be viewed as an approximation of the full KennyJudd model for nonlinear effects of latent variables.
In the first step of Ping's method, the model without the product indicators is analyzed. Results from this analysis are used to calculate parameters of the measurement model of the product indicators. These parameters include the factor loadings of the product indicators, their measurement error variances, and the variances of latent product variables that represents a nonlinear effect (e.g., X^{2}, XW). These calculated values are then specified as fixed parameters in the analysis of the original model with the product indicators in the second step of Ping's method. From this second analysis, one obtains the path coefficients that estimate the linear and nonlinear effects of the latent variables.
Ping (1996a) created artificial data that are summarized with the covariance matrix presented in Table A.3. The variables include two indicators of a latent exogenous factor, X; three product indicators of a latent quadratic factor, X^{2}; and an observed endogenous variable y. The first step in Ping's method is analyze a standard hybrid model that specifies factor X as a cause of y and that x_{1} and x_{2} are the indicators of X (i.e., model (a) of Figure A.3). One records from this analysis the unstandardized estimates of the parameters of the measurement model for x_{1} and x_{2}. The analysis of the covariance matrix of x_{1}, x_{2}, and y in Table A.3 with a modelfitting program yielded the unstandardized ML estimates that are reported in the middle of the table.

With estimates of the measurement model of the nonproduct indicators in hand, one then calculates the parameters of the measurement model for the three product indicators, x_{1}^{2}, x_{2}^{2}, and x_{1}x_{2} using the formulae that are also presented in Table A.3. These formulae basically integrate Equations [A.2] and Equations [A.3] such that one works with a simplified measurement model for the product indicators. The parameters of this simplified measurement model include only the loadings of the product indicators on X^{2} and the variances of X^{2} and the measurement error terms. The calculated values of the parameters of this simplified measurement model are presented in the lower part of the table. For example, the variance of the latent quadratic factor X^{2} is calculated as two times the squared variance of the latent linear factor X, or 2(1.009)^{2} = 2.036. This calculated value simulates the nonlinear constraint that restricts the variance of X^{2} to 2(Var X)^{2} as per Equations [A.3]. In the upcoming analysis of the whole model with the product indicators, the variance of X^{2} is fixed to equal 2.036. Likewise, the loadings of the product indicators on X^{2} are the respective products of the loadings of the nonproduct indicators , x_{1} and x_{2} on X. The derivations for the measurement error variances of the product indicators are more complex, but they eliminate the need to represent all of the latent variables that underlie the product indicators implied by Equations [A.2].
The second step is to take the calculated estimates for the measurement model of the product indicators and specify them as fixed parameters in the analysis of the whole model. All of the other parameters in the whole model are freely estimated. Given an acceptable measurement model of the nonproduct indicators, one should see only trivial changes in the values of these parameters in the analysis of the whole model. If not, then interpretational confounding is suggested (Chapter 8, section 8.3). The other freely estimated parameters from this analysis include the direct effects of the latent linear and quadratic effects of X on y and the disturbance variance of the latter. The whole covariance matrix at the top of Table A.3 was submitted to a modelfitting program to analyze both the product and nonproduct indicators with the parameters of the product indicators fixed to equal the calculated values in the table. The results of this analysis are presented in Figure A.4. Note that the estimates for the measurement model of the product indicators in the figure match the calculated values in Table A.3. Also, estimates for the measurement model of the indicators of X from this analysis are similar to those from the first step of Ping's method (Table A.3), which is a desired result. The unstandardized (and standardized) estimates of the linear and quadratic effects of X on y are respectively .296 (.31) and .521 (.79). These results indicate that the although there is a positive linear effect of X on y, there is also a negative quadratic aspect to their relation that is greater in absolute magnitude than the linear trend. A negative quadratic relation is shaped like a rainbow, in this case one that is inverted. Although X alone accounts for some portion of y's variance, prediction is better when X^{2} is also included in the model.
Figure
A.4 A model with a quadratic effect of
latent variable X estimated with Ping's (1996a) method. Note: Latent variables are represented by uppercase letters and indicators by lower case letters. The values in parentheses are standardized estimates. The standardized value for the disturbance is the proportion of unexplained variance. 
Figure A.4 A model with a quadratic
effect of latent variable X estimated with Ping's (1996a) method. Note: Latent variables are represented by uppercase letters and indicators by lower case letters. The values in parentheses are standardized estimates. The standardized value for the disturbance is the proportion of unexplained variance. 
The estimation of an interactive effect of latent variables X and W using Ping's method is demonstrated in Table A.4. The covariance matrix of five observed variables and four product indicators presented at the top of the table was created by Ping (1996a). In the first step of Ping's method, a standard hybrid model was tested. This model featured x_{1} and x_{2} as indicators of X; w_{1 }and w_{2} are indicators of W; an unanalyzed association between X and W; and direct effects of X and W on y. The covariance matrix for these five observed variables was submitted to a modelfitting program for ML estimation. The unstandardized estimates for the measurement model of the nonproduct indicators are reported in the middle of Table A.4. These values were used to calculate the parameters of the measurement of the product indicators x_{1}w_{1}, x_{1}w_{2}, x_{2}w_{1}, and x_{2}w_{2} according to the formulae listed in the table. These calculated values were then specified as fixed parameters in the analysis of the whole covariance matrix.

The unstandardized estimates from the analysis of the nonproduct and product indicators is presented in Figure A.5. Note that the estimates from the measurement model of the product indicators are the same as the calculated values in Table A.4. The only really new results are the path coefficients for the linear and interactive effects of X and W on the endogenous variable and the disturbance variance. The standardized estimates of the interaction effect indicate that its absolute value (.87) is greater than either linear effect of X or W on y (.14, .28, respectively). Thus, the effect of X seems to depend upon the level of W and viceversa. How to more precisely interpret an interaction effect is demonstrated later.
Figure
A.5 A model with an interactive effect
of latent variables X and W estimated with ping's method. Note: Latent variables are represented by uppercase letters and indicators by lowercase letters. The unstandardized solution is shown. The values in parentheses are standardized estimates. The standardized value for the disturbance is the proportion of unexplained variance. 
Figure A.5 A model with an interactive
effect of latent variables X and W estimated with ping's method. Note: Latent variables are represented by uppercase letters and indicators by lowercase letters. The unstandardized solution is shown. The values in parentheses are standardized estimates. The standardized value for the disturbance is the proportion of unexplained variance. 
Three other points about estimation of nonlinear effects of latent variables before we consider an example with actual data. First, the models in Figure A.4 and Figure A.5 (and later in Figure A.6, too) cannot be estimated in a single step. That is, one could not specify these models as they appear in the figures and analyze them with covariance matrices of product and nonproduct indicators without going through both steps of Ping's (1996a) method. The models in these figures only approximate the full models implied by Equations [A.1] through Equations [A.6].
Second, the assumption of the KennyJudd and Ping methods that the nonproduct factors and measurement errors are normally distributed is important. If it is violated, then the constraints imposed in each method may be incorrect. Normality of the nonproduct factors and error terms can be evaluated by examining the distributions of the nonproduct observed variables. If these distributions are obviously nonnormal, then something should be done such as the use of a transformation (Chapter 4, section 4.6). Even if the distributions of the nonproduct variables are normal, those of the product variables are not. Thus, the multivariate normality assumption of ML estimation may be violated. Although estimates generated by ML are reasonably accurate when the data are nonnormal, recall that tests of significance may be positive too often. The use of corrected test statistics such as the SatorraBentler statistic and robust standard errors may reduce some of this bias (Chapter 7, section 7.6). The use of estimation procedures with alternative distributional assumptions is also possible (Chapter 5, section 5.11).
Finally, the assumption of the KennyJudd and Ping methods that the means of all nonproduct latent variables are zero may be inappropriate in some cases. These include longitudinal studies in which means are expected to change over time and multisample analyses in which group means may be different. See Jöreskog and Yang (1996) for additional discussion of this issue.
Example
This example involves the application of Ping’s (1996a) method to estimate a model with an interactive effect of two latent variables. The data for this problem are from Kenny and Judd (1984), who used them to demonstrate their analytical method for models with product variables. The model they analyzed was quite complex in that it contained a quadratic effect and two interaction effects, one linear×linear and the other linear×quadratic. Of these three nonlinear effects, however, the one for the linear×linear interaction effect was clearly the largest. For this reason and for clarity’s sake, the model analyzed here contains only the linear×linear interaction. This simplified model is thus nested under the one evaluated by Kenny and Judd (1984).
The main question of this analysis is whether the relation of voters’ views on issues to their perceptions of a candidate’s positions on the same issues varies as a function of voters’ sentiment toward the candidate. For example, voters who dislike a candidate may overestimate the discrepancies between their views and those of the candidate. Likewise, voters may tend to overestimate their agreement with a candidate they personally like. The data analyzed here are from a survey conducted in 1968 of 1,160 voters in the United States. Note that this sample size is not extraordinarily large for the analysis of an interaction effect of latent variables. Large samples (e.g., N > 500) may be needed in such analyses for the results to be statistically stable. The voters in this study were asked their views on two issues, the Vietnam War and crime. The same subjects were asked to rate the views of two presidential candidates, Hubert Humphrey and Richard Nixon, on the same issues and to indicate their sentiment toward each candidate. Kenny and Judd (1984) conceptualized ratings of each issue as the indicators of two latent variables, one that represents voters’ views (V) and the other their perceptions of a candidate’s views (C). There was only a single measure of sentiment. Accordingly, this observed variable is represented with a lowercase s.
The model presented in Figure A.6 represents C (perception of candidates’ views) as affected by V (voters’ views) and s (sentiment). The model also contains unanalyzed associations between the exogenous variables V and s and between the measurement errors of voter and candidate views of the same issue. Also note that the loadings of the indicators of V and of C are all constrained to equal one. Because the V and C factors each have only two indicators, this constraint is necessary to identify the model given the measurement error correlations. However, these particular equality constraints are unrelated to Ping’s estimation method. The exogenous product factor Vs with its product indicators (v_{1}s and v_{1}s) represents the interactive effect of voters’ views and sentiment. If the path coefficient for the direct effect of Vs on C is substantial, then the relation between voters’ views and their judgements of candidates’ positions is moderated by sentiment. (By the same token, the effect of s on C would also vary with V.) The data are summarized in Table A.5 are for one presidential candidate (Nixon) and they include the two product variables that are the indicators of the latent interaction factor. Consistent with the assumption of their method that the means of the nonproduct latent variables are all are zero, Kenny and Judd (1984) rescaled the observed variables so that their means were zero before deriving the product variables.
Figure
A.6 A model with a latent interactive
effect of voters' positions (V) and their sentiment toward a candidate (s) on the perception of a candidate's positions (C) estimated with Ping's method. 
Figure A.6 A model with a latent
interactive effect of voters' positions (V) and their sentiment toward a candidate (s) on the perception of a candidate's positions (C) estimated with Ping's method. 
The first step of Ping’s method is to analyze the model in Figure A.6 without the product variables. From the results of this analysis, one records the unstandardized parameter estimates for the measurement model of the observed variables, v_{1}, v_{2}, and s. Because s is the sole indicator of a sentiment factor, its factor loading is assumed to be 1.0 and its measurement error variance zero. Also recall that the loadings of v_{1} and v_{2} are both fixed to 1.0. With these fixed parameters, the other pertinent estimates of the measurement model for v_{1}, v_{2}, and s include the variances of s and the latent voter factor V, the covariance of these two exogenous variables, and the measurement error variances of the indicators of V.
A covariance matrix assembled from the data reported in Table A.5 for the five nonproduct observed variables was analyzed with a modelfitting program to derive the aforedescribed estimates with ML. These estimates are reported in the middle of Table A.5. Estimates for the parameters of the measurement model of the product indicators were then calculated using the formulae summarized in Table A.4. These calculated values include the variance of the latent product variable Vs; the loadings of the product indicators v_{1}s and v_{2}s on Vs; and the variances of their measurement errors. These calculated values are reported in the bottom part of Table A.5. For example, the calculated variance of Vs is 9.08, which equals Var V × Var s + (Cov V, s)^{2}, which is 1.80(4.96) + (.39)^{2}. The derivations of the other parameters of the measurement model of the product indicators are also listed in the table.

The next step in Ping’s method is to treat the values calculated for the product variables as fixed parameters in a second analysis, this one of the whole model. The whole covariance matrix for all five nonproduct variables and two product variables assembled from the correlations and standard deviations reported in Table A.5 was submitted to a modelfitting program. The overall chisquare (15) statistic for the whole model is 164.39 and the chisquare/df ratio is 10.96. Values so high for a standard hybrid model would suggest poor fit, but these values seem fairly typical for models with product variables estimated in twosteps. Values of other fit indexes seem more consistent with expectations. For example, the JöreskogSörbom Goodness of Fit Index is .96 and the standardized root mean squared residual is .01.
Reported in Table A.6 are the ML estimates for the whole model. Estimates that are fixed due to the application of Ping’s method are reported in the top part of the table followed by estimates that are fixed due to general requirements for the identification of this particular example. Reported next in the table are the estimates of the free parameters. All of these unstandardized estimates are significant at the .05 level. The main estimates of interest here are those of the linear and interactive effects of sentiment (s) and voters’ views (V) on judgement of the candidate’s positions (C). Kenny and Judd’s (1984) unstandardized estimates of V=>C, s=>C, and Vs=>C were, respectively, .18, .11, and .21. These values are very close to those derived here using Ping’s method (.20, .08, and .21; see Table A.6), which is encouraging in that Ping’s method is simpler. Overall, the magnitude of the interaction effect is sizable (standardized path coefficient = .81). A method to interpret the nature of this interaction effect is outlined below.
Table A.6 Parameter estimates for the full interactive model of voters’ perception  
Variable(s)  Parameter  Estimate^{a} 
Parameters Fixed as Part of Ping’s Method  
Vs  Variance of product factor  9.08 
V, Vs  Covariance of product factor (Vs) with voter (V) factor  0 
s, Vs  Covariance of product factor (Vs) with Sentiment (s)  0 
v_{1}s  Loading of product indicator on product factor (Vs)  1.00 (.64) 
Measurement error variance of product indicator  13.17 (.59)  
v_{2}s  Loading of product indicator on prroduct factor (Vs)  1.00 (.63) 
Measurement error variance of product indicator  13.61 (.60)  
Parameters Fixed Due to General Requirements for Identification  
v_{1}  Loading of Vietnam War indicator on voter factor (V)  1.00 (.54) 
v_{2}  Loading of crime indicator on voter factor (V)  1.00 (.53) 
c_{1 }  Loading of Vietnam War indicator on candidate factor (C)  1.00 (.49) 
c_{2}  Loading of crime indicator on candidate factor (C)  1.00 (.52) 
Free Parameters  
V  Variance of voter factor (V)  1.08 (1.00) 
s  Variance of observed sentiment variable (s)  4.96 (1.00) 
V, s  Covariance of voter factor (V) and sentiment (s)  .39 (.16) 
v_{1}  Measurement error variance of indicator of voter factor (V)  2.64 (.71) 
v_{2}  Measurement error variance of indicator of voter factor (V)  2.75 (.72) 
c_{1}  Measurement error variance of indicator of candidate factor (C)  1.97 (.76) 
c_{2}  Measurement error variance of indicator of candidate factor (C)  1.72 (.73) 
E_{v}_{1}, E_{c}_{1}  Measurement error covariance  .60 (.26) 
E_{v}_{2}, E_{c}_{2}  Measurement error covariance  .62 (.28) 
C  Direct effect of voter factor (V) on candidate factor (C)  .20 (.26) 
Direct effect of sentiment (s) on on candidate factor (C)  .08 (.23)  
Direct effect of product factor (Vs) on candidate factor (C)  .21 (.81)  
Disturbance variance  .15 (.24)  
Note. All unstandardized estimates are significant at the .05 level.  
^{a}Unstandardized
(standardized). The standardized values for the disturbance and measurement variances are proportions of unexplained variance.  
Table A.6 Parameter estimates for the full interactive model of voters’ perception 
Interpretation of an Interactive Effect
The method described here is an extension of a strategy for interpreting an interaction effect of observed variables (e.g., Aiken & West, 1991; Cohen & Cohen, 1983). The starting point for the present example is to write out the structural equation for the endogenous variable C (perception of candidate’s positions), which has the following form:
Equation [A.7] 
C = P_{V} (V) + P_{s} (s) + P_{Vs} (Vs) + D, 
where P represents unstandardized path coefficients; V stands for the voter’s views latent variable; s is the single indicator of sentiment; Vs is the voter view × sentiment interaction factor; and D is the disturbance term. Dropping the disturbance (which represents unexplained variance) transforms Equation [A.7] to an equation that expresses predicted perception of a candidate’s position (C_{pred}) as a function of the other three product or nonproduct factors (V, s, Vs). Writing this prediction formula with the actual values of the path coefficients in it yields:
Equation [A.8] 
C_{pred }=.20 V  .08 s + .21 Vs. 
Equation [A.8] above is similar to a regression equation but one that generates predicted scores on a dependent latent variable. The above prediction equation can be rearranged so that the Vs interaction term is eliminated. For example, the following version is algebraically identical to Equation [A.8]:
Equation [A.9] 
C_{pred} = (.20 + .21 s) V  08 s. 
Note that multiplying V by s in Equation [A.9] above yields the product variable Vs. The term (.20 + .21s) in the above equation can be viewed as regression coefficient for V in the prediction of C but one that varies with the level of s, which also describes an interactive effect. The second term of this equation, (.08 s), is analogous to an intercept that, like V’s coefficient, also takes s into account. This rearranged prediction equation thus captures the essence of the interactive effect between V (voter’s view) and s (sentiment) in their joint effect on C (perception of candidate view).
Using Equation [A.9], one substitutes various values of s and inspects the effect on the coefficients for V. For instance, let’s plug into this prediction equation five values of s, ones that correspond to +2, +1, 0, 1, and 2 standard deviations away from its mean. Because the mean of s is already zero and its standard deviation is 2.23 (Table A.5), these five values are 4.46 (i.e., 2×2.23), 2.23, 0, 2.23, and 4.46. Substituting these values for s into Equation [A.9] yields the following results:
Value of s  Meaning  Prediction Formula C_{pred} = (.20 + .21s) V  08 s 
4.46  +2 SDs  (.20+.21(4.46)) V  .08 (4.46) = 1.14 V  .36 
2.23  +1 SD  (.20+.21(2.23)) V  .08 (2.23) = .69 V  .18 
0  mean  (.20+.21(0)) V  .08 (0) = .20 V 
2.23  1 SD  (.20+.21(2.23)) V  .08 (2.23) = .27 V + .18 
4.46  2 SDs  (.20+.21(4.46)) V  .08 (4.46) = .74 V + .36 
Note that the coefficients for V in the prediction formulae in the right side of the above table are positive at high levels of s but negative at lower levels. Also note that predicted values of C are adjusted downward at higher levels of s and upward at lower levels, which reflects the overall negative relation between s and C. Expressed more directly: As sentiment toward a candidate declines, the relation between voter’s views and his or her judgement about the candidate’s positions on the same issues becomes increasingly negative. That is, voter’s perceive dissimilarity between their views and those of a disliked candidate. Voters who like the candidate, however, tend to see their views as being similar to those of the candidate, which yields a positive relation between the two.
A popular stereotype of social scientists is that in response to the question, “How is X related to Y?” (insert your favorite pair of constructs: mores and conformity, handedness and neural organization, etc.), they respond, “Well, it depends.” The representation of curvilinear or interactive effects in structural models provides a way to evaluate “it depends” relations. Both types of nonlinear effects involve statistical effects that are not uniform (linear). A curvilinear relation between two variables means that the slope of the line that describes their scatterplot changes as scores increase. An interaction effect involves, at minimum, a relation of one variable to another that is not uniform across the levels of a third. The relation may change in direction or magnitude, but it varies with the other variable.
When a structural model includes only observed variables (i.e., a path model), curvilinear or interactive effects can be represented directly by product variables. For example, the product term X^{2} represents the quadratic effect of X. When both X and X^{2} are specified as causes of Y, the path coefficients of these direct effects respectively estimate the linear and quadratic effects of X on Y. Interactive effects of observed variables are likewise represented with product variables. For instance, the product term XW represents the interactive effect of X and W. When represented in a path model along with its component variables (X and W) as causes of Y, the path coefficient for the direct effect of XW on Y estimates the interactive (joint) effect of X and W on Y. Adding product variables to a path model makes it more complicated. This is especially true if higherorder interactive effects are estimated, which may require the presence of numerous product variables that represent all lowerorder effects. Product variables should thus not be added to path models willynilly.
The analysis of nonlinear effects of latent variables in hybrid models is based on a similar logic. The indicators of latent product variables that represent curvilinear or interaction effects are the products of the corresponding indicators of the latent linear variables. For example, if X is a latent factor with two indicators x_{1} and x_{2}, then the product variables x_{1}^{2}, x_{2}^{2}, and x_{1}x_{2} are the indicators of X^{2}, the latent product variable that represents the quadratic effect of X. Along the same lines, if w_{1} and w_{2} are the indicators of W, then the four product variables x_{1}w_{1}, x_{1}w_{2}, x_{2}w_{1}, and x_{2}w_{2} are the indicators of the latent interaction factor XW.
The measurement models of product indicators actually include more latent product variables than just, say, a single quadratic or interaction factor. Also, the parameters of these measurement models are related to the parameters of the measurement models of the nonproduct indicators. Some of the relations between these two sets of parameters are nonlinear, which means that one parameter is a multiplicative product of other parameters. There are two estimation methods that deal with the nonlinear constraint problem. Both methods assume that the nonproduct latent variables and the measurement errors of their indicators are normally distributed and that the means of these variables are all zero. The KennyJudd method features specification of all the latent variables that underlie the product indicators and the imposition of all nonlinear constraints. Not all modelfitting programs allow nonlinear constraints, however.
In contrast, Ping’s method can be used with virtually any modelfitting program. Ping’s method uses twostep estimation to simulate the imposition of nonlinear constraints. In the first step, the model is analyzed without the product variables. Estimates of the measurement model for indicators that are components of product indicators are recorded. These results are then used to calculate parameter estimates for the measurement model of the product indicators. These calculated values are specified as fixed parameters in the second step, the analysis of all the variables, product and nonproduct. The path coefficients for the latent product variables that represent quadratic or interactive effects on endogenous variables are obtained from this second analysis.
If the nonproduct observed variables are severely nonnormal, then the constraints implied by the KennyJudd and Ping methods may lead to incorrect results. It is thus important to carefully screen the data when nonlinear effects of latent variables are estimated with either method. Large samples (e.g., N > 500) may also be required in order for the results to be reasonably stable. Finally, neither the KennyJudd or Ping methods may be suitable if the assumption that the means of the latent nonproduct variables are zero is not tenable. For example, if latent variables A_{1} and A_{2} represent the same construct measured at different times, then it may not be reasonable to assume that both of their means are zero if the average level of subjects’ standing on this construct is expected to increase over time.
The analysis of nonlinear effects is one way to expand the usefulness of SEM. Another is to analyze means along with covariances, which is introduced in the next supplemental chapter (B).
Baron and Kenny (1986) and James and Brett (1984) provide helpful starting points for understanding the conceptual implications of the difference between interaction and indirect effects in structural models.
Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
Baron, R. M., & Kenny, D. A. (1986). The moderatormediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 11731182.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Frederiksen, N., & and Melville, S. D. (1954). Differential predictability in the use of test scores. Educational and Psychological Measurement, 14, 647656.
Hartmann, W. M. (1992). The CALIS procedure: Extended user's guide. Cary, NC: SAS Institute.
Hayduk, L. A. (1987). Structural equation modeling with LISREL. Baltimore, MD: John Hopkins University Press.
Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348357.
James, L. R., & Brett, J. M. (1984). Mediators, moderators and tests for mediation. Journal of Applied Psychology, 69, 307321.
Jöreskog, K. G., and Yang, F. (1996). Nonlinear structural equation models: The KennyJudd model with interaction effects. In G. A. Marcoulides and R. E. Schumacker (Eds.), Advanced structural equation modeling (pp. 5788). Mahwah, NJ: Lawrence Erlbaum.
Kenny, D. A., & Judd, C. M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 96, 201210.
Keppel, G., & Zedeck, S. (1989). Data analysis for research designs. New York: W. H. Freeman.
Kline, R. B. (1994, August). Validity of a math skills pretest for introductory statistics. Presented at the 102nd Annual Meeting of the American Psychological Association, Los Angeles.
Koestler, A. (1989). The act of creation. New York: Penguin Books. (Original work published 1964)
Lance, C. E. (1988). Residual centering, exploratory and confirmatory moderator analysis, and decomposition of effects in path models containing interaction effects. Applied Psychological Measurement, 12, 163175.
Loehlin, J. C., Horn, J. M., & Willerman, L. (1990). Heredity, environment, and personality change: Evidence from the Texas Adoption Project. Journal of Personality, 58, 221243.
MacCallum, R. C., & Mar, C. M. (1995). Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin, 118, 405421.
McArdle, J. J. (1994). Structural factor analysis experiments with incomplete data. Multivariate Behavioral Research, 29, 409454.
McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interaction and moderator effects. Psychological Bulletin, 114, 376390.
Ping, R. A. (1996a). Interaction and quadratic effect estimation: A twostep technique using structural equation analysis. Psychological Bulletin, 119, 166175.
Ping, R. A. (1996b). Latent variable regression: A technique for estimating interaction and quadratic coefficients. Multivariate Behavioral Research, 31, 95120.
Suinn, R. M. (1988). Mathematics Anxiety rating Scale (MARS). Fort Collins, CO: RMBSI.
Wood, P. K. (1994). The effect of unmeasured variables and their interactions on structural models. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis (pp. 109130). Thousand Oaks, CA: Sage Publications.