MetaAnalytic Structural Equation Modeling
 Mike W.L. CheungMike W.L. CheungDepartment of Psychology, National University of Singapore
Summary
Metaanalysis and structural equation modeling (SEM) are two popular statistical models in the social, behavioral, and management sciences. Metaanalysis summarizes research findings to provide an estimate of the average effect and its heterogeneity. When there is moderate to high heterogeneity, moderators such as study characteristics may be used to explain the heterogeneity in the data. On the other hand, SEM includes several special cases, including the general linear model, path model, and confirmatory factor analytic model. SEM allows researchers to test hypothetical models with empirical data. Metaanalytic structural equation modeling (MASEM) is a statistical approach combining the advantages of both metaanalysis and SEM for fitting structural equation models on a pool of correlation matrices. There are usually two stages in the analyses. In the first stage of analysis, a pool of correlation matrices is combined to form an average correlation matrix. In the second stage of analysis, proposed structural equation models are tested against the average correlation matrix. MASEM enables researchers to synthesize researching findings using SEM as the research tool in primary studies. There are several popular approaches to conduct MASEM, including the univariater, generalized least squares, twostage SEM (TSSEM), and onestage MASEM (OSMASEM). MASEM helps to answer the following key research questions: (a) Are the correlation matrices homogeneous? (b) Do the proposed models fit the data? (c) Are there moderators that can be used to explain the heterogeneity of the correlation matrices? The MASEM framework has also been expanded to analyze large datasets or big data with or without the raw data.
Introduction
It is of methodological and practical importance to understand how seemingly unrelated statistical methods are connected. Consider the development of the dominant modeling technique called structural equation modeling (SEM) as an example (e.g., Bentler, 1986; Bollen, 1989; Jöreskog, 1970). SEM provides a flexible modeling framework for testing complicated models involving latent and observed variables. It integrates ideas of latent variables in psychology, path models in sociology, and structural models in economics. The general linear model, path analysis, and confirmatory factor analytic model are some special cases of SEM. It has been found that some specialized models such as item response theory (e.g., GlocknerRist & Hoijtink, 2003; Takane & Deleeuw, 1987), categorical data analysis (Muthén, 1984), and multilevel models (e.g., Bauer, 2003; Curran, 2003) can be integrated into the SEM framework. Analysis of missing data and robust test statistics on nonnormal data are also integrated into SEM. SEM provides a unified framework for social, behavioral, and management scientists to test their models with empirical data.
Metaanalysis, a term coined by Gene Glass, is “the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings” (Glass, 1976, p. 3). After Glass’s introduction to the scientific communities, metaanalysis has become the de facto standard in synthesizing research findings in many disciplines, including medicine, psychology, and management sciences. Many of the highly cited papers are metaanalyses. Metaanalysis and SEM are generally treated as two unrelated topics in the literature (see Cheung, 2008). They have their own journals—Structural Equation Modeling: A Multidisciplinary Journal and Research Synthesis Methods. Researchers with knowledge in one area may not be aware of the benefits, techniques, and issues surrounding knowledge of the other area. The separation of these two techniques hinders the development of methods utilizing the benefits of both techniques.
Metaanalytic structural equation modeling (MASEM) is a new development to combine metaanalysis with SEM. It has also been known as, for example, metaanalytic path analysis (Colquitt, LePine, & Noe, 2000), metaanalysis of factor analysis (Becker, 1996), metaanalytical structural equations analysis (Hom, CaranikasWalker, Prussia, & Griffeth, 1992), path analysis of metaanalytically derived correlation matrices (Eby, Freeman, Rush, & Lance, 1999), SEM of a metaanalytic correlation matrix (Conway, 1999), path analysis based on metaanalytic findings (Tett & Meyer, 1993), and modelbased metaanalysis (Becker, 2009) in the literature. Here, the generic term MASEM is used to represent this class of techniques.
Phenomena in the social, behavioral, and management sciences are complicated, involving different constructs and sources of influences. To study these phenomena, researchers have to simplify them into models involving a few key theoretical constructs. SEM is one popular modeling framework to test the empirical relationships among the constructs. Besides representing structural equation models as mathematical equations, research questions can also be formulated as graphical models. Figure 1 displays a twofactor confirmatory factor analytic model. One latent variable (${f}_{1}$) is measured by three indicators (${x}_{1}$ to ${x}_{3}$), whereas the other latent variable (${f}_{2}$) is measured by another three indicators (${x}_{4}$ to ${x}_{6}$). Observed and latent variables are represented by squares (or rectangles) and circles (or ellipses), respectively. Oneway arrows represent the direct impact or regression coefficient, whereas twoway arrows show the covariances among the variables. When a twoway arrow applies to the same variable, it represents either the variance or its error variance. Using this set of notations, researchers may translate many complicated mathematical to graphical models for ease of understanding.
After collecting data, researchers may test their proposed model by comparing the model against the data. If the proposed models are consistent with the data, they can be tentatively used to describe the processes. On the other hand, if the proposed models are not consistent with the data, it suggests that the proposed models may not be good candidates to explain the underlying processes. Because the proposed models are only approximations of the complicated phenomena, researchers rarely believe that the proposed models are correct. Instead of relying on the statistical tests, many researchers prefer to use goodnessoffit indices such as the root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and nonnormed fit index (NNFI, also known as the TuckerLewis index), to assess how good the proposed models are.
Although SEM is a powerful tool in testing hypothesized models, it is recognized that findings in a single study rarely provide enough evidence on a topic of interest. There are several reasons. First, researchers may propose different theoretical models supported by their own data, and it is difficult to compare and synthesize them systematically. It means that it may not be easy to tell which model has better support empirically. Second, many researchers are reluctant to consider alternative models (e.g., MacCallum & Austin, 2000). Researchers may stop considering other theoretically plausible models once the data do not reject the proposed models, even though there may be better models to explain the data. Hence, conducting more empirical research does not necessarily decrease the uncertainty surrounding a particular topic if the findings from that research are inconsistent (National Research Council, 1992). More importantly, it has recently been argued that many published studies are not replicable (e.g., Francis, 2012; Open Science Collaboration, 2015). Researchers need to synthesize a pool of studies in order to advance science on a particular topic. MASEM provides a tool to address these issues by testing various theoretical models on a pool of correlation matrices.
Ideas for combining metaanalysis with SEM appeared in the literature in the late 1980s. For instance, using a total sample size of 1,474 participants from four studies, Schmidt, Hunter, and Outerbridge (1986) compared several path models of job experience on job knowledge, performance capability as measured by job sample tests, and supervisory ratings of job performance. Premack and Hunter (1988) studied models of six variables on the process of individual unionization decisions and tested their models against 14 studies with a total of 53,768 participants. More recent applications are the association between task conflict and group performance (de Wit, Greer, & Jehn, 2012), the relationship between competition and performance (Murayama & Elliot, 2012), and the effect of psychological ownership on attachment in the workspace (Zhang, Liu, Zhang, Xu, & Cheung, 2020). Several methodological papers (Bergh et al., 2016; Cheung & Hong, 2017; Landis, 2013; Sheng, Kong, Cortina, & Hou, 2016) and a special issue (Cheung & Hafdahl, 2016) have been published to summarize methodological issues in MASEM. Cheung (2015a) provides a booklength treatment on the statistical details on MASEM, whereas Jak (2015) gives a tutorial focusing on the twostage SEM (TSSEM) approach, which will be introduced later.
Approaches to Conducting MASEM
MASEM usually involves two stages of analysis. In the first stage, the correlation matrices are metaanalytically combined to form an average correlation matrix. This stage of analysis helps to address the potential missing data (correlation matrices) in the primary studies. In the second stage of analysis, the average correlation matrix is used to fit structural equation models. Exact and approximate fit indices may be used to evaluate the appropriateness of the proposed models. It seems straightforward to conduct MASEM—conducting a metaanalysis and then fitting a structural equation model on the average correlation matrix. However, many researchers ignore the fact that metaanalysis and SEM are based on different terminologies and statistical assumptions. Blindly combining them may lead to incorrect statistical influences (Cheung, 2019). Next, some of these approaches to conducting MASEM are reviewed.
There are two models in metaanalyses—fixed and randomeffects models. A fixedeffects model, also known as a commoneffects model, assumes that the sample effect sizes are functions of the true population effect sizes and the sampling errors. On the other hand, a randomeffects model hypothesizes that each study has its own studyspecific population effect sizes. Furthermore, it is assumed that the studyspecific population effect sizes follow a multivariate distribution with the average population effects and heterogeneity variances and covariances. This article mainly focuses on the randomeffects model as the fixedeffects model is a special case of the randomeffects model when the heterogeneity variances and covariances are zero.
Before introducing the specific approaches, the article outlines some key procedures and decisions in conducting MASEM summarized by Jak and Cheung (2020) (see Cheung [2015a], and Viswesvaran & Ones [1995] for more details, and also Cooper [2010] for some general issues in metaanalysis):
Identify key research questions, constructs, measurements, and structural equation models: Before conducting the MASEM, researchers have to formulate research questions and identify all the relevant key constructs, measurement models, and structural equation models. If the proposed models are too small with too few variables, it may limit the usefulness of the findings. On the other hand, there may be lots of incomplete data if the proposed models are too large. In the worstcase scenario, there may be no study or just a few studies in some of the elements in the correlation matrix. If this happens, the proposed models have to be revised or trimmed down to remove the constructs without data. Researchers may have to find a balance in formulating the models. Moreover, it is also advisable to formulate several theoretically meaningful models based on the literature for future testing. This makes the analyses more theoretically driven.
Formulate clear inclusion and exclusion criteria. This step is essential in all metaanalyses, including MASEM because it provides theoretical justifications on whether the selected studies can be meaningfully combined. For example, researchers may need to answer whether it makes sense to combine studies from different populations, such as student samples versus working samples. The inclusion and exclusion criteria will affect the generalizability of the findings. If the data are based on student samples, it cannot be generalized to the work environment with adults.
Identify and extract the relevant data, including correlation matrices, sample sizes, and study characteristics (moderators). The primary inputs in a metaanalysis are the effect sizes and their sampling variances. In the context of MASEM, the correlation matrices and their sample sizes are needed. There are likely incomplete data because the primary studies are conducted by different researchers independently. Different approaches may handle incomplete data differently. Moreover, researchers should attempt to retrieve unpublished data, such as dissertations, conference presentations, and technical reports, to minimize publication bias’s influence. One common test is to treat the published versus unpublished data as a moderator. Whether the findings are influenced by the publication type can then be tested.
Choose an appropriate approach to combine the correlation matrices and fit the structural equation models. There are several approaches to conduct MASEM. Generally speaking, they can be classified under the univariate approach and multivariate approach. The univariate approach, for instance, the univariater (Viswesvaran & Ones, 1995), treats the elements of the correlation matrices as if they were independent in combining the correlation matrices. The average correlation matrix is then used to fit structural equation models as if it was an observed covariance matrix. On the other hand, the multivariate approach, for example, the generalized least squares (GLS; Becker, 1992, 1995), the TSSEM (Cheung, 2014a; Cheung & Chan, 2005b, 2009) and the onestage MASEM (OSMASEM; Jak & Cheung, 2020), takes the dependence of the correlations in the correlation matrix into account when metaanalyzing the correlation matrices. More importantly, the multivariate approach also considers the estimation uncertainty in fitting the structural equation models. Readers may refer to Cheung (2015b) and Cheung and Hafdahl (2016) for details on the differences between these approaches.
Univariate Approach
The most popular method is the univariater approach proposed by Viswesvaran and Ones (1995). The first step of the univariater is to conduct several univariate metaanalyses (e.g., Hunter & Schmidt, 1990) on the correlation coefficients. Suppose that the structural equation model involves five variables, there is a total of $5\times 4/2$, or $10$, correlation coefficients of the bivariate relationships in the model. Researchers conduct $10$ univariate metaanalyses on these bivariate relationships. After the first stage of analysis, there is a $5\times 5$ average correlation matrix. This correlation matrix is then used as if it was an observed covariance matrix in fitting structural equation models using standard SEM packages such as Mplus or Amos in stagetwo analysis. The main advantage of this approach is its ease of application—researchers just need to conduct several metaanalyses and SEM. It was found (Rosopa & Kim, 2017; Sheng et al., 2016) that the univariater approach dominates the literature of human resource management, management, and organizational studies.
Although the univariate approach is easy to apply, it suffers from several statistical difficulties (Cheung, 2015b; Cheung & Chan, 2005b). The first step of metaanalysis in combining correlation matrices is fine if researchers are only interested in testing the average correlation coefficients. However, there are several statistical issues in fitting structural equation models. Because of the incomplete data in the primary studies, pairwise aggregation (or deletion) is usually used to metaanalyze the correlation coefficients in the univariate$\text{r}$ approach. Elements of the average correlation matrix are likely based on different numbers of studies, whereas a single sample size is required in SEM. Several adhoc procedures have been used to calculate an “average” sample size in SEM. These include, for example, the arithmetic mean (Premack & Hunter, 1988), the harmonic mean (Conway, 1999), the median (Brown & Peterson, 1993), or the total (Tett & Meyer, 1993) of the sample sizes based on the synthesized correlation coefficients. As the test statistics, some goodnessoffit indices, and the standard errors of parameter estimates depend on the sample size used in SEM (Bollen, 1989, 1990). Using different sample sizes in the analysis may lead to different results and conclusions. No matter which sample size is used, it is hard to obtain the correct test statistics and standard errors.
The average correlation matrix elements are likely combined using a pairwise aggregation because of missing data. It is found that such matrices may be nonpositive definite, meaning that the observed correlations are either impossible or inconsistent. One example of nonpositive definite matrix is that the correlation is larger than 1. If this happens, these correlation matrices cannot be used to fit structural equation models. Even if the average correlation matrix is positive definite, its statistical properties are still questionable in SEM because different elements of the pooled correlation matrix are probably based on different samples (Marsh, 1998; Wothke, 1993).
The third problem is ignoring the sampling uncertainty across studies. After obtaining an average correlation matrix, the average correlation coefficients’ precision is not incorporated in fitting the structural equation models. Let us consider a simple example with a $3\times 3$ average correlation matrix with three correlation coefficients. For ease of illustration, suppose that the average correlation coefficients of these three correlation coefficients are all $0.5$ but with different precision (sampling variances = $0.1$, $0.2$, and $0.3$). When this average correlation matrix is used in fitting the structural equation model, the SEM packages, for instance, Mplus or Amos, do not know that some correlation coefficients are more precise than others. The lack of information on the precision will lead to biased test statistics and standard errors in SEM.
The last issue is analyzing a correlation matrix as if it was a covariance matrix. It is wellknown in the SEM literature that it is advisable not to analyze the correlation matrix as if it was a covariance matrix. It is because the diagonals of a correlation matrix are always one, whereas the diagonals of a covariance matrix can be any nonnegative values. The diagonals of a covariance matrix carry information, whereas the diagonals of a correlation matrix do not. Specifically, the chisquare statistics, goodnessoffit indices, and the standard errors of parameter estimates may be incorrect if a correlation matrix is analyzed as if it was a covariance matrix (Cudeck, 1989; Jöreskog & Sörbom, 1996). The impact of analyzing a correlation matrix is more noticeable when there are constraints imposed on the models. It should be noted that correlation matrices can be correctly analyzed in SEM (Bentler & Savalei, 2010). The problem here is that these techniques cannot be implemented in the univariater approach. Simulation studies (Cheung & Chan, 2005b; Jak & Cheung, 2020) have shown that the test statistics of the univariater approach are overestimated while the standard errors are underestimated.
Generalized Least Squares
Becker (1992, 1995) proposed a GLS approach to conduct MASEM. Essentially, it is also a twostep approach. In the first step of the analysis, the correlation matrices are metaanalyzed via a multivariate approach by taking the sampling variance–covariance matrices of the correlations into account. Suppose that ${R}_{i}$ is a sample correlation matrix in the ith study. It can be transformed into a vector of correlation coefficients via the $\text{vechs}()$ operator, which takes the elements of the lower triangle without the diagonals by a columnorder major, that is, ${r}_{i}=\text{vechs}\left({R}_{i}\right)$. For example, if ${R}_{i}$ is a $5\times 5$ correlation matrix, ${r}_{i}$ will be a column vector with $10$ correlation coefficients. The model for a multivariate metaanalysis is
where $\rho $ is the vector of average correlations, ${u}_{i}$ is the vector of random effects, and ${e}_{i}$ is the vector of error variances. The heterogeneity matrix ${T}_{\rho}{}^{2}=\text{Var}\left({u}_{i}\right)$ indicates how heterogeneous the population correlation coefficients are, whereas the variance–covariance matrix of ${V}_{i}=\text{Var}\left({e}_{i}\right)$ is assumed known and calculated before the analysis (Olkin & Siotani, 1976). The variance–covariance matrices of the sampling errors ${V}_{i}$ are getting smaller and smaller when the sampling size is getting bigger and bigger. In contrast, the heterogeneity matrix ${T}_{\text{\rho}}{}^{2}$ represents the true variability of the studyspecific correlation coefficients in the population level.
After the estimation, there is an average correlation vector $\widehat{\rho}$, its asymptotic sampling covariance matrix ${\widehat{V}}_{p}$, which indicates the precision of the estimates, and a heterogeneity matrix ${\widehat{T}}_{\rho}^{2}$. Becker used some matrix calculations to obtain the parameter estimates of regression models and some path models. She also showed how the regression coefficients’ standard errors could be obtained via the multivariate delta method. The GLS approach solves all the problems mentioned in the univariate approach. However, the main limitation is that it is not easy to apply it as matrix calculations are required. Moreover, it is also limited to fitting regression models and some path models without any latent variables.
TwoStage Structural Equation Modeling
Building on Becker’s idea, Cheung (2014a; Cheung & Chan, 2005b, 2009) proposed a TSSEM approach to conduct MASEM. The statistical model is nearly the same as in Becker (1992). The critical difference is that the TSSEM approach uses the SEM framework in both stages of analysis. Therefore, advanced features, including handling missing data with fullinformation maximum likelihood (FIML) estimation, likelihoodbased confidence intervals, multiplegroup analysis, and robust standard errors, can be applied in MASEM.
In the first stage of analysis, the model is the same as that in equation (1). Instead of using the GLS as in Becker’s approach, the TSSEM approach uses FIML estimation. FIML is unbiased and efficient in handling missing data (correlation coefficients in MASEM) when the missingness is either missing completely at random (MCAR) or missing at random (MAR; Enders, 2010). Theoretical and computer simulations support that it works better than the listwise deletion or pairwise deletion in the univariate approach. When the missing data are missing not at random (MNAR), for example, in the presence of publication bias, no approach is unbiased (Furlow & Beretvas, 2005). However, the bias of the FIML is still less than that of the pairwise deletion (Jamshidian & Bentler, 1999). Thus, FIML is usually the preferred choice in handling missing data in data analysis (Enders, 2010).
In stagetwo analysis, researchers propose a structural equation model, which can be a regression model, path model, confirmatory factor analytic model, or a full structural equation model, on the correlation matrix. The notation for the proposed correlation structure is $\rho =\rho (\theta )$, meaning that the population correlation vector $\rho $ is a function of some unknown parameters $\theta $. Because the estimates from stageone stage one analysis are being used as inputs in stagetwo analysis, the “hat” in $\widehat{\rho}$ and ${\widehat{V}}_{\rho}$ must be gotten rid of to avoid confusion in the notations. Here, $r=\widehat{\rho}$ and ${V}_{r}={\widehat{V}}_{\rho}$ are used as the data inputs for the stage two analysis. Now, the structural equation model is fit by minimizing the fit function $F(\widehat{\theta})$ with the weighted least squares (WLS) estimation method (Browne, 1984),
In words, the authors of this article try to find the parameter estimates that minimize the discrepancy between their data $r$ and the proposed model $\rho (\theta )$ by taking the precision matrix ${V}_{r}$ into account. Although the WLS estimation was developed in the SEM literature, it follows the same principle of the estimation method in the metaanalysis (Cheung, 2010)—the average effect sizes are estimated by weighing the precision of the effect sizes. Standard test statistics and various goodnessoffit indices may be used to test the exact and approximate fit of the proposed model. The parameter estimates divided by its standard error approximately follows a standard normal distribution.
In comparison to the univariate$\text{r}$ approach, the authors of this article find that there are several advantages to the TSSEM approach. They have mentioned that there are four technical issues in the univariater approach, namely, (a) adhoc choice in choosing a sample size in SEM; (b) potential nonpositive definite in the average correlation matrix; (c) ignoring of the sampling variance in SEM; and (d) analysis of the correlation matrix as if it was a covariance matrix in SEM. The second issue is resolved by the use of FIML estimation to handle the missing data. Research has found that the listwise aggregation is more likely to create nonpositive definite matrices than other approaches such as FIML (Wothke, 1993). Regarding the other issues, the use of the WLS method addresses all of them. As can be seen in equation (2), the fit function does not involve any sample size. Therefore, the parameter estimates, standard errors, and the model fit statistics are identical regardless of the sample size used in the analysis (Cheung & Chan, 2005b). The sample size used in SEM, on the other hand, still slightly influences some goodnessoffit indices that explicitly included the sample size in the calculations. In both the TSSEM and OSMASEM approaches, the total sample size is used for convenience.
When the degree of heterogeneity is enormous, an average effect may not be very useful. Researchers may want to identify moderators that may help to explain the heterogeneity. This step is similar to identifying the moderating effect in multiple regression. If the moderators are categorical, for instance, types of samples and regions, a subgroup analysis may be used (Cheung & Chan, 2005a; Jak & Cheung, 2018). In the stageone analysis, the correlation matrices are metaanalyzed separately according to their groups. In the stagetwo analysis, the same structural equation model is fitted on the averaged correlation matrices via a multiplegroup SEM. Some of the parameters can be constrained equally to test research hypotheses related to the moderators. If the subgroup analysis fits much better than one analysis, it suggests that the categorical moderator may be used to explain the heterogeneity.
OneStage MASEM
The TSSEM works pretty well when there is no moderator or with categorical moderators. When the moderators are continuous, researchers may have to categorize the continuous moderator into several categories. However, categorizing continuous moderators may lead to several issues, such as loss of information about individual differences, loss of power, the occurrence of spurious significant main effects or interactions, and risks of overlooking nonlinear effects (MacCallum, Zhang, Preacher, & Rucker, 2002). Jak and Cheung (2020) proposed a novel approach to address these issues. The model of OSMASEM is very similar to that in the GLS and TSSEM approaches by replacing $\rho $ with $\rho (\theta )$, that is,
In plain language, the model means that the sample correlation matrix ${r}_{i}$ is a function of the proposed correlation structure $\rho ({\theta}_{i})$ after taking the heterogeneity variance ${T}_{\rho}{}^{2}=\text{Var}\left({u}_{i}\right)$ and known sampling error ${V}_{i}=\text{Var}\left({e}_{i}\right)$ into account. Although the changes look minor, there are two huge differences. First, the proposed correlation structure $\rho ({\theta}_{i})$ is included in equation (3). Instead of two stages of analyses, as in the TSSEM approach, there is only one stage of estimation with the FIML estimation method. In other words, the proposed structural equation models are fitted directly on the correlation matrices without estimating an average correlation matrix. Second, there is a subscript $i$ in $\rho ({\theta}_{i})$, meaning that the correlation structure may vary according to the moderators in the ith study. Categorical and continuous moderators can be added to the proposed structural equation models by the use of definition variables. Definition variables allow researchers to set the structural parameters, for example, path coefficients as functions of observed data, for example, moderators. In other words, moderators can be used to predict the structural parameters in the models.
After fitting the OSMASEM, researchers may test the model fit by using the exact and approximate fit indices. When there are moderators, researchers may test whether the moderators are statistically better than the models without the moderators. Moreover, explained variances on the correlation coefficients might be calculated. It helps to study how good the moderators are in explaining the heterogeneity of the correlation coefficients. Both the TSSEM and OSMASEM approaches are implemented in an R package called metaSEM (Cheung, 2015b), which is freely available in the opensource R environment (R Development Core Team, 2020).
Other Approaches
There are also a few other alternatives to conduct MASEM. Cheung and Cheung (2016) differentiated two different conceptualizations of MASEM. The first one is the correlationbased MASEM. The idea is that the correlation coefficient is used as effect sizes in the metaanalyses. The unit of heterogeneity is the correlation coefficient. The univariate$\text{r}$, GLS, TSSEM, and OSMASEM fall into this category. The second type is the parameterbased MASEM. Researchers first fit a structural equation model on each study to get the parameter estimates. The parameter estimates or functions of parameters such as regression coefficient, factor loading, indirect effect, ${R}^{2}$, and reliability estimate are treated as effect sizes in the subsequent metaanalyses.
There are pros and cons of the correlationbased and parameterbased MASEM. The primary strength of the correlationbased MASEM is its ability to handle incomplete correlation coefficients in the first stage of analysis. For example, the TSSEM and OSMASEM use FIML to handle incomplete data effectively. The other strength of the correlationbased MASEM is that competing structural models can be tested and compared easily in the stagetwo analysis. This step is crucial in MASEM as it allows researchers to test and compare several theoretically completing models. The major limitation of the correlationbased MASEM is that its heterogeneity is defined in the correlation coefficients rather than the structural parameters, such as regression coefficients and factor loadings. However, some researchers may have research questions targeting the heterogeneity of the structural parameters rather than the correlation coefficients.
On the other hand, parameterbased MASEM has some strengths. First and most important, research questions may involve functions of the structural parameters. For example, researchers conducting a metaanalysis on a mediation model may be more interested in how the direct effect and the indirect effect vary across studies. Second, parameterbased MASEM quantifies the heterogeneity of the parameter estimates across studies. Moderators may be used to explain the variability of these effect sizes. Despite its advantages, several key limitations are making the parameterbased MASEM less useful. First and essential, parameterbased MASEM does not work well when there are missing effect sizes. It is because a model, such as multiple regression, has to be applied in each primary study. When some correlation coefficients are missing, the multiple regression cannot be fitted. Another limitation is that parameterbased MASEM may not be appropriate for overidentified SEM models. If the proposed model does not fit the sample correlation matrix well, the validity of the parameter estimates (regression coefficients or factor loadings) and the subsequent analyses are questionable. Moreover, it is difficult to compare competing models with parameterbased MASEM as there is no overall model fitness in the models. By weighing the pros and cons, the correlationbased MASEM is more prevalent in MASEM.
Yu, Downes, Carter, and O’Boyle (2016) suggested an alternative approach called the fullinformation MASEM (FIMASEM) to address the heterogeneity issues in MASEM. In the first stage of analysis, either the univariater or TSSEM is used to metaanalyze the correlation matrices. In the second stage of analysis, the average correlation matrix and its heterogeneity are treated as population values to generate correlation matrices with a parametric bootstrap. The simulated correlation matrices are used to fit the proposed SEM. The parameter estimates and various goodnessoffit indices of the bootstrapped correlation matrices are obtained. Cheung (2018) identified a couple of human errors and statistical issues in Yu et al. (2016). By running a new simulation study using the same conditions in Yu et al. (2016), Cheung (2018) found that the heterogeneity of the parameter estimates (e.g., path coefficients) of the FIMASEM is relatively unbiased. However, the goodnessoffit indices of the FIMASEM are seriously misleading (see Yu, Downes, Carter, & O’Boyle, 2018, for their replies).
Ke, Zhang, and Tong (2019) proposed a Bayesian approach to conduct MASEM. The key idea is to treat the parameters ${\theta}_{i}$ as random. The model is
where ${\overline{\theta}}_{i}$ is the average of the parameters, ${T}_{\theta}{}^{2}=\text{Var}\left({\theta}_{i}\right)$ is the variance–covariance matrix (heterogeneity) of the parameters, and ${V}_{i}=\text{Var}\left({e}_{i}\right)$ is the known variance–covariance matrix of the sampling errors. However, one practical limitation is that it is not easy to implement the model as it involves lots of programming. As it is quite new, more studies are required to investigate the pros and cons of the Bayesian approach.
An Illustration
In this section, a data set is used by Mathieu, Kukenberger, D’Innocenzo, and Reilly (2015) to demonstrate how to conduct the TSSEM and OSMASEM approaches in order to answer empirical research questions in management science. It should be noted that this article mainly focuses on how to illustrate the techniques. The analyses are slightly different from those reported in Mathieu et al. (2015), which used the univariate$\text{r}$ approach as the statistical method. Moreover, readers should refer to Mathieu et al. (2015) for substantive interpretations. These authors were interested in the reciprocity in the cohesion–performance ($\text{C}\text{P}$) relationship. Fourbyfour correlation matrices between twotime points of cohesion and performance of 15 studies were extracted. Moreover, two additional longitudinal studies on the same topic were conducted by the authors. As there is no missing data, there is a total of 17 fourbyfour correlation matrices (see Mathieu et al., 2015, Table 1) with a total sample size of 737. Their Table 1 also provides information on the type of samples. These samples can be classified as either student or nonstudent.
Figure 2 shows the proposed path model (please ignore the estimates for the time being). The primary research questions are as follows: (a) What are the average correlation matrix and its heterogeneity variances? (b) What are the estimated parameters in fitting the model in Figure 2? (c) Are the effects between cohesion and performance symmetric? That is, is the effect $\text{C}1\to \text{C}2$ the same as $\text{P}1\to \text{P}2$, and is $\text{C}1\to \text{P}2$ the same as $\text{P}1\to \text{C}2$? (d) Does sample type (students versus nonstudents) moderate the path coefficients? The TSSEM approach is used to answer the first three research questions, whereas the OSMASEM approach is used to address the last research question. All the analyses were conducted in the metaSEM package (Cheung, 2015b) in the R statistical environment (R Development Core Team, 2020). The complete R code is available on mikewlcheung/codein articles.
Note. C1 and C2 represent Cohesion measured at times 1 and 2. P1 and P2 represent Performance measured at times 1 and 2.
Table 1 shows the average correlation matrix (lower triangle in the matrix) and the estimated heterogeneity variances (upper triangle in the matrix). The average correlations between cohesion and performance vary from $0.25$ to $0.50$, which are small to moderate. The estimated heterogeneity variances vary from $0.00$ to $0.05$, which are typical in applied psychology (Bosco, Aguinis, Singh, Field, & Pierce, 2015). In stagetwo analysis, the proposed model is fitted on the average correlation matrix. Because it is a justidentified model, there are no fit indices. The parameter estimates are shown in Figure 2. All of them are statistically significant at $\text{\alpha}=0.05$. For the third research question, the hypothesis of equal regression coefficients of ($\text{C}1\to \text{C}2$ and $\text{P}1\to \text{P}2$) are ($\text{C}1\to \text{P}2$ and $\text{P}1\to \text{C}2$) is tested. The likelihood ratio statistic is not statistically significant, ${\chi}^{2}(df=2)=0.34,p=0.84$. It suggests that the effects are symmetric between cohesion and performance. The estimated coefficient for $\text{C}1\to \text{C}2$ and $\text{P}1\to \text{P}2$ is $0.50$, whereas the estimated coefficient for $\text{C}1\to \text{P}2$ and $\text{P}1\to \text{C}2$ is $0.13$.
Table 1. Parameter Estimates of StageOne Analysis With the TSSEM Approach
C1 
P1 
C2 
P2 


C1 
1.00 
0.05 
0.05 
0.00 
P1 
0.25 
1.00 
0.00 
0.05 
C2 
0.55 
0.26 
1.00 
0.03 
P2 
0.25 
0.50 
0.37 
1.00 
Note. C1 and P1 are Cohesion and Performance measured at time 1. C2 and P2 are Cohesion and Performance measured at time 2. Elements in the lower triangle are the average correlation coefficients. Elements in the upper triangle are the estimated heterogeneity variances.
The OSMASEM approach was used to answer the fourth research question on whether the sample type (students versus nonstudents) moderates the path coefficients. The models were compared with and without the sample type; the likelihood ratio test is statistically significant with ${\chi}^{2}\left(df=4\right)=12.18,p=0.02$. It suggests that some of the path coefficients may be different in the sample type. Further analyses using the $z$ statistic show that the path coefficient of $P1\to C2$ are different in the two groups ($0.35$ for nonstudent samples and $0.08$ for student samples) with $\text{p}2\text{c}\_1=0.28,\phantom{\rule{0.2em}{0ex}}\text{z}=2.60,p=0.0094$ in the R output, whereas the other path coefficients are not statistically different in these two samples.
Further Applications
MASEM is primarily used to summarize correlation matrices to test theoretical models. The techniques, however, are not limited to analyze “summary statistics.” Researchers may apply MASEM even if raw data are available. There are at least two such applications. First, raw data may involve private or confidential information, which makes it challenging to share with other researchers. Second, the datasets are too big for typical workstations. It may take a long time to process. Given this approach only relies on the summary statistics (correlation matrices), researchers in management sciences are more likely to test models based on the correlation matrices.
Based on the popular splitapplycombine approach (Wickham, 2011) in R, Cheung and Jak (2016) proposed a split/analyze/metaanalyze (SAM) approach to analyze large datasets or data with only summary statistics such as correlation matrices. There are three steps in conducting the SAM analysis. In the first step, the data were split into many independent “studies.” There are two methods to split the data, depending on the research questions and the nature of the data. If the data are already structured in a hierarchical way, for instance, geographic locations or years, the data can be split based on these characteristics, which is termed “stratified split” in their paper. If there are no nested structures, an arbitrary (random) split on the data can be applied, which is termed “random split” in their paper. How the data are split has implications for how the results are to be combined in the final step.
In the second step, each “study” is independently analyzed with the appropriate statistical models, such as multiple regression, path model, structural equation model, and reliability analysis. As the large datasets have been broken down into smaller pieces, the stagetwo analysis is more management. In the final step, the parameter estimates in each “study” are treated as effect sizes and combined by a multivariate metaanalysis. A randomeffects model is used when the data are split by the stratified split; otherwise, a fixedeffects model is used. Sample characteristics may be used as moderators in the multivariate metaanalysis. Cheung and Jak (2016) shown how the SAM approach could be used to analyze data in the social and behavioral sciences. It is of interest to see if this approach can be applied to answer research questions in management sciences.
Another line of research in integrating metaanalysis and SEM is known as the SEMbased metaanalysis (Cheung, 2008). The SEMbased metaanalysis utilizes the SEM framework to conceptualize and conduct metaanalyses. In words, standard metaanalytic models such as those outlined in Borenstein, Hedges, Higgins, and Rothstein (2009) and Hedges and Olkin (1985) can be formulated as structural equation models and analyzed with SEM packages. The SEMbased metaanalysis has been extended to multivariate metaanalysis (Cheung, 2013), a threelevel metaanalysis (Cheung, 2014b), and network metaanalysis (Tu & Wu, 2017). There are several advantages of integrating metaanalysis into the SEM framework. First, many existing SEM functions in handling missing data and nonnormal data with robust statistics can be directly applied to metaanalytic data. Second, researchers may build structural equation models on the effect sizes with study characteristics as covariates (e.g., Shadish, 1992; Shadish & Sweeney, 1991). For example, Cheung (2015a) demonstrated how effect sizes could be used as mediators and moderators in predicting other effect sizes under the SEMbased metaanalysis.
Conclusion
MASEM is a useful technique to synthesize correlation matrices to fit and compare structural equation models. It helps to accumulate research findings to test theoretical models. MASEM has also been extended to analyze big datasets. Researchers are still actively developing new methods to improve the methodology in MASEM. To summarize, the combination of metaanalysis with SEM opens up many new research avenues.
Link to Digital Materials
 The complete R code for the illustration is available at mikewlcheung/codein articles.
Further Reading
 Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17(4), 341–362.
 Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models.” Journal of Educational and Behavioral Statistics, 20(1), 100–102.
 Cheung, M. W.L. (2014). Fixed and randomeffects metaanalytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), 29–40.
 Cheung, M. W.L. (2019). Some reflections on combining metaanalysis and structural equation modeling. Research Synthesis Methods, 10(1), 15–22.
 Cheung, M. W.L., & Chan, W. (2005). Metaanalytic structural equation modeling: A twostage approach. Psychological Methods, 10(1), 40–64.
 Cheung, M. W.L., & Hafdahl, A. R. (2016). Special issue on metaanalytic structural equation modeling: Introduction from the guest editors. Research Synthesis Methods, 7(2), 112–120.
 Cheung, M. W.L., & Hong, R. Y. (2017). Applications of metaanalytic structural equation modelling in health psychology: Examples, issues, and recommendations. Health Psychology Review, 11(3), 265–279.
 Jak, S., & Cheung, M. W.L. (2020). Metaanalytic structural equation modeling with moderating effects on SEM parameters. Psychological Methods, 25(4), 430–455.
References
 Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28(2), 135–167.
 Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17(4), 341–362.
 Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models.” Journal of Educational and Behavioral Statistics, 20(1), 100–102.
 Becker, B. J. (2009). Modelbased metaanalysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and metaanalysis (2nd ed., pp. 377–395). New York, NY: Russell Sage Foundation.
 Becker, G. (1996). The metaanalysis of factor analyses: An illustration based on the cumulation of correlation matrices. Psychological Methods, 1(4), 341–353.
 Bentler, P. M. (1986). Structural modeling and psychometrika: An historical perspective on growth and achievements. Psychometrika, 51(1), 35–51.
 Bentler, P. M., & Savalei, V. (2010). Analysis of correlation structures: Current status and open problems. In S. Kolenikov, D. Steinley, & L. Thombs (Eds.), Statistics in the social sciences (pp. 1–36). New York, NY: John Wiley & Sons.
 Bergh, D. D., Aguinis, H., Heavey, C., Ketchen, D. J., Boyd, B. K., Su, P., . . . Joo, H. (2016). Using metaanalytic structural equation modeling to advance strategic management research: Guidelines and an empirical illustration via the strategic leadershipperformance relationship. Strategic Management Journal, 37(3), 477–497.
 Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
 Bollen, K. A. (1990). Overall fit in covariance structure models: Two types of sample size effects. Psychological Bulletin, 107(2), 256–259.
 Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to metaanalysis. New York, NY: John Wiley & Sons.
 Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100(2), 431–449.
 Brown, S. P., & Peterson, R. A. (1993). Antecedents and consequences of salesperson job satisfaction: Metaanalysis and assessment of causal effects. Journal of Marketing Research, 30(1), 63–77.
 Browne, M. W. (1984). Asymptotically distributionfree methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37(1), 62–83.
 Cheung, M. W.L. (2008). A model for integrating fixed, random, and mixedeffects metaanalyses into structural equation modeling. Psychological Methods, 13(3), 182–202.
 Cheung, M. W.L. (2010). Fixedeffects metaanalyses as multiplegroup structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 17(3), 481–509.
 Cheung, M. W.L. (2013). Multivariate metaanalysis as structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 429–454.
 Cheung, M. W.L. (2014a). Fixed and randomeffects metaanalytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), 29–40.
 Cheung, M. W.L. (2014b). Modeling dependent effect sizes with threelevel metaanalyses: A structural equation modeling approach. Psychological Methods, 19(2), 211–229.
 Cheung, M. W.L. (2015a). Metaanalysis: A structural equation modeling approach. New York, NY: John Wiley & Sons.
 Cheung, M. W.L. (2015b). MetaSEM: An R package for metaanalysis using structural equation modeling. Frontiers in Psychology, 5, 1521.
 Cheung, M. W.L. (2018). Issues in solving the problem of effect size heterogeneity in metaanalytic structural equation modeling: A commentary and simulation study on Yu, Downes, Carter, and O’Boyle (2016). Journal of Applied Psychology, 103(7), 787–803.
 Cheung, M. W.L. (2019). Some reflections on combining metaanalysis and structural equation modeling. Research Synthesis Methods, 10(1), 15–22.
 Cheung, M. W.L., & Chan, W. (2005a). Classifying correlation matrices into relatively homogeneous subgroups: A cluster analytic approach. Educational and Psychological Measurement, 65(6), 954–979.
 Cheung, M. W.L., & Chan, W. (2005b). Metaanalytic structural equation modeling: A twostage approach. Psychological Methods, 10(1), 40–64.
 Cheung, M. W.L., & Chan, W. (2009). A twostage approach to synthesizing covariance matrices in metaanalytic structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 28–53.
 Cheung, M. W.L., & Cheung, S. F. (2016). Randomeffects models for metaanalytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods, 7(2), 140–155.
 Cheung, M. W.L., & Hafdahl, A. R. (2016). Special issue on metaanalytic structural equation modeling: Introduction from the guest editors. Research Synthesis Methods, 7(2), 112–120.
 Cheung, M. W.L., & Hong, R. Y. (2017). Applications of metaanalytic structural equation modelling in health psychology: Examples, issues, and recommendations. Health Psychology Review, 11(3), 265–279.
 Cheung, M. W.L., & Jak, S. (2016). Analyzing big data in psychology: A split/analyze/metaanalyze approach. Frontiers in Psychology, 7, 738.
 Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A metaanalytic path analysis of 20 years of research. Journal of Applied Psychology, 85(5), 678–707.
 Conway, J. M. (1999). Distinguishing contextual performance from task performance for managerial jobs. Journal of Applied Psychology, 84(1), 3–13.
 Cooper, H. M. (2010). Research synthesis and metaanalysis: A stepbystep approach (4th ed.). Thousand Oaks, CA: SAGE.
 Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105(2), 317–327.
 Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38(4), 529–569.
 de Wit, F. R. C., Greer, L. L., & Jehn, K. A. (2012). The paradox of intragroup conflict: A metaanalysis. Journal of Applied Psychology, 97(2), 360–390.
 Eby, L. T., Freeman, D. M., Rush, M. C., & Lance, C. E. (1999). Motivational bases of affective organizational commitment: A partial test of an integrative theoretical model. Journal of Occupational and Organizational Psychology, 72(4), 463–483.
 Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
 Francis, G. (2012). The psychology of replication and replication in psychology. Perspectives on Psychological Science, 7(6), 585–594.
 Furlow, C. F., & Beretvas, N. S. (2005). Metaanalytic methods of pooling correlation matrices for structural equation modeling under different patterns of missing data. Psychological Methods, 10(2), 227–254.
 Glass, G. V. (1976). Primary, secondary, and metaanalysis of research. Educational Researcher, 5(10), 3–8.
 GlocknerRist, A., & Hoijtink, H. (2003). The best of both worlds: Factor analysis of dichotomous data using item response theory and structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 10(4), 544–565.
 Hedges, L. V., & Olkin, I. (1985). Statistical methods for metaanalysis. Cambridge, MA: Academic Press.
 Hom, P. W., CaranikasWalker, F., Prussia, G. E., & Griffeth, R. W. (1992). A metaanalytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77(6), 890–909.
 Hunter, J. E., & Schmidt, F. L. (1990). Methods of metaanalysis: Correcting error and bias in research findings. Thousand Oaks, CA: SAGE.
 Jak, S. (2015). Metaanalytic structural equation modelling. New York, NY: Springer International.
 Jak, S., & Cheung, M. W.L. (2018). Testing moderator hypotheses in metaanalytic structural equation modeling using subgroup analysis. Behavior Research Methods, 50(4), 1359–1373.
 Jak, S., & Cheung, M. W.L. (2020). Metaanalytic structural equation modeling with moderating effects on SEM parameters. Psychological Methods, 25(4), 430–455.
 Jamshidian, M., & Bentler, P. M. (1999). ML estimation of mean and covariance structures with missing data using complete data routines. Journal of Educational and Behavioral Statistics, 24(1), 21–24.
 Jöreskog, K. G. (1970). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences (pp. 85–112). Princeton, NJ: Seminar Press.
 Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: A user’s reference guide. Lincolnwood, IL: Scientific Software International.
 Ke, Z., Zhang, Q., & Tong, X. (2019). Bayesian metaanalytic SEM: A onestage approach to modeling betweenstudies heterogeneity in structural parameters. Structural Equation Modeling: A Multidisciplinary Journal, 26(3), 348–370.
 Landis, R. S. (2013). Successfully combining metaanalysis and structural equation Modeling: Recommendations and strategies. Journal of Business and Psychology, 28(3), 251–261.
 MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51(1), 201–226.
 MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19–40.
 Marsh, H. W. (1998). Pairwise deletion for missing data in structural equation models: Nonpositive definite matrices, parameter estimates, goodness of fit, and adjusted sample sizes. Structural Equation Modeling: A Multidisciplinary Journal, 5(1), 22–36.
 Mathieu, J. E., Kukenberger, M. R., D’Innocenzo, L., & Reilly, G. (2015). Modeling reciprocal team cohesionperformance relationships, as impacted by shared leadership and members’ competence. Journal of Applied Psychology, 100(3), 713–734.
 Murayama, K., & Elliot, A. J. (2012). The competition–performance relation: A metaanalytic review and test of the opposing processes model of competition and performance. Psychological Bulletin, 138(6), 1035–1070.
 Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115–132.
 National Research Council. (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academies Press.
 Olkin, I., & Siotani, M. (1976). Asymptotic distribution of functions of a correlation matrix. In S. Ideka (Ed.), Essays in probability and statistics (pp. 235–251). Tokyo, Japan: Shinko Tsusho.
 Open Science Collaboration. (2015). Estimating the reproducibility of psychological. Science, 349(6251), aac4716.
 Premack, S. L., & Hunter, J. E. (1988). Individual unionization decisions. Psychological Bulletin, 103(2), 223–234.
 R Development Core Team. (2020). R: A language and environment for statistical computing.
 Rosopa, P. J., & Kim, B. (2017). Robustness of statistical inferences using linear models with metaanalytic correlation matrices. Human Resource Management Review, 27(1), 216–236.
 Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986). Impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology, 71(3), 432–439.
 Shadish, W. R. (1992). Do family and marital psychotherapies change what people do? A metaanalysis of behavioral outcomes. In T. D. Cook, H. M. Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light, … F. Mosteller (Eds.), Metaanalysis for explanation: A casebook (pp. 129–208). New York, NY: Russell Sage Foundation.
 Shadish, W. R., & Sweeney, R. B. (1991). Mediators and moderators in metaanalysis: There’s a reason we don’t let dodo birds tell us which psychotherapies should have prizes. Journal of Consulting and Clinical Psychology, 59(6), 883–893.
 Sheng, Z., Kong, W., Cortina, J. M., & Hou, S. (2016). Analyzing matrices of metaanalytic correlations: Current practices and recommendations. Research Synthesis Methods, 7(2), 187–208.
 Takane, Y., & Deleeuw, J. (1987). On the relationship between item response theory and factoranalysis of discretized variables. Psychometrika, 52(3), 393–408.
 Tett, R. P., & Meyer, J. P. (1993). Job satisfaction, organizational commitment, turnover intention, and turnover: Path analyses based on metaanalytic findings. Personnel Psychology, 46(2), 259–293.
 Tu, Y.K., & Wu, Y.C. (2017). Using structural equation modeling for network metaanalysis. BMC Medical Research Methodology, 17, 104.
 Viswesvaran, C., & Ones, D. S. (1995). Theory testing: Combining psychometric metaanalysis and structural equations modeling. Personnel Psychology, 48(4), 865–885.
 Wickham, H. (2011). The splitapplycombine strategy for data analysis. Journal of Statistical Software, 40(1), 1–29.
 Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 256–293). Thousand Oaks, CA: SAGE.
 Yu, J., Downes, P. E., Carter, K. M., & O’Boyle, E. H. (2016). The problem of effect size heterogeneity in metaanalytic structural equation modeling. Journal of Applied Psychology, 101(10), 1457–1473.
 Yu, J., Downes, P. E., Carter, K. M., & O’Boyle, E. (2018). The heterogeneity problem in metaanalytic structural equation modeling (MASEM) revisited: A reply to Cheung. Journal of Applied Psychology, 103(7), 804–811.
 Zhang, Y., Liu, G., Zhang, L., Xu, S., & Cheung, M. W.L. (2020). Psychological ownership: A metaanalysis and comparison of multiple forms of attachment in the workplace. Journal of Management.