Multilevel Models Evaluating Multilevel Models in Cross-Cultural

Multilevel Models

1

Running head: Multilevel Models

Evaluating Multilevel Models in Cross-Cultural Research: An Illustration with Social Axioms

Mike W.-L. Cheung The University of Hong Kong (Now at National University of Singapore)

Kwok Leung The City University of Hong Kong

Kevin Au The Chinese University of Hong Kong

This is a post-print version of the following paper: Cheung, M. W.-L., Leung, K., & Au, K. (2006). Evaluating multilevel models in cross-cultural research an illustration with social axioms. Journal of Cross-Cultural Psychology, 37(5), 522–541. http://doi.org/10.1177/0022022106290476

Multilevel Models

2

Abstract To assess how culture influences the behavior of people, multilevel models seem to be an immediate choice for modeling the relationship at the levels of the individual and the culture. In this paper we propose the use of structural equation modeling (SEM) to test the universality of psychological processes at the individual- and culture-levels. Specifically, the structural equivalence of the measurement (where the instrument is measuring the same construct across countries) is first tested with meta-analytic structural equation modeling (MASEM). If the measurement is structurally equivalent, cross-level equivalence (where the instrument is measuring similar constructs at different levels) will then be tested with multilevel structural equation modeling (MLSEM). A large data set on social axioms with 7,590 university students from 40 cultural groups was used to illustrate the procedures. The results showed that the structural equivalence of the social axioms was well supported at the individual level across 40 cultural groups, while the cross-level equivalence was partially supported. The superiority of the SEM approach and the theoretical meaning of its solution are discussed.

Keywords: social axioms, multilevel models, structural equation models, structural equivalence, cross-level equivalence.

Multilevel Models

3

Evaluating Multilevel Models in Cross-Cultural Research: An Illustration with Social Axioms Culture influences how people think, behave, and communicate. Hofstede (1980) defined culture as “the collective programming of the mind which distinguishes the members of one human group from another” (p. 25). This definition suggests that culture is shared among individuals. It has long been recognized that people within the same culture share similar languages, beliefs, values, and other psychological attributes when compared with people in other cultures. From a statistical point of view, people and culture are considered on two distinct levels, but people are nested within culture. One of the key research questions in cross-cultural research is whether psychological processes are universal across cultures. To answer this question, the dominant approaches are the individual-level analysis and the culture-level analysis (Leung, 1989). For example, based on over 60,000 individuals from 40 nations or regions, Hofstede (1980) factor-analyzed societal means (the average scores of people in each society) and identified four cultural dimensions. These cultural dimensions, especially individualism-collectivism, have been frequently used to explain cross-cultural similarities and differences. While Hofstede’s approach was based on the culture-level analysis, Schwartz and Bilsky (1987) proposed a value-based model (the Schwartz value survey) presented at the individual level. Schwartz and Bilsky identified fifty-six values that could be grouped into ten types. Further research has showed that these value types can be applied to explain the psychology of people from a diverse range of cultures (Schwartz, 1992).

Multilevel Models

4

As international collaborations have increased, it has become popular to conduct research using large cross-cultural data sets (e.g., Diener & Diener, 1995; Hofstede, 1980; House et al., 2002; Inglehart, Basañez, & Moreno, 1998; International Social Survey Program, 1997; Leung et al., 2002; Schwartz & Sagie, 2000). These large cross-cultural data sets provide much richer information than conventional two-country comparisons, and permit multilevel analyses. In particular, two levels－individual and culture－are obvious in these data sets. When analyzing these data sets, the issues of level and the non-independence of people within cultures need to be addressed properly (Chan, 1998; Klein, Dansereau, & Hall, 1994; Smith, 2002). The appropriate, and probably best, method is to analyze individual- and culture-level data simultaneously via multilevel models (e.g., Klein, Tosi, & Cannella, 1999; Raudenbush & Bryk, 2002). Theoretically, multilevel models provide a more accurate way of conceptualizing how culture influences people within cultures (Chao, 2000). That is, such models enable researchers to develop theories capturing the dynamic processes at different levels (Klein et al., 1994; Klein et al., 1999). On the other hand, ignoring the presence of a multilevel data structure may lead to incorrect statistical inferences, even if the research questions concern the individual level only. For example, standard errors may be underestimated because data within the same culture are likely to be more similar than data across the cultures. Thus, the Type I error is usually much higher than the predefined one (Raudenbush & Bryk, 2002). Multilevel structural equation modeling (MLSEM) is a statistical technique to investigate factor structures and structural relationships at different levels (Goldstein, 2003; Muthén, 1994). It can be used to study the factor structures of a measurement instrument with data spanning two levels. By analyzing a large data set from the International Social Survey Program (ISSP, 1997)

Multilevel Models

5

with 15,244 full time employees from 27 countries, Cheung and Au (in press) showed that MLSEM could be applied to cross-cultural research to evaluate factor structures at the levels of the individual and the culture. However, one critical, but rarely addressed, assumption in MLSEM is that the proposed within-structure model (the model at the individual level) is assumed to be the same across all cultures (see Assumption A1 in Liang & Bentler, 2004, p. 103). This is an untenable assumption for cross-cultural research, as psychological processes cannot be assumed to be universal across cultures. If this assumption is not met, the well-fitted results of MLSEM are misleading. Cross-cultural researchers may incorrectly conclude that the psychological processes at the culture level and at the individual level are similar. To address this methodological limitation in applying MLSEM in cross-cultural research, we propose to use meta-analytic structural equation modeling (MASEM; Cheung & Chan, 2005) to test whether the individual-level model holds in all cultures. If the individual-level model holds in all other cultures, MLSEM can then be used to study the factor structures at the levels of the individual and the culture. There are two objectives in the present study. First, we propose a two-step approach in the use of structural equation modeling (SEM) to test the universality of psychological processes in cross-cultural research with multilevel data. By showing that SEM is useful in cross-cultural research, we hope to provide an integrated framework for analyzing multilevel data. Second, to illustrate our approach, we apply these procedures to a large cross-cultural data set on social axioms gathered by Leung and Bond and their collaborators (Bond, et al., 2004; Leung, et al., 2002; Leung & Bond, 2004). Their results, which were obtained with independent individualand culture-level analyses, were evaluated with our integrated approach, which is more stringent

Multilevel Models

6

and more statistically elegant. A replication of their previous results with our procedure would provide support to our approach as well as stronger evidence of the validity of their study.

Searching for Cross-cultural Differences van de Vijver and Leung (1997; see also 2000, 2001) summarized two major orientations in cross-cultural studies: Structure-oriented and level-oriented. Structure-oriented approaches explore the relationship among variables within cultures. In level-oriented approaches, the focus is on the cultural means of the variables. A similar taxonomy on cross-cultural meta-analysis (instrument-based versus domain-based meta-analyses) was also suggested by van Hemert (2003). Although van de Vijver and Leung, and van Hemert did not link their taxonomies directly to multilevel models, a multilevel framework was implied. Putting their taxonomies into the framework of multilevel models, the structure-oriented studies and instrument-based metaanalysis focused on the individual level, while the level-oriented studies and domain-based metaanalysis focused on the culture level because cultural means were involved. Structural Equivalence of Individual-level Models across Cultures Before comparing whether the score of one country is higher than that of another, it is necessary to address whether the instrument is measuring the same construct across countries (van de Vijver & Leung, 1997). Structural equivalence means that the construct is defined similarly in different cultures (van de Vijver & Leung, 1997). In measurement theory, this issue is typically handled under the topic of measurement equivalence (e.g., Byrne & Campbell, 1999; Byrne & Watkins, 2003). If there is measurement equivalence, the instrument has the same

Multilevel Models

7

meaning and psychometric properties across cultures; otherwise, it measures different constructs in different cultures. Exploratory factor analysis (EFA) and SEM are usually used to study structural equivalence. EFA approach. To study structural equivalence in cross-cultural research, McCrae et al. (1996; see also van de Vijver & Leung, 1997) suggested that cross-cultural researchers could compare the factor structure of an instrument against a reference population using the Procrustes rotation. First, separate EFA are conducted with the same number of factors in each culture. One culture is treated as the target, against which other cultures are compared. Congruence coefficients derived from the factor solutions for each culture can be calculated. One of the most frequently used congruence coefficients is Tucker’s phi (Tucker, 1951) which measures the identity of two factors, up to a positive, multiplying constant (McCrae et al., 1996). It is similar to the correlation coefficient that is not sensitive to the absolute difference between the factor patterns. If the congruence coefficients are high enough, researchers may infer that the measurement is structurally equivalent across cultures. For example, Barrett et al. (1998) used the Procrustes rotation and showed that the Eysenck Personality Questionnaire had a similar meaning across 34 countries. Since the procedure is based on a pairwise comparison between two cultures, the number of pairwise comparisons becomes prohibitively large when the number of cultures increases. For example, there are 780 pairwise comparisons for 40 cultural groups, and it is often not easy to obtain a general pattern based on these results. Recently, Welkenhuysen-Gybels and van de Vijver (2001) proposed two approaches to “cluster” the cultures into groups with a similar data structure. Their approach may be useful when EFA is being applied in cross-cultural research.

Multilevel Models

8

SEM approach. A different approach to assessing structural equivalence is based on the traditions of SEM and meta-analysis. Cheung and Chan (2005) proposed a unified approach, termed meta-analytic structural equation modeling (MASEM), to conduct a meta-analysis of studies based on SEM, and their approach can be applied to assess structural equivalence across cultures. They proposed using a special case of the multiple-group confirmatory factor analytic model to test the homogeneity of correlation matrices across cultures (see the Appendix for the technical details). If the correlation matrices are reasonably homogeneous, a pooled correlation matrix will be obtained. The homogeneity of correlation matrices does not tell us what the model is because no factor structure is imposed when testing for homogeneity. To study the factor structure of the measurement, a SEM model is proposed and fitted against the pooled correlation matrix. If the proposed model fits well, we have strong evidence to support the universality of the model across cultures. In other words, the model is structurally equivalent across cultures. If the correlation matrices are heterogeneous, Cheung and Chan (in press) proposed a cluster analytic approach to classify cultures according to their correlation matrices. Within each cluster, cultures are similar in their structural model. This approach also provides a clear description of how structural models vary across clusters. Cross-Level Equivalence across Individual and Culture Levels Structural equivalence ensures that the constructs in different cultures are comparable. That is, the items are measuring the same construct in different cultures. In order to interpret the aggregated (cultural) means with the same meaning as those in the individual level, the similarity of meaning at both levels is a prerequisite (van de Vijver & Poortinga, 2002). If the variables have similar correlates as other variables at both levels, they are regarded as showing cross-level

Multilevel Models

9

equivalence. Putting it back into the framework of EFA or SEM, the factor structures are required to be similar at both levels. Leung and Bond (1989) call dimensions of this type a strong etic dimension. It is also called “cross-level equivalence,” “structural equivalence across aggregation levels” in van de Vijver and Poortinga (2002, p. 141) or “functional equivalence” in van Hemert, van de Vijver, and Poortinga (2002, p. 257). For consistency throughout this paper, the term cross-level equivalence is used here. EFA approach. Based on the MLSEM of Muthén (1994), van de Vijver and Poortinga (2002) proposed using EFA with a Procrustes rotation to study the cross-level equivalence in the individual and culture levels. They proposed calculating a pooled within (individual-level) correlation matrix and a between (culture-level) correlation matrix. EFA is then used to factoranalyze the correlation matrices at both levels separately. Then, the Procrustes rotation is used to evaluate the similarity between these two factor structures. If the congruence coefficient is high enough, the constructs are said to be equivalent across both levels. For example, van Hemert, van de Vijver, and Poortinga (2002) found that the Beck Depression Inventory appears to have the same correlation pattern at the individual and culture levels, which supports the equivalence of the Beck Depression Inventory across both levels. In another study, van Hemert, van de Vijver, Poortinga, and Georgas (2002) found that two Eysenck Personality Questionnaire scales (Extraversion and Neuroticism) were equivalent across the individual and culture levels, whereas the other two scales (Psychoticism and Lie) were not. MLSEM approach. A more sophisticated approach in analyzing factor structures with individual- and culture-level data is MLSEM (e.g., Cheung & Au, in press; Goldstein, 2003; Muthén, 1994). Similar to the method of van de Vijver and Poortinga (2002), the scores observed

Multilevel Models 10 at the individual level are decomposed into between- (or culture-) level scores and within- (or individual-) level scores. After decomposing the data into a pooled-within and a between component, researchers can propose a within- and between-structural model for the data. If the within- and between-structures are similar and hold at both levels, cross-level equivalence is said to be achieved. If not, different models may be required for different levels, and different psychological processes may be involved at different levels (see the Appendix for the technical details). MLSEM is a sophisticated method that also tests the within-structure at the individual level. MASEM is, however, necessary for two reasons. First, in cross-cultural research structural equivalence is commonly assessed with procedures based on the correlation matrix, such as factor analysis (e.g., McCrae et al., 1996; van de Vijver & Leung, 1997; Welkenhuysen-Gybels & van de Vijver, 2001, among others). As MLSEM is based on the covariance matrix, it is possible that even though MLSEM shows a poorly fitted within-structure at the individual level, the structural equivalence based on the correlation matrix may still be reasonable according to conventional procedures. Second, the factor structure at the individual level of MLSEM is assumed to be equal across cultures, but it is not assessed empirically (see Assumption A1 in Liang & Bentler, 2004, p. 103; Muthén, 1994). Therefore, it is possible that the “pooled” withinstructure in MLSEM fits well at the individual level, but that structural equivalence does not hold for some cultural groups. Comparison between the EFA and SEM approaches The SEM approach has some advantages over the EFA approach in assessing structural and cross-level equivalence, especially in addressing complicated research questions and data.

Multilevel Models 11 Exploratory versus confirmatory research. One of the arguments for using EFA rather than SEM, is that researchers may not have strong theories about their data. Their focus is to explore the structure of the data rather than to test an a priori model. It is correct to say that EFA is most suitable for exploratory research while SEM is usually used for confirmatory research. However, it should be pointed out that SEM can also be used in an exploratory manner (e.g., Loehlin, 2004). Modification indices are routinely provided in most SEM packages for searching for better models. Moreover, systematic model specification searches are available in SEM (e.g., Marcoulides & Drezner, 2003). Thus, it is possible to search for an optimal factor structure even without an a priori model in SEM. Of course, researchers should be careful in interpreting the goodness-of-fit indices when they conduct exploratory analyses with SEM. Rule of thumb versus statistical test. SEM has been criticized as being based on a formal statistical theory that often requires unattainable assumptions. For example, models are usually not acceptable based on the chi-square test statistic (Jöreskog & Sörbom, 1996). Researchers in SEM are also aware of this problem, and have paid more attention to achieving a “close fit” rather than an “absolute fit.” Several goodness-of-fit measures have been proposed, and some practical guidelines have been suggested (see Hu & Bentler, 1998). In fact, EFA with the Procrustes rotation is not free from this problem, and some rules of thumb are employed in its application. For example, when the congruence coefficient is larger than some cutoff value, say 0.90 (McCrae et al., 1996), the factor structures are usually considered to be similar. However, this rule of thumb has been challenged by Chan et al. (1999), who found in their simulation studies that even though the target structure and the replication structure were different, using 0.90 as the cutoff would result in a wrong conclusion of congruence. They proposed a bootstrap

Multilevel Models 12 approach to empirically estimate the distribution of the congruence coefficient. In this sense, the EFA and SEM approaches are not different when assessing model fit based on practical guidelines instead of rigid statistical criteria. Measurement model versus full model. The major problem with the EFA approach is that the analysis is restricted to factor analysis with continuous data only, or what is known as the measurement model. If researchers are interested in how some constructs predict other constructs, separate regression analyses are required. The advantage of the SEM approach is that it is readily extendable to other statistical modeling. We may include antecedent and consequent variables at the individual and country levels. Categorical and missing data can be handled properly and efficiently within SEM (Muthén & Muthén, 2004). Moreover, it is also possible to analyze ipsative data, which can be used to reduce the response bias in cross-cultural research (Cheung, 2004; in press; Cheung & Chan, 2002). Another point to note is that EFA can only fit first-order models, but SEM can fit higher-order factor models. The advantage to testing a second-order factor structure will be demonstrated in the present study. To summarize, we propose to use SEM as a unified framework to test the universality of psychological processes in cross-cultural research. The first step is to use MASEM to test the structural equivalence of the variables at the individual level across cultures. If it is supported, we can test the cross-level equivalence of a structure at the individual and culture levels with MLSEM. Cross-cultural Factor Structures of Social Axioms Background on Social Axioms

Multilevel Models 13 Leung and Bond and their collaborators (Leung, et al., 2002) initiated a project on social axioms, or general beliefs, which are the basic premises that people endorse and use to guide their behavior in a variety of situations. Based on a scale derived from an extensive literature review as well as on interviews and a content analysis of major beliefs concerning health, marriage, family, politics, and religion, they identified five general factors of social axioms at the individual level, namely, social cynicism, social complexity, reward for application, religiosity, and fate control. They proposed that social axioms are applicable to people across cultures. These guiding principles can be used to predict different types of behavior within a particular culture or across cultures (Fraser & Gaskell, 1990). The first version had 60 items that were developed based on five countries (Leung et al., 2002). Using EFA and the Procrustes rotation, Leung and Bond (2004) found that the five-factor structure was universal across 40 cultures. In an ecological factor analysis (a factor analysis based on cultural means) on 41 cultural means, Bond et al. (2004) found that there were only two cultural dimensions. Societal cynicism consists of items only from social cynicism. It reflects the culture-level cognitive apprehension of the cruelty of the world and the belief that powerful people and institutions have suppressed the citizenry for selfish and malignant purposes. Dynamic externality consists of items from the other four dimensions. It denotes the culture-level belief structures of how people have mobilized themselves psychologically to confront environmental difficulties and expected to succeed. The results of Bond et al. (2004) and Leung and Bond (2004) were based on EFA. To illustrate our proposed approach to multilevel modeling, and to provide a more rigorous test of the factor structures of social axioms at the individual and culture levels, the SEM approaches

Multilevel Models 14 described above were applied to the social axiom data for testing the cross-level equivalence of the social axioms at the individual and culture levels. After establishing the structural equivalence of the five-factor model (Leung, et al., 2002) at the individual level with MASEM, three competing models at the individual and culture levels were tested with MLSEM. Although there could be other possible models, we have limited our analyses to these three models because of their theoretical basis. The first model, a first-order five-factor model at the culture level, was based on the direct consensus model. As shown in Figure 1, this model assumes that psychological meanings and processes are similar across levels (Chan, 1998). The second model was based on Bond et al.’s (2004) first-order two-factor model (see Figure 2). The five-factor and two-factor models are incompatible with each other. The five-factor model suggests that all of the five social axioms are equivalent across both levels, whereas the two-factor model suggests that only social cynicism is equivalent across both levels. For simplicity, only the first and last items for each factor are shown in the Figures, although all items were included in the analyses. ---------------------------------------Insert Figures 1 and 2 about here ---------------------------------------The third model is the second-order, two-factor model. As shown in Figure 3, this model specifies that the relationships between the items and the five factors are equivalent across both levels. Social complexity, reward for application, religiosity, and fate control are clustered under the second-order factor of dynamic externality, while social cynicism constitutes the second-

Multilevel Models 15 order factor of societal cynicism. We shall discuss the significance of the second-order factor below. ---------------------------------------Insert Figure 3 about here ---------------------------------------Method Sample. The data set was obtained from the global study of the social axiom project (Leung & Bond, 2004), comprising 7,590 university students from 40 societies. Sample sizes within these societies varied, with a minimum of 80 in the United Kingdom to a maximum of 710 in India (for details, see Table II in Leung and Bond, 2004). The samples were genderbalanced, with excess respondents of either sex randomly discarded before the data were analyzed. Instrument. Thirty-nine items were selected based on the individual-level factor analysis results of the 60-item Social Axiom Survey from the 40 societies. Leung and Bond (2004) selected these 39 items based on two criteria: they had a minimum loading of .35 on the primary factor and there were no sizable secondary loadings (>.30). Five social axiom dimensions were measured by these 39 items. Social cynicism was measured by 11 items, e.g., “Powerful people tend to exploit others.” Social complexity was measured by 6 items, e.g., “People may have opposite behaviors on different occasions.” Reward for application was measured by 9 items, e.g., “Knowledge is necessary for success.” Religiosity was measured by 7 items, e.g., “Belief in a religion helps one understand the meaning of life.” Finally, fate control was measured by 6 items, e.g., “Individual characteristics, such as appearance and birthday, affect one’s fate.”

Multilevel Models 16 Statistical analysis. First, MASEM was conducted to study the structural equivalence of the model across cultures. Second, MLSEM was used to investigate the cross-level equivalence across the individual and cultural levels. LISREL 8.54 (Jöreskog & Sörbom, 2003) was used to implement the MASEM procedures proposed by Cheung and Chan (2005), while Mplus 3.11 (Muthén & Muthén, 2004) was used to conduct MLSEM because of its powerful ability to handle multilevel data and missing data. Criteria to evaluate the models. Since SEM was used in this study, goodness-of-fit indices could be used to evaluate the proposed models. One problem of using the chi-square statistic is that it is too sensitive to the sample size used. As the sample size is usually very large in a typical cross-cultural data set (N=7,590 in our study), it is likely that all proposed models will be rejected based on the chi-square test. Thus, the use of chi-square statistics is not considered here. Based on extensive simulation studies, Hu and Bentler (1998) suggested several goodness-of-fit indices for identifying incorrect models. These indices include the standardized root-mean-square residual (SRMR), the root-mean-square error of approximation (RMSEA), the Tucker-Lewis Index (TLI; also known as the non-normed fit index, NNFI), and the comparative fit index (CFI). They suggested a cutoff value close to .95 for TLI and CFI; a cutoff value close to .08 for SRMR; and a cutoff value close to .06 for RMSEA. If non-nested models are involved, the Akaike information criterion (AIC) and Bayesian information criterion (BIC) can be used to help compare models (Loehlin, 2004). Smaller values on AIC and BIC suggest better models in terms of model fit and parsimony. Results and Discussion

Multilevel Models 17 Testing structural equivalence with MASEM. The fit indices for the first step of testing the homogeneity of all correlation matrices are 2(28,899, N=7,590) = 41,132, p < .0001, RMSEA = 0.047, 90 percent confidence intervals (CIs) for the RMSEA = (0.046; 0.048), TLI = 0.79, CFI = 0.79. RMSEA shows that the proposed model fits the data well, whereas CFI and TLI suggest a poor fit. These apparently contradictory results deserve further discussion. There are two arguments supporting the view that RMSEA and SRMR as more appropriate than CFI and TLI in evaluating the model fit in MASEM. Comparing CFI and RMSEA, Rigdon (1996) showed that CFI and other incremental fit indices (e.g., TLI) are less stable across different estimation methods because a null model is involved in the calculation of the indices. On the other hand, RMSEA is insensitive to changes in sample size, especially when the sample size is large. Rigdon concluded that “CFI [is] better suitable to more exploratory, small sample cases, and RMSEA [is] better suited to more confirmatory, large sample situations” (p. 376). Because the sample size in the present study is large (N=7,590), RMSEA is more appropriate than CFI in evaluating the model fit. In addition, Browne et al. (2002) found a discrepancy between the standard chi-squarebased fit indices (CFI and TLI) and the residual-based fit indices (SRMR) when the model being tested has a small or no unique variance. They found that CFI and TLI may indicate a poor fit when it is indeed otherwise. In testing the homogeneity of the correlation matrices with Cheung and Chan’s (2005) approach, it is assumed that there is no unique variance (see Equation 2 in Appendix). Therefore, CFI and TLI may erroneously indicate a poor fit when the model is

Multilevel Models 18 indeed a good fit in MASEM. In light of these findings, RMSEA and SRMR are better indicators of model fit than CFI and TLI in evaluating MASEM. In the second step, the five-factor model at the individual level as defined in Leung and Bond (2004) was fitted upon the pooled correlation matrix. The fit indices for the second stage are 2(692, N=7,590) = 6,653, p < .0001, RMSEA = 0.034, CIs = (0.033; 0.034), SRMR = 0.046, TLI=0.68, CFI=0.70. Again, both RMSEA and SRMR suggest a good fit for the five-factor model. These results suggest that the five-factor model based on EFA (Leung & Bond, 2004) has structural equivalence across the 40 cultures. However, these findings pertain to the individuallevel structure only, and not to the culture-level structure. Testing cross-level equivalence with MLSEM. The average intra-class correlation (ICC) for the variables is .12, which suggests that societal level, on average, explains about 12% of the variance in the variables. This value is comparable to Cheung and Au’s (in press) findings (average ICC=.085) with the ISSP (1997) data. James (1982) and Muthén (1994) suggested that the typical ICC values ranged from .00 to .50, while Bliese (2000) estimated the typical values at between .05 and .20. No matter which reference is used, our ICCs are quite typical of those in other MLSEM studies. Table 1 shows the ICCs for individual items. Several observations can be made. Religiosity items have the largest average ICC, whereas social complexity items have the smallest. In other words, people across cultures are more similar in social complexity and more dissimilar in religiosity. In sum, the moderate value of the ICCs suggests that any analysis that ignores the culture level is inappropriate. ----------------------------------------

Multilevel Models 19 Insert Table 1 about here ---------------------------------------As mentioned before, three culture-level models were tested. The first-order five-factor model in Figure 1 ran into a non-convergent problem, and no result could be obtained for this model. Based on EFA, Bond et al. (2004) also did not find support for this five-factor model. The speculation is that the first-order five-factor model was probably seriously misspecified, and hence, inappropriate. The first-order two-factor model (Figure 2) converged, and the goodness-of-fit indices are 2(1,393, N=7,590) = 8,647, p < .0001, RMSEA = 0.026, SRMR (between structure) = 0.152, SRMR (within structure) = 0.042, CFI = 0.77, TLI = 0.75, AIC = 780,372, and BIC = 781,800. Moreover, the second-order two-factor model (Figure 3) also fits the data well with 2(1,389, N=7,590) = 8,569, p < .0001, RMSEA = 0.026, SRMR (between structure) = 0.148, SRMR (within structure) = 0.042, CFI = 0.77, TLI = 0.75, AIC = 780,292, and BIC = 781,749. The fit indexes of the first-order two-factor model and the second-order two-factor model are similar. Due to its simplicity and conceptual appeal (see details in the General Discussion section), we chose to discuss the second-order factor model. In this model, the within-structure SRMR is much smaller than the between-structure SRMR, which suggests that the within structure model fits well at the individual level while the between structure fits only marginally well at the culture level. Since the social axioms were derived at the individual level, this may be the reason why the model fits better at the individual level than at the culture level. Another possible reason is the inclusion of more factors at the individual level: five first-order factors at the individual

Multilevel Models 20 level and only two second-order factors at the culture level. More factors can explain covariation better then fewer factors.1 Table 2 shows the factor correlations at the individual level based on MASEM and MLSEM, which are quite similar. Note that these factor correlations are based on latent variables, which means that measurement errors are minimized. There is only one minor discrepancy in the factor correlation between social cynicism and fate control between the results of MASEM and MLSEM. It should be noted that MASEM uses correlation matrices as the input while MLSEM uses covariance matrices as the data. When the scales of the variables differ largely across variables or across cultures, the results between MASEM and MLSEM may not be the same. If this happens, researchers know that the construct is defined similarly in different cultures, but that the scales (in terms of standard deviation) are not the same across cultures. In other words, the construct is structurally equivalent but not metrically equivalent (see van de Vijver & Leung, 1997). It should be noted that testing for structural equivalence should be based on the results of MASEM. A comparison of these two sets of results suggests that the scales of the social axiom variables are quite comparable across cultures. This supports the view that people from different cultures regard the measurement scales in a similar manner. ---------------------------------------Insert Table 2 about here ---------------------------------------Table 3 shows the standardized factor loadings at the individual level based on MASEM and MLSEM. All factor loadings are reasonably high, which suggests that the measurement is very good and that the items are properly loaded on their corresponding constructs. Table 3 also

Multilevel Models 21 shows the standardized factor loadings at the culture level. There are only two factor loadings at the culture level smaller than 0.4 (com4 and fat1) while other factor loadings are very high. This pattern indicates that the items are good at tapping the culture-level dimensions. Indeed, the factor loadings at the culture level are usually higher than the factor loadings at the individual level. ---------------------------------------Insert Table 3 about here ---------------------------------------The structural loadings of the second-order two-factor model at the culture level are shown in Figure 4. Social cynicism is uniquely loaded on societal cynicism while the other four factors are loaded on dynamic externality. All of the structural paths from dynamic externality are very high, and this may be the reason why there is no obvious difference in the goodness-offit indices between the first-order two-factor and the second-order two-factor models. At the culture level, cultures with high dynamic externality are defined as characterized by high reward for application, religiosity, fate control, and having low social complexity. The factor correlation between social cynicism and dynamic externality is .52, which is substantial. Discussion on the Factor Structure of Social Axioms By analyzing the social axioms with MASEM, the within-culture structural equivalence of the social axioms was well supported. The five-factor structure fits well in 40 cultures, and these five factors are relatively uncorrelated except for social complexity and reward for application at the individual level. These results support the generality of the five social axiom dimensions across cultures.

Multilevel Models 22 The cross-level equivalence of the model at the individual and the cultural levels, which was tested with MLSEM, was partially supported. The factor loadings are all reasonably good at the culture level, suggesting that aggregated means are good indicators for measuring the social axioms at the culture level. Social cynicism shows strong equivalence across both levels. In comparison, the other four factors are highly correlated at the culture level, whereas they are quite orthogonal at the individual level. These results are similar to those of Leung and Bond (2004) and Bond et al. (2004), which are based on independent individual- and culture-level analyses. Our more rigorous procedure is superior in its ability to identify a second-order twofactor model. The fact that social axioms exhibit different structures across the two levels of analysis may suggest that social and instrumental problems have driven individuals and societies to develop different ways of coping with external challenges (Leung & Bond, 2004). More specifically, the five axiom dimensions identified at the individual level are useful to guide individual behavior, but the two dimensions identified at the societal level suggest that societal responses to external challenges are less differentiated. This is another demonstration that individual-level and culture-level analyses may tap different types of constructs (c.f. Schwartz, 1992). In the case of social axioms, dynamic externality encompasses the items of four distinctive factors at the individual level. This culture-level factor indicates a universal belief structure for which societies use to deal with a difficult external environment. That is probably why dynamic externality is strongly related to poverty (Bond et al., 2004). Societies use institutions and rules, as a package, to promote fatalism, the belief in a supreme being, hard work, and low social complexity in their response to hostile surroundings. In contrast, individuals seem

Multilevel Models 23 to vary more widely than societies on how they react to hostile environments. For instance, it is more common to find rich and poor individuals who are religious or fatalistic. Hence, the four individual-level factors of social axioms do not cluster together as they do at the culture level. This is a clear demonstration of the shift in meaning when moving from the individual level to the culture level. Interested readers may refer to Chan (1998) for the theoretical implications of the shift of meanings across levels and for more examples of the process. Finally, societal cynicism is defined by the same content as social cynicism at the individual level and constitutes a “strong etic” (Leung & Bond, 1989). This convergence implies that individuals have a belief system that concerns mistrust towards and disengagement from people and social systems, and such an individual-level belief system corresponds to a cultural dimension of the same nature. Although individual-level and culture-level factors are usually consequences of different dynamics and processes (Leung, 1989), cynicism happens to be one of the very few exceptions. Social and political turmoil and instability may have promulgated a cynical belief system among individuals and at the same time caused societies to develop cynical characteristics (Bond et al., 2004). General Discussion Comparing the similarity of psychological processes across cultures and between the individual and culture levels is an important issue in cross-cultural research. In the present study, SEM is proposed as an integrated framework for assessing the universality of psychological processes in cross-cultural studies with multilevel data. This study can be considered as an integrated approach to implementing the methodology addressed in van de Vijver and Leung (1997) and van de Vijver and Poortinga (2002). It also has the theoretical advantage (Chao, 2000)

Multilevel Models 24 of addressing the issues of level and the non-independence of people within cultures (Chan, 1998; Klein, Dansereau, & Hall, 1994; Smith, 2002). Specifically, in the first step, MASEM is proposed for use in examining the structural equivalence of a model across cultures. After establishing structural equivalence, MLSEM is proposed for investigating the equivalence of the model across levels. The social axiom data set was used to demonstrate this two-step approach in a real-life context. It is argued that EFA yields better results than does CFA, especially in personality and cross-cultural research (e.g., Church & Burke, 1994; McCrae et al., 1996). However, there are many counterexamples showing that SEM can generate interpretable results in cross-cultural research (e.g., Byrne & Campbell, 1999; Byrne & Watkins, 2003; Cheung & Rensvold, 2000; Little, 1997). Taking our social axiom data as an example, the results show that the five-factor model fits well across 40 cultures, even though the tested model, which has no double loading, is very restrictive. Therefore, it is hard to believe that SEM is generally sub-optimal for an analysis of cross-cultural data. Researchers may instead consider whether poor SEM results are attributable to poor measurement or conceptualization. In weighing SEM and EFA, researchers may also consider the capability of SEM in testing solutions of higher-order factors. Second-order factors are broader factors than first-order factors, and usually a small number of them explain more variance and provide an economical account for a complex field (Kline, 1998, p. 102). For this reason, second-order factors have been proposed for studying personality traits and human abilities (e.g., Carroll, 1993), the primary factors of which are numerous and difficult to exhaust (Kline, 1998). The second-order two-factor model here seems complex, but it exhibits conceptual and structural simplicity.

Multilevel Models 25 Perhaps, as Kline suggests, a higher-order factor analysis is what is needed for a complex field such as values and beliefs, whose primary factors are also difficult to exhaust, especially across cultures. The SEM procedures proposed here will be useful when research like Leung and Bond (2004) turns up more values and beliefs across cultures. Further Directions The present study focuses on the structural and cross-level equivalence of a measurement instrument only. In some cases, researchers are interested in how culture-level variables predict or moderate individual-level relationships. For example, Oishi et al. (1999) found that the wealth of nations (a culture-level variable) moderated the strength of the association between wealth and life satisfaction at the individual level. This approach can be viewed from the perspective of meta-analysis or multilevel models in which culture-level variables serve as moderators (e.g., Au & Cheung, 2004; de Leeuw & Hox, 2003; van Hemert, 2003; Raudenbush & Bryk, 2002). Applications of these more complex models to cross-cultural research are still rare, and we hope that our paper will stimulate more applications of multilevel models in cross-cultural research.

Multilevel Models 26 References Au, K., & Cheung, M. W. L. (2004). When do trustors thrive? A cross-level study on generalized trust. Manuscript submitted for publication. Barrett, P. T., Petrides, K. V., Eysenck, S. B. G., & Eysenck, H. J. (1998). The Eysenck Personality Questionnaire: An examination of the factorial similarity of P, E, N, and L across 34 countries. Personality and Individual Differences, 25, 805-819. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341-362. Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models”. Journal of Educational Statistics, 20, 100-102. Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein, S. W. J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 349-381). San Francisco, CA: Jossey-Bass. Bond, M. H., Leung, K., Au, A., Tong, K. K., Reimel de Carrasquel, S., Murakami, F., et al. (2004). Culture-level dimensions of social axioms and their societal correlates across 41 cultures. Journal of Cross-Cultural Psychology, 35, 548-570. Browne, M. W. MacCallum, R. C., Kim, C. T., Andersen, B. L., & Glaser, R. (2002). When Fit Indices and Residuals Are Incompatible. Psychological Methods, 7, 403-421. Byrne, B. M., & Campbell, T. L. (1999). Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure: A look beneath surface. Journal of Cross-Cultural Psychology, 30, 555-574. Byrne, B. M., & Watkins, D. (2003). The issue of measurement invariance revisited. Journal of Cross-Cultural Psychology, 34, 155-175.

Multilevel Models 27 Carroll, J. B. (1993). Human cognitive abilities. Cambridge, Cambridge University Press. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246. Chan, W., Ho, R. M, Leung, K., Chan, D. K.-S., & Yung, Y.-F. (1999). An alternative method for evaluating congruence coefficients with Procrustes rotation: A bootstrap procedure. Psychological Methods, 4, 378-402. Chao, G. T. (2000). Multilevel issues and culture: An integrative view. In K. J. Klein, & S. W. J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 308-346). San Francisco, CA: Jossey-Bass. Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equation modeling. Journal of CrossCultural Psychology, 31, 187-212. Cheung, M. W. L. (2004). A direct estimation method on analyzing ipsative data with Chan and Bentler's (1993) method. Structural Equation Modeling, 11, 217-243. Cheung, M. W. L. (2005). TSSEM: A LISREL syntax generator for two-stage structural equation modeling (Version 1.0) [Computer software]. Available online at http://web.hku.hk/~mikewlch/tssem.zip. Cheung, M. W. L. (in press). Recovering preipsative information from ipsatized data: A factor score approach. Educational and Psychological Measurement. Cheung, M. W. L., & Au, K. (in press). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling.

Multilevel Models 28 Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple group confirmatory factor analysis. Structural Equation Modeling, 9, 55-77. Cheung, M. W. L., & Chan, W. (2005). Meta-analytic structural equation modeling: A twostage approach. Psychological Methods, 10, 40-64. Cheung, M. W. L., & Chan, W. (in press). Classifying correlation matrices into relatively homogeneous subgroups: A cluster analytic approach. Educational and Psychological Measurement. Church, A. T., & Burke, P. J. (1994). Exploratory and confirmatory tests of the Big Five and Tellegen’s three- and four-dimensional models. Journal of Personality and Social Psychology, 66, 93-114. de Leeuw, E. D., & Hox, J. J. (2003). The use of meta-analysis in cross-national studies. In J. A. Harkness, F. J. R. van de Vijver & P. Ph Mohler (Eds.). Cross-cultural survey methods (pp. 329-345). New York: Wiley. Diener, E., & Diener, M. (1995). Cross-cultural correlates of life satisfaction and self-esteem. Journal of Personality and Social Psychology, 68, 653-663. Fraser, C., & Gaskell, G. (1990). The social psychological study of widespread beliefs. Oxford, UK: Glarendon. Goldstein, H. (2003). Multilevel statistical models (3rd Ed.). London: Arnold. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press. Hofstede, G. (1980). Culture’s consequences. Beverly Hills, CA: Sage.

Multilevel Models 29 House, R., Javidan, M., Hanges, P., & Dorfman, P. (2002). Understanding cultures and implicit leadership theories across the globe: An introduction to project GLOBE. Journal of World Business, 37, 3-10. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage. Inglehart, R., Basañez, M., & Moreno, A. (1998). Human values and beliefs: A cross-cultural sourcebook. Ann Arbor: The University of Michigan Press. International Social Survey Program (ISSP) (1997). International social science program: Work orientations II, 1997 [Computer file]. ICPSR version. Koeln, Germany: Zentralarchiv fuer Empirische Sozialforschung [producer], 1999. Koeln, Germany: Zentralarchiv fuer Empirische Sozialforschung/Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributiors], 2000. James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229. Jöreskog, K. G., & Sörbom, D (2003). LISREL 8.54 [Computer software]. Chicago, Ill.: Scientific Software International. Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: A user's reference guide. Chicago, IL: Scientific Software International, Inc. Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229. Klein, K. J., Tosi, H., & Cannella, Jr., A. A. (1999). Multilevel theory building: Benefits, barriers, and new developments. Academy of Management Review, 24, 243-248.

Multilevel Models 30 Kline, P. (1998). The new psychometrics: Science, psychology, and measurement. London; New York: Routledge. Leung, K. (1989). Cross-cultural differences: Individual-level vs. culture-level analysis. International Journal of Psychology, 24, 703-719. Leung, K., & Bond, M. H. (1989). On the empirical identification of dimensions for cross-cultural comparisons. Journal of Cross-Cultural Psychology, 20, 133-151. Leung, K., & Bond, M. H. (2004). Social axioms: A model for social beliefs in multicultural perspective. Advances in Experimental Social Psychology, 36, 119-197. Leung, K., Bond, M. H., Carrasquel, S. R., Munoz, C., Hernadez, M., Murakami, F., Yamaguchi, S., Bierbrauer, G., & Singelis, T. M. (2002). Social axioms: The search for universal dimensions of general beliefs about how the world functions. Journal of Cross-Cultural Psychology, 33, 286-302. Liang, J., & Bentler, P. M. (2004). An EM algorithm for fitting two-level structural equation models. Psychometrika, 69, 101-122. Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretically issues. Multivariate Behavioral Research, 32, 53-76. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th Ed.). Mahwah, N. J.: London. Marcoulides, G. A., & Drezner, Z. (2003). Model specification searches using ant colony optimization algorithms. Structural Equation Modeling, 10, 154-164. McCrae, R. R., Zonderman, A. B., Costa, P. T., Bond, M. H., & Paunonen, S. V. (1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus procrustes rotation. Journal of Personality and Social Psychology, 70, 552-566.

Multilevel Models 31 Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods and Research, 22, 376-398. Muthén, L. K., & Muthén, B. O. (2004). Mplus user's guide (3rd Ed.). Los Angeles, CA: Muthén & Muthén. Oishi, S., Diener, E. F., Lucas, R. E., & Suh, E. M. (1999). Cross-cultural variations in predicting life satisfaction: Perspectives from needs and values. Personality and Social Psychology Bulletin, 25, 980-990. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd Ed.). Thousand Oaks, CA: Sage Publications. Rigdon, E. E. (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling, 3, 369-379. Schwartz, S. H., & Bilsky, W. (1987). Towards a psychological structure of human values. Journal of Personality and Social Psychology, 53, 550-562. Schwartz, S. H. (1992). The universal content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25, 1-65. Schwartz, S. H., & Sagie, G. (2000). Value consensus and importance: A cross-national study. Journal of Cross-Cultural Psychology, 31, 465-497. Smith, P. B. (2002). Levels of analysis in cross-cultural psychology. In W. J. Lonner, D. L. Dinnel, S. A. Hayes, & D. N. Sattler (Eds.), Online Readings in Psychology and Culture (Unit 2, Chapter 7), (http://www.wwu.edu/~culture), Center for CrossCultural Research, Western Washington University, Bellingham, Washington USA.

Multilevel Models 32 Tucker, L. R. (1951). A method for synthesis of factor analysis studies (Personnel Research Section Report No. 984). Washington, DC: U.S. Department of the Army. van de Vijver, F. J. R., and Leung, K. (1997). Methods and data analysis for cross-cultural research. Thousand Oaks, Ca: Sage. van de Vijver, F. J. R., & Leung, K. (2000). Methodological issues in psychological research on culture. Journal of Cross-Cultural Psychology, 31, 33-51. van de Vijver, F. J. R., & Leung, K. (2001). Personality in cultural context: Methodological Issues. Journal of Personality, 69, 1007-1031. van de Vijver, F. J. R., & Poortinga, Y. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33, 141-156. van Hemert, D. A. (2003). Patterns of cross-cultural differences in psychology: A metaanalytic approach. Unpublished doctoral dissertation. Tilburg University, the Netherlands. van Hemert, D. A., van de Vijver, F. J. R., & Poortinga, Y. H. (2002). The Beck Depression Inventory as a measure of subjective well-being: A cross-national study. Journal of Happiness Studies, 3, 257-286. van Hemert, D. A., van de Vijver, F. J. R., Poortinga, Y. H., & Georgas, J. (2002). Structural and functional equivalence of the Eysenck Personality Questionnaire within and between countries. Personality and Individual Differences, 33, 1229-1249. Welkenhuysen-Gybels, J. G. J., & van de Vijver, F. J. R. (2001). A comparison of methods for the evaluation of construct equivalence in a multigroup setting. In 2001 proceedings. American Statistical Association [CD-ROM]. Opladen: Leske+budrich.

Multilevel Models 33 Appendix Model for the Meta-Analytic Structural Equation Modeling (MASEM) To test whether a structural model is the same across studies, Cheung and Chan (2005) proposed a two-stage SEM approach. In the first stage, the homogeneity of the correlation matrices is tested with a multiple-group confirmatory factor analytic model. Mathematically, a typical multiple-group confirmatory factor analytic model for the population covariance matrix is: (θ) ( k )  ( k )  ( k ) ( k ) T   ( k ) ,

(1)

where , ,  and  are the parameter vector, factor loadings, factor covariance, and error variance matrices in the kth study, respectively. Since the statistical theory of SEM is based on the distribution of covariance matrices, Cheung and Chan (2005) proposed the following constraints to fit correlation matrices correctly: (k) is a p(k)p(k) diagonal matrix, (k) is a p(k)p(k) standardized matrix, i.e., diag[(k)] = I and (k) is a p(k)p(k) zero matrix,

(2)

where p(k) is the number of variables in the kth study and I is an identity matrix. Then, the correlation matrix in the kth group is stored in  (k), and (k) is a nuisance scaling variable storing the standard deviations. If no constraint is applied, the factor correlation matrices would be exactly equal to the sample correlation matrices. By applying the equality constraints on the factor correlation matrices over all groups, we force all correlation matrices over the cultural groups to be the same. The homogeneity of the correlation matrices can then be tested by using the chi-square difference test. After testing the homogeneity of the correlation matrices across cultures, Cheung and Chan (2005) proposed to test the proposed SEM model with the pooled correlation matrix. If

Multilevel Models 34 the structural model fits the data well, we can conclude that structural equivalence is supported across cultures. A computer program and a working example have been prepared to facilitate the above analysis (Cheung, 2005). Cheung and Chan (2005) showed with simulation studies that their proposed method was better than the conventional methods, such as the methods based on averaging correlations (Hunter & Schmidt, 1990), averaging Fisher’s z scores (Hedges & Olkin, 1985) and Generalized Least Squares (Becker, 1992, 1995). Their method obtains results with higher statistical power, and appropriate Type I error and standard errors. Model for Multilevel Structural Equation Modeling (MLSEM) In multilevel models, the observed scores at the individual level data ygi are generally decomposed into a between-cultural component yg (which equals the aggregated cultural mean) and a within-culture component yw (which equals the centered score from the corresponding cultural mean). Mathematically, the model is: y gi  y g  y w .

(3)

Since yg and yw are uncorrelated, the total population covariance matrix ( T) can be decomposed into a between-group population covariance matrix (B) and a pooled withingroup population covariance (W):  T   B  W .

(4)

A within-structure model and a between-structure model can then be proposed and fitted against the pooled within-group covariance matrix and the between-group covariance matrix, respectively (see Cheung & Au, in press; Muthén, 1994). If the models fit well, we can conclude that the cross-level equivalence of the model is supported.

Multilevel Models 35 Author Note Mike W.-L. Cheung, Department of Psychology, the University of Hong Kong; Kwok Leung, Department of Management, the City University of Hong Kong; Kevin Au, Department of Management, the Chinese University of Hong Kong. Preparation of the work was partially supported by a University Development Fund from the University of Hong Kong. Portions of this article were presented at the 2004 Academy of International Business Annual Conference, 10-13 July 2004, Stockholm, Sweden. We would like to thank Mr. Al Au for preparing the data set. Correspondence concerning this article should be sent to Mike W.-L. Cheung, Department of Psychology, the University of Hong Kong, Pokfulam Road, Hong Kong; Tel.: (852) 2857-8621; Fax: (852) 2858-3518; E-mail: [email protected].

Multilevel Models 36 Footnote 1

We thank an associate editor for this suggestion.

Multilevel Models 37 Table 1 Intra-class Correlations for Individual Items Variable ICC Variable ICC Variable ICC Variable ICC Variable ICC CYN1 0.06 COM1 0.06 REW1 0.10 REL1 0.19 FAT1 0.07 CYN2 0.07 COM2 0.06 REW2 0.13 REL2 0.27 FAT2 0.07 CYN3 0.13 COM3 0.08 REW3 0.09 REL3 0.30 FAT3 0.17 CYN4 0.12 COM4 0.05 REW4 0.11 REL4 0.27 FAT4 0.06 CYN5 0.08 COM5 0.11 REW5 0.11 REL5 0.15 FAT5 0.09 CYN6 0.07 COM6 0.09 REW6 0.19 REL6 0.09 FAT6 0.23 CYN7 0.06 REW7 0.11 REL7 0.13 CYN8 0.13 REW8 0.17 CYN9 0.10 REW9 0.10 CYN10 0.10 CYN11 0.09 Mean 0.09 0.07 0.12 0.20 0.12 Note. ICC = intra-class correlation; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT = fate control.

Multilevel Models 38 Table 2 Factor Correlations at the Individual Level Method Factor correlation MASEM MLSEM CYN, COM 0.13 0.09 CYN, REW 0.10 0.08 CYN, REL -0.03 0.01 CYN, FAT 0.19 0.36 COM, REW 0.36 0.33 COM, REL -0.04 -0.05 COM, FAT 0.01 -0.03 REW, REL 0.20 0.20 REW, FAT 0.16 0.19 REL, FAT 0.23 0.24 Note. MASEM = meta-analytic structural equation modeling; MLSEM = multilevel structural equation modeling; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control.

Multilevel Models 39 Table 3 Standardized Factor Loadings at the Individual and Culture Levels Individual level based Individual level based Culture level based on Items on MASEM on MLSEM MLSEM cyn1, CYN 0.58 0.53 0.56 cyn2, CYN 0.52 0.51 0.59 cyn3, CYN 0.51 0.52 0.71 cyn4, CYN 0.54 0.47 0.48 cyn5, CYN 0.48 0.39 0.70 cyn6, CYN 0.43 0.37 0.80 cyn7, CYN 0.41 0.35 0.43 cyn8, CYN 0.35 0.36 0.75 cyn9, CYN 0.43 0.35 0.76 cyn10, CYN 0.40 0.35 0.50 cyn11, CYN 0.39 0.29 0.72 com1, COM 0.34 0.50 0.64 com2, COM 0.52 0.43 0.58 com3, COM 0.43 0.37 0.78 com4, COM 0.41 0.40 0.24 com5, COM 0.35 0.26 0.55 com6, COM 0.28 0.24 0.80 rew1, REW 0.31 0.56 0.95 rew2, REW 0.61 0.47 0.78 rew3, REW 0.53 0.47 0.49 rew4, REW 0.48 0.43 0.78 rew5, REW 0.49 0.40 0.82 rew6, REW 0.46 0.34 0.82 rew7, REW 0.48 0.32 0.70 rew8, REW 0.38 0.33 0.66 rew9, REW 0.43 0.29 0.62 rel1, REL 0.33 0.74 0.99 rel2, REL 0.78 0.69 0.96 rel3, REL 0.77 0.54 0.88 rel4, REL 0.65 0.58 0.94 rel5, REL 0.69 0.43 0.90 rel6, REL 0.51 0.40 0.78 rel7, REL 0.44 0.45 0.76 fat1, FAT 0.48 0.46 0.36 fat2, FAT 0.38 0.44 0.76 fat3, FAT 0.43 0.38 0.48 fat4, FAT 0.30 0.36 0.66 fat5, FAT 0.40 0.39 0.88 fat6, FAT 0.44 0.39 0.70 Note. MASEM = meta-analytic structural equation modeling; MLSEM: multilevel structural equation modeling; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control.

Multilevel Models 40 Figure Captions Figure 1. Multilevel model of social axioms with a first-order five-factor model at the culture level. Figure 2. Multilevel model of social axioms with a first-order two-factor model at the culture level. Figure 3. Multilevel model of social axioms with a second-order two-factor model at the culture level. Figure 4. Standardized structural path loadings of social axioms in a second-order two-factor model at the culture level.

Multilevel Models 41

Between structure at the culture level of analysis

CYN

cyn1

cyn1

cyn11

cyn11

CYN

COM

REW

REL

FAT

com1

com6

rew1

rew9

rel1

rel7

com1

com6

rew1

rew9

rel1

rel7

COM

REW

REL

fat1

fat1

fat6

fat6

FAT

Within structure at the individual level of analysis Note. CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control. To simplify the presentation, only the first and last items for each factor are shown.



SOC_CYN

cyn1

cyn1

cyn11

cyn11

CYN

DYN_EXT

com1

com6

rew1

rew9

rel1

com1

com6

rew1

rew9

rel1

COM

REW

rel7

rel7

REL

fat1

fat1

fat6

fat6

FAT

Within structure at the individual level of analysis Note. SOC_CYN = societal cynicism; DYN_EXT = dynamic externality; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control. To simplify the presentation, only the first and last items for each factor are shown.


Between structure at the culture level of analysis SOC_CYN

DYN_EXT

CYN

cyn1

cyn1

cyn11

cyn11

CYN

COM

REW

REL

com1

com6

rew1

rew9

rel1

com1

com6

rew1

rew9

rel1

COM

REW

rel7

rel7

REL

FAT

fat1

fat1

fat6

fat6

FAT

Within structure at the individual level of analysis Note. SOC_CYN = societal cynicism; DYN_EXT = dynamic externality; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control. To simplify the presentation, only the first and last items for each factor are shown.



0.52 SOC_CYN

DYN_EXT

-0.73

1.00*

0.93 0.89

CYN

COM

REW

0.88

REL

FAT

Note. SOC_CYN = societal cynicism; DYN_EXT = dynamic externality; CYN = social cynicism; COM = social complexity; REW = reward for application; REL = religiosity; FAT= fate control. The value with an asterisk is a fixed parameter.