Development and Validation of the College Mattering Inventory With ...

Measurement and Evaluation in Counseling and Development http://mec.sagepub.com

Development and Validation of the College Mattering Inventory With Diverse Urban College Students Esau Tovar, Merril A. Simon and Howard B. Lee Measurement and Evaluation in Counseling and Development 2009; 42; 154 DOI: 10.1177/0748175609344091 The online version of this article can be found at: http://mec.sagepub.com/cgi/content/abstract/42/3/154

Published by: http://www.sagepublications.com

On behalf of:

Association for Assessment in Counseling and Education

Additional services and information for Measurement and Evaluation in Counseling and Development can be found at: Email Alerts: http://mec.sagepub.com/cgi/alerts Subscriptions: http://mec.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations http://mec.sagepub.com/cgi/content/refs/42/3/154

Downloaded from http://mec.sagepub.com by Esau Tovar on September 18, 2009

Article

Development and Validation of the College Mattering Inventory With Diverse Urban College Students

Measurement and Evaluation in Counseling and Development Volume 42 Number 3 October 2009 154-178 © 2009 The Author(s) 10.1177/0748175609344091 http://mec.sagepub.com

Esau Tovar Santa Monica College, Santa Monica, California

Merril A. Simon Howard B. Lee California State University, Northridge The perceived impact of mattering is quite accepted in higher education yet has not been well measured with diverse college students. This article describes the development and validation of the College Mattering Inventory with community college and university students. Results and implications for faculty and student affairs professionals are discussed. Keywords: mattering; marginality; diversity; scale; validity; factor analysis; factorial invariance

H

igher education practitioners and researchers alike consistently espouse the value and impact of the concept of mattering on college students. This has been done notwithstanding the relatively few attempts to validate a measure of mattering (versus marginality) with those students, and those studies that have been conducted have focused on an older college population (Schlossberg, Lasalle, & Golec, 1990), on a noncollege population (Marshall, 2001), or with a related but not identical construct—namely, sense of belonging (Hoffman, Richmond, Morrow, & Salomone, 2002–2003). In a discussion on the precursors to the development of self-esteem, Coopersmith (1967) included a sense of significance to self and others as one of the components of self-esteem. In an attempt to assess selfesteem, he did not include within the measurement tool a measure of significance distinct from success or worthiness. In contrast, Rosenberg and McCullough (1981) elaborated on the more specific construct of

significance, which they termed mattering, indicating that it is not the same construct as self-esteem. They indicated that the construct of self-esteem is a form of internally based validation, as opposed to mattering, which is a form of external validation: “Mattering is a motive: the feeling that others depend on us, are interested in us, are concerned with our fate, or experience us as an ego-extension exercises a powerful influence on our actions” (p. 165). They also discussed the inherent impact that the stages of late adolescent development may have in terms of mattering to others. As adolescents recognize and experience the importance of others in their lives (Erikson, 1968), the sense of mattering that they desire Authors’ Note: An earlier version of this article was presented at the 2008 annual meeting of the American Educational Research Association. Correspondence concerning this article should be addressed to Esau Tovar, Counseling Department, Santa Monica College, 1900 Pico Boulevard, Santa Monica, CA 90405; e-mail: [email protected].

154 154 Downloaded from http://mec.sagepub.com by Esau Tovar on September 18, 2009

Tovar et al. / College Mattering Inventory 155

may increase (Rosenberg, 1985). Even when controlling for self-esteem, Rosenberg and McCullough found a statistically significant difference between levels of hostility, anxiety, and depression and low levels of perceived parental mattering. This finding further confirmed that mattering and self-esteem are different constructs, although with some potential overlap between the two. Like Rosenberg and McCullough (1981), Josselson (1998; who described identity and the interconnectedness of women) discussed mattering but only from an experiential perspective, rather than an empirical one. Building from Kohut’s psychoanalytic work (1977), Josselson suggested that mattering is an aspect of identity, in terms of relationships that result in validation from others to create a sense of certainty about identity. In other words, without that connection with others, there would be no sense of mattering, no complete sense of self, and no realized self-construct. Based on Rosenberg and McCullough’s earlier work (1981), Schlossberg’s work (see Schlossberg, Lynch, & Chickering, 1989) provided increased attention to the construct of mattering, particularly in higher education. Following her seminal work addressing life transitions (Schlossberg, 1984), Schlossberg (1989) extended this work to describe mattering as “the beliefs people have, whether right or wrong, that they matter to someone else, that they are the object of someone else’s attention, and that others care about them and appreciate them” (Schlossberg et al., 1989, p. 21) because of her perception that people in transition often feel marginalized, particularly college students.

Theoretical Framework of Mattering Although a detailed review of the literature involved with perceived mattering—both its

formation and its maintenance—is beyond the scope of this article, some background of the conceptual framework is needed to facilitate understanding of the construct validation process. The perception of mattering is emotionally and cognitively related to a sense of belonging to and with others, thereby reducing feelings of marginality or disconnect edness to the social context (Baumeister & Leary, 1995). Schlossberg (1989) referred to this perception as a sense of insignificance that may lead to a greater sense of overall existential meaninglessness for some people. In contrast, those who perceive a sense of social purpose and meaning for their life are more likely to feel significant to the key people in their lives (Marshall, 2001). Ascribing meaning to one’s interactions with others is based not only on the individual’s sense of valuing those interactions but also on how many interactions there are; it is a socially learned construct that has different meanings within various cultures (Hart, 1988). One study (Marshall, 2001) found that gender socialization was related to mattering to specific others (mother and friends versus father and friends), but further studies under different conditions seem to be warranted.

Applications of the Construct of Mattering Outside of Higher Education Mattering as a construct has been applied in the psychological and nursing literature. For example, Elliott, Colangelo, and Gelles (2005) defined mattering as the “belief that one makes a difference in the life of others” (p. 223). In looking at the factors that relate to suicidal ideation in early- to midadolescent youth, the researchers found that perception of mattering to others influences adolescents’ self-esteem, which then directly affects their levels of depression.


156 Measurement and Evaluation in Counseling and Development

Unfortunately, the measure of mattering developed by those authors considered only the relationships between individuals and their families and friends; as well, it linked the two subscales, thereby obscuring the differentiation of mattering type. The authors did not include mattering for college-aged students nor that within an academic setting. Marshall and a colleague assessed adolescents’ perceptions of mattering with respect to their relationships with friends and parents (Marshall, 2001) and in terms of romantic relationships (Mak & Marshall, 2004). Their work found support for the operationalization of mattering in terms of specific characteristics and, further, that global self-esteem seems to be a distinct topic from perceived mattering. Having a sense of purpose for life and a sense of relatedness with others was strongly connected with perceived mattering. Although Marshall’s sample (2001) included college and high school students, the development of the mattering inventory focused on parental and friend-based relationships rather than on connections within the educational settings. Marshall found that males and females viewed themselves as mattering more to their mothers than their fathers. However, compared to females, males tended to perceive themselves as mattering less to friends, which suggests some gender role socialization and varying expectations of friendships. In their study of a monocultural population, perceived mattering does appear to be related to psychosocial well-being. Overall, Marshall (2001) found positive attention from others to be related to a greater sense of mattering from those significant others and that peer rejection is negatively associated with perceived sense of mattering. Given that these significant relationships are developed over years (versus those initiated as students are entering college), it would be inappropriate to assume that the sense of

perceived mattering is the same for newer relationships (peer or otherwise), such as those that occur during the first term of college and/ or in more diverse college environments. Marcus (1991, unpublished study, as cited in Connolly & Myers, 2003; DeForge & Barclay, 1997) developed the General Mattering Scale as a part of a master’s thesis. The scale comprised five items: “How important do you feel you are to other people?” “How much do you feel other people pay attention to you?” “How much do you feel others would miss you if you went away?” “How interested are people generally in what you have to say?” and “How much do people depend on you?” For this study, Cronbach’s alphas were .73 for males, .75 for females, and .74 for the total sample. In a study with 199 homeless men in a midsize eastern seaboard city, DeForge and Barclay (1997) utilized the General Mattering Scale and reported a Cronbach’s alpha of .85. Using the General Mattering Scale with 82 employees, Connolly and Myers (2003) studied the connection between mattering and wellness to job satisfaction. These workers, in various levels and positions in business, were mostly Euro-Americans (82%), with the nextlargest group being that of African Americans (10%); 60% were married; and the mean age of the sample was 38.2. Using four of the five items of the General Mattering Scale, wellness and mattering partially explained the variance (r = .22, p < .05) of commonly reported variables related to job satisfaction (i.e., age, job tenure, education, skill variety, and feedback). However, the added variance of mattering and wellness was not a statistically significant predictor over and above the other established predictors of job satisfaction. In hypothesizing whether these results were due to sampling problems, the authors encouraged further research with the constructs to better understand the relationship of wellness and mattering with job satisfaction.



Mattering in Adult College Student Population Beyond bringing the concept of marginality to the attention of many in the higher education community, Schlossberg et al. (1990) developed the Mattering Scales for Adult Students in Postsecondary Education. This tool was developed to identify the mattering perceptions of adult college students. Scales were created to measure students’ perception of their experiences with faculty, administration, and peers—namely, the impact that the complexity of their lives had owing to potential role strain and advising. Schlossberg et al. developed the scale for students older than 25 years of age—so-called adult students. Although the researchers presented normative scales and noted that they used factor analysis to determine the validity of each scale, they included no factor analysis or principle components analysis results in the manual. In addition, this work (Schlossberg et al., 1990) was conducted approximately two decades ago, since which time the issues and perceptions of adult students are likely to have changed. Colleges and universities have adapted to a broader spectrum of students, so not all the items developed for the Mattering Scales for Adult Students in Postsecondary Education may even be valid for adult students of today, much less with traditional-age students. For example, Item 1 (which describes policy issues that affect courses taken in preparation for transfer), Item 17 (which addresses the time to complete the administrative tasks), and Item 20 (focusing on perceived inconvenient hours of operation for administration offices) may no longer be issues because of, respectively, more and readily available articulation agreements between 2- and 4-year institutions, the greater availability of online technology for administrative requirements, and an overall greater perception of the need of

higher education institutions to provide better customer service to all students by having more hours available for such service, provided in synchronous and asynchronous modes. Overall, the Mattering Scales for Adult Students in Postsecondary Education has provided a model for potential scales and items that may be measuring different aspects of mattering for current students enrolled in higher education. However, further factor analysis is needed to verify the tool and develop any other related tool to measure the perception of mattering in college students.

Why Does Mattering Matter? The impact of the construct of mattering has been studied—directly via single items or indirectly as a similar variation of the construct (e.g., sense of belonging)—in identifying its relationship to retention, persistence, and success in college. It has not, though, been directly measured in terms of an empirically developed and tested tool. With college students, the measured construct of mattering has not been meaningfully included in past retention/persistence studies (Hurtado & Carter, 1997). Not including this concept when developing a model explaining why students leave college may explain part of the relatively low levels of explained variance found in most studies (Pascarella & Terenzini, 2005). Although college personnel broadly espouse the importance of mattering for student retention, that is currently based on the face validity of the construct rather than on the reality of its measurement. Meaningful assessment of the construct would lead to empirical support of its importance and impact on students’ success in college. With that information, appropriate interventions to promote mattering could be developed, implemented, and assessed.



Sense of Belonging Anant (1966) described belongingness as one’s having a “sense of personal involvement in a social system so that persons feel themselves to be an indispensible and integral part of the system” (p. 21). Baumeister and Leary (1995) further indicated that belongingness is characterized by the need for frequent, nonaversive interactions within an ongoing relational bond, to maintain relationships in which the individual feels a part of something greater than himself or herself. This need is consistent with the higher education literature that has found that cocurricular and classroom relationships significantly affect a student’s decision to stay in or leave college (Pascarella & Terenzini, 2005; Tinto, 1993). Alford (1998) suggested that there is similarity across ethnic, racial, gender, and sexual preference lines in students’ desire to belong within the college setting; in particular, she described the even greater need to feel a sense of belonging for those who are commuters, living bifurcated lives, as compared to traditional, residential students. When effective social integration into the college community via the perception of belongingness occurs for the student, there is a direct positive impact on student retention (Alford, 1998); as such, a student’s failing to achieve a positive sense of belonging may lead to more negative consequences in terms of his or her experiences and outcome (Baumeister & Leary, 1995; Hausmann, Schofield, & Woods, 2007; Johnson et al., 2007).

Need for a Valid Measure of Mattering for a Cross-Section of College Students It is in the first year of college that the greatest attrition of students occurs, particularly

within the first 6 weeks (Tinto, 1993). Students’ involvement and connection during that time has been shown to increase the retention rates of first-year students in their initial term. However, the intrapersonal impact of students’ involvement and connection is not yet completely understood. Interventions intended to increase the persistence of first-year students by increasing their social and academic integration have been developed and implemented widely across U.S. higher education institutions (e.g., learning communities and firstyear seminars). Students who participate in these type of experiences—even with great variation in their operationalization—have greater overall success in their first term of college (Tinto, 1993). What is not well researched as yet is the impact of psychosocial factors, such as the perceived experience of a greater sense of mattering (Barefoot, Warnock, Dickinson, Richardson, & Roberts, 1998; Upcraft, Gardner, & Barefoot, 2004). To this end, the purpose of this study is to further define the construct of mattering as applied in higher education settings. Specifically, this article addresses the development and initial validation of the College Mattering Inventory on a sample of university and community college students. In it, we first describe the construction and initial factor structure of the inventory, as determined by exploratory factor analysis (Study 1); then, we describe the results of a confirmatory factor analysis and invariance analysis (Study 2). Our construct definition of mattering relies heavily on the theoretical propositions of Rosenberg and McCullough (1981) and Schlossberg (1989), and it integrates those elements that they believe are essential to establishing intrapersonal mattering: the feeling that others are dependent on us, are interested in us, are concerned with our fate, care about us, and appreciate us. Unlike these authors, however, we



believe that mattering is contextual and object specific. For example, whereas a general feeling of mattering to the college may be indicative of students’ general place within the university or college, it is their relationships to specific others (e.g., faculty, counselors/advisors, other students) that may best attest to their feelings of mattering or not mattering.

Method Participants and Procedures Data collection took place at two Southern California institutions: a community college (Carnegie category: associate level, public, urban serving, single campus) and a university (Carnegie category: master’s level, large, urban). Both may be characterized as large urban multiply diverse commuter colleges. After approval was secured from the two schools’ institutional research boards, each institutional research office generated a database of randomly selected, currently enrolled students. The database contained e-mail addresses and select demographic characteristics. E-mail invitations were sent, asking students to participate in a 30-minute online survey during the 11th week of a 16-week semester. Students were informed that participation was voluntary; as incentive, they were offered the opportunity to enter a drawing for an iPod Shuffle. Two e-mail reminders were sent 1 week apart. The online survey was closed to participants at the conclusion of week 14. Our initial sample consisted of 3,139 participants who completed the mattering inventory (described below) in its entirety. The sample was randomly divided into two groups but stratifying for gender, race/ ethnicity, and college attended. The first subsample, for the exploratory factor analysis, consisted of 1,578 students (50.3%). The

second subsample, for the confirmatory factor analysis, consisted of 1,561 students (49.7%). Table 1 presents the demographic characteristics of our final sample for each study conducted. Overall participants included 1,199 males (38.2%) and 1,940 females (61.8%). With respect to ethnicity, 1,115 (35.5%) were White; 744 (23.7%), Latino/Latina; 580 (18.5%), Asian/ Pacific Islander; 196 (6.2%), other; 167 (5.3%), multiracial; 155 (4.9%), Black/ African American; 18 (0.6%), American Indian; and 164 (5.2%) declined to state. Participants ranged in age from 18 to 87, with a mean of 24.4 (SD = 7.59). Community college students were enrolled in an average of 10.4 units (SD = 4.29); the university students, in 12.6 (SD = 4.12).

Instrumentation College Mattering Inventory. As stated in our review of the literature, the construct of mattering within the higher education field has been studied and measured on a limited basis, primarily with nontraditional (i.e., older) college students. The purpose of this study was to further attempt to measure the construct of mattering with a diverse group of students and to determine the factor structure of these items. Following the literature review and, in particular, the works of Rosenberg and McCullough (1981) and Schlossberg (1989), we proposed a construct definition of mattering that incorporated the following themes: being the object of attention of others (faculty, counselors/ advisors, students), perception of support in various student endeavors by others (faculty, counselors/advisors, students), supportive learning environment, sense of fit within the college, and perceived marginality owing to personal characteristics. Fifty-five items based on these themes were generated and incorporated into the



Table 1 Demographic Characteristics for Final Sample by Student Type Characteristic

Study 1: EFA (n = 1,578) n

%

Study 2: CFA (n = 1,561) n

%

Total (n = 3,139) n

College setting Community college 1,105 70.0 1,095 70.1 2,200 University 473 30.0 466 29.9 939 Ethnicity American Indian 10 0.60 8 0.50 18 Asian/Pacific Islander 291 18.40 289 18.50 580 Black/African American 78 4.90 77 4.90 155 Latino/Latina 373 23.60 371 23.80 744 White 559 35.40 556 35.60 1,115 85 5.40 82 5.30 167 Multiracial Other 99 6.30 97 6.20 196 83 5.30 81 5.20 164 Decline to state Marital status Single, never married 1,277 80.90 1,281 82.10 2,558 152 9.60 156 10.00 308 Married Divorced 62 3.90 56 3.60 118 1 0.10 1 0.10 2 Widow/widower In a domestic partnership 48 3.00 42 2.70 90 38 2.40 25 1.60 63 Decline to state Grade point average 3.50–4.00 511 32.60 479 30.80 990 3.00–3.49 498 31.70 511 32.80 1,009 2.50–2.99 313 19.90 321 20.60 634 177 11.30 187 12.00 364 2.00–2.49 < 2.00 68 4.30 51 3.30 119 Semesters attending (community college sample only) 145 13.20 151 13.90 296 Only this semester so far 2 semesters 301 27.40 284 26.10 585 3–4 semesters 327 29.80 355 32.60 682 185 16.80 173 15.90 358 5–6 semesters 7 ≤ semesters 140 12.80 126 11.60 266 Class standing (university sample only) Freshman 69 4.40 64 4.10 133 Sophomore 91 5.80 76 4.90 167 Junior 151 9.60 164 10.50 315 Senior 162 10.30 162 10.40 324 English as first language Yes 961 60.90 941 60.30 1,902 No 617 39.10 620 39.70 1,237

% 70.00 30.00 0.60 18.50 4.90 23.70 35.50 5.30 6.20 5.20 81.50 9.80 3.80 0.10 2.90 2.00 31.70 32.30 20.30 11.60 3.80 13.50 26.70 31.20 16.40 12.20 4.20 5.30 10.00 10.30 60.60 39.40

Note: CFA = confirmatory factor analysis; EFA = explanatory factor analysis.

online survey. The order of the items was randomly determined. Response format was based on a Likert-type scale ranging from 1

(not at all) to 5 (very much). Table 2 presents the 55 items, along with initial descriptive statistics. Seventeen items were negatively



Table 2 Initial College Mattering Inventory Items and Descriptive Statistics (n = 1,577) Item

M

M02. There are people at the college who are determined to see me succeed. 3.41 M03. Instructors generally do not care about students’ well-being.a 3.82 M04. I often feel my instructor(s) care more about other things than me as a student.a 3.81 M05. Knowing that other people at the college care for me motivates me to do better. 3.82 M06. There are people at the college that sincerely appreciate my involvement as a student. 3.42 M07. It is comforting to know that my contributions are valued by my instructors. 3.87 M08. Sometimes I feel that I am not interesting to anyone at the college.a 3.72 M09. Instructors sometimes tell me how much they appreciate my comments and 3.10 participation in class. M10. Students in my classes show interest in me because I make good contributions. 3.15 M11. Most of my professors would not miss me if I suddenly stopped attending classes.a 3.35 M12. I believe that my counselor(s) would miss me if I suddenly stopped attending college. 2.28 M13. Based on my experience so far, there seem to be many people on campus 3.06 who wish to see me succeed. M15. It is comforting to know that my contributions are valued by other students. 3.31 M16. People on campus are generally accepting of me. 3.76 M17. My counselor is generally receptive to what I have to say. 3.41 M18. I sometimes feel alone and isolated at the college because of my gender.a 4.62 M19. I sometimes feel my instructor(s) want me to hurry up and finish speaking.a 4.30 M20. Other students rely on me for support. 2.68 M21. If I stopped attending college, most of my instructors would be disappointed. 2.88 M24. If I stopped attending college, my counselor(s) would be disappointed. 2.51 M25. Sometimes my instructors simply do not listen to what I have to say.a 4.16 M26. Other students are happy for me when I do well in exams or projects. 3.30 M27. My instructors are sensitive to my non-school responsibilities. 2.75 M28. Sometimes I get so busy with my off-campus activities that 2.86 I distance myself from others at the college.a M29. Instructors appear genuinely happy when I do well in class. 3.86 M30. Sometimes I feel that no one at the college notices me.a 3.74 M31. Sometimes I feel alone at the college.a 3.55 M32. Sometimes I get so wrapped up in my personal problems that 3.32 I isolate myself from others at the college.a M33. Other students in my classes would miss me if I suddenly went away. 2.76 M34. I often feel isolated when involved in student activities (e.g., clubs, events).a 3.89 M35. I receive thoughtful and timely comments on my work from my instructors. 3.42 M37. Sometimes other students simply do not listen to what I have to say.a 4.02 M38. My instructors sometimes ignore my comments or questions.a 4.26 M39. If I had a personal problem, I believe that counselors would be willing 3.35 to discuss it with me. M40. People on campus seem happy about my accomplishments. 3.13 M41. It is good to know that others at the college care about my well-being. 3.65 M42. When in groups, other students tend to rely on my contributions. 3.41 M43. If I asked my instructors for help, I am confident that I would receive it. 4.06 M44. Students in my classes have shown interest in my personal well-being. 3.10 M45. There are enough social or academic opportunities for me to get connected 3.41 with others at the college. M48. There are people on campus who are sad for me when I fail in something 2.77 I set out to do.

SD 1.16 1.08 1.10 1.17 1.19 1.06 1.21 1.26 1.11 1.24 1.31 1.19 1.14 0.94 1.22 0.85 0.97 1.12 1.29 1.33 0.97 1.13 1.22 1.30 1.02 1.25 1.36 1.32 1.20 1.16 1.09 1.00 0.94 1.33 1.13 1.12 1.08 0.97 1.11 1.17 1.21

(continued) Downloaded from http://mec.sagepub.com by Esau Tovar on September 18, 2009


Table 2 (continued) Item M49. I often feel that I do not belong at this college.a M50. It is important for me that my professors notice my presence or participation in class. M51. Some students are dependent on my guidance or assistance to help them succeed. M52. Some people on campus are disappointed in me when I do not accomplish all I should. M53. People on campus are generally supportive of my individual needs. M54. There are people at the college who are genuinely interested in me as a person. M55. I sometimes feel alone and isolated because of my race/ethnicity.a M56. I often feel socially inadequate at school.a M57. Most of my instructors know my name. M58. If I had a personal problem, I believe that instructors would be willing to discuss it with me. M59. Counselors at the college generally show their concern for students’ well-being. M60. My opinions are generally valued at the college. M61. There are people at the college who are concerned about my future. M62. I sometimes feel pressured to do better because people at the college would be disappointed if I did not.

M

SD

4.06 3.62 2.81 2.51

1.17 1.22 1.18 1.14

3.02 3.33 4.42 3.97 3.82 3.31

1.09 1.15 1.03 1.21 1.18 1.27

3.27 3.18 3.28 2.46

1.25 1.09 1.18 1.23

a. Item reverse-scored.

stated, to decrease the likelihood of response acquiescence (Jayanti, McManamon, & Whipple, 2004), and were reverse-scored before data analysis. Thus, higher scores on resulting scales were thought to measure an increased sense of mattering versus marginality and so resulted in the current version of the College Mattering Inventory. Note that reverse-worded items have been found to decrease the reliability and internal consistency of scales and sometimes lead to a form of method factor in which negative items result in one factor and positive items in another (Hazlett-Stevens, Ullman, & Craske, 2004; Woods, 2006). Sense of Belonging Scale. The Sense of Belonging Scale (Hoffman et al., 2002– 2003) consists of 26 items distributed among five subscales assessing aspects of student belongingness in a college setting: perceived peer support (8 items; α = .87), perceived faculty support/comfort (6 items; α = .87), perceived classroom comfort (4 items; α = .90), perceived isolation (4 items; α = .82) and

empathetic faculty understanding (4 items; α = .85). Items are measured using a Likerttype scale anchored by 1 (completely true) to 5 (completely untrue). A sample item for perceived classroom comfort is “It is difficult to meet other students in class.” Although the scale shows promise and is characterized by moderate degrees of internal consistency, its factorial structure remains to be confirmed. The Sense of Belonging Scale was derived using principal components analysis. Lower scores on the scale are indicative of a higher sense of belonging. Other measures. Although not central to this article, the online survey incorporated additional exploratory measures of interpersonal relatedness to institutional agents (i.e., instructors, counselors) and quality of transition assistance and support to college (which we developed). Additional sociodemographic items were included, as were six validity items directing respondents on how to answer. Surveys were deemed accurate only when these items were answered in the



prescribed manner. A sample validity item within the mattering inventory is “Please mark #4 as your response for this row.”

Statistical Analyses An exploratory factor analysis was conducted in Study 1 using principal axis factoring with varimax rotation in SPSS 17.0. The Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett’s test of sphericity were used to determine if factor analysis was appropriate for the cases and variables in the study. Item and factor retention decisions were guided by the following criteria: A minimum communality of .30 must be established for each item retained; all items had to have a minimum loading of .40 on a single factor; items loading on any given factor must be theoretically related to the other items; Horn’s parallel analysis (1965) and Velicer’s minimum average partial test (1976) would guide the factor extraction decision, with the understanding that a minimum of three items should compose each factor. The scree test and eigenvalue greaterthan-one rule were used on an ad hoc basis because these methods are not recommended for stand-alone use (Lee & Comrey, 1979; Wood, Tataryn, & Gorsuch, 1996). Based on a number of studies, parallel analysis has been determined to be most accurate at pinpointing the number of factors to retain (Thompson & Daniel, 1996; Velicer, Eaton, & Fava, 2000)—followed by the minimum average partial test—particularly, when compared to the eigenvalue greater-than-one test or the scree test. Statistical analyses for Study 2 were guided by the derived factor structure stemming from the exploratory factor analysis. We decided a priori to pursue model respecification if the hypothesized model did not adequately fit the data. Multiple indices of fit were used to assess model fit. Once an

acceptable model was identified, a confirmatory factor analysis for males and females as single groups was pursued, leading to the assessment of structural invariance. According to the guidelines offered by Byrne (2001) and Dimitrov (2006), the focus on invariance was to test increasingly restrictive models commencing with the invariance of factor loadings. Given invariant parameters, the analysis continued by imposing additional equality constraints in error covariances and factor covariances. Where statistically significant differences were found, the parameter in question was allowed to be freely estimated. The baseline model that we used was respecified Model 2.

Results Study 1: Exploratory Factor Analysis Principal axis factoring with varimax rotation was performed on the 55 proposed mattering items. Before analyzing the results, we assessed the adequacy of our data to ensure that factor analysis was appropriate. The Kaiser–Meyer–Olkin measure of sampling adequacy (for the set and for individual items) and Bartlett’s test of sphericity were reviewed, indicating that factor analysis was appropriate. As stated earlier, parallel analysis and the minimum average partial test were used as our primary indicators for the number of factors to extract, in addition to their theoretical salience and factor interpretability. Accordingly, parallel analysis (95th percentile) indicated that seven factors be retained, whereas the minimum average partial test suggested six to seven factors. These procedures were carried out using O’Connor’s SPSS programs (2000). We next examined communalities and items cross-loading on more than one factor. After five iterations, items M11, M18, M28,



M45, M50, M55 were removed owing to their having communalities less than .30. Items M02, M13, M15, M33, M37, M41, M43, M44, M60 were removed owing to their cross-loading on two factors, whereas Items M03, M09, M16, M21, M27, and M57 were removed because they had loadings of less than .40. It was clear by the sixth iteration that six factors with 34 items be retained, which together accounted for 48.2% of the variance postrotation. Communalities ranged from .30 to .68 (mean h2 = .48) and all items loaded on a single theoretically consistent and interpretable factor. The final Kaiser–Meyer–Olkin measure of sampling adequacy was .94, and Bartlett’s test of sphericity was significant, χ2(666, n = 1,577) = 26,301, p < .001. Table 3 presents factor loadings for the retained items, along with communalities. The first factor consisted of eight items and was labeled General College Mattering; it accounted for 9.94% of the variance. Items in the factor reflect students’ perceptions that there are individuals at the college, which is interested in their success and well-being. This interest may be expressed affectively or behaviorally—positively and negatively—by demonstrating concern for students, lending attention to their actions, noticing their presence or absence, appearing genuinely happy with their accomplishments and achievements, and expressing disappointment when warranted. The second factor, consisting of seven items, was labeled Mattering Versus Marginality and so accounted for 9.52% of the variance. Items in this factor reflect perceptions of belongingness, fit, acceptance, or rejection. The third factor consisted of five items; it was labeled Mattering to Counselors (or advisors); and it accounted for 8.34% of the variance. These items reflect counselors’ support, attention, receptivity, and concern for students. The fourth factor, Mattering to

Instructors, consisted of seven items and accounted for 7.85% of the variance. Items in the factor reflect students’ perceptions of support and attention given to them by instructors, inside and outside the classroom. The fifth factor, consisting of four items, was labeled Mattering to Students, and it accounted for 7.43% of the variance. Items in this factor reflect students’ perception that other students at the college pay attention to them and depend or rely on them academically. The last factor, consisting of three items, was labeled Perception of Value, and it accounted for 5.12% of the variance. Items in the factor reflect the perception that others at the college value students’ contributions and involvement.

Study 2: Confirmatory Factor Analysis and Multigroup Invariance Analysis Study 2 encompassed a confirmatory factor analysis on the factor structure of the College Mattering Inventory as derived through exploratory factor analysis (presented in Table 3). The hypothesized model consisted of six factors and 34 items. Maximum likelihood estimation was used to perform the confirmatory factor analysis. Given the lack of consensus for preferred indices of fit (e.g., Bentler, 1990; Byrne, 2001; Hu & Bentler, 1999; Kline, 1998; MacCallum, Browne, & Sugawara, 1996; Ullman, 1996), model fit was assessed with multiple goodness-of-fit indices, residual error terms, modification indices, and their accompanying expected parameter change (Arbuckle & Wothke, 1999). Given that the chi-square statistic is significantly influenced by large sample sizes (Byrne, 2001), other goodness-of-fit indices were examined to evaluate fit. These indices included the ratio of chi-square to degrees of freedom (χ2/df), the normed fit index (NFI), the



Table 3 Principal Axis Factoring With Varimax Rotation Solution for the College Mattering Inventory

Factor

Item

1

M48. There are people on campus who are sad for me when I fail in something I set out to do. M52. Some people on campus are disappointed in me when I do not accomplish all I should. M53. People on campus are generally supportive of my individual needs. M40. People on campus seem happy about my accomplishments. M62. I sometimes feel pressured to do better because people at the college would be disappointed if I did not. M61. There are people at the college who are concerned about my future. M54. There are people at the college who are genuinely interested in me as a person. M26. Other students are happy for me when I do well in exams or projects. M31. Sometimes I feel alone at the college. M30. Sometimes I feel that no one at the college notices me. M56. I often feel socially inadequate at school. M08. Sometimes I feel that I am not interesting to anyone at the college. M32. Sometimes I get so wrapped up in my personal problems that I isolate myself from others at the college. M34. I often feel isolated when involved in student activities (e.g., clubs, events). M49. I often feel that I do not belong at this college. M24. If I stopped attending college, my counselor(s) would be disappointed. M59. Counselors at the college generally show their concern for students’ well-being. M17. My counselor is generally receptive to what I have to say. M12. I believe that my counselor(s) would miss me if I suddenly stopped attending college. M39. If I had a personal problem, I believe that counselors would be willing to discuss it with me. M25. Sometimes my instructors simply do not listen to what I have to say. M38. My instructors sometimes ignore my comments or questions. M35. I receive thoughtful and timely comments on my work from my instructors. M04. I often feel my instructor(s) care more about other things than me as a student.

2

3

4

5

6

h2

.678

.123 .203 .107

.235 .119 .60

.628

.001 .199 –.015

.246 .077 .50

.589

.216 .250 .232

.270 .185 .62

.567

.220 .276 .211

.266 .260 .63

.558 –.047 .239 .034

.124 .161 .41

.538

.155 .376 .247

.152 .260 .61

.527

.305 .157 .186

.318 .180 .56

.473

.135 .122 .095

.242 .150 .35

.156 .170 .000 .103

.798 .731 .660 .602

.080 .168 .098 .182

.023

.583 .052 .133

.134 .025 .38

–.002

.579 .032 .181

.023 –.009 .37

.052 .050 .002 .105

.090 .112 .091 .161

.013 .076 .010 .103

.68 .61 .45 .45

.130 .239

.471 .106 .255 –.033 .143 .34 .044 .759 .045 .148 .062 .66

.211

.103 .720 .282 –.007 .106 .66

.145

.063 .602 .139

.025 .161 .43

.193

.042 .598 .032

.117 .094 .42

.164

.073 .570 .287

.000 .074 .44

.026

.177 .086 .656 –.039 .040 .47

.013

.182 .060 .656 –.072 –.026 .47

.146

.078 .176 .503

.077

.143 .131 .498 –.009 .106 .30

.261 .188 .41

(continued)



Table 3 (continued)

Factor

Item

1

2

3

4

M19. I sometimes feel my instructor(s) want me to hurry up .031 .255 .087 .465 and finish speaking. M29. Instructors appear genuinely happy when I do well .308 .058 .203 .463 in class. M58. If I had a personal problem, I believe that instructors .199 .144 .327 .458 would be willing to discuss it with me. M51. Some students are dependent on my guidance .340 .097 .086 –.089 or assistance to help them succeed. M42. When in groups, other students tend to rely on .204 .107 .019 –.007 my contributions. M20. Other students rely on me for support. .291 .138 .088 –.115 M10. Students in my classes show interest in me because .214 .256 .010 .032 I make good contributions. M07. It is comforting to know that my contributions are .151 .046 .170 .205 valued by my instructors. M06. There are people at the college that sincerely appreciate .314 .194 .193 .144 my involvement as a student. M05. Knowing that other people at the college care for me .280 .000 .160 –.022 motivates me to do better. 10.76 3.32 2.71 1.68 Eigenvalue before rotation Percentage of variance before rotation 29.09 8.96 7.34 4.54 Percentage of variance after rotation 9.94 9.52 8.35 7.85

5

6

h2

–.096 –.025 .30 .274 .305 .52 .162 .171 .43 .640 .025 .55 .630 .085 .46 .605 .099 .50 .543 .298 .50 .208 .671 .59 .201 .585 .58 .071 .525 .39 1.22 1.18 3.29 3.19 7.43 5.12

Note: Bold font indicates items per factor.

comparative fit index (CFI), the Tucker– Lewis index (TLI), and the root mean square error of approximation (RMSEA). Table 4 reports the optimal values for these indices. We sought a simple structure of the College Mattering Inventory based on the confirmatory factor analysis model assessment guidelines provided by Byrne (2001) and Elliott, Kao, and Grant (2004) in the validation of their interpersonal mattering index. The first congeneric variable for each factor was fixed to a 1.0 loading and to a zero loading on all other factors. The error terms associated with the observed variables were assumed to be independent of all other error terms. Cross-loading items on two or more factors were deemed a threat to discriminant validity and were deleted from further consideration when the modification index

value exceeded 15 units and the expected parameter change was larger than .200. These deletions were also guided by mattering theory. The same guidelines were followed for error disturbances, whose modification indices and expected parameter change suggested that allowing them to correlate might improve model fit. MacCallum (1995) has cautioned that substantive and theoretical consideration must guide error correlations. Error covariances often may derive from characteristics of the items in question (e.g., tapping into similar constructs; redundancy) or the respondents (Aish & Jöreskog, 1990).

Evaluation of Model Fit We now present the results of the confirmatory factor analysis, first discussing



Table 4 Summary of Goodness-of-Fit Indices for Alternate Models of the College Mattering Inventory

Goodness-of-Fit Measures χ2

Model

df

χ2/df

NFI

CFI

TLI

Optimal values — — < 3.0a > .90b > .95c > .95d Hypothesized six-factor model 3,550.12 512 6.93 .85 .87 .86 Respecified six-factor Model 1: 1,917.70 386 4.97 .91 .92 .91 Items 10, 29, 35, 49 deleted; error correlations allowed: 12 and 24, 38 and 25, 52 and 62, and 58 and 39. Respecified six-factor Model 2: 1,558.09 359 4.34 .92 .94 .93 Items 10, 29, 35, 49, 58 deleted; error correlations allowed: 12 and 24, 38 and 25, 52 and 62. Gender Male 811.06 359 2.26 .89 .94 .93 Female 1,205.33 359 3.36 .90 .93 .92 Gender Invariance: Model Description (Model Number) 1. Hypothesized model: baseline (Model 1) 2. Factor loadings, variances, and covariances constrained equal (Model 2) 3. Factor loadings constrained equal (Model 2a) 4. Factor loadings constrained equal, covariance e52-e62 constrained equal (Model 2b) 5. Factor loadings constrained equal, covariance e52-e62 freely estimated, covariance e12-e24 constrained equal (Model 2c) Model 2c with covariance e25-e38 constrained equal (Model 3) Model 3 with covariance GCM and MVM (Model 3a) Model 3a with covariance GCM and MC constrained equal (Model 3b) Model 3b with covariance with all factors (Model 3c) Model 3c with covariance MVM and MC (Model 3d) Model 3d with covariance MVM and MI (Model 3e)

Comparative Model

RMSEA < .06d .06 .05

.05

.05 .05

χ2

df

∆χ2

∆df

p

2,016.42

718

—

—

—

Model 1

2,085.70

765

69.28

47

.019

Model 1

2,030.82

741

14.41

23

.915

Model 2a

2,040.80

742

9.98

1

.002

Model 2a

2,032.37

742

1.55

1

.213

Model 2c

2,032.42

743

0.05

1

.830

Model 3

2,032.85

744

0.44

1

.509

Model 3a

2,033.16

745

0.30

1

.582

Model 3b

2,037.11

748

3.95

3

.266

Model 3c

2,037.48

749

0.37

1

.542

Model 3d

2,042.94

750

5.46

1

.019

—

(continued)



Table 4 (continued) Gender Invariance: Model Description (Model Number)

Comparative Model

Model 3c with covariance MVM and MI freely estimated, covariance MVM and MS constrained equal (Model 3f) Model 3f with covariance MVM and VAL constrained (Model 3g) Model 3g with covariance MVM and VAL freely estimated, covariance MC with MI, MS, VAL constrained equal (Model 3h) Model 3h with covariance MI and MS, MI and VAL constrained equal (Model 3i) Model 3i with covariance MS and VAL (Model 3j) Model 3j with variance CGM constrained equal (Model 3k) Model 3k with variance MVM constrained equal (Model 31) Model 3k with variance MVM freely estimated, variance MC constrained equal (Model 3m) Model 3k with variance MVM, MC freely estimated and MI constrained equal (Model 3n) Model 3k with variance MVM, MC, MI freely estimated, MS constrained equal (Model 3o) Model 3o with variance VAL constrained equal (Model 4)

χ2

df

∆χ2

∆df

p

Model 3d

2,037.49

750

0.01

1

.944

Model 3f

2,045.30

751

7.81

1

.005

Model 3f

2,044.48

753

6.99

3

.072

Model 3h

2,044.49

755

0.02

2

.992

Model 3i

2,044.67

756

0.18

1

.671

Model 3j

2,045.82

757

1.14

1

.285

Model 3k

2,049.75

758

3.94

1

.047

Model 3k

2,052.67

758

6.85

1

.009

Model 3k

2,050.91

758

5.10

1

.024

Model 3k

2,048.34

758

2.53

1

.112

Model 3o

2,050.59

759

2.242

1

.134

Note. NFI = normed fit index; CFI = comparison fit index; TLI = Tucker-Lewis index; RMSEA = root mean square error of approximation; GCM = General College Mattering; MVM = Mattering Versus Marginality; MC = Mattering to Counselors; MI = Mattering to Instructors; MS = Mattering to Students; VAL = Perception of Value. a. Kline (1998). b. Bentler (1992). c. Bentler (1990). d. Hu and Bentler (1999).

outcomes for the hypothesized model, then the two respecified models. All analyses were conducted using AMOS 17.0 (SPSS, Chicago, IL). Hypothesized six-factor model. Results of the initial confirmatory factor analysis

suggested that the hypothesized model was of poor fit for the data and should thus be rejected, χ2(512, n = 1,561) = 3,550.12, p < .001, χ2/df = 6.93, NFI = .85, CFI = .87, TLI = .86, RMSEA = .06. An examination of the standardized residuals, modification indices, and their expected parameter change



further supported the model’s significant misfit. The standardized residual covariance matrix included multiple residual values in excess of 2.58, which are considered large and indicative of model misfit (Jöreskog & Sörbom, 1988). The largest residual of 8.06 was between Indicators M10 and M35. Large modification indices suggested the crossloading of four items on multiple factors (Items M10, M29, M35, and M49; modification indices ranged from 19.09 to 93.67), and large error covariances between Items M12 and M24, M38 and M25, M52 and M62, and M58 and M39 (modification indices ranged from 60.14 to 289.16). Given the large standardized residuals, the large error covariance, and the cross-loading described, respecification of the model was pursued. Respecified Model 1. Post hoc model adjustment was pursued in an attempt to develop a better-fitting model. Table 4 pre sents fit statistics for the respecified model, along with the originally hypothesized model for comparison. The respecified model encompassed the deletion of items M10, M29, M35, and M49 from their respective factors. Upon consideration of item content for indicators with large error covariances as described previously, we determined that a degree of content redundancy (Aish & Jöreskog, 1990) was likely; therefore, we allowed errors to correlate. Results indicated significant improvement over the originally hypothesized model, χ2(386, n = 1,561) = 1,917.70, p < .001, χ2/df = 4.97, NFI = .91, CFI = .92, TLI = .91, RMSEA = .05. The examination of standardized residuals, modification indices, and expected parameter change further indicated that additional modifications might be possible. Item M58 was found to cross-load with the General College Mattering factor (modification index = 114.97, expected parameter change = 0.35). Respecification was pursued one last time, as noted next.

Respecified Model 2: Accepted model. An additional post hoc model adjustment was carried out by deleting cross-loading Item M58. This action resulted in the elimination of the error correlation between Items M58 and M39. Fit statistics indicated significant improvement over respecified Model 1, χ2(359, n = 1,561) = 1,558.09, p < .001, χ2/df = 4.34, NFI = .92, CFI = .94, TLI = .93, RMSEA = .046 (90% confidence interval = 0.044–0.049). Additional model adjustments were not pursued, in the interest of parsimony and modification indices with expected parameter change less than .200. Respecified Model 2 is proposed as the best-fitting model, and Figure 1 presents the standardized parameter estimates. Accord ingly, the factor structure of the College Mattering Inventory consists of six factors and 29 items: General College Mattering (8 items), Mattering Versus Marginality (6 items), Mattering to Counselors (5 items), Mattering to Instructors (4 items), Mattering to Students (3 items), and Perception of Value (3 items).

Invariance Testing for Gender Following the confirmatory factor analysis discussed above, two confirmatory factor analyses were conducted for males and females using the same factor structure. As presented in Table 4, fit indices supported the structure of the general model. Table 4 also presents a summary of goodness-of-fit statistics for the assessment of gender invariance in the measurement and structural models. To determine whether the increasingly restrictive models were significantly different (not invariant) from the baseline model—no constraints; χ2(718, n = 1,561) = 2,016.42—we compared their respective chi-square values. The initial invariance analysis encompassed a model in which all factor loadings, variances, and covariances were constrained equal; however, this model was of poor fit to the data (p < .05). Using



Figure 1 Standardized Coefficients for the Revised Factor Structure of the College Mattering Inventory Based on Gender Invariance—Males and (Females) e48

.50 (.48) .41 (.35)

e52

.19(.32)

e53

.64 (.61) .68 (.64)

e40 e62

.31 (.29) .61 (.59)

e61 e54

.59 (.58) .37 (.37)

e26

M48

.71 (.69)a

M52

.64 (.59)

M53 M40 M62 M61 M54

M31 .64 (.68)

M30

e30 .38 (.41) e56 e08 e32

.42 (.47) .28 (.30) .28 (.30)

e34

.56 (.54) .78 (.77)

GCM

.77 (.76) .60 (.60)

M26

.62 (.69) e31

.80 (.78) .82 (.80)

M56

.54 (.50)

a

.79 (.83) .80 (.82) .61 (.64) .65 (.69)

.67 (.62)

MVM

M08

.53 (.52)

M32

.53 (.75)

M34

.38 a (.34)

.34 (.29) .43 (.49)

M24

e24

.47 (.48)

e59 e17 e12 e39

.22 (.20)

e25

.72 (.73) .46 (.48) .24 (.27) .52 (.56)

.48 (.49) .50 (.54)

e38 .31 (.35) e04 e19

.36 (.38)

M59 M17 M12

.66 (.70)a .85 (.86) .68 (.69)

MC

.49 (.52)

.70 (.70)

.72 (.75)

M39

.46 (.39)

a

M25 M38 M04

.69 (.70)

.28 (.26)

.66 (.64)

e42 e20

.36 (.38) .56 (.57)

.44 (.44) e07

.56 (.59) .60 (.61)

.21 (.19)

MI

M19

M51 M42

.44 (.44)

M07 M06

e05

.29 (.29)

.81 (.80)a .60 (.62) .75 (.76)

.30 (.27)

MS .43 (.43)

M20

.67 (.68) e06

.31 a (.44)

.71 (.74)

.00

e51

.62 (.62)

.45 (.50)

.66 (.67)a .82 (.82) .54 (.54)

VAL

M05

Note: All factor loadings are statistically significant (p < .001). GCM = General College Mattering, MVM = Mattering Versus Marginality, MC = Mattering to Counselors, MI = Mattering to Instructors, MS = Mattering to Students; VAL = Perception of Value. a. First congeneric loading for each factor set to 1.0. Parameter not freely estimated.



Byrne’s guidelines (2001), we next tested a model where all factor loadings were constrained equal, which was not statistically significant (p > .05), thus signaling that factor loadings were gender invariant. As Table 4 notes, 19 additional models were conducted, varying in their degree of constraints imposed. The final model (Model 4) resulted in a chisquare value of 2,050.59 with 759 degrees of freedom. When this value was compared to the initial group model (Model 1), the resulting chi-square difference was 34.17 (df = 41; p > .05). Indices of fit also supported the adequacy of the model, χ2/df = 2.70, NFI = .90, CFI = .93, TLI = .93, RMSEA = .033 (90% confidence interval = 0.031–0.035). To summarize the model, equality constraints were imposed on all but the following parameters (which were freely estimated): error covariance e52 and e62; covariance Mattering Versus Marginality and Mattering to Instructors, covariance Mattering Versus Marginality and VAL; variance Mattering Versus Marginality, variance Mattering to Counselors, and variance Mattering to Instructors. Hence, the final model resulted in partial invariance. Given the lack of hard rules on the tenability of partial invariance (e.g., Dimitrov, 2006), we proceeded to assess mean differences on the factors encompassing the College Mattering Inventory.

Latent Mean Structures Analysis Using Model 4 above, we conducted latent mean structures analysis to determine if the scores on the latent constructs of the mattering inventory significantly differed by gender. Males were coded as the reference group. Results indicated that the mean differences for females were not statistically significant for all six factors (p > .05): General College Mattering = .058, Mattering Versus Marginality = –.013, Mattering to Instructors = .012, Mattering

to Counselors = –.06, Mattering to Stu dents = –.026, VAL = .004.

Internal Consistency Reliability Based on the structure derived from the confirmatory factor analysis discussed above, six subscales were constructed using the corresponding items. In addition, a Total Mattering Scale was derived on the basis of the 29 retained items. The Cronbach’s alpha coefficient of internal consistency reliability for the scales was found to be as follows: Total Mattering Scale, α = .91; General College Mattering, α = .89; Mattering Versus Marginality, α = .83; Mattering to Counselors, α = .84; Mattering to Instructors, α = .76; Mattering to Students, α = .77; and Per ception of Value, α = .72. Table 5 presents means, standard deviations, and intercorrelations for the scales. For reference, Table 6 provides means and standard deviations for gender and race/ethnicity. Future research should assess whether the factor structure of the College Mattering Inventory is race/ ethnicity invariant. Generally speaking, Asian students appeared to have lower perceptions of mattering on most scales. Black/ African American students, however, expressed higher levels of mattering (versus marginality), mattering to counselors, and perception of value.

Convergent Evidence of Validity Convergent evidence of the external aspect of validity of the College Mattering Inventory was assessed using the Sense of Belonging Scale (Hoffman et al., 2002–2003). As a reminder, the scale was administered to only the university sample, and only Study 2 participants were incorporated into this analysis (n = 466). Given that lower scores on the scale indicate a greater degree of perceived sense of belonging whereas higher scores on


172


SD

20.00 7.65 15.73 5.50 9.57 4.30 12.74 3.98 10.24 3.82 68.28 15.38

96.98 18.03 23.99 6.99 21.91 5.58 14.55 5.00 16.38 3.11 9.05 2.78 11.11 2.70

M

8 6 4 4 4 26

43 8 6 5 5 3 3

2

3

4

5

6

7 –.41 –.42 –.34 –.12 –.11b –.34 –.25

8 –.50 –.40 –.37 –.32 –.35 –.20 –.25

9 –.38 –.29 –.40 –.13 –.23 –.23 –.17

10

.39 .35 .41 .10b .14 .25 .22

11

–.53 –.44 –.32 –.39 –.42 –.15 –.33

12

40 .49 .52 –.68 .44 30 .61 –.38 .80 20 –.42 .52 20 –.33 20 121

144 .87 .67 .70 .52 .52 .62 40 .39 .54 .27 .52 .54 30 .26 .40 .22 .26 25 .31 .17 .36 20 .02c .19 15 .32 15

Min Max

.75 .88 .77 –.41 .82 —

–.52 –.45 –.39 –.28 –.31 –.28 –.28

13

Note: All measures are subscales, unless designated as a full scale. a. Sample size for College Mattering Inventory: n = 1,561; all correlations significant at p < .01, unless otherwise noted. b. p < .05. c. Not significant. d. Intercorrelations based on university sample only: n = 466; all correlations significant at p < .01. The Sense of Belonging Scale uses a Likert-type scale where 1 = completely true and 5 = completely untrue—thus, the negative correlations with the mattering scales.

8. Perceived Peer Support 9. Perceived Faculty Support/Comfort 10. Perceived Classroom Comfort 11. Perceived Isolation 12. Empathetic Faculty Understanding 13. Total scale

.88 .84 .93 .78 .87 .87

.91 .89 .83 .84 .76 .77 .72

1. Total Mattering Scale 2. General College Mattering 3. Mattering Versus Marginality 4. Mattering to Counselors 5. Mattering to Instructors 6. Mattering to Students 7. Perception of Value

Sense of Belonging Scaled

α

College Mattering Inventorya

Table 5 Descriptive Statistics, Internal Reliability, and Scale Intercorrelations for the College Mattering Inventory and the Sense of Belonging Scale


Table 6 Mean College Mattering Inventory Subscale Scores for Gender and Ethnicity

Gender

Race/Ethnicity

College Mattering Male Female Asian Inventory Subscales (n = 596) (n = 965) (n = 289) General College Mattering Mattering Versus Marginality Mattering to Instructors Mattering to Counselors Mattering to Students Perception of Value

Black / African Latino / American Latina White Multiracial (n = 77) (n = 371) (n = 556)

Other

(n = 82)

(n = 186)

M SD

23.84 6.99

24.09 6.99

23.48 6.56

23.90 7.26

24.08 7.30

24.10 6.92

24.32 6.87

24.18 7.19

M SD

21.95 5.43

21.88 5.67

21.07 5.68

21.79 5.92

22.21 5.77

21.99 5.46

22.59 4.87

22.09 5.44

M SD M SD M SD M SD

14.68 4.89 16.34 3.00 9.09 2.64 11.09 2.67

14.47 5.06 16.41 3.18 9.02 2.87 11.12 2.72

14.29 4.42 16.01 2.99 8.76 2.59 10.78 2.55

15.56 5.80 16.56 3.02 9.14 3.09 11.61 2.57

15.18 5.15 16.79 2.93 8.60 2.77 11.20 2.66

14.49 4.92 16.34 3.22 9.38 2.76 11.07 2.82

14.21 5.16 17.00 2.83 8.91 2.97 11.04 2.77

13.58 5.14 15.92 3.37 9.39 2.83 11.35 2.67

the College Mattering Inventory reflect a higher sense of perceived mattering, a negative correlation was expected with four subscales of the Sense of Belonging Scale: Perceived Peer Support, Perceived Faculty Support/Comfort, Perceived Classroom Com fort, and Empathetic Faculty Understanding. By contrast, we expected that the scale’s Perceived Isolation subscale would correlate positively. As expected, the intercorrelations for the mattering and sense of belonging scales followed the hypothesized outcome (see Table 5). Overall, the pattern of correlations lends support to the convergent evidence for the validity of the College Mattering Inventory. As noted, all correlations were significant at p < .01, except one, which was significant at p < .05. The smallest correlation occurred between the Mattering to Counselors and Perceived Isolation subscales (r = .10). The largest correlation was between General

College Mattering and Empathetic Faculty Understanding (r = –.44). The pattern of intercorrelations for both instruments support the notion that mattering and sense of belonging are related but different constructs.

Discussion The aim of this study was to assess the construct validity of the College Mattering Inventory on a multiply diverse college student sample. Based on the pioneering works of Rosenberg and McCullough (1981) and Schlossberg (1989), items for the College Mattering Inventory were crafted to reflect various aspects of perceived mattering by college students: importance, attention, support, dependence, ego extension, and marginality. The items were written to reflect student interactions with the college environment, instructors, counselors, and other students. Results



of the confirmatory factor analysis of the College Mattering Inventory support a welldefined, simple factorial structure—although the analyses do suggest a small degree of measurement noninvariance for gender. The small to medium (Cohen, 1992) scale intercorrelations between the College Mattering Inventory and the Sense of Belonging Scale support the instruments’ convergent evidence of external validity. Whereas the correlation between the total score of the College Mattering Inventory and the total score of the Sense of Belonging Scale is moderate (r = –.52), we remain confident that mattering and sense of belonging are two distinct but related constructs. Future research can assert or refute this claim, and an additional study is in the planning stages to explore this. This study is the first of its kind in attempting to measure the construct of mattering with such a diverse college student population—from students attending community college to students attending a master’s-level university. The study begins to fill in a gap in the literature by furthering the development and validation of a psychometrically sound mattering instrument that will aid higher education researchers, as well as academic and student affairs professionals, to assess students’ perceptions of mattering at their institution. Although much has been written about the impact of college and students’ interactions with institutional agents, mattering has attracted little attention when compared to the related constructs of sense of belonging, fit, involvement, and engagement (e.g., Chickering & Gamson, 1987; Kuh, 2001; Pascarella & Terenzini, 1991, 2005; Pike & Kuh, 2005; Schroeder, 2003). Thus, the theoretical and construct validation of mattering still remains a viable area for research, as does the assessment of mattering experiences of diverse college students.

Now that a validated mattering instrument is available—although its factorial structure should be confirmed with a greater number of college and university samples— colleges and universities may begin to systematically assess students’ perception of mattering versus not mattering at their respective institutions. A perceived sense of not mattering has been identified as precursor to dropping out from college (Schlossberg, 1989). Creating and sustaining a supportive, welcoming environment where students perceive that they matter will enhance the opportunities they have for involvement and success. Positive student experiences with the learning environment, with institutional agents such as instructors and counselors, and with student-centered programs, policies, and procedures will help increase mattering. As Schlossberg (1989) posits, institutions that focus on mattering and greater student involvement will be more successful in creating campuses where students are motivated to learn, where their retention is high, and ultimately, where their institutional loyalty for the short- and long-term future is ensured. (p. 14)

Limitations Several limitations may affect the generalizability of the factorial structure of the College Mattering Inventory. First, data collection took place exclusively through an online survey. Although e-mail invitations were sent to a predetermined number of students, we were unable to confidently determine a response rate, because we do not know how many actually received or read the invitations. In addition, we cannot ascertain that the intended student accessed and answered the survey. The offer of an incentive to participate could have also resulted in selection bias. Given that data collection



took place at only one community college and one university in Southern California, additional research needs to be conducted to determine whether our results are generalizable to institutions and college students across the country, although this sample is the most diverse of its sort to date.

Implications for Practice According to Dixon Rayle (2006), perceived importance to others is likely the foundation of mattering to others. In contrast, not mattering may lead individuals to engage in socially undesirable behaviors to get noticed. This view is consistent with Rosenberg and McCullough’s findings (1981) on adolescents. Those who perceived themselves as being unimportant, essentially not mattering, were more likely to engage in antisocial behaviors. In the college setting, lower levels of perceived mattering resulted in increased levels of academic stress for students (Dixon Rayle & Chung, 2007– 2008), negatively associated with alienation but positively related to self-esteem, selfmonitoring, and social support (Elliot et al., 2004). Given these findings and the issues discussed above, we can reasonably conclude that counselors, professors, and other college personnel can in fact positively affect students. They may do so by directly demonstrating how important students are to them individually or to the college as a whole and how they rely on them to contribute to the successful experiences of the class and other students. A thoughtful instructor will readily make students aware that their contributions are valued, even when they are not the best quality; furthermore, they will communicate their appreciation to the student for having the courage to participate. In short, if students perceive that they matter, they will be more likely to participate; that is, they will become part of the we

classroom culture (participative) versus the they classroom culture (observer). Similarly, within the context of counseling, the relationship between the student and the counselor or advisor in the cocurricular environment has the potential to positively influence the student. Being attended to and having one’s specific needs met is likely to create a greater sense of mattering and connection to the institution. Hence, having a validated instrument will avail institutions with the opportunity to readily assess students’ perceptions of mattering to the college and to instructors, counselors, and other students. This assessment may in turn lead to needed changes toward creating a more student-centered learning environment where students are more likely to fully reap the benefits of a college education.

Future Research Future studies should focus on the replication of the factorial structure of the College Mattering Inventory through confirmatory factor analysis and with a larger representation of students from different institutions and institutional types. The prospect for conducting research on the link among college student retention, success, and mattering is now empirically possible. Research assessing mattering differences among students of diverse socioeconomic backgrounds and statuses has yet to be conducted. For example, do international students and American students (U.S. citizens) differ in their perceptions of mattering? Do students from traditionally marginalized communities (e.g., immigrants, sexual minorities) experience mattering differently? As a starting point, future studies should expand on our work on gender invariance by assessing if the factorial structure of the College Mattering Inventory is also race/ethnicity invariant. Last, other lines of research may



pursue if mattering, sense of belonging, marginality, and alienation, for example, are the same or simply related constructs.

References Aish, A. M., & Jöreskog, K. G. (1990). A panel model for political efficacy and responsiveness: An application of LISREL 7 with weighted least squares. Quality and Quantity, 19, 716–723. Alford, S. M. (1998). The impact of inner-city values on student social adjustment in commuter colleges. NASPA Journal, 35(3), 225–233. Anant, S. S. (1966). The need to belong. Canada’s Mental Health, 14, 21–27. Arbuckle, J. L., & Wothke, W. (1999). Amos 4.0 user’s guide. Chicago, IL: SPSS. Barefoot, B. O., Warnock, C. L., Dickinson, M. P., Richardson, S. E., & Roberts, M. R. (Eds.). (1998). Exploring the evidence: Reporting outcomes of firstyear seminars. Columbia, SC: National Resource Center for the Freshman Year Experience. Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. Bentler, P. M. (1992). On the fit of models to covariances and methodology to the Bulletin. Psychological Bulletin, 112, 400–404. Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Hillsdale, NJ: Erlbaum. Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3–7. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. Connolly, K. M., & Myers, J. E. (2003). Wellness and mattering: The role of holistic factors in job satisfaction. Journal of Employment Counseling, 40, 152–159. Coopersmith, S. (1967). The antecedents of selfesteem. San Francisco: Freeman. DeForge, B. R., & Barclay, D. M. (1997). The internal reliability of a general mattering scale in homeless men. Psychological Reports, 80, 426–430. Dimitrov, D. M. (2006). Comparing groups on latent variables: A structural equation modeling approach.

WORK: A Journal of Prevention, Assessment, and Rehabilitation, 26, 429–436. Dixon Rayle, A. (2006). Mattering to others: Implications for the counseling relationship. Journal of Counseling and Development, 84, 483–487. Dixon Rayle, A., & Chung, K. Y. (2007–2008). Revisiting first-year college students’ mattering: Social support, academic stress, and the mattering experience. Journal of College Student Retention, 9, 21–37. Elliott, G. C., Colangelo, M. F., & Gelles, R. J. (2005). Mattering and suicide ideation: Establishing a relationship. Social Psychology Quarterly, 68(3), 223–238. Elliott, G. C., Kao, G., & Grant A. M. (2004). Mattering: Empirical validation of a socialpsychological concept. Self and Identity, 3, 339–354. Erikson, E. (1968). Identity, youth, and crisis. New York: Norton. Hart, D. (1988). Self-understanding and social– cognitive development. Early Child Development and Care, 40, 50–23. Hausmann, L. R. M., Schofield, J. W., & Woods, R. L. (2007). Sense of belonging as a predictor of intentions to persist among African American and White first-year college students. Research in Higher Education, 48(7), 803–839. Hazlett-Stevens, H., Ullman, J. B., & Craske, M. G. (2004). Factor structure of the Penn State Worry Questionnaire: Examination of a method factor. Assessment, 11, 361–370. Hoffman, M., Richmond, J., Morrow, J., & Salomone, K. (2002–2003). Investigating “sense of belonging” in first-year college students. Journal of College Student Retention, 4(3), 227–256. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria to fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. Hurtado, S., & Carter, D. F. (1997). Effects of college transition and perceptions of the campus racial climate on Latino college students’ sense of belonging. Sociology of Education, 70, 324–345. Jayanti, R. K., McManamon, M. K., & Whipple, T. W. (2004). The effects of aging on brand attitude measurement. Journal of Consumer Marketing, 21(4), 264–273.


Tovar et al. / College Mattering Inventory 177 Johnson, D. R., Soldner, M., Leonard, J. B, Alvarez, P., Inkelas, K. K., Rowan-Kenyon, H., et al. (2007). Examining sense of belonging among first-year undergraduates from different racial/ethnic groups. Journal of College Student Development, 48(5), 525–541. Jöreskog, K. G., & Sörbom, D. (1988). LISREL 7: A guide to the program and applications. Chicago: SPSS. Josselson, R. (1998). Revising herself: A story of women’s identity from college to midlife. New York: Oxford University Press. Kline, R. B. (1998). Principles and practice of structural equation modeling. New York: Guilford. Kohut, H. (1977). The restoration of the self. New York: International Universities Press. Kuh, G. D. (2001). Assessing what really matters to student learning: Inside the National Survey of Student Engagement. Change, 33(3), 10–17, 66. Lee, H. B., & Comrey, A. L. (1979). Distortions in a commonly used factor analytic procedure. Multivariate Behavioral Research, 14, 301–321. Mak, L., & Marshall, S. K. (2004). Perceived mattering in young adults’ romantic relationships. Journal of Social and Personal Relationships, 21, 469–486. Marshall, S. K. (2001). Do I matter? Construct validation of adolescents’ perceived mattering to parents and friends. Journal of Adolescence, 24, 473–490. MacCallum, R. C. (1995). Model specification: Procedure, strategies, and related issues. In R. H. Hoyle (Ed.), Structural equation modeling (pp. 16–36). Thousand Oaks, CA: Sage. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, and Computers, 32, 396–402. Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students. San Francisco: JosseyBass. Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (2nd ed.). San Francisco: Jossey-Bass. Pike, G. R., & Kuh, G. D. (2005). First- and secondgeneration college students: A comparison of their engagement and intellectual development. Journal of Higher Education, 76, 276–300.

Rosenberg, M. (1985). Self-concept and psychological well-being in adolescence. In R. L. Leahy (Ed.), The development of self (pp. 205–246). Toronto, Ontario, Canada: Academic Press. Rosenberg, M., & McCullough, B. C. (1981). Mattering: Inferred significance and mental health among adolescents. Research in Community and Mental Health, 2, 163–182. Schlossberg, N. K. (1984). Counseling adults in transition. New York: Springer. Schlossberg, N. K. (1989). Marginality and mattering: Key issues in building community. New Directions for Student Services, 48, 5–15. Schlossberg, N. K., Lasalle, A., & Golec, R. (1990). Mattering Scales for Adult Students in Post secondary Education. Washington, DC: American Council on Education, Center for Adult Learning and Educational Credentials. (ERIC Document Reproduction Service No. ED341772) Schlossberg, N. K., Lynch, A. Q., & Chickering, A. W. (1989). Improving higher education environments for adults: Responsive programs and services from entry to departure. San Francisco: Jossey-Bass. Schroeder, C. (2003). How are we doing at engaging students? Charles Schroeder talks to George Kuh. About Campus, 8, 9–15. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Allyn & Bacon. Thompson, B., & Daniel, L. G. (1996). Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56, 197–208. Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago: University of Chicago Press. Ullman, J. B. (1996). Structural equation modeling. In B. G. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (3rd ed., pp. 748–753). New York: HarperCollins. Upcraft, M. L., Gardner, J. N., & Barefoot, B. O. (Eds.). (2004). Challenging and supporting the first-year student: A handbook for improving the first year of college. San Francisco: Jossey-Bass/ Wiley. Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321–327. Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review of alternative procedures for


178 Measurement and Evaluation in Counseling and Development determining the number of factors or components. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy (pp. 41–71). Norwell, MA: Kluwer Academic. Wood, J. M., Tataryn, D. J., & Gorsuch, R. L. (1996). Effects of under- and overextraction on principal axis factor analysis with varimax rotation. Psychological Methods, 1, 354–365. Woods, C. M. (2006). Careless responding to reverseworded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28, 189–194.

Esau Tovar is Associate Professor of Counseling and Assessment Center Director at Santa Monica College. Merril A. Simon is an Associate Professor in the Department of Educational Psychology and Counseling at California State University, Northridge, where she coordinates the College Counseling/Student Services specialty program. Her research interests are in the area of college student transition and career indecision/indecisiveness. Howard B. Lee is Professor of Psychology at California State University, Northridge.

For reprints and permissions queries, please visit SAGE’s Web site at http://www.sagepub.com/journals Permissions.nav.
