K E V I N H. C . C HE N G
Perceived Interpersonal Dimensions and Its Effect on Rating Bias: How Neuroticism as a Trait Matters in Rating Creative Works ABSTRACT Understanding any inter- and intra-personal dynamic that affects bias in the judgment of creative output is among numerous areas of focus for researchers in Psychology (or subset field). Notably, as a result of changes in interpersonal dynamics, conditions of subjective construal can induce bias either leniently or severely without any due awareness by the evaluator. Based on 153 Cantonese-English bilinguals, the present study explored two potentially relevant conditions in each of two separate experiments. The first condition (N = 90) examined whether the perceived dominance of the creator differentially affected individuals high in neuroticism in rating involving creativity performance. A generic and robust measure of creativity performance was operationalized and transformed to a score representing the extent the bias was embedded in the rating. In the second condition (N = 63), the perceived friendliness of the evaluation target was manipulated and neuroticism was also assessed. The results indicated raters low in neuroticism exhibited lenient rating bias irrespective of the perceived friendliness or dominance of the rating target. Raters high in neuroticism reported severe rating bias when their targets were perceived as non-dominant. The study demonstrates the application of the meta-theoretical framework of the cognitive-affective personality system. The discussion also emphasizes how the results contribute to development in creativity research—namely the perception and evaluation of creativity. Practical implications of using the findings in managing public expectations from art works to human resource management are posited. Keywords: neuroticism, creativity, rating bias, dominance, friendliness, CAPS, LEGO. Using the cognitive-affective processing system (CAPS) as the meta-framework (Cervone, 2004; Mischel & Shoda, 1995), the present study focuses effort in the area of creativity and factors affecting biases in the judgment of creative output. The void in the field of creativity is explored by operationalizing the CAPS using existing models of interpersonal behavior (Wiggins, 2003) and individual difference (Costa & McCrae, 1992). In creativity research, Kasof (1995, p. 313) noted, “not much is known or even hypothesized about the situational factors that influence the reception of original products, . . . and systematic departures from such normative criteria (i.e., biases) in the evaluation. . .”. Today, discernments in creative judgments lack consensus (Kozbelt, 2007; Silvia, 2008). The use of the Consensual Assessment Technique (CAT) involves groups of expert judges who rate a set of creative products (Amabile, 1983; Kaufman, Baer, Cole, & Sexton, 2008). In its original design, CAT judges receive no information about the creators of the product. In gender and race stereotype research (Kaufman, Baer, Agars, & Loomis, 2010; Lebuda & Karwowski, 2013), the CAT has been specially used in two major ways. First, expert raters were replaced by novices. Second, information about creators becomes the manipulation variable so that the raters judge the product knowing that a profile of the creators which includes: gender, ethnicity, and name (i.e., common or uncommon names). The use of the CAT (or according to the above “special use”) has resulted in claims that it is either robust (Kaufman et al., 2010) or problematic (Kaufman, Baer, & Cole, 2009; Lebuda & Karwowski, 2013). The present study is relevant to the above in that: 1 A design similar to the CAT is used (to be described in methods). 2 Interpersonal variables are used.1 1
Interpersonal variables are used rather than more commonly used variables such as gender, race or name familiarity. The main reason is to examine whether rating bias (RBias) is affected by the interaction effect between the perception of interpersonal dimensions and personality traits.
The Journal of Creative Behavior, Vol. 0, Iss. 0, pp. 1–12 © 2016 by the Creative Education Foundation, Inc. Ó DOI: 10.1002/jocb.156
1
Personality and Rating Bias in Creative Work
The following section discusses relevant literature on interpersonal theories by integrating it with personality and the CAPS framework. According to the CAPS, the coherence of personality and its behavioral manifestations can be conceived as the result of the interaction between cognitive-affective units (CAU) (such as one’s values, beliefs, or attitudes) and situations. A situation, however, refers not to the physical aspect of the environment but instead, to psychological features of a given situation. The psychological features of a situation provides a pre-condition (or if-condition) which determines which CAU configurations are likely to be triggered. The CAPS postulates that differences in individual predispositions arises from the organization and chronic accessibility of CAUs and these give rise to a stable pattern of situation-behavior contingencies, known as “if-then” behavioral signatures. In defining the specifics in the CAPS from what we know about interpersonal behavior and the trait of neuroticism (refer to interpersonal theory and neuroticism in supplement material), we may consider that individuals high in neuroticism who are chronically sensitive and apprehensive on potential threats, may respond accordingly by approaching or avoiding (then) a situation (if). When they encounter a creative situation (if), where there are no clear signs of doing or making things that are right or wrong, when they are made to serve as a critic or rater of others’ creativity, they are more likely to be cautious or even lenient (then) because they want to avoid being judged as harsh and/or critical (CAU), which may upset the recipient (or the creative person). Empirical studies support the association between neuroticism and submissive behavior (Cote & Moskowitz, 1998) as well as with the submissiveness pole of the interpersonal trait circumplex (Trapnell & Wiggins, 1990). In forming the first hypothesis, we consider individuals high in neuroticism in a state of non-dominance (if) as a rater of creative output, which according to interpersonal theory implies the absence of power, respect or aggression and possibly the disposition of deference. Because the avoidance and sensitive systems are motivated toward the avoidance of threat and punishment (CAU), any form of behavior (or judgment) falls in accord with the consequence. To reduce a potential threat for a person in a state of non-dominance, the accord behavior provides more lenient ratings than what the actual output deserves (then). Evidence has indicated that fear of backlash results and is related to rating bias (Spence & Keeping, 2010). What remains unknown for individuals high in neuroticism is how their behavior may change in contexts where they are in a position of dominance. That is, how will they behave or judge others in such circumstances? Dominance refers to the prominence, status, or personal influence to control or exert influence in the way others think, feel, and behave (Bales, 1999). It is customary to assume that individuals who are raters are relatively more dominant than individuals, who are ratees, or those receiving the rating (Bass, 1960; French & Raven, 2004). Ng, Koh, Ang, Kennedy, and Chan (2011) argued that raters in the position of power are less likely to produce rating bias because there is no social costs when severe ratings are allocated to ratees. Based on these findings, the author asks, Is the rating of creative output more or less lenient compared to the above when the perceived threat becomes a lesser concern? The threat avoidance system may remain relevant to individuals high in neuroticism even though a state of power or dominance is perceived (if). Alternatively, perceived dominance could change behavior from one aimed to eliminate threats by avoidance to one that eliminates threats by approaching or facing the threat (CAU). The approach can be a form of self-defense by individuals high in neuroticism and this results in harsh or severe judgment (then). While it is common for people high in neuroticism to be submissive, it is also reported that they exhibit quarrelsome behaviors (Bolger & Zuckerman, 1995). Caprara, Barbaranelli, and Zimbardo (1996) noted neurotics in specific social dynamics can be highly predictive in demonstrating aggressive behavior. In this context, the CAPS is useful as it affords the kind of environmental stimulus that is salient or will become relevant in triggering a series of CAU interactions. Unlike other models that assess the interaction between the environment and its people (Cable & Edwards, 2004; Pervin, 1989; Schneider, 1987), the CAPS defines and confines the environment to features that are psychologically relevant. In this case, it is the interpersonal dimension of dominance (Study 1) and friendliness (Study 2) that are investigated. This feature of the meta-theory channels its applicants to think of the environment at the level and mode of the CAUs and it offers a distinctive advantage over others. Based on the above discussions, the following hypothesis is proposed: H1: Across the polar ends of the dominant vs. non-dominant condition, individuals high in neuroticism are more likely, across these conditions, to produce rating bias than individuals low in neuroticism.
2
Journal of Creative Behavior
One implication from the findings in this study is that by knowing factors that may affect the reception of creative work, especially if they are endogenous, we can then forecast how likely inaccurate evaluations are to occur and, if possible, avoid flourishing works from unfair comments. Furthermore, we can also nurture relevant factors that are conducive to the development of creativity.
STUDY 1 (DOMINANT VS. NON-DOMINANT) (DND) METHODS Participants There were 90 participants in this study (25 male and 65 female). All respondents were between 19 and 25 years of age. All students were bilingual (Canton-Chinese and English) and local Hong Kong residents. There were 47 and 43 participants in the dominant and non-dominant condition respectively. Two participants did not finish the study and left the session for personal reasons. These students also received grade credits as indicated in the consent form. Measures Five factor model The Five factor model (FFM) dimensions were measured by a version of International Personality Item Pool in Chinese (Traditional) (Zheng et al., 2008). The instrument used a 100-item scale and asked the participants to respond to items ranked on a 5-point Likert scale (“1” and “5” representing “not very accurate” and “very accurate” respectively). Reliability and concurrent validity of the instrument have been previously demonstrated in Zheng et al. (2008). In this study, Cronbach’s a for neuroticism, extraversion, openness-toexperience, agreeableness, and conscientiousness, were calculated as .93, .89, .86, .82, and .90 respectively. Rating of creativity Five items on the model’s creativity, complexity, artistic impression, novelty and aesthetic quality made up the rating of creativity (RoC). It is a robust measure of the model’s creativity. These terms were chosen after referencing work by Christiaans (2002) and Simonton (1998, 2000). In the former, creative criteria were related to aesthetic means such as attractiveness, technical quality, and mental representation of the design (Christiaans, 2002). In the latter, creative criteria referred to originality, adaptiveness, novelty, complexity, and aesthetic quality. These studies acknowledged that creativity arises from specific demand of the social and environmental constraints and it leads to non-homogenous criteria in different situations. Recently, Simonton (2012) proposed creative works as those that are new, useful, and non-obvious. This perspective on creativity offers a sophisticated analysis of creativity and provides diversity in appreciating different kinds of creative elements, such as surprise (Boden, 2004), relevance (Kaufman & Sternberg, 2010), or efficiency (Amabile, 1983). The measure used in this study is diverse in that it offers a robust measure of creativity. In the present study, participants used a 10-point Likert scale to express their level of agreement (or disagreement) with each term. Beyond a simple semantic explanation of what each term meant, participants were left to interpret the elements of the model that corresponded to specific aspects of creativity. A principal components analysis and scree plot revealed only one general factor, which accounted for more than 48% of the total variance. These five items were aggregated to comprise the RoC measure. The Cronbach’s a for these items was .93. Rating bias The present study calculates disagreement between two sources of the RoC: individual rater (or recipient) and an aggregate score of all raters. Though the participants are considered as novices, the design ensured that they were not unfamiliar with the ratings scales. A bias measure (RBias) denotes rating leniency or severity when the rater’s evaluation deviates from the aggregated norm. Grohman, Wodniecka, and Kłusak (2006) also used a similar subtraction method in deriving a score for leniency (over-estimation) and severity (under-estimation). The idea of accuracy which involves the disagreement between a subjective measure and a criterion objective measure was proposed by Hastie and Rasinski (1988). Note that the aggregate norm infers a point of relative reference and it is different from any point of absolute certainty. It is not the gold standard criterion even when the judgments are based on expert opinions (Cronbach, 1955; Silvia & Gendolla, 2001). The gold standard is tied to the various notions of accuracy but the author does not intend to weigh in on the debate. According to Silvia (2008), it is difficult if not impossible, to provide a point of 3
Personality and Rating Bias in Creative Work
absolute reference (or an inapt criterion) for creative products (e.g., music, poem, art). It is worth noting that studies such as those which indicate the presence of an objective measure or the gold standard (Grohman et al., 2006; Hastie & Rasinski, 1988), do not escape from the same argument made by Silvia (2008) and Silvia and Gendolla (2001). That is, disagreement among expert judges on creative evaluations are not uncommon especially if the criterion is heterogeneous or if the raters’ criteria arise from complex socio-cultural and historical processes (Grohman et al., 2006; Runco & Smith, 1992; Sawyer, 2006). Having said that, agreements do occur when the judgment concerns the rarity of the creativity solution (Chand & Runco, 1992). Perhaps a probable resolution is to treat accuracy as a special case of agreement. The readers should consult Silvia (2008) for a thorough discussion on the merits of various operationalizations of rating accuracy. Eventually, the numeric difference between (a) RoC made by individual ratings of the common model (OM) and (b) the mean score of all ratings on the OM (using RoC) becomes the measure of RBias. The operationalization of the measure, where levels with high (or low) ratings are found refers to more leniency (or severity). ROCaggregate is operationalized by using average ratings by all participants. It is different from the formative operationalization of creative judgment (Groborz & Necka, 2003; Grohman et al., 2006) in three aspects: an aggregation of judgment from the novice creators/participants form the criteria measure; the objects of judgment are not constructed by the creators, they are provided by the experimenter; and creators’ creation and self-judgment of creation were part of the experiment but it serves as practice and belongs not to any part of the data analyses. Below is an algebraic expression of the bias score, which is the only dependent variable in the two studies: R1 þ R2 þ . . . þ Rp RoCaggregate ¼ p where p is the total number of participants; R refers to individual RoC. RBiask = Rk RoCaggregate where k = 1 through N participants. Experimental manipulation The experiments referred to two conditions: dominant and non-dominant. Over these two conditions, photos of a Lego enthusiast and his creative models were presented in a fictitious format aimed to achieve the above state toward the enthusiasts (the creator). In the non-dominant condition, a man is described as an experienced LEGO hobbyist who enters his first exhibition since his teen years. Participants were shown photos of the man’s past Lego models. Participants’ were then informed that the man also built his version of the model (i.e., rock legend or angry boss). The participants were rated on Figure S3c in terms of creativity measures described in RoC. (Further details are provided in Supporting Information as well as Figures S1, S2a, S2b, S3a, S3b, and S3c.) In other words, the participants first completed their models then rated their own creativity. They were then introduced to the enthusiast’s (or the creator’s) photo, his models (Figure S3c) and then rated his creativity. Because the model by the enthusiast was most likely to be better built, the participants may have felt less dominant in the context of creative performance. In the dominant condition, a boy was described as a first time exhibitionist who was accompanied by his father. He was described as the son of the enthusiast man above. The enthusiast’s son was described as having undeveloped (or amateur-like) skills. By describing the boy as the son of the enthusiast, this allows every photo (other than the boy’s photo) as well as the fictitious story, to be the same as the non-dominant condition. To measure the efficiency of the above manipulation, a set of 10 items were presented. The items aimed to measure how participants perceived their own dominance with respect to: the enthusiast in the non-dominant condition and the enthusiast’s son in the dominant condition. The method was analogous to that reported by Leikas, L€ onnqvist, and Verkasalo (2012), with a slight difference. A relative measure was used by assessing different scores between participants’ ratings of dominance for self against the creative person (or the creator). The manipulation check was taken immediately after the rating of the common model target—that is, after rating Figure S3c (i.e., rock-legend or angry boss). Participants responded to items using a 10-point Likert scale to measure degree of agreement with the statement (a score of 1 indicated high in agreement; a score of 10 indicated not-in agreement). To ensure the items were properly read, two methods were used to reduce careless or malingering responses. First, the agreement and disagreement poles were switched between two versions. Second, some items were inversely phrased. The original items, in Chinese traditional typeset,
4
Journal of Creative Behavior
appeared in a contextualized personality scale (called the Chinese Personality at Work) in which issues on validity and reliability were reported (Hui, Gan, & Cheng, 2000). Sample translated items are as follows: I feel I am not as good as others in many ways; I like to seek other peoples’ opinions when we talk; In meetings, I am the first one to speak or address others; I like to take a leading role when I am working with others. An analysis of variance between the conditions supported the effect of the manipulation in the intended direction (F (1, 62) = 6.83, p < .01; sample mean (Kid/Man) = .41/1.32; sample SD (Kid/Man) = 1.43/1.73). The Cronbach’s a for the 10 items was .81. Procedure Allocation of students into groups was arranged prior to the commencement of the study. Students read, signed, and received a copy of the consent form. Each experimental condition consisted of a practice session followed by a creative session (procedural details are provided in the supplementary materials as well as supplement figures, i.e., Figure S1, S2a, S2b, S3a, S3b, and S3c). At the end, participants provided self-ratings of their LEGO model followed by peer-rating. The experimental manipulation was applied after participants listened to the fictitious story (see experimental manipulation above). Participants were asked to rate Figure S3c using the same rating scale (left for rock-legend and right for angry boss). The common model (OM) was rated by every participant across all groups and conditions. This rating measure was used to derive the RBias variable used to test the hypothesis of each study (note: the hypothesis of the second study is identified below). RESULTS Table 1 shows descriptive statistics, Cronbach’s a and Pearson’s r for all relevant variables in Study 1. For Study 1, the five significant correlations were as follows: conscientiousness is positively associated with RBias; agreeableness is positively associated with neuroticism; extroversion is positively associated with openness-to-experience; conscientiousness is positively associated with extroversion and agreeableness. Correlation between neuroticism and RBias was not significant (r = .07, p > .05; N = 90). A measure of non-independence within group was conducted by examining the intra-class correlation coefficient. Both g2 and intra-class correlation ICC1, as computed through a random intercepts model, indicated the proportion of total variance between teams (Bliese, 2000). According to Bliese (2000), ICC1 values are typically under .20 and are usually smaller than the corresponding g2 values. In this study, the ICC1 and g2 were .30
TABLE 1.
RBias N E O A C
Descriptive Statistics and Correlation Table Study
Mean (SD)
RBias
N
E
O
A
C
1 2 1 2 1 2 1 2 1 2 1 2
8.03 8.32 3.02 3.18 3.06 3.11 3.35 3.00 3.59 3.59 3.14 3.26
.93 .89 .07 .04 .05 .01 .05 .04 .14 .33* .29** .22
.93 .93 .08 .14 .14 .05 .24*** .14 .09 .16
.89 .89 .41*** .22 .36*** .26* .27*** .28*
.86 .87 .19** .16 .10 .09
.82 .82 .45*** .20
.90 .89
(1.04) (1.05) (.71) (.65) (.57) (.58) (.48) (.50) (.40) (.29) (.58) (.54)
Note. N = 90/63 (Study-1/2). Diagonal cells (in gray) in correlation matrix refers to Cronbach’s a; N = Neuroticism, E = Extroversion, O = Openness to Experience, A = Agreeableness, C = Conscientiousness. *p < .05, **p < .01, ***p < .001.
5
Personality and Rating Bias in Creative Work
(range = .17–.39) and .08 respectively. ICC2 was used to estimate the reliability of the mean values that were obtained from aggregate measures. The ICC2 was .54. To statistically evaluate H1, interpersonal condition (DnD) was treated as a moderator variable with N as the independent variable and RBias as the dependent variable in a stepwise hierarchical regression analysis. The N variable was standardized. The categorical variable of DnD is referenced at “D”—that is, the dominant condition was coded as zero; non-dominant condition coded as one. The moderating effect was tested according to the procedures outlined by Baron and Kenny (1986). The regression analysis indicated main effects for DnD and N on RBias (see Table 2). When RBias was regressed on neuroticism, with DnD as the moderating variable, the results were significant (DF (1, 89) = 6.73, p < .01). A simple slope analysis showed that when DnD equals zero (i.e., dominant condition; solid line in Figure 1), the relationship between RBias and neuroticism is significant and inversely related (bstandardized = .40, bun-standardized = .46, p < .005). When DnD equals one, the relationship is insignificant (bstandardized = .09, bun-standardized = .08, p = .56). The unstandardized beta value for neuroticism is the usual t-test of beta (Winnifred, 2015). An illustration of the interaction effect is shown in Figure 1. Note the two discrete groups on the x-axis resulted from the partition of the neuroticism score at the median point. The purpose of this was to allow the author to graphically present the interaction effect. As described above, RBias, the y-axis, is derived by subtracting individual OM scores from the average score of all OM ratings across individuals. Positive values from zero denote ratings above the mean value (i.e., leniency) and negative values from zero denote ratings below the mean value (i.e., severity). Because the zero point was the mean value calculated by averaging all values made by individual raters of the common model (OM), positive and negative values were taken as a measure of RBias. As the figure illustrates, individuals who are low in neuroticism tend to rate close to the point of neutrality (or the zero point). Individuals high in neuroticism were swayed by the dominant conditions —that is, there was negative RBias (or harsh) when individuals perceived themselves as more dominant than their ratees. Conversely in a non-dominant condition, individuals high in neuroticism were more likely to provide a lenient rating—that is, producing positive RBias. The evidence supports H1.
STUDY 2 (FRIENDLY VS. NON-FRIENDLY) (FNF) The second study used perceived friendliness toward the creative person as the interpersonal dimension. As mentioned in Study 1, friendliness is a main interpersonal factor and is a facet of the communal dimension in the interpersonal theory of behavior. It is an orthogonal dimension of dominance in the circumplex model (Wiggins, 2003). These dimensions allow the study to classify different social situations in a TABLE 2.
Stepwise Regression Analysis by Hypotheses
Study
Hypothesis
1
1
Model
Variables
DnD, N
Adjusted R2
DF
0.05
3.53
DnD N DnD, N, DnD 9 N
2
0.11
FnF, N
6.73
FnF N FnF 9 N
df2
p
2
88
1
87
.03 .02 .10 .01
.24 .21 .26 0.07
2.55
0.16
5.51
FnF N FnF, N, FnF 9 N
df1
.24 .17
DnD N DnD 9 N 2
b
.34 .05
2
61
1
60
.28 .003 .34
.02 .05 .01 .09 .03 .73 .02
Hypothesis confirmed? Yes
Yes
.06 .99 .02
Note. DnD = Dominant vs. Non-dominant; N = Neuroticism; FnF = Friendly vs. Non-friendly; Dominant (D) and Friendliness (F) are the point of reference (nD and nF are coded at “1”). 6
Journal of Creative Behavior
FIGURE 1. Neuroticism and DnD interactions on rating bias. meaningful way (Fournier, Moskowitz, & Zuroff, 2008; Funder, 2009), which is grounded in the theory of interpersonal behavior (Moskowitz, 1994; Wiggins, 2003). Similar to Study 1, the CAPS guides how competing hypotheses might be realized by focusing on what psychologically relevant features of the situation might be salient for individuals high in neuroticism. In forming the second hypothesis, we consider individuals high in neuroticism who perceive friendliness toward the ratee (if). According to the theory of interpersonal behavior, friendliness denotes the presence of warmth, affiliation, and sociability. As discussed in Study 1, the avoidance mechanism is geared toward the reduction of threat and negative outcomes (i.e., CAU). Therefore, individuals high in neuroticism should react with approach rather than avoidant behavior by providing lenient ratings (then). Wong and Kwong (2007) found that students who had been primed to maintain group harmony would provide an elevated evaluation of their peers. People who were socially oriented, friendly, and concerned about other people’s feelings tried not to make the target feel badly, even when a low rating was justified. Thus, raters who placed a high value on social interactions and sensitivity to colleagues’ needs were less likely to be unduly harsh. As Jawahar (2001) suggested, a concern for maintaining social relationships with others may contribute to leniency in rating others. Conversely, in the non-friendly situation, cues representing cold-heartedness, non-affiliation, or uncooperativeness (if) are likely to be salient to an avoidance and sensitive system aimed to avoid threat or punishment. It is likely to assume that the discomfort arises from rating others in less friendly situations which may result in unwanted stress leaving the rater feeling overwhelmed (i.e., CAU). The corresponding behavior, in order to avoid this unpleasant feeling, is to retaliate by providing a harsh rating (then). Caprara et al. (1996) indicated that hostile or aggressive behavior is highly predictive of neuroticism under specific situations. Using this operationalization of the CAPS, the second hypothesis is as follows: H2: Across the polar ends of the friendly-non-friendly condition, individuals characterized by high neuroticism are more likely, across these conditions, to produce RBias than individuals low in neuroticism. METHOD Participants Study-2 also enlisted student participants from those enrolled in the introductory psychology course. All recruitment procedures were the same as Study 1. There were 63 participants (14 male and 49 female) of 7
Personality and Rating Bias in Creative Work
which there were 35 and 28 participants in the friendly and non-friendly condition respectively. One participant withdrew due to personal reasons but was given grade credit nevertheless. Materials With the exception of manipulation pictorials (i.e., Figure S3b), all other pictorials used in the previous study remained the same in Study 2. Measures Five factor model The same measures of the FFM dimensions were used. In Study 2, Cronbach’s a for neuroticism, extraversion, openness-to-experience, agreeableness, and conscientiousness were .93, .89, .87, .82, and .89 respectively. Rating of creativity The items used in Study 1 also applied in Study 2. The Cronbach’s a based on all the rating sources was .93. This rating is different from Study 1 by a small value of .003. The ratings from each study appear equal due to a rounding error. Experimental manipulation In the friendly-non-friendly condition (FnF), participants in different conditions were either shown photos of a man looking friendly (left of Figure S3b) or the same man looking non-friendly (right of Figure S3b). The friendly man was described as a LEGO enthusiast but is now described as approachable, kind, and friendly. The same fictitious story, where the experimenter met the enthusiast in a LEGO exhibition, also applied in Study 2. In the non-friendly condition, the same man was described as unapproachable, arrogant, and uncooperative. When asked to help the experimenter with his study, the man was described as greedy and opportunistic. The participants were told that the man requested a profitable fee for his consultation/assistance. In the friendly condition, the enthusiast kindly complied without any monetary requests. Manipulation check was based on a set of items aimed to measure how the participants perceived their friendliness with respect to: the likeable man in the friendly condition and the unlikeable man in the non-friendly condition. The friendliness scale also was taken from a measure of affiliation to others (Hui et al., 2000). It assumes that those who are facilitative are also friendly (Bluhm, Widiger, & Miele, 1990). A sample of translated items are as follows: I want to meet new people; I like talking to strangers; I am a sociable person; I share important things with friends immediately. An analysis of variance between the conditions supported the effect of the manipulation (F (1, 86) = 54.36, p < .0001; Mean (Fr/nFr) = 1.52/2.49; SD (Fr/nFr) = 1.89/1.59). The Cronbach’s a for the 10 items was .94. Procedure Similar to Study 1, all participants read, signed, and received a copy of the consent form prior to the experiment. Debriefing followed at the end of the experiment. Unless specified, the procedures remained the same as in Study 1. See experimental manipulation above where applicable. RESULTS Table 1 shows descriptive statistics, Cronbach’s a and Pearson’s r for all relevant variables in this study. The three significant correlations were as follows: RBias is positively associated with agreeableness; extroversion is positively associated with agreeableness and negatively associated with conscientiousness. The correlation between neuroticism and RBias was not significant (r = .04, p > .05; N = 63) (see Table 1). Intra-class correlation coefficients and effect-size (i.e., ICC1, ICC2 and g2) were .17 (range = .00–.28), .37, and .11 respectively. Interpersonal condition (FnF) was treated as a moderator variable with N (standardized) as an independent variable and RBias as a dependent variable in a stepwise hierarchical regression analysis. The categorical variable of FnF is referenced as “F”—that is, the friendly condition was coded as zero; the non-friendly condition was coded as one. Similar to the testing for H1, RBias was regressed on neuroticism with FnF as the moderating variable. The regression analysis indicated no main effects for FnF and N on RBias (see Table 2). However, there was a significant interaction effect between FnF and N on RBias (DF (1, 39) = 5.51, p < .05). A simple slope analysis showed that when FnF equals zero (i.e., friendly condition; 8
Journal of Creative Behavior
solid line in Figure 1), the relationship between RBias and neuroticism is significantly and positively related (bstandardized = .38, bun-standardized = .35, p = .06). When FnF equals one, the relationship is insignificant (bstandardized = .34, bun-standardized = .44, p = .17). An illustration of the interaction effect is shown in Figure 2. Note the two discrete groups on the x-axis resulted from the partition of the neuroticism score at the median point. The purpose was to allow the author to graphically present the interaction effect. As Figure 2 shows, individuals who are low in neuroticism tend to rate close to the point of neutrality. On the other hand, ratings from individuals high in neuroticism were swayed across the conditions of friendliness and non-friendliness. Specifically in the friendly conditions, individuals high in neuroticism were more likely to give a lenient rating than those on the other end of the dimension. By contrast, non-friendly conditions appeared to produce a harsh rating, as hypothesized above. This evidence supports H2. DISCUSSION The results speak to the issues outlined by Kasof (1995), which highlight three aspects in creativity research. First, studies have put too much attention toward intra-personal dynamics or the dispositional approach to creativity. Furthermore, it appears that there is an under-emphasis of socio-, inter-personal, cultural-, or environmental impact on creativity. Because past studies do not take into account social or interpersonal influence on creativity, it continues to be undermined or confined by simplistic one-to-one or many-to-one research assumptions. In the current study, Table 1 indicates that a five-factor-model of personality and RBias on creativity output does not bear a clear and simple correlation. Second, past studies that focus on the social aspect of creative output have been promising. The current study demonstrates the promise of using a theoretical framework grounded in social psychology. That is, in using existing models/ theories in operationalizing the CAPS, it is found that: (a) psychologically meaningful situational cues (if) and (b) cognitive-affective personality dynamics (i.e., CAU) of the meta-framework can support logical constraint and conceptual flexibility in the development and consolidation of working hypotheses. The CAPS framework allows findings from creativity (Kaufman et al., 2010), interpersonal (Wiggins, 2003), and personality (Costa & McCrae, 1992) research to be integrated and also provides a level of flexibility for further development in situational specific manner. Third, the focal aspect of Kasof’s (1995) article that concerns the evaluation (or reception) of creative output by non-creative appraisers. In the current study, RoC output was either lenient or severe depending on interpersonal factors. The dimension of neuroticism was
FIGURE 2. Neuroticism and FnF interaction on rating bias. 9
Personality and Rating Bias in Creative Work
moderated by two interpersonal situations: dominance and friendliness. For instance, individuals high in neuroticism were more likely to provide lenient ratings when faced with a rating target that was perceived to be friendly. By contrast, ratings were more severe when the target was perceived as less friendly or unfriendly. Such individuals would be designated as low in neuroticism. These individuals low in neuroticism were not affected in both interpersonal situations—that is, their ratings were near the point of neutrality. Overall, the two hypotheses were confirmed. The CAPS provides the meta-theoretical underpinnings to explain behavior in terms of the relevant situational features that are psychologically meaningful and afford cognitive-affective interplay. According to the CAPS, in the non-friendly condition, cues denoting unfriendliness were psychological features of the situation that were relevant and perhaps more salient to individuals high in neuroticism. The reason could be that individuals high in neuroticism, as they are more responsive to threats than praise, are less able to cope or they are wary of any dire consequence in unfriendly conditions. In friendly conditions they respond to cues denoting friendliness and warmth by a lenient rating. While it is not possible given the current design methodology to test what and how CAUs are relevant in the interaction, it points to future potential to investigate the dynamic nature of rating bias. This is the delineation process of a meta-theory and the results lead to two prospective questions for future research: the first question is whether unfriendliness is a psychologically relevant cue for neuroticism at all and if so, do friendly cues have less impact? The second question is, do the biased results come from salience or psychologically relevant features of situations as the CAPS predicts, or do the biased results come from implied hypersensitivity of neuroticism on the appraisal of external cues? The dominant condition features cues denoting submission to those high in neuroticism because there are psychological features of the situation that are common and habitual for individuals generally known to lack self-assurance and confidence. In the presence of dominance or features that signal the need to be nondominant, the resultant response is to rate leniently. Another perspective is that the rater conformed to the behavior of an inferior who is rating another superior—that is, to appraise positively. However, rating bias also occurs when the cues denote one of submissiveness—that is, when the neurotic is made to feel dominant and thus placed in an unusual position. The condition here is similar to an inferior looking down to another perceived as more inferior or in a more submissive position. The bias exhibited here is by several orders away from the zero point. It is unlike the above results where the neurotics feel submissive and subsequently provide a lenient rating. The results here lead to one future question that deserves empirical enquiry: Why is the bias more severe in the incongruent condition for neurotics than in the congruent condition? The figure indicates that three other points (i.e., neurotic-nondominant, non-neurotic-nondominant, non-neurotic-dominant) do not sway exuberantly compared to neurotics in a dominant position. One may speculate neurotics are so accustomed to being subservient that their rating response in the nondominant condition results in no rating bias and the emotionally stable individuals’ ratings are invariant to bias because the cues are psychologically irrelevant. One practical implication is, because the evaluators in the current study are non-experts/laypersons, it gives practitioners insights into what and how situational cues and personality traits sway evaluations in a form of rating inflation or deflation. This extents knowledge on the use of the CAT in interpersonal settings. In previous research, it had been focus on limitations and applications regarding the creators’ race and gender (Kaufman et al., 2008, 2009, 2010; Lebuda & Karwowski, 2013). Creative works are conceived by marketing creatives, architects, painters, sculptors, photographers, or multi-media personas. Because their products are massively received and evaluated by the general public (more of whom are novices), the study’s results also suggest perceived legitimacy (in a form of interpersonal dominance) and likeability (in a form of friendliness) of the creative individual that are entrenched in the evaluative outcome. In addition, the outcomes are convoluted by individual differences in neuroticism. In the opinion of the author, extreme forms of criticism or outburst from online evaluations about creative works can be attributed to the above dynamics. There remains a possible gap for future studies to address; that of the expert evaluators and how situations and their personality can sway creative judgments. Another implication is the significance in the industrial psychology domain. The nature of the evaluation presently is one related to creativity. Yet, it is possible to extend the findings to performance appraisal in general. That is, How would personality (neuroticism) (CAU) and situations (dominance) (if) affect appraisal of work performance (then)? A plausible enquiry would be: Would the appraisal of work performance be better served by an individual low or high in neuroticism? The results from the current study would suggest the former—that is, appraisal would be better served by an individual low in neuroticism. Furthermore, given that appraisals at work are predominantly 10
Journal of Creative Behavior
made by superiors, the results suggest that those in a state of dominance who are also high in neuroticism may produce the most biased appraisals. On this basis, human resource management may need to conduct appraisals based on multi-source systems. The author notes that, due to the limits imposed by the relatively low sample size in both studies, the stability of the trends shown will rest on future studies in similar and different contexts. The same also applies to the use of different stimuli as the basis of RBias. Currently, the results applied to tangible items— that is, the use of LEGO blocks for creativity performance. However, it may not apply to non-tangible or non-creative tasks such as perceived interpersonal outcomes/dynamics.
REFERENCES Amabile, T.M. (1983). The social psychology of creativity. New York: Springer Verlag. Bales, R.F. (1999). Social interaction systems: Theory and measurement. New Brunswick, NJ: Transaction. Baron, R.M., & Kenny, D.A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. Bass, B.M. (1960). Leadership, psychology, and organizational behavior. New York: Harper. Bliese, P.D. (2000). Within-group agreement, non-independence and reliability: Implications for data aggregation and analysis. In K.J. Klein & S.W.J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 351–424). San Francisco: Jossey-Bass. Bluhm, C., Widiger, T.A., & Miele, G.M. (1990). Interpersonal complementarity and individual differences. Journal of Personality and Social Psychology, 58, 464–471. Boden, M.A. (2004). The creative mind: Myths and mechanisms. Hove, England: Psychology Press. Bolger, N., & Zuckerman, A. (1995). A framework for studying personality in the stress process. Journal of Personality and Social Psychology, 69, 890–902. Cable, D.M., & Edwards, J.R. (2004). Complementary and supplementary fit: A theoretical and empirical integration. Journal of Applied Psychology, 89, 822–834. Caprara, G.V., Barbaranelli, C., & Zimbardo, P.G. (1996). Understanding the complexity of human aggression: Affective, cognitive, and social dimensions of individual differences in propensity toward aggression. European Journal of Personality, 10, 133–155. Cervone, D. (2004). The architecture of personality. Psychological Review, 111, 183–204. Chand, I., & Runco, M.A. (1992). Problem finding skills as components of the creative process. Personality and Individual Differences, 14, 155–162. Christiaans, H.H.C.M. (2002). Creativity as a design criterion. Creativity Research Journal, 14, 41–54. Costa, P.T., & McCrae, R.R. (1992). Revised NEO personality inventory (NEO PI-R) and NEO five-factor inventory (NEO FFI): Professional manual. Lutz, FL: Psychological Assessment Resources. C^ ote, S., & Moskowitz, D.S. (1998). On the dynamic covariation between interpersonal behavior and affect: Prediction from neuroticism, extraversion, and agreeableness. Journal of Personality and Social Psychology, 75, 1032–1046. Cronbach, L.J. (1955). Processes affecting scores on “understanding of others” and “assumed similarity”. Psychological Bulletin, 52, 177–193. Fournier, M.A., Moskowitz, D.S., & Zuroff, D.C. (2008). Integrating dispositions, signatures, and the interpersonal domain. Journal of Personality and Social Psychology, 94, 531–545. French, J.R., & Raven, B. (2004). The bases of social power. Modern Classics of Leadership, 2, 309–326. Funder, D.C. (2009). Naive and obvious questions. Perspectives on Psychological Science, 4, 340–344. Groborz, M., & Necka, E. (2003). Creativity and cognitive control: Explorations of generation and evaluation skills. Creativity Research Journal, 15, 183–197. Grohman, M., Wodniecka, Z., & Kłusak, M. (2006). Divergent thinking and evaluation skills: Do they always go together? Journal of Creative Behavior, 40, 125–145. Hastie, R., & Rasinski, K.A. (1988). The concept of accuracy in social judgment. In D. Bar-Tal & A.W. Kruglanski (Eds.), The social psychology of knowledge (pp. 193–208). New York: Cambridge University Press. Hui, C.H., Gan, Y., & Cheng, K. (2000). The Chinese Personality at Work Questionnaire: Development, validation, and significance. Acta Psychologica Sinica, 32, 443–452. Jawahar, I.M. (2001). Attitudes, self-monitoring, and appraisal behaviors. Journal of Applied Psychology, 86, 875–883. Kasof, J. (1995). Explaining creativity: The attributional perspective. Creativity Research Journal, 8, 311–366. Kaufman, J.C., Baer, J., Agars, M.D., & Loomis, D. (2010). Creativity stereotypes and the consensual assessment technique. Creativity Research Journal, 22, 200–205. Kaufman, J.C., Baer, J., & Cole, J.C. (2009). Expertise, domains, and the consensual assessment technique. Journal of Creative Behavior, 43, 223–233. Kaufman, J.C., Baer, J., Cole, J.C., & Sexton, J.D. (2008). A comparison of expert and nonexpert raters using the consensual assessment technique. Creativity Research Journal, 20, 171–178. Kaufman, J.C., & Sternberg, R.J. (Eds.) (2010). The Cambridge handbook of creativity. Cambridge: Cambridge University Press.
11
Personality and Rating Bias in Creative Work
Kozbelt, A. (2007). A quantitative analysis of Beethoven as self-critic: Implications for psychological theories of musical creativity. Psychology of Music, 35, 144–168. Lebuda, I., & Karwowski, M. (2013). Tell me your name and I’ll tell you how creative your work is: Author’s name and gender as factors influencing assessment of products’ creativity in four different domains. Creativity Research Journal, 25, 137–142. Leikas, S., L€ onnqvist, J.E., & Verkasalo, M. (2012). Persons, situations, and behaviors: Consistency and variability of different behaviors in four interpersonal situations. Journal of Personality and Social Psychology, 103, 1007–1022. Mischel, W., & Shoda, Y. (1998). Reconciling processing dynamics and personality dispositions. Annual Review of Psychology, 49, 229–258. Moskowitz, D.S. (1994). Cross-situational generality and the interpersonal circumplex. Journal of Personality and Social Psychology, 66, 921–933. Ng, K.Y., Koh, C., Ang, S., Kennedy, J.C., & Chan, K.Y. (2011). Rating leniency and halo in multisource feedback ratings: Testing cultural assumptions of power distance and individualism-collectivism. Journal of Applied Psychology, 96, 1033–1044. Pervin, L.A. (1989). Persons, situations, interactions: The history of a controversy and a discussion of theoretical models. Academy of Management Review, 14, 350–360. Runco, M.A., & Smith, W.R. (1992). Interpersonal and intrapersonal evaluations of creative ideas. Personality and Individual Differences, 13, 295–302. Sawyer, R.K. (2006). Explaining creativity: The science of human innovation. New York: Oxford University Press. Schneider, B. (1987). The people make the place. Personnel Psychology, 40, 437–453. Silvia, P.J. (2008). Discernment and creativity: How well can people identify their most creative ideas? Psychology of Aesthetics, Creativity, and the Arts, 2, 139–146. Silvia, P.J., & Gendolla, G.H.E. (2001). On introspection and self perception: Does self-focused attention enable accurate self-knowledge? Review of General Psychology, 5, 241–269. Simonton, D.K. (1998). Creativity and genius. In H.S. Friedman (Ed.), Encyclopedia of mental health (Vol. 1, pp. 607–617). San Diego, CA: Academic Press. Simonton, D.K. (2000). Cognitive, personal, developmental, and social aspects. American Psychologist, 55, 151–158. Simonton, D.K. (2012). Taking the US Patent Office criteria seriously: A quantitative three-criterion creativity definition and its implications. Creativity Research Journal, 24, 97–106. Spence, J.R., & Keeping, L.M. (2010). The impact of non-performance information on ratings of job performance: A policy-capturing approach. Journal of Organizational Behavior, 31, 587–608. Trapnell, P.D., & Wiggins, J.S. (1990). Extension of the Interpersonal Adjective Scales to include the Big Five dimensions of personality. Journal of Personality and Social Psychology, 59, 781–790. Wiggins, J.S. (2003). Paradigms of personality assessment. New York: Guilford Press. Winnifred, R.L. (2015, Jan 29). Moderated multiple regression: Work notes and syntax. Available from: https://www2.psy.uq.edu.au/~uqwloui1/stats/moderationv6.doc [last accessed January 29, 2016]. Wong, K.F.E., & Kwong, J.Y.Y. (2007). Effects of rater goals on rating patterns: Evidence from an experimental field study. Journal of Applied Psychology, 92, 577–585. Zheng, L., Goldberg, L.R., Zheng, Y., Zhao, Y., Tang, Y., & Liu, L. (2008). Reliability and concurrent validation of the IPIP BigFive-Factor markers in China: Consistencies in factor structure between Internet-obtained heterosexual and homosexual samples. Personality and Individual Difference, 45, 649–654. Kevin H. C. Cheng, Tung Wah College Correspondence concerning this article should be addressed to Kevin H. C. Cheng, Department of Rehabilitation and Social Sciences, Tung Wah College, 31 Wylie Rd., Homantin, KLN, Hong Kong. E-mail:
[email protected]
AUTHOR NOTE The author extends his gratitude to C Harry Hui and Wayne Cascio. Their mentorships inspired this study. I wish the very best for Harry and hope he will continue to be productive when his retirement commences this year.
SUPPORTING INFORMATION Additional Supporting Information may be found in the online version of this article at the publisher’s web-site: Figure S1. Lego blocks used for design exercise. Figure S2. (a) Practice pictorials. (b) Creative exercise pictorials. Figure S3. (a) Dominant versus Non-dominant Manipulation (Study 1). (b) Friendly versus Non-friendly Manipulation (Study 2). (c) Objective Model (OM) for rock-legend versus angry boss.
12
