Visual mismatch negativity indicates automatic, task

Biological Psychology 136 (2018) 76–86

Contents lists available at ScienceDirect

Biological Psychology journal homepage: www.elsevier.com/locate/biopsycho

Visual mismatch negativity indicates automatic, task-independent detection of artistic image composition in abstract artworks

T

Claudia Menzela,1, Gyula Kovácsb,c, Catarina Amadob,2, Gregor U. Hayn-Leichsenringa,3, ⁎ Christoph Rediesa, a

Experimental Aesthetics Group, Institute of Anatomy I, Jena University Hospital, University Jena School of Medicine, Jena, Germany Institute of Psychology, Friedrich Schiller University Jena, Jena, Germany c Brain Imaging Centre, Research Centre for Natural Sciences, Hungarian Academy of Sciences, Budapest, Hungary b

A R T I C LE I N FO

A B S T R A C T

Keywords: Visual mismatch negativity (vMMN) Art Image composition EEG

In complex abstract art, image composition (i.e., the artist’s deliberate arrangement of pictorial elements) is an important aesthetic feature. We investigated whether the human brain detects image composition in abstract artworks automatically (i.e., independently of the experimental task). To this aim, we studied whether a group of 20 original artworks elicited a visual mismatch negativity when contrasted with a group of 20 images that were composed of the same pictorial elements as the originals, but in shuffled arrangements, which destroy artistic composition. We used a passive oddball paradigm with parallel electroencephalogram recordings to investigate the detection of image type-specific properties. We observed significant deviant-standard differences for the shuffled and original images, respectively. Furthermore, for both types of images, differences in amplitudes correlated with the behavioral ratings of the images. In conclusion, we show that the human brain can detect composition-related image properties in visual artworks in an automatic fashion.

1. Introduction In the field of aesthetics, there has been a long-standing debate as to whether visual artworks possess a characteristic image composition (“significant form”; Bell, 1914), by which we mean a formal image structure that is intended by the artist and contributes to the sensory visual appeal of the artwork. The search for such objective image properties in artworks has gained new momentum during the last decade, mainly due to technological advances in computer-based image analysis (“computational aesthetics”; Brachmann & Redies, 2017; Hoenig, 2005). The notion that there are specific visual features in artworks has been advanced also by artists, philosophers and art critics (Dowling, 2014; Fechner, 1876; Greenberg, 1955; Kandinsky, 1912; Malevich, 1927). Some visual psychologists have put forward similar ideas (“good composition”, Arnheim, 1954; “visual rightness”, Locher, Stappers, & Overbeeke, 1999). In support of the formalist concept of visual aesthetic judgment, a number of objective image properties have been identified in large sets of artworks in recent years. Most of these features reflect global image

⁎

structure. Examples are luminance and color statistics, image complexity, pictorial balance, Fourier spectral properties, fractality and selfsimilarity, as well as regularities in the orientations of luminance gradients, edges and lines (for reviews, see Brachmann & Redies, 2017; Graham & Redies, 2010). Besides the objective properties of visual stimuli, the observers' responses and aesthetic judgments have also been the subject of modern aesthetics research. A major challenge in this field has been the lack of clear definitions of the terms that human observers use to describe artworks or other visual stimuli. There have been several exploratory studies on the usage of the diverse aesthetic terms (Augustin, Wagemans, & Carbon, 2012; Jacobsen, Buchta, Kohler, & Schroger, 2004; Locher, Krupinski, Mello-Thoms, & Nodine, 2007; Markovic & Radonjic, 2008) as well as their relation to image properties (Lyssenko, Redies, & Hayn-Leichsenring, 2016; Mallon, Redies, & HaynLeichsenring, 2014; Redies, Brachmann, & Hayn-Leichsenring, 2015). These studies have shown that different observers associate a vast number of terms with beauty or artworks (Jacobsen et al., 2004; Lyssenko et al., 2016). To highlight such terms that refer to subjective

Corresponding author at: Institute of Anatomy I, Jena University Hospital, D-07740, Jena, Germany. E-mail address: [email protected] (C. Redies). 1 Present address: Social, Environmental and Economic Psychology, Department of Psychology, University of Koblenz-Landau, Fortstraße 7, 76829, Landau, Germany. 2 Present address: Experimental Cognitive Science, Department of Computer Science, Eberhard Karls University of Tübingen, Tübingen, Germany. 3 Present address: Center for Cognitive Neuroscience, University of Pennsylvania, 3720 Walnut Street, Philadelphia, PA, 19104-6241, U.S.A. https://doi.org/10.1016/j.biopsycho.2018.05.005 Received 31 October 2017; Received in revised form 2 March 2018; Accepted 2 May 2018 Available online 06 May 2018 0301-0511/ © 2018 Elsevier B.V. All rights reserved.


C. Menzel et al.

cognitive processing of depicted content because, by definition, cognitive processing is absent for abstract art. By studying artworks of a single artist, previously unknown to the participants, we minimized differences in the art-historical context between the two types of images. In summary, the two types of images (original and shuffled) differed in their form (with and without artistic composition) but not in their content or context. In a previous study, human observers rated the two types of images differently (Redies et al., 2015). The original drawings were rated as more harmonious and ordered as well as less interesting than their shuffled counterparts. With respect to their statistical image properties, selfsimilarity (defined as the similarity of the histogram of gradient orientations in parts of the images compared to the entire image; Amirshahi, Koch, Denzler, & Redies, 2012), was higher in the original drawings than in the shuffled versions. The original drawings, but not the shuffled versions, had a self-similarity comparable to a control set of 200 graphic artworks of Western provenance. The calculated luminance gradient density, which relates to subjective complexity, and the distribution of edges across orientations (anisotropy) were also different between the two types of stimuli, but to a lesser degree. A small difference was observed also for the slope of log-log plots of radially averaged Fourier power (Redies et al., 2015). Thus, shuffling the pictorial elements and thereby destroying the intended artistic composition in the images affected objective image properties, which tend to be processed automatically at a mid-level of the visual system (Graham & Redies, 2010; Redies, 2015). We here define processing as automatic if it is independent of the experimental task and, hence, demands little or no attention. In the present study, we speculated that the presence or absence of artistic composition in otherwise similar images elicits a differential response in the human visual system. Therefore, we presented the original and shuffled images in a passive oddball paradigm, which is a well-suited tool to investigate automatic detection of stimulus-specific regularities (“categories”). The paradigm is designed to elicit a visual mismatch negativity (vMMN) when the brain detects a difference between two categories. A vMMN is a (mostly, but not exclusively negative) differential brain response measured by event-related brain potentials (ERPs). It is defined as the difference in elicited activity between a frequently presented stimulus or category (“standard”) and an infrequently interjected one (“deviant”; for reviews on vMMN, see Bodnar, File, Sulykos, Kecskés-Kovác, & Czigler, 2017; Czigler, 2007; Kimura, 2012). The mechanisms that underlie vMMN are still debated (e.g., Kimura, 2012; Stefanics, Kremlacek, & Czigler, 2014), but it has become clear that both stimulus repetition-related adaptation as well as expectation-related mechanisms play a role (Amado & Kovács, 2016; Kimura, Katayama, Ohira, & Schroger, 2009). Previous studies found vMMN for different stimulus properties such as orientation (Kimura et al., 2009), vertical mirror symmetry (Kecskés-Kovács, Sulykos, & Czigler, 2013b) or categories such as face gender (Kecskés-Kovács, Sulykos, & Czigler, 2013a), among others. For our stimuli, we speculate that the brain automatically detects composition in the artworks so that the original images built an own category, which does not include the shuffled images. We would thus expect a vMMN for shuffled images if they are deviants in a series of original images. If vice versa, the shuffled images are the standard and the original images are the deviants, we can imagine two possible scenarios. First, if the shuffled images do not form an own category, the original images do not deviate in this context and we would expect no vMMN. Previously, Kecskés-Kovács et al. (2013a, 2013b) obtained this type of result in their study on vertical mirror symmetry. They found that a random pattern elicited a vMMN when presented among symmetric patterns, but a symmetric pattern among random patterns did not elicit a vMMN. The authors concluded that the random patterns built no own category and thus the symmetric patterns were not deviating in this context. In this scenario, which we expect based on the above results by Kecskés-Kovács et al. (2013a, 2013b), there is no

impressions or aesthetic judgments, we use italic script in the present study (e.g., harmonious, ordered, liking etc.). Some of these terms are more generally applied to objects or scenes (e.g. beauty; Jacobsen et al., 2004; Augustin et al., 2012) while others are applied more specifically (for example, wonderful to landscapes; Augustin et al., 2012). Some terms are used to describe the object's physical features (e.g. structured; Lyssenko et al., 2016) while others refer more closely to emotions that are elicited in the viewer (e.g., warm; Lyssenko et al., 2016). Note that, due to their subjective nature, most of these terms lack a precise definition. Besides formalist approaches to aesthetics, contextual theories have been advanced in aesthetics, especially during the last century. According to these ideas, the aesthetic experience of artworks is based predominantly, if not exclusively, on their social and art-historical context, for example, in relation to the circumstances of their creation, the intentions of the artist and their mode of presentation (Danto, 1981; Dickie, 1974; Goodman, 1968). Such ideas are fundamental to most psychological models of aesthetic experience, in particular for (post-) modern and contemporary art (Bullot & Reber, 2013; Conway & Rehding, 2013; Pearce et al., 2016). Therefore, even the question of what constitutes an artwork can by no means be answered unequivocally. However, formal and contextual theories of aesthetic experience do not need to be mutually exclusive and can be combined in an integrative model, wherein two independent channels process the specific form of artworks (lower-level, sensory “perceptual” processing) and the contextual information of artworks (higher-level, “cognitive” processing), respectively (Redies, 2015). Some formal theories of aesthetic experience postulate that the significant form of artworks (Bell, 1914) elicits a particular state of neural resonance in the visual system (Taylor, Newell, Spehar, & Clifford, 2005) or causes the activation of a specific neural mechanism in (beauty-responsive) brain regions or circuits (Redies, 2015). It is thought that this specific activity is the neural correlate of visual preference or the perception of beauty (Redies, 2007). Whereas the role of cultural context and depicted content in aesthetic experience has been subject of many experimental studies (for a review, see Pearce et al., 2016), the response of the human brain to the formal composition of artworks, i.e. specific statistical image properties that are associated with artworks, has been studied less frequently by experimental manipulations (Graham & Redies, 2010; Locher, Overbeeke, & Stappers, 2005; McManus, Cheema, & Stoker, 1993). One reason for this paucity may lie in the difficulty in generating well-controlled stimuli that differ only in their image properties but not in depicted content matter, so that the role of formal composition in artworks can be investigated experimentally in isolation from issues of content or context. Moreover, formal aspects of image composition have been studied experimentally in geometrical patterns, but rarely in more complex stimuli or artworks (Götz, Borisy, Lynn, & Eysenck, 1979). For example, Jacobsen (2004) studied perceived complexity and vertical/horizontal mirror symmetry in simple graphic patterns. Wilson and Chatterjee (2005) investigated balance, which they defined as a pictorial structure in which the visual forces of the elements compensate for each other, in images composed of just a few squares or circles. To overcome the shortage of studies on how brain responses to artworks relate to formal image composition, we used a set of 20 original abstract artworks that was created by one of the authors (Redies et al., 2015). Each drawing consisted of 52–127 abstract pictorial elements (patches, lines and dots or small groups thereof), which were arranged in a way so as to satisfy the aesthetic objective of the artist. We compared each original image to a modified version of the same image wherein the pictorial elements were randomly arranged (shuffled) by a computer program. By shuffling the pictorial elements, we deliberately destroyed the specific image composition, which the artist conceived and realized in his drawings (artistic composition). Therefore, we assume that the shuffled versions of the images lacked artistic composition. Moreover, by using abstract artworks, we minimized the 77


C. Menzel et al.

elements per image) could be moved independently with a custommade computer program. In the shuffled versions of the images, each element was placed at a random position (Fig. 1). The shuffling process was limited so that the layering of the elements in each image (i.e., large elements in the background and smaller elements in the foreground) was largely preserved. As a result, the two image categories (original and shuffled) differed only in their artistic composition, which we here define as the arrangement of elements within the image that was intended by the artist. Due to the different organization of elements in the original and shuffled versions, some image properties differed between the two conditions (see Introduction). The image properties and details of their calculations are described in Redies et al. (2015). Images were 800 × 600 pixels large (corresponding to 11.9° × 8.9° visual angle at a viewing distance of 100 cm). The luminance histograms of each image pair (original and shuffled) were equalized using the histMatch function of the SHINE toolbox in MATLAB (adapted settings: rescaling of absolute values, no SSIM optimization; Willenbockel et al., 2010). Images were presented on a white computer screen. We used an LCD-monitor (Eizo Foris FS2333 with 1920 × 1080 pixels) that was calibrated before the start of the experiment by using the i1Display Pro device and the i1Profiler software (both X-Rite; Grand Rapids, USA). The calibration included gamma linearization. A red fixation cross was shown on top of the images. The cross was asymmetric and its orientation changed within the experiment (see below and Fig. 2). The longer side of the cross was 30 pixels long while the shorter side was 10 pixels in length.

vMMN for original images in the context of shuffled ones. As an alternative, one could speculate that the shuffled images do form an own category that is different from the category of the original images. In this case, we would expect a vMMN. In all cases, we anticipate an early emergence of the vMMN over visual processing areas because our stimulus categories differ in their mid-level image properties (see above). 2. Materials & methods 2.1. Participants Twenty-two participants took part in the study. Five of them were excluded from the analyses because their EEG data were of poor quality (extreme drifts and/or alpha waves). Of the remaining 17 participants, four were male and all were right-handed. Their age ranged from 19 to 31 years, with a mean of 23.4 ( ± 3.1 SD) years. No participant reported a history of neurological or psychiatric illness. All stated normal or corrected-to-normal vision. None of the participants was an art expert or had studied art, design or art history. Participants received either partial course credit or monetary reimbursement. They gave their written consent before the start of the experiment. The experiment was carried out in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Jena University Hospital, Jena, Germany. 2.2. Stimuli Stimuli were 20 digital grayscale abstract artworks that were created by one of the authors (Fig. 1; Redies et al., 2015). In composing the drawings, the author did not consciously follow any explicit rules. Moreover, he did not introduce any figurative or contextual meaning in the drawings. His only intention was to create visually pleasing images that represented a harmonious visual composition. All stimuli were shown in the supplemental material of a previous study (Redies et al., 2015). Images were generated so that all constituent pictorial elements in the image (blotches, lines and dots or combinations thereof; 52–127

2.3. Design and procedure We applied an oddball paradigm, in which stimuli of one type were presented as standards (with a probability of 0.8). Presentation was alternated by a few interspersed stimuli of the other type (deviants; probability of 0.2). In one condition, original images were the standards and the shuffled versions were the deviants, while in the other condition, it was the other way around. Standard images within one trial block consisted of different images from the same type and were

Fig. 1. Example stimuli. (A–C) Original images that were produced with the intention to create a harmonious, visually pleasing arrangement of the pictorial elements in each image (artistic composition). (D–F) Shuffled images that contain the same elements as the original images shown on the top, respectively, but at randomized positions (without artistic composition). Reproduced with permission. © Christoph Redies, 2018. 78


C. Menzel et al.

Fig. 2. Procedure of the main experiment. In this example, original images were standards (presented in 80% of the trials) and shuffled images were deviants (presented in 20% of the trials). Participants were asked to detect a change in a red fixation cross that was presented at the center of the stimuli. The two versions of the fixation cross can be seen at the bottom left. Note that, for better visibility, the cross is depicted here larger than in the experiment. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

presented in sequences of three to five images until a deviant image appeared. For the temporal parameters of stimulus presentation, see Fig. 2. In a given block, the sequence started always with a standard and ended with a deviant. Each block consisted of a sequence of 100 images: four times exemplars from the 20 images that were shown as a standard and once an exemplar from the 20 images of the other type. For each condition (original as standard or shuffled as standard), participants ran ten blocks. Thus, all images were presented at the same frequency during the experiment (i.e., 50 times). The stimuli were irrelevant to the participants’ task. Participants were asked to respond to a change in orientation of the centrally presented fixation cross (Fig. 2). The cross changed every 5–15 trials (i.e., every 5–15 s on average). Trials, in which the cross was changed, were not included in the analysis. After application of the EEG cap and electrodes, participants were seated in a quiet, dimly lit and electrically shielded room. The viewing distance of 100 cm was assured by a chin rest. Participants were introduced to the task and performed a 30-s long training phase without image presentation. Within this time period, the fixation cross changed its orientation three to fifteen times. During the training, we gave visual feedback (“correct” when change was detected, “false” when key was pressed although no change occurred or “you missed the change” when no key press was registered). After the training, participants started the experiment self-initiated. There was no visual feedback during the main experiment. After each of the 20 blocks, participants were allowed to take a break of desired length. The main experiment took about 35 min plus breaks.

artifact rejection, we excluded trials that contained an amplitude change larger than ± 70 μV on any of the recorded electrodes, including the eye electrodes. Therefore, trials with eye movements were also excluded from further analyses. For analysis of the vMMN, the ERP of the last standard in a sequence was compared with the ERP of the deviant. Using this strategy, the trial numbers were comparable in the conditions. After artifact rejection, mean trial numbers were 140.5 ( ± 30.4 SD) for original last standards, 138.6 ( ± 28.8 SD) for original deviants, 137.9 ( ± 31.2 SD) for shuffled last standards and 139.2 ( ± 30.2 SD) for shuffled deviants.

2.4. ERP recording and processing

2.6. Rating

We used the 64-channel Active II system (BioSemi, Amsterdam, The Netherlands) with a sampling rate of 512 Hz. The electrode sites were Fp1, FT9, AF3, F1, F3, F5, F7, FT7, TP9, FC3, FC1, C1, C3, C5, T7, TP7, PO9, CP3, CP1, P1, P3, O9, P7, P9, PO7, PO3, O1, Iz, Oz, POz, Pz, CPz, Fpz, Fp2, FT10, AF4, Afz, Fz, F2, F4, F6, F8, FT8, TP10, FC4, FC2, FCz, Cz, C2, C4, C6, T8, TP8, PO10, CP4, CP2, P2, P4, O10, P8, P10, PO8, PO4 and O2. The BioSemi system includes a CMS/DRL feedback loop (see http://www.biosemi.com/faq/cms&drl.htm). Additionally, we recorded the horizontal electro-oculogram from the outer canthi of both eyes. The vertical electro-oculogram was recorded from above and below the left eye. The EEG signal was analyzed offline using the EEGLAB software from MATLAB. Data were re-referenced to the average of the 64 electrodes. We applied a 0.1 Hz high-pass filter (12 dB/octave, Butterworth, zero-phase shift) and a 25 Hz low-pass filter (24 dB/octave, Butterworth, zero-phase shift). The EEG signal was then segmented in 1000 ms epochs, including a 100 ms baseline. By the use of automatic

After the main experiment, participants ran three rating blocks: rating of harmony, order and hedonic value (“liking”), respectively. The order of the three blocks was randomized between participants. We asked participants to choose the image of each pair (original or its shuffled version), which they perceived as more harmonious, ordered or liked more. We gave no definition or further explanation for the rating terms. The two images of each pair were presented sequentially and the order of the version (original/shuffled) was randomized (for the experimental schedule, see Fig. 3). Participants indicated their choice for the image of each pair, which they perceived as more harmonious, ordered or liked more, via key press after the presentation of the second image. For each participant, we counted how many of the respective images were selected from the total of 20 pairs. Preference towards either image type was then calculated using binominal tests. Additionally, the proportion of original images that were chosen in a given rating block was calculated for each participant. Then, this value

2.5. Statistical analyses For the statistical analyses of the data, we focused on the lateral and posterior electrodes only (O1/2, P1/2, P3/4, P7/8, P9/10, PO3/4, PO7/8, PO9/10, CP1/2, CP3/4, T7/8, TP7/8, TP9/10, Oz, POz, Pz and CPz). At these electrodes, we applied point-by-point t-tests to estimate the differences of the evoked responses for standards and deviants. A difference was considered significant when it reached a level of p < .05 at least at two neighboring electrode sites for a minimum of 20 data points in a row (i.e., 40 ms). Results are presented in time-byelectrode plots in which the significant difference between the amplitudes of the deviant and standard are color-coded. Moreover, we calculated similar point-by-point t-tests to compare the absolute vMMNs of original and shuffled images. This comparison was done only in the time windows, in which both conditions revealed significant vMMNs.

79


C. Menzel et al.

Fig. 3. Procedure of rating task. Each image of a pair was shown for 400 ms. Following the second image, participants were asked to indicate which image they found either more ordered, more harmonious, or which they liked more, respectively.

participants perceived a difference in order and harmony between the conditions (Table 2). Ratings on liking were not different from chance although six of the participants showed a significant preference for the shuffled images (Table 2).

was correlated with the amplitude differences (deviant-minus-standard) that were averaged over the time windows in which vMMN was significant. This analysis was done exclusively at electrode sites, at which we found vMMN. We corrected for multiple comparisons with the Bonferroni-Holm method.

3.4. Correlation of ratings with vMMN 3. Results We calculated Pearson correlation coefficients between the proportion of original images that were chosen in the rating tasks and the vMMN amplitude differences (deviant-minus-standard; Tables 3 and 4). The proportion of the original images that were chosen as more harmonious correlated positively with the amplitude differences between original deviant and standard images at the electrode Pz (Table 3). The proportion of original images chosen as more ordered and harmonious correlated positively with the amplitude differences between shuffled deviants and standards at electrodes Pz and PO4 (Table 4).

3.1. Behavioral results Participants detected 98.53% (SD = 3.26) of fixation cross changes. There was no difference in accuracy between conditions (original versus shuffled images as deviants) in the change detection task (t [16] = −.46, p = .65). On average, participants needed 434 ms (SD = 48.5) to detect the change in cross orientation. There was no difference in reaction time between the conditions (t[16] = .80, p = .43).

4. Discussion 3.2. ERP results In the present study, by using a paradigm to elicit a vMMN, we observed differences in the neural processing of deviant and standard presentation of both original abstract artworks and their shuffled counterparts. For the original images, differences between deviant and last standards were positive, while for the shuffled images, the difference was negative. These results show that the brain reacts differently to the two image categories, which differ only in whether or not they have an image composition, which the artist intended to be harmonious and, presumably, in an undetermined image property that is associated with it. Furthermore, the present results confirm findings that the original drawings are rated as more ordered and more harmonious than the shuffled versions (Redies et al., 2015). Here, we show that these ratings correlate with the amplitude of the vMMN over specific electrode positions.

Time-by-electrode plots show significant deviant-minus-standard differences for both original and shuffled images (Fig. 4A, B). These differences were positive for the original images and negative for the shuffled images. Thus, original deviants elicited more positive amplitudes than original standards, and shuffled deviants elicited more negative amplitudes than shuffled standards (Fig. 4C, D). For both image types, differences were most pronounced over the occipital midline, with significant differences at electrodes POz, Pz, PO3 and PO4 for both stimulus types, and at electrodes Oz, Cpz, O1, O2, P1, P2, P3, P4 and TP7 for either type (Fig. 4A, B; E, F). The first significant difference started at 146 ms at PO4 or 92 ms at PO3, for the original or shuffled images, respectively (Fig. 4A, B; Table 1). For the original images, the differences between deviants and standards were significant in a time window between 146 and 561 ms, with two short periods in which they did not reach significance (Table 1). For the shuffled images, vMMN was significant in four time windows, which can be separated into an early (92–322 ms) and a late (666–871 ms) phase (Table 1). The comparison of original and shuffled vMMNs revealed that, independent of polarity, original images elicited a significantly larger vMMN at POz (t > 3.78, p < .002; time range: 277–297 ms), Pz (t > 4.00, p < .001; time range: 279–303 ms) and PO4 (t > 2.79, p < .005; time ranges: 147–188 ms, 217–248 ms and 258–307 ms).

4.1. Localization and time course of vMMN signal To the best of our knowledge, this study is the first to use a vMMN paradigm in order to test whether the human central nervous system differentiates between complex abstract images with artistic composition and their non-artistic (shuffled) counterparts. Our results suggest the existence of early and late vMMN effects. The earlier time window (up to around 200 ms) corresponds to what previous studies found mostly over parietal and occipital electrodes (Amado & Kovács, 2016; Korpilahti, Krause, Holopainen, & Lang, 2001; Morlet & Fischer, 2014). These early differences might be related to image properties that are associated with the composition of our stimuli, such as self-similarity (Redies et al., 2015). The later component, observed over more central and centro-parietal electrodes, appears to

3.3. Rating data For the harmonious and ordered ratings, the participants chose the original image significantly more often than the shuffled ones. Thus, the 80


C. Menzel et al.

Fig. 4. Summary of ERP results for original (A, C, E) and shuffled images (B, D, F). (A, B) Time-by-electrode plots. Amplitude differences between deviant and standard stimuli are color-coded. (C, D) Representative ERPs at channel Pz (for original images in C) and Oz (for shuffled images in D). (E, F) Scalp maps for the time windows 217–457 ms (for original images in E) and 92–248 ms (for shuffled in F).

81


C. Menzel et al.

sustained posterior negativity in the symmetric condition. Additionally, all conditions elicited late positive potentials. The authors argued that, while the earlier frontomedian differences reflect impression formation, the later posterior components signal the evaluative categorization of the stimuli. In our current study, the earlier difference at around 300 ms was present for both the original and the shuffled vMMNs. However, the later vMMN difference between 666 and 900 ms was only present for the shuffled images. Whether this later difference reflects processes similar to those proposed by Jacobsen and Höfel (2003), will require further studies. It is worth noting that the polarity of the observed difference waves was different for the original and shuffled images. While it was positive for the original images, the negative differences observed for the shuffled ones may correspond to traditional vMMN responses. Such polarity differences have been described previously (Amado & Kovács, 2016; Sulykos & Czigler, 2011). While the exact nature of this phenomenon is unknown, the experiments of Sulykos and Czigler (2011) suggest a cortical origin in early, retinotopic visual areas and a shift of the cortical generator across the calcarine sulcus. In the following discussion, we will refer to amplitude differences of either polarity with the abbreviation vMMN.

Table 1 Time windows (ms), during which the ERPs differed significantly between deviant and standard at one or more electrodes. original

shuffled

146–201 217–457 516–561

092–248 258–322 666–734 770–871

have two distinct phases, one around 300 ms and another around 700–800 ms. Such late vMMN components have been described previously over central electrodes, for example, for characters (Amado & Kovács, 2016), but also for other stimulus types (Kimura, 2012; Kreegipuu et al., 2013; Stefanics, Csukly, Komlosi, Czobor, & Czigler, 2012; Zhao and Li, 2006) or stimulus features such as vertical mirror symmetry (Kecskés-Kovács et al., 2013b). Thus, it seems that the vMMN effects, elicited by our abstract complex stimuli, which differ only in their artistic composition, lead to similar automatic change detection mechanisms as the previously tested visual stimuli. The observed vMMN amplitude is relatively small compared to that of some prior studies. However, this finding is not surprising for several reasons. First, instead of repeating the same identical stimulus, different standard stimuli were presented in a row in the present study. Second, the standard and deviant stimulus categories were composed of identical pictorial elements and differed in their spatial arrangement only. In fact, the observed vMMN amplitudes were similar to those of previous studies, in which several different standards were presented and the standards and deviants were also relatively similar (e.g., faces of both genders and asymmetric/symmetric patterns; Kecskés-Kovács et al., 2013a, 2013b). Because the present study is the first to use automatic change detection paradigms and vMMN measurements in order to investigate the neural correlates of the perception of abstract images with and without artistic composition, the comparison to prior studies is limited to the few available ERP recording studies. Sbriscia-Fioretti et al. (2013) measured ERPs for abstract paintings of Franz Kline and computergenerated images that consisted of the same elements and arrangement of the images, but lacked their dynamic visual (and presumably artistic) attributes. This study revealed a direct involvement of sensorimotor cortical areas, of the reward-related orbitofrontal cortex, as well as of cognitive categorization-related prefrontal areas in the perception of these stimuli. Pang, Nadal, Müller-Paul, Rosenberg, and Klein (2013) presented original and differentially filtered paintings as stimuli and found that the filtered images, in which high spatial frequency information is lost, led to smaller amplitudes in the P3b/late positivity complex (LPC) when compared to the original paintings. This component peaked between 245 and 390 ms after stimulus onset over posterior electrodes, a time-window and location that corresponds to the major vMMN difference of the current study. Jacobsen and Höfel (2003) used simple geometric stimuli and asked their participants to rate them subjectively according to their symmetry and beauty. The ERPs that were recorded in parallel revealed a phasic, early frontal negativity for the not-beautiful judgements as well as a

4.2. Automatic processing of image composition In order to study whether the processing of the formal image structure of the abstract artworks occurs automatically, we performed our experiments under the following conditions: First, we used two categories of stimuli that differed in whether or not they possessed an artistic composition, but not in any contextual features or the displayed content. While the pictorial elements relate to each other in an artistically purposeful manner in the original drawings (artistic composition), we assume that randomizing the position of the pictorial elements largely destroyed this specific artistic composition (shuffled images; see Introduction). Note that differences in the adaptation at a given location during the course of the experiment should be negligible because the location of the pictorial elements varies greatly between original and shuffled images and there are only four presentations of each standard stimulus during each block. A similar probability of deviants has been used in previous vMMN studies (Kimura et al., 2009; Li, Lu, Sun, Gao, & Zhao, 2012; Stefanics, Kimura, & Czigler, 2011; Stefanics et al., 2014) and a lower probability of deviants would have likely resulted in a larger vMMN (Lopez-Caballero, Zarnowiec, & Escera, 2016; Näätänen, Paavilainen, Rinne, & Alho, 2007). We propose that global image properties are at the basis of the observed vMMN. Although our stimuli were highly controlled, we previously identified statistical image properties that differ between the categories. Specifically, the original drawings have a higher self-similarity and a lower anisotropy than the shuffled ones (see Introduction and Redies et al., 2015). This pattern of image properties has been associated with artworks (Braun, Amirshahi, Denzler, & Redies, 2013; Redies, Amirshahi, Koch, & Denzler, 2012; Redies, Hasenstein, & Denzler, 2007). While it is difficult, if not impossible, to disentangle image properties and artistic composition in an image, we assume that these or other statistical image properties accompany artistic composition and thereby contribute to the vMMN effects, especially in the

Table 2 Rating data and differences between original and shuffled images (t-test).

Proportion original image was chosen ± SD t statistic p-value Number of participants with significant preference for original images (binomial test, p < .05) Number of participants with significant preference for shuffled images (binomial test, p < .05)

82

harmonious

ordered

liking

.68 ± .20 t(16) = 3.45 p = .003 5 0

.78 ± .16 t(16) = 6.91 p < .001 11 0

.43 ± .20 t(16) = −1.43 p = .171 0 6

252–359 r = .15 p = .570 r = .22 p = .397 r = .42 p = .097

Significant after Bonferroni-Holm correction.

227–279 r = .07 p = .776 r = .05 p = .852 r = .28 p = .283

P1 246–295 r = .35 p = .166 r = .31 p = .226 r = .47 p = .056

P3 230–307 r = .24 p = .362 r = .33 p = .198 r = .41 p = .101

PO3 318–389 r = −.05 p = .848 r = .13 p = .613 r = .36 p = .152

PO3 152–201 r = .10 p = .704 r = −.09 p = .731 r = .39 p = .126

POz 227–406 r = .15 p = .554 r = −.10 p = .699 r = .19 p = .457

POz 256–457 r = .42 p = .095 r = .21 p = .414 r = .31 p = .227

Pz 516–561 r = .59a p = .014 r = .22 p = .405 r = .37 p = .148

Pz 270–457 r = .31 p = .227 r = .16 p = .542 r = .24 p = .362

CPz 256–303 r = .13 p = .628 r = −.10 p = .700 r = .17 p = .506

P2 250–295 r = −.23 p = .383 r = −.01 p = .977 r = −.03 p = .904

P4 146–188 r = −.20 p = .445 r = −.20 p = .437 r = −.14 p = .592

PO4

83

92–135 r = .07 p = .561 r = −.02 p = .925 r = .07 p = .802

time window in ms harmonious

a

176–215 r = .16 p = .534 r = .10 p = .713 r = .12 p = .639

O1 131–213 r = .10 p = .692 r = −.09 p = .725 r = −.09 p = .728

Oz

Significant after Bonferroni-Holm correction.

liking

ordered

PO3

shuffled 695–734 r = −.04 p = .868 r = .14 p = .597 r = −.22 p = .396

Oz 770–871 r = .06 p = .812 r = .11 p = .675 r = −.13 p = .609

Oz 96–148 r = −.21 p = .417 r = −.05 p = .859 r = −.13 p = .616

POz 277–322 r = .08 p = .751 r = .31 p = .218 r = −.26 p = .306

POz 783–850 r = −.07 p = .804 r = .34 p = .182 r = −.13 p = .625

POz

279–318 r = .24 p = .364 r = .33 p = .193 r = .12 p = .639

Pz

666–734 r = .39 p = .119 r = .67a p = .003 r = .15 p = .555

Pz

781–830 r = .43 p = .088 r = .48 p = .051 r = .15 p = .560

Pz

135–248 r = .46 p = .066 r = .32 p = .204 r = .05 p = .834

PO4

258–322 r = .52a p = .032 r = .49 p = .045 r = .12 p = .641

PO4

783–857 r = .20 p = .437 r = .54a p = .025 r = .29 p = .253

PO4

139–244 r = .09 p = .741 r = .06 p = .805 r = −.02 p = .930

O2

217–307 r = −.20 p = .436 r = −.26 p = .317 r = −.19 p = .477

PO4

Table 4 Pearson correlation coefficients r (N = 17) between the average of the amplitude differences for shuffled images and the proportion of original images that were chosen in the behavioral task.

a

liking

ordered

time window in ms harmonious

TP7

Table 3 Pearson correlation coefficients r (N = 17) between the average of the amplitude differences for original images and the proportion of original images that were chosen in the behavioral task.

781–854 r = .09 p = .717 r = .30 p = .235 r = .13 p = .619

O2

328–383 r = −.36 p = .162 r = −.33 p = .189 r = −.30 p = .243

PO4

C. Menzel et al.



C. Menzel et al.

early time windows. Our results are reminiscent of findings by Durant, Sulykos, and Czigler (2017) who obtained vMMNs for oriented line stimuli when the global statistics of orientations differed, but only when disordered stimuli were presented as deviants. In contrast to a previous study (Kecskés-Kovács et al., 2013b; see above), we found vMMN for both types of stimuli: the original drawings and the shuffled versions. The authors of the previous study argued that their lack of a vMMN for mirror symmetrical patterns might indicate that asymmetrical patterns did not build a category and, therefore, symmetric stimuli did not deviate (for similar results with oriented line stimuli, see Durant et al., 2017). In our study, however, it seems that the shuffled images lead to the emergence of an own perceptual category as well. In fact, the vMMN was larger for original compared to shuffled images, indicating that original images are more surprising among shuffled ones than the other way around. However, the larger vMMN for original images can also be caused by larger adaptation to original compared to shuffled images. To what extent surprise/expectation and adaptation mechanisms contribute to the finding of a larger vMMN for original images, remains to be investigated. The notion that shuffled images build an own category is also supported by the finding that six participants preferred the shuffled images when they were asked to indicate which image they liked more. Moreover, in a previous study (Redies et al., 2015), participants rated the shuffled images as more interesting. Interest and pleasure are thought to represent independent aesthetic dimensions (Cupchik and Gebotys, 1990; Graf & Landwehr, 2015; Locher et al., 2007). It is therefore conceivable that the shuffling of the otherwise identical pictorial elements resulted in a transformed image structure with distinct perceptual qualities. The correlation analyses of the rating and vMMN revealed an association between explicit appraisal of the images and automatic brain responses (Tables 3, 4). The more harmonious the original images were rated, the larger the amplitude differences for original images (i.e., larger vMMN). However, the more ordered or harmonious the original images were rated, the less negative the amplitude differences for shuffled images (i.e., smaller vMMN). A larger/smaller vMMN can be caused by either a larger/smaller surprise of deviants among standards, or by a stronger/weaker adaptation to the standard images or both (cf., Amado & Kovács 2016). Thus, the positive correlation for the original images implies that the more harmonious original images were perceived, the more surprising they are among shuffled ones, and/or there is a stronger adaptation to the (standard) original images. In contrast, the correlations for the shuffled images indicate that participants who perceive the original images as more harmonious and ordered – and thus the shuffled images as less harmonious and ordered – experience a reduced surprise enhancement for shuffled deviants among original standards and/or a smaller adaptation to the shuffled standard images, compared to those participants who rated the image types more similar (i.e., rating proportion closer to 0.5). Because the arrangement of elements is more random and clustered in the shuffled images than in the original images (Fig. 1), local adaptation to the shuffled images may be lower than to the original images. Possibly, this reduced adaptation results in the smaller vMMN for shuffled images in individuals who perceive the shuffled images as less harmonious and ordered. The mechanisms that underlie such correlations remain to be studied. Interestingly we found no significant correlation of the vMMN with liking. This suggests that the vMMN relates more to ratings that are associated with image structure and composition (i.e., harmonious and ordered) than with the more subjective rating of liking. It was found previously − and is also suggested by the current rating data (Table 2) – that there are inter-individual differences in the more subjective ratings of the two image types (Redies et al., 2015). In general, however, the observed correlations and conclusions are limited because they are based on only 17 participants and, thus, lack good power. Moreover, the correlations were significant only at two electrodes, Pz and/or PO4, which are more medial and slightly more superior than the electrodes that typically show the largest vMMNs.

Also, the correlations were not significant in all time windows (however, correlation coefficients were similar for each electrode in the other time windows; Tables 3, 4). Finally, behavioral ratings and vMMN were measured in different experiments, which might weaken the relation between the two and thus yield a low number of significant correlations. Second, we minimized any differences between these two image categories that might be introduced by a cognitive evaluation of image context or cultural context. This goal was achieved by using a set of similar drawings of abstract (meaningless) content, which were created by a virtually unknown artist. Contextual differences were further minimized by presenting the images under well-controlled conditions in the laboratory, rather than by free viewing in an art museum. Although the aesthetic experience in the laboratory is generally less intense than in a museum environment (Brieber, Nadal, & Leder, 2015; Specker, Tinio, & Van Elk, 2017), the relatively small differences that were measured in the ERP demanded identical experimental conditions for the presentation of each image, which are difficult to ensure during free visual exploration in a museum. Third, the participants' attention was diverted away from the drawings to the detection of a change in the orientation of a cross that was superimposed on the images (Fig. 2). This task was unrelated to the artistic and non-artistic nature of the stimuli, respectively. Nevertheless, depending on the context (shown as a standard or deviant), the images elicited a differential electrophysiological response. We thus conclude that the detection of the change from artistic to non-artistic stimuli (and vice versa) occurred automatically (i.e. independent of the task) and was based on perceptual features that differed between the stimuli, such as the artistic composition, or any image property associated with it (see above). A role for automatic, stimulus-based processing in the evaluation of artworks has been suggested, for example, by results of a priming study (Duckworth, Bargh, Garcia, & Chaiken, 2002) and an implicit association test (Pavlovic & Markovic, 2012). Moreover, several researchers have shown that an aesthetic evaluation can be reliably reached after exposure times as short as 50–500 ms (Cupchik and Berlyne, 1979; Lindgaard, Fernandes, Dudekx, & Brown, 2006; Locher et al., 2007; Mullin, Hayn-Leichsenring, Redies, & Wagemans, 2017; Verhavert, Wagemans, & Augustin, 2017). Such short exposure times are enough to form an initial holistic impression of the structure and semantic meaning of images (Oliva & Torralba, 2006). The time period after stimulus onset, during which we observe vMMN in the present study, thus roughly coincides with the time required for first impression formation. In summary, our results indicate that artistic image composition or any associated structural image property is automatically detected by the human visual system. Our well-controlled experimental conditions have the drawback that the conclusions are restricted to a very narrow set of abstract artworks that were created by a single artist. Whether our findings apply to other abstract artworks, representational art or art viewing in a museum environment remains to be investigated in future research. Moreover, it remains unclear whether the observed effects were elicited by repetition suppression and/or surprise (violation of prediction). We deliberately omitted an equiprobable control condition because every stimulus thinkable was either composed or shuffled and, thus, belonged to one of the two stimulus categories of interest. Another limitation is related to the change detection task. Because this task was not very demanding and stimuli were presented in central vision, it cannot be ruled out that participants spend some attention on the stimuli. In fact, all participants reported to have seen the images and most of them gave some indications on what they believed to have seen in them; the descriptions ranged from lines and patches to animals and ships. However, we chose this paradigm because such an arrangement was used in similar studies before (e.g., Amado & Kovács, 2016; Kecskés-Kovács et al., 2013a) and artworks are usually observed with central vision. 84


C. Menzel et al.

Last but not least, we should emphasize that the specific terms, which participants ascribe to the stimuli in a laboratory experiment, play a role in the outcome of any aesthetic judgment task. For example, we have shown previously that participants evaluate the original (artistic) versions of our abstract images on average as more harmonious and ordered, and the shuffled versions as more interesting (Redies et al., 2015). Remarkably, the participants in that study differed in which type of images they found more aesthetic. Some participants considered the original versions to be more aesthetic, others the shuffled versions. In the present study, 6 out of the 22 participants liked the shuffled versions more (Table 2), possibly because they also preferred interesting images over ordered/harmonious ones. The reason for such differences between participants remains unclear, but might relate to individual differences in the usage and interpretation of the terms (Lyssenko et al., 2016), in the sensitivity to perceive artistic image structure (Götz et al., 1979; Wilson & Chatterjee, 2005), and/or in personality traits (Chamorro-Premuzic, Reimers, Hsu, & Ahmetoglu, 2009; Furnham & Walker, 2001; Lyssenko et al., 2016).

specificity of word usage in visual aesthetics. Acta Psychologica, 139, 187–201. Bell, C. (1914). Art. London: Chatto & Windus. Bodnar, F., File, D., Sulykos, I., Kecskés-Kovács, K., & Czigler, I. (2017). Automatic change detection in vision: Adaptation, memory mismatch, or both? II: Oddball and adaptation effects on event-related potentials. Attention, Perception, & Psychophysics, 79, 2396–2411. http://dx.doi.org/10.3758/s13414-017-1402-x. Brachmann, A., & Redies, C. (2017). Computational and experimental approaches to visual aesthetics. Frontiers in Computational Neuroscience, 11, 1–17. http://dx.doi.org/ 10.3389/fncom.2017.00102. Braun, J., Amirshahi, S. A., Denzler, J., & Redies, C. (2013). Statistical image properties of print advertisements, visual artworks and images of architecture. Frontiers in Psychology, 4, 808. http://dx.doi.org/10.3389/fpsyg.2013.00808. Brieber, D., Nadal, M., & Leder, H. (2015). In the white cube: Museum context enhances the valuation and memory of art. Acta Psychologica (Amsterdam), 154, 36–42. http:// dx.doi.org/10.1016/j.actpsy.2014.11.004. Bullot, N. J., & Reber, R. (2013). The artful mind meets art history: Toward a psychohistorical framework for the science of art appreciation. Behavioral and Brain Sciences, 36, 123–137. http://dx.doi.org/10.1017/S0140525x12000489. Chamorro-Premuzic, T., Reimers, S., Hsu, A., & Ahmetoglu, G. (2009). Who art thou? Personality predictors of artistic preferences in a large UK sample: The importance of openness. British Journal of Psychology, 100, 501–516. http://dx.doi.org/10.1348/ 000712608x366867. Chatterjee, A., & Vartanian, O. (2014). Neuroaesthetics. Trends in Cognitive Sciences, 18, 370–375. http://dx.doi.org/10.1016/j.tics.2014.03.003. Conway, B. R., & Rehding, A. (2013). Neuroaesthetics and the trouble with beauty. PLoS Biology, 11, e1001504. http://dx.doi.org/10.1371/journal.pbio.1001504. Cupchik, G. C., & Berlyne, D. E. (1979). The perception of collative properties in visual stimuli. Scandinavian Journal of Psychology, 20, 93–104. Cupchik, G. C., & Gebotys, R. J. (1990). Interest and pleasure as dimensions of aesthetic response. Empirical Studies of the Arts, 8, 1–14. Czigler, I. (2007). Visual mismatch negativity – violation of nonattended environmental regularities. Journal of Psychophysiology, 21, 224–230. Danto, A. C. (1981). The transfiguration of the commonplace: A philosophy of art. Cambridge, MA: Harvard University Press. Dickie, G. (1974). Art and the aesthetic: An institutional analysis. Ithaca, NY: Cornell University Press. Dowling, C. (2014). Aesthetic formalism. Website: http://www.iep.utm.edu/aes-form/. (date accessed: 16 July 2017). Duckworth, K. L., Bargh, J. A., Garcia, M., & Chaiken, S. (2002). The automatic evaluation of novel stimuli. Psychological Science, 13, 513–519. http://dx.doi.org/10.1111/ 1467-9280.00490. Durant, S., Sulykos, I., & Czigler, I. (2017). Automatic detection of orientation variance. Neuroscience Letters, 658, 43–47. http://dx.doi.org/10.1016/j.neulet.2017.08.027. Fechner, G. T. (1876). Vorschule der Ästhetik. Leipzig: Breitkopf and Härtel. Furnham, A., & Walker, J. (2001). Personality and judgements of abstract pop art, and representational paintings. European Journal of Personality, 15, 57–72. Götz, K. O., Borisy, A. R., Lynn, R., & Eysenck, H. J. (1979). A new visual aesthetic sensitivity test: I. Construction and psychometric properties. Perceptual and Motor Skills, 49, 795–802. Goodman, N. (1968). Languages of art an approach to a theory of symbols. Indianapolis: Hackett Publishing Company. Graf, L. K., & Landwehr, J. R. (2015). A dual-process perspective on fluency-based aesthetics: The pleasure-interest model of aesthetic liking. Personality and Social Psychology Review, 19, 395–410. http://dx.doi.org/10.1177/1088868315574978. Graham, D. J., & Redies, C. (2010). Statistical regularities in art: Relations with visual coding and perception. Vision Research, 50, 1503–1509. http://dx.doi.org/10.1016/j. visres.2010.05.002. Greenberg, C. (1955). American-type painting. Partisan Review, 22, 179–196. Höfel, L., & Jacobsen, T. (2007). Electrophysiological indices of processing aesthetics: Spontaneous or intentional processes? International Journal of Psychophysiology, 65, 20–31. http://dx.doi.org/10.1016/j.ijpsycho.2007.02.007. Hoenig, F. (2005). Defining computational aesthetics. In L. Neumann, M. Sbert, B. Gooch, & W. Purgathofer (Eds.). Computational aesthetics in graphics, visualization and imaging (pp. 13–18). Geneva: Eurographics Association. Jacobsen, T., & Höfel, L. (2003). Descriptive and evaluative judgment processes: Behavioral and electrophysiological indices of processing symmetry and aesthetics. Cognitive, Affective and Behavioral Neuroscience, 3, 289–299. Jacobsen, T., Buchta, K., Kohler, M., & Schroger, E. (2004). The primacy of beauty in judging the aesthetics of objects. Psychological Reports, 94, 1253–1260. Jacobsen, T. (2004). Individual and group modelling of aesthetic judgment strategies. British Journal of Psychology, 95, 41–56. Kandinsky, W. (1912). Über das Geistige in der Kunst, insbesondere in der Malerei. München: Piper. Kecskés-Kovács, K., Sulykos, I., & Czigler, I. (2013a). Is it a face of a woman or a man? Visual mismatch negativity is sensitive to gender category. Frontiers in Human Neuroscience, 7. Kecskés-Kovács, K., Sulykos, I., & Czigler, I. (2013b). Visual mismatch negativity is sensitive to symmetry as a perceptual category. European Journal of Neuroscience, 37, 662–667. Kimura, M., Katayama, J., Ohira, H., & Schroger, E. (2009). Visual mismatch negativity: New evidence from the equiprobable paradigm. Psychophysiology, 46, 402–409. Kimura, M. (2012). Visual mismatch negativity and unintentional temporal-context-based prediction in vision. International Journal of Psychophysiology, 83, 144–155. Korpilahti, P., Krause, C. M., Holopainen, I., & Lang, A. H. (2001). Early and late mismatch negativity elicited by words and speech-like stimuli in children. Brain and Language, 76, 332–339. http://dx.doi.org/10.1006/brln.2000.2426.

4.3. Conclusions and implications for modelling aesthetic experience The present results suggest that artistic image composition in a special set of abstract artworks is processed automatically in human observers, when attention is diverted to a change detection task that is irrelevant for aesthetic judgment. In a companion study, we used the same set of stimuli to show that human observers can reach consistent and stable aesthetic ratings on harmonious and ordered with short presentation times of 50 ms (Kana Schwabe and co-workers, unpublished data). Together, these results support the notion that automatic, lowand/or mid-level neural mechanisms contribute to aesthetic perception under conditions when differences in the cultural context and displayed content of the stimuli are minimized (Höfel & Jacobsen, 2007). This automatic detection of artistic image composition is compatible with a model of aesthetic experience, in which fast, lower-level sensory (“perceptual”) mechanisms play a pivotal role in aesthetic judgments (Redies, 2015). In this model, perceptual processing is independent of higher-level “cognitive” processing of cultural or contextual information, which can take place in parallel or can even be absent. It remains to be studied in detail how the automatic processing of image composition interacts with processing of contextual and art-historical information (Conway & Rehding, 2013; Leder & Nadal, 2014; Markovic & Radonjic, 2008; Pearce et al., 2016), as well as affective processing (Leder, Gerger, Brieber, & Schwarz, 2014; Markovic, 2010), all of which can contribute to aesthetic experience (Chatterjee & Vartanian, 2014). Conflicts of interest None. Acknowledgements We are grateful to Bettina Kamchen and Kathrin Rauscher for their help in data collection, Anselm Brachmann for his support in programming the trial lists, and Carolin Altmann for critical reading of the manuscript. This work was supported by grants from the German Research Council (grant numbers RE616/7-1 and KO 3918/1-2; 2-2). References Amado, C., & Kovács, G. (2016). Does surprise enhancement or repetition suppression explain visual mismatch negativity? Perception, 45 98-98. Amirshahi, S. A., Koch, M., Denzler, J., & Redies, C. (2012). PHOG analysis of self-similarity in esthetic images. Proceedings of SPIE (Human Vision and Electronic Imaging XVII), 8291, 82911J. http://dx.doi.org/10.1117/12.911973. Arnheim, R. (1954). Art and visual perception: A psychology of the creative eye. Berkeley, CA: University of California Press. Augustin, M. D., Wagemans, J., & Carbon, C. C. (2012). All is beautiful? Generality vs.

85


C. Menzel et al.

2012.10.013. Pavlovic, M., & Markovic, S. (2012). Automatic processes in aesthetic judgment: Insights from the Implicit Association Test. Psihologija, 45, 377–393. Pearce, M. T., Zaidel, D. W., Vartanian, O., Skov, M., Leder, H., Chatterjee, A., et al. (2016). Neuroaesthetics: The cognitive neuroscience of aesthetic experience. Perspectives on Psychological Science, 11, 265–279. http://dx.doi.org/10.1177/ 1745691615621274. Redies, C., Hasenstein, J., & Denzler, J. (2007). Fractal-like image statistics in visual art: Similarity to natural scenes. Spatial Vision, 21, 137–148. http://dx.doi.org/10.1163/ 156856807782753921. Redies, C., Amirshahi, S. A., Koch, M., & Denzler, J. (2012). PHOG-derived aesthetic measures applied to color photographs of artworks, natural scenes and objects. In A. Fusiello, V. Murino, & R. Cucchiara (Vol. Eds.), Computer Vision – ECCV 2012. Workshops and Demonstrations: 7583, (pp. 522–531). Berlin Heidelberg: Springer. http://dx.doi.org/10.1007/978-3-642-33863-2_54. Redies, C., Brachmann, A., & Hayn-Leichsenring, G. U. (2015). Changes of statistical properties during the creation of graphic artworks. Art & Perception, 3, 93–116. Redies, C. (2007). A universal model of esthetic perception based on the sensory coding of natural stimuli. Spatial Vision, 21, 97–117. http://dx.doi.org/10.1163/ 156856807782753886. Redies, C. (2015). Combining universal beauty and cultural context in a unifying model of visual aesthetic experience. Frontiers in Human Neuroscience, 9, 219. http://dx.doi. org/10.3389/fnhum.2015.00218. Sbriscia-Fioretti, B., Berchio, C., Freedberg, D., Gallese, V., & Umilta, M. A. (2013). ERP modulation during observation of abstract paintings by Franz Kline. PLoS One, 8, e75241. http://dx.doi.org/10.1371/journal.pone.0075241. Specker, E., Tinio, P. P. L., & Van Elk, M. (2017). Do you see what I see? An investigation of the aesthetic experience in the laboratory and museum. Psychology of Aesthetics, Creativity, and the Arts, 11, 265–275. http://dx.doi.org/10.1037/aca0000107. Stefanics, G., Kimura, M., & Czigler, I. (2011). Visual mismatch negativity reveals automatic detection of sequential regularity violation. Frontiers in Human Neuroscience, 5, 46. http://dx.doi.org/10.3389/fnhum.2011.00046. Stefanics, G., Csukly, G., Komlosi, S., Czobor, P., & Czigler, I. (2012). Processing of unattended facial emotions: A visual mismatch negativity study. Neuroimage, 59, 3042–3049. http://dx.doi.org/10.1016/j.neuroimage.2011.10.041. Stefanics, G., Kremlacek, J., & Czigler, I. (2014). Visual mismatch negativity: A predictive coding view. Frontiers in Human Neuroscience, 8, 666. http://dx.doi.org/10.3389/ fnhum.2014.00666. Sulykos, I., & Czigler, I. (2011). One plus one is less than two: Visual features elicit nonadditive mismatch-related brain activity. Brain Research, 1398, 64–71. http://dx.doi. org/10.1016/j.brainres.2011.05.009. Taylor, R. P., Newell, B., Spehar, B., & Clifford, C. W. G. (2005). Fractals: A resonance between art and nature. In M. Emmer (Ed.). The Visual Mind II. Cambridge: MIT Press. Verhavert, S., Wagemans, J., & Augustin, M. D. (2017). Beauty in the blink of an eye: The time course of aesthetic experiences. British Journal of Psychology, 109, 63–84. http:// dx.doi.org/10.1111/bjop.12258. Willenbockel, V., Sadr, J., Fiset, D., Horne, G. O., Gosselin, F., & Tanaka, J. W. (2010). Controlling low-level image properties: The SHINE toolbox. Behavior Research Methods, 42, 671–684. http://dx.doi.org/10.3758/brm.42.3.671. Wilson, A., & Chatterjee, A. (2005). The assessment of preference for balance: Introducing a new test. Empirical Studies of the Arts, 23, 165–180. Zhao, L., & Li, J. (2006). Visual mismatch negativity elicited by facial expressions under non-attentional condition. Neuroscience Letters, 410, 126–131. http://dx.doi.org/10. 1016/j.neulet.2006.09.081.

Kreegipuu, K., Kuldkepp, N., Sibolt, O., Toom, M., Allik, J., & Naatanen, R. (2013). vMMN for schematic faces: Automatic detection of change in emotional expression. Frontiers in Human Neuroscience, 7, 714. http://dx.doi.org/10.3389/fnhum.2013.00714. Leder, H., & Nadal, M. (2014). Ten years of a model of aesthetic appreciation and aesthetic judgments: The aesthetic episode − developments and challenges in empirical aesthetics. British Journal of Psychology, 105, 443–464. http://dx.doi.org/10.1111/ bjop.12084. Leder, H., Gerger, G., Brieber, D., & Schwarz, N. (2014). What makes an art expert? Emotion and evaluation in art appreciation. Cognition & Emotion, 28, 1137–1147. http://dx.doi.org/10.1080/02699931.2013.870132. Li, X., Lu, Y., Sun, G., Gao, L., & Zhao, L. (2012). Visual mismatch negativity elicited by facial expressions: New evidence from the equiprobable paradigm. Behavioral and Brain Functions, 8, 7. http://dx.doi.org/10.1186/1744-9081-8-7. Lindgaard, G., Fernandes, G., Dudekx, C., & Brown, J. (2006). Attention web designers: You have 50 milliseconds to make a good first impression. Behaviour & Information Technology, 25, 115–126. Locher, P. J., Stappers, P. J., & Overbeeke, K. (1999). An empirical evaluation of the visual rightness theory of pictorial composition. Acta Psychologica, 103, 261–280. Locher, P., Overbeeke, K., & Stappers, P. J. (2005). Spatial balance of color triads in the abstract art of Piet Mondrian. Perception, 34, 169–189. Locher, P., Krupinski, E. A., Mello-Thoms, C., & Nodine, C. F. (2007). Visual interest in pictorial art during an aesthetic experience. Spatial Vision, 21, 55–77. http://dx.doi. org/10.1163/156856807782753868. Lopez-Caballero, F., Zarnowiec, K., & Escera, C. (2016). Differential deviant probability effects on two hierarchical levels of the auditory novelty system. Biological Psychology, 120, 1–9. http://dx.doi.org/10.1016/j.biopsycho.2016.08.001. Lyssenko, N., Redies, C., & Hayn-Leichsenring, G. U. (2016). Evaluating abstract art: Relation between term usage, subjective ratings, image properties and personality traits. Frontiers in Psychology, 7, 973. http://dx.doi.org/10.3389/fpsyg.2016.00973. Malevich, P. (1927). Die gegenstandslose Welt. Bauhausbücher, 11, 65–98. Mallon, B., Redies, C., & Hayn-Leichsenring, G. U. (2014). Beauty in abstract paintings: Perceptual contrast and statistical properties. Frontiers in Human Neuroscience, 8, 161. http://dx.doi.org/10.3389/fnhum.2014.00161. Markovic, S., & Radonjic, A. (2008). Implicit and explicit features of paintings. Spatial Vision, 21, 229–259. http://dx.doi.org/10.1163/156856808784532563. Markovic, S. (2010). Aesthetic experience and the emotional content of paintings. Psihologija, 43, 47–64. McManus, I. C., Cheema, B., & Stoker, J. (1993). The aesthetics of composition: A study of Mondrian. Empirical Studies of the Arts, 11, 83–94. Morlet, D., & Fischer, C. (2014). MMN and novelty P3 in coma and other altered states of consciousness: A review. Brain Topography, 27, 467–479. http://dx.doi.org/10.1007/ s10548-013-0335-5. Mullin, C., Hayn-Leichsenring, G., Redies, C., & Wagemans, J. (2017). The gist of beauty: an investigation of aesthetic perception in rapidly presented images. Paper presented at Electronic Vision - Human Vision and Electronic Imaging 2017. Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology, 118, 2544–2590. http://dx.doi.org/10.1016/j.clinph.2007.04.026. Oliva, A., & Torralba, A. (2006). Building the gist of a scene: The role of global image features in recognition. Progress in Brain Research, 155, 23–36. http://dx.doi.org/10. 1016/S0079-6123(06)55002-2. Pang, C. Y., Nadal, M., Müller-Paul, J. S., Rosenberg, R., & Klein, C. (2013). Electrophysiological correlates of looking at paintings and its association with art expertise. Biological Psychology, 93, 246–254. http://dx.doi.org/10.1016/j.biopsycho.

86