Augmented Metacognition: Exploring Pupil Dilation Sonification to Elicit Metacognitive Awareness Introduction Metacognition and its augmentation
The pupil dilation-cognition link
• Metacognition is the awareness and control of one’s own cognitive processes [1]
• Pupil dilation rapidly tracks cognitive processing, e.g. cognitive load, focusing, arousal [3]
• When changes in cognitive processes enter awareness, decision making about one’s cognitive behaviour, e.g., for task performance goals, can take place
• People do not have access to their pupil dilations
• Technologies that effectively augment metacognition are scarce [2]
• These correlations could serve as embodied information about if, when, and what cognitions change during a task [cf. 5], i.e., help elicit metacognitive awareness [6]
• We propose such a technology that is based on making changes in pupil dilation audible
• Perceiving these changes via sound, one might be able to learn correlations between one’s pupil dilations and related cognitions [cf. 4]
Method
Results
An exploratory study was conducted to investigate our conjectures.
• The results suggest that people correlate sounds based on pupil dilation with their cognitions, even without instructions 4
Frequency
• Participants: N=20, Mage=23, SDage=5, 16♀, 4♂ • Pupil size (A1) is mapped to the amplitude (A2) of a sine wave (ν=440Hz) • This makes the user’s pupil size audible during problem solving tasks
3 2
2 1
1
1
1
1
1
1
1
2
• When instructed that the sounds correlate with the amount of cognitive resources allocated, fifteen participants (75%) experienced a correlation, whereas five (25%) did not Participants solve mathematics problems on a computer screen while listening to a sonification of their own pupil dilation data that is captured and streamed by an eye-tracker in realtime. • Part 1: Twelve mathematics problems, no information about the link between the sounds and cognition • Part 2: Five mathematical problems preceded by the instruction that there exists a link between “...the amount of cognitive resources allocated to the task and the changes in volume of the sound”
Discussion • Our preliminary study suggests that pupil dilation sonification can elicit metacognitive awareness • Exploration with other sound designs is needed • Replication with a control condition (i.e. a fake sonification) is needed
References [1] Baker, L. (2010). Metacognition. International encyclopedia of education, 204-210. [2] Goldberg et al., (2014). Enhancing self-regulated learning through metacognitively-aware intelligent tutoring systems. In Proceedings of the 11th International Conference of the Learning Sciences. [3] Reimer, J., McGinley, M. J., Liu, Y., Rodenkirch, C., Wang, Q., McCormick, D. A., & Tolias, A. S. (2016). Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nature communications, 7, 13289. [4] Novich, N. (2015). Sound-to-touch sensory substitution and beyond. Doctoral thesis. Rice University. [5] Storbeck, J., & Clore, G. L. (2008). Affective arousal as information: How affective arousal influences judgments, learning, and memory. Social and personality psychology compass, 2(5), 1824-1843. [6] Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American psychologist, 34(10), 906.
Alwin de Rooij, Hanna Schraffenberger, & Mathijs Bontje Tilburg University, Department of Communication and Cognition Warandelaan 2, Tilburg, The Netherlands Contact:
[email protected]