Accepted Manuscript Computational Psychosomatics and Computational Psychiatry: Towards a joint framework for differential diagnosis Frederike H. Petzschner, Lilian A.E. Weber, Tim Gard, Klaas E. Stephan PII:
S0006-3223(17)31584-6
DOI:
10.1016/j.biopsych.2017.05.012
Reference:
BPS 13207
To appear in:
Biological Psychiatry
Received Date: 1 January 2017 Revised Date:
14 April 2017
Accepted Date: 15 May 2017
Please cite this article as: Petzschner F.H., Weber L.A.E., Gard T. & Stephan K.E., Computational Psychosomatics and Computational Psychiatry: Towards a joint framework for differential diagnosis, Biological Psychiatry (2017), doi: 10.1016/j.biopsych.2017.05.012. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Computational Psychosomatics and Computational Psychiatry: Towards a joint framework for differential diagnosis Frederike H. Petzschner1, Lilian A.E. Weber1, Tim Gard1,2, Klaas E. Stephan1,3*
3
*
M AN U
Corresponding author: Translational Neuromodeling Unit (TNU) Institute for Biomedical Engineering University of Zurich & ETH Zurich Wilfriedstrasse 6 CH-8032 Zurich Switzerland Phone: +41 44 6349111
[email protected]
RI PT
2
Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & ETH Zurich, 8032 Zurich, Switzerland. Center for Complementary and Integrative Medicine, University Hospital Zurich, Switzerland. Wellcome Trust Centre for Neuroimaging, University College London, London, WC1N 3BG, UK.
SC
1
Short title: Computational Psychosomatics
TE D
Keywords: hierarchical Bayesian model, inference, cybernetics, prediction error, homeostasis, allostasis, metacognition
AC C
EP
Abstract: 93 words Main text: 4000 words Number of tables: 0 Number of figures: 4 Number of supplementary material: 1
ACCEPTED MANUSCRIPT Abstract This article outlines how a core concept from theories of homeostasis and cybernetics, the inference-control loop, may be used to guide differential diagnosis in In particular we
RI PT
computational psychiatry and computational psychosomatics.
discuss (i) how conceptualizing perception and action as inference-control loops yields a joint computational perspective on brain-world and brain-body interactions
SC
and (ii) how the concrete formulation of this loop as a hierarchical Bayesian model points to key computational quantities that inform a taxonomy of potential disease
AC C
EP
TE D
concrete clinical applications.
M AN U
mechanisms. We consider the utility of this perspective for differential diagnosis in
Petzschner et al.
2
ACCEPTED MANUSCRIPT INTRODUCTION Psychiatry faces major challenges: its nosology is agnostic about mechanisms, lacks predictive validity, and leads to trial-and-error treatment (1, 2). Strikingly,
RI PT
neuroscientific advances have hardly affected nosology or clinical practice (3). One response to this disconnect is computational psychiatry, with its emerging focus on clinical applications (4-11).
SC
One strategy for computational psychiatry is to learn from internal medicine where mechanistic frameworks for differential diagnosis enable targeted treatment
M AN U
decisions for individual patients. Importantly, differential diagnosis does not necessarily require molecular mechanisms. Much coarser distinctions – such as inflammatory, infectious, vascular, neoplastic, autoimmunological, or hereditary causes of disease – can provide crucial guidance for treatment as they disclose
TE D
fundamentally distinct disease processes.
This paper outlines a framework for differential diagnosis that is motivated by a
EP
general computational perspective on brain function. While not the first attempt of its kind (4, 12-14), this paper makes three contributions. First, we adopt a disease-
AC C
independent motif – the inference-control loop as fundament of cybernetic theories (15-18) – and consider how this may help systematizing computational perspectives on brain-world and brain-body interactions. Second, we consider a hierarchical Bayesian implementation that suggests three possible computational quantities (predictions, prediction errors, and their precisions) at five potential failure loci (sensation, perception, metacognition, forecasting, action). Third, we discuss the potential clinical utility of this taxonomy for differential diagnosis in computational psychiatry and psychosomatics; cf. (14, 19-21). Petzschner et al.
3
ACCEPTED MANUSCRIPT
INFERENCE-CONTROL LOOPS Different theories of adaptive behavior exist, but they share common themes. Here,
RI PT
we focus on the closed loop between sensations and actions that is at the core of classical cybernetic theories (15, 16) and homeostatic principles (22, 23). We first summarize extended cybernetic/homeostatic theories (Figure 1) before considering
M AN U
mechanisms in psychiatry and psychosomatics.
SC
one particular implementation as a foundation for a joint taxonomy of disease
A useful starting point to reflect on adaptive behavior is the observation that it must be constrained by requirements of bodily homeostasis. In the simplest case, actions can be purely reactive. For instance, to maintain constant body temperature, sensor information can be compared to a pre-defined setpoint (e.g., 37°). Actions, such as
TE D
heating or cooling the body, are then selected to bring sensory inputs closer to that setpoint. This reflex arc – which implements the same feedback control as a simple
EP
thermostat – is illustrated in Figure 1A.
If biological systems were like thermostats, with unambiguous sensory inputs and
AC C
purely reactive in nature, simple feedback control would be sufficient. However, biological systems face three major challenges: First, sensations (inputs from sensory channels; see Glossary in Supplementary Material) are noisy and often highly ambiguous – because the world’s states (body or environment) that excite sensors can interact nonlinearly and/or hierarchically (24, 25). It has long been recognized that the world’s true state is not directly accessible for the brain and needs to be inferred (26, 27). This notion of perception as inference Petzschner et al.
4
ACCEPTED MANUSCRIPT renders perception an interpretation of sensations, guided by prior beliefs and a model of the world (28, 29). Among many findings supporting this notion, illusions prominently illustrate how learned physical regularities can shape perception profoundly (30, 31); Figure 2A. When sensations are ambiguous, perception can
RI PT
expand the capacity for control, particularly when action selection requires information about hierarchically deep states of the world that relate nonlinearly to sensations. For example, in social interactions, inferring the nature of others’ acts
SC
that generated visual input may not be sufficient; instead, inference on deeper states, such as the intentions of others’ that generated their acts may be required
M AN U
(32, 33).
A second challenge is that inference on current states of the world can only finesse reactive control. By contrast, prospective control requires predicting the world’s future states (forecasting), taking into account both the influence of possible actions
TE D
(34, 35) and the world’s endogenous dynamics (36) (Figure 1C). Third, action selection and execution is influenced by beliefs about one’s abilities
EP
(37). This self-monitoring of one’s level of mastery in acting upon the world is part of metacognition and can be seen as a high-level form of inference about one’s
AC C
capacity for control (19, 38) (Figure 1C). Finally, given an inferred (or forecast) state of the world, actions can be selected to achieve a particular goal (optimize some objective function). This objective function can defined differently – for example, in terms of utility (39), reward (34), cost (40), loss (41), or surprise (42). Figure 1C depicts a schematic illustration of the extended inference-control loop. Importantly, any given action alters the world, thus shaping future sensory input. In Petzschner et al.
5
ACCEPTED MANUSCRIPT other words, sensation, perception, forecasting and actions form a closed loop between the brain and its external world. For brevity, we refer to this entire cycle as inference-control loop. Its closed-loop nature is fundamentally important as it creates problems of circular causality that are at the core of diagnostic challenges we
RI PT
examine below.
SC
COMPUTATIONAL MODELING OF INFERENCE-CONTROL LOOPS
M AN U
We now consider how inference-control loops can be formalized as concrete computational models. Here, we adopt hierarchical Bayesian models (HBMs) but emphasize that this is not the only possible perspective; for forecasting and control in particular, alternative (and arguably more established) modeling approaches exist, e.g. (34, 43-45). We prefer a hierarchical Bayesian view for two main reasons. First,
TE D
it uses the same formalism and quantities – precision-weighted predictions and prediction errors – for implementing perception, forecasting, reactive/prospective alike. This suggests
a
compact
taxonomy of
EP
control and metacognition
computational dysfunctions and their differential diagnosis; cf. (2, 14). Second, the
AC C
formulation of control in HBMs is intimately connected to concepts of homeostatic (reactive) and allostatic (prospective) control, which are of central importance for psychosomatics.
Bayesian Inference A widely adopted concept of perception is the Bayesian framework (27, 28, 46, 47). This casts perception as inference, where prior beliefs about hidden states of the Petzschner et al.
6
ACCEPTED MANUSCRIPT world are updated in the light of sensory data to yield a posterior belief (Figure 3A) (24, 27). A popular notion is that this computation rests on a (hierarchical) “generative model” of how sensory data are caused by hidden states of the world (28, 29, 48, 49); see Figure 3A. Inverting this model under beliefs about the states’ a
RI PT
priori probability allows for inferring the causes of sensations.
Bayesian models explain phenomena across the spectrum of perception, for
SC
example, how humans combine multi-sensory information (50, 51) and how biases and illusions result from prior beliefs and experience (24, 30, 52, 53).
M AN U
A key point for this paper is that Bayesian belief updates have, for most probability distributions, a generic form: the change in belief is proportional to prediction error (PE) – the difference between actual (sensory) data and predicted data (under the prior) – weighted by a precision ratio (54). The latter is critical, as it determines the
TE D
relative influence of prior and sensory data: precise predictions (priors reduce, while precise sensory inputs increase belief updates (Figure 3A). Generally, abnormal computations and/or signaling of any of these three quantities – PEs, predictions,
AC C
EP
precisions – could disrupt inference.
Hierarchical Bayesian Models The hierarchical structure of the external world suggests an equivalent (mirrored) structure of the brain’s generative model (28, 48). Anatomically, this “hierarchical Bayesian” idea is supported by structural hierarchies in cortex (55-57). Popular hierarchical Bayesian models (HBMs) include hierarchical filtering (HF; (54, 58)) and predictive coding (PC; (28, 49)). In these models, each level holds a belief Petzschner et al.
7
ACCEPTED MANUSCRIPT (prediction) about the state of the level below (PC) or its rate of change (HF). This prediction is signaled to the lower level where it is compared against the actual state, resulting in a PE. This PE is sent back up the hierarchy to update the prediction – and thus reduce future PEs. Critically, again, this update is weighted by a precision
lower precision
of
predictions
lead
to more
RI PT
ratio (Figure 3A): higher precision of bottom-up signals (sensory inputs or PEs) or pronounced belief
updates.
Neurobiologically, in cortex, predictions are likely signaled via NMDA receptors
SC
(NMDARs) at descending connections, PEs via AMPA receptors (and possibly NMDARs) at ascending connections, while precision-weighting depends on
M AN U
postsynaptic gain; this is determined by neuromodulators (e.g., dopamine, acetylcholine; (59)) and GABAergic inhibition (for reviews, (60-64)). In an HBM context, the brain’s objective function can be seen as minimizing PEs (as a proxy to surprise) under its generative model (65). Notably, PEs can not only be
TE D
reduced by updating the generative model (as above), but also by changing the precision of sensory channels (attention), or by actions that fulfil predictions. The
EP
latter is “active inference” (35, 42, 66)), a concept in line with the cybernetic notion
AC C
that “... control systems control what they sense, not what they do.” (17)
Forecasting, action and metacognition in HBMs While HBMs are popular models of perceptual inference, they can also implement forecasting, action and metacognition; again, this rests on precision-weighted PEs (pwPEs). Switching from inference to forecasting/actions requires “switching off” sensory precision (sensory attenuation; (66-68)); this abolishes belief updates, while PEs are now used as simulation or action signals (19, 36, 42). Petzschner et al.
8
ACCEPTED MANUSCRIPT While different formalisms of forecasting exist (34, 43, 44, 69), their common theme is a “forward simulation” under a given model. Bayesian implementations of forecasting include “planning by inference” (36, 70) and inference on trajectories of states
(generalized
coordinates;
(42,
71)).
One
challenge
for
RI PT
psychiatric/psychosomatic applications is that the model often not only needs to predict the effects of chosen actions, but also the intrinsic dynamics of environment
SC
and body (36, 45).
Turning to action, HBMs can implement both reactive and prospective control. The
M AN U
former occurs through a reflex arc at the bottom of the hierarchy (Figure 3B). Specifically, replacing classical cybernetic setpoints with beliefs about hidden states that cause sensory inputs, reactive control can be cast as a reflex where PEs elicit corrective actions that minimize surprise about sensory inputs (19, 42). Importantly, the belief’s precision determines the vigor of these actions (19); a property that
TE D
allows for new explanations of psychosomatic phenomena and placebo effects (see below). Prospective control can be implemented by dynamically adjusting this belief
EP
(e.g., its mean or precision) as a function of predicted future states (19, 72). These predictions could be signaled from higher levels in the HBM that implement
AC C
forecasting.
Action selection in HBMs could, in principle, proceed with respect to optimizing any chosen objective function, e.g., a subject-specific utility function (73). We focus on active inference (35, 42, 66) as a specific proposal. Simply speaking, this postulates that actions serve to minimize PEs by changing the world (environment or body) in order to fulfil the brain’s expectation of sensory inputs. We focus on this idea because it is closely related to cybernetics (e.g., perceptual control theory; (17)) and
Petzschner et al.
9
ACCEPTED MANUSCRIPT represents a probabilistic formulation of the core principle of homeostasis – that regulatory actions minimize discrepancies between expected and actual inputs. It thus provides a basis for formal models of brain-body interactions (19) and a bridge
RI PT
to psychosomatics. Finally, metacognition could be incorporated into HBMs through an additional layer that holds expectations about the level of PEs throughout an inference hierarchy
SC
(19); Figure 1. This layer infers the performance of the inference-control loop as a
M AN U
whole, enabling a representation (and updating) of mastery or self-efficacy beliefs.
INTEROCEPTION AND HOMEOSTATIC/ALLOSTATIC CONTROL HBMs have been used for more than two decades to investigate perceptual
TE D
inference on environmental states (exteroception) (48, 49, 54, 74, 75). However, the same inference challenge exists with regard to bodily states (interoception; (76-78)). Signals from bodily sensors (interosensations) – such as blood oxygenation and
EP
osmolality, temperature, pain, heartrate, or plasma concentrations of metabolites and hormones – reach the brain through various afferent pathways that converge on
AC C
posterior/mid insula cortex (79-81), a region regarded as interoceptive cortex (82, 83). Several lines of evidence – in particular from pain and placebo research (for reviews, (79, 84-86)) – indicate that interosensations are not processed “raw” but shaped by prior beliefs (Figure 2B). Supported by anatomical and physiological findings (for review, (77, 87)), it has been proposed that perception and control of bodily states follow the same hierarchical Bayesian principles as for environmental states (76-78). This implies that a joint Petzschner et al.
10
ACCEPTED MANUSCRIPT computational approach to characterizing disease mechanisms in exteroceptive (psychiatry) and interoceptive (psychosomatics) domains (Figure 4). Notably, regulation of bodily states comes in two forms. Homeostatic control (22) is a
RI PT
form of reactive control (23) that is classically formalized as cybernetic feedback control (15). The more recent concept of allostasis (“stability through change”; (88)) refers to prospective control, where actions are taken before homeostasis is violated.
SC
Put differently, allostasis is a self-initiated temporary change in homeostatic setpoints to prepare for a predicted external perturbation (89). When replacing classical
M AN U
setpoints with homeostatic beliefs (expectations about bodily states), both can be cast formally as active inference (19). Homeostatic control can then be understood as reflex-like emission of corrective actions that fulfill beliefs about bodily states; and allostatic control as changing homeostatic beliefs under guidance by higher beliefs or
TE D
forecasts about future perturbations of bodily states (19).
Neuroanatomically concrete circuits for interoception and homeostatic/allostatic control have been suggested (19, 78, 80, 81, 87). Anterior insula (AI) and anterior
EP
cingulate cortex (ACC) play a central role in these proposals as they are thought to represent current and predicted states of the body within the external world (80, 90,
AC C
91). Equipped with projections to regions with homeostatic reflex arcs (e.g., hypothalamus, brainstem), AI and ACC may signal the forecasts that guide allostatic control (19, 80, 87). Furthermore, they likely interface interoceptive and exteroceptive systems and mediate their interactions, such as the influence of interoceptive signals on exteroceptive judgments (92-94) (Figure 4).
A TAXONOMY OF FAILURE LOCI AND COMPUTATIONAL DYSFUNCTIONS Petzschner et al.
11
ACCEPTED MANUSCRIPT Our general thesis is that conceptualizing adaptive behavior in terms of inferencecontrol loops and their concrete implementation as hierarchical Bayesian models systematizes potential failure loci and associated computational dysfunctions. The ensuing taxonomy of disease mechanisms could guide differential diagnosis, in
RI PT
analogous ways for computational psychiatry and computational psychosomatics. That is, in the general inference-control loop outlined above, maladaptive behavior
sensory inputs (sensations),
(ii)
inference (perception),
(iii)
forecasting,
(iv)
control (action),
(v)
metacognition.
TE D
M AN U
(i)
SC
could arise from primary disruptions at five major loci (Figure 3B):
Clearly, each of these processes could be conceptualized under different
EP
computational frameworks. In the specific case of HBMs, failures at any of these
3A): (i)
AC C
levels can arise from disturbances in a small set of computational quantities (Figure
bottom-up signals (sensory input or PEs),
(ii)
top-down signals (expectations or predictions),
(iii)
their precision (inverse uncertainty).
These
two
axes
may lend
useful overarching structure
to
pathogenetic
considerations and provide a conceptual grid for classifying disease mechanisms in Petzschner et al.
12
ACCEPTED MANUSCRIPT computational psychiatry and computational psychosomatics. However, this requires that the above levels and quantities can be inferred non-invasively in individual patients, using computational assays that can be applied to behavioral, (neuro)physiological and neuroimaging data (6, 95). Suitable techniques for model-
SC
limitations, we discuss them in the Supplementary Material.
RI PT
based inference on pwPE signaling in cortical hierarchies exist (96); due to space
M AN U
Computational Psychosomatics
Psychosomatic medicine is concerned with somatic diseases that are caused or influenced by mental processes (97), for example, bodily symptoms caused by beliefs. Classic examples for the influence of beliefs on bodily states are placebo/nocebo effects (84-86) (Figure 2B). Here, expectations about the effects of
TE D
an intervention trigger reactions that fulfil the expectation. Importantly, the strength of placebo is known to depend not only on beliefs about effect amplitude, but also on
EP
the precision of this belief (85). Our framework offers a formal explanation for this empirical phenomenon because in HBM implementations of homeostatic control the
AC C
vigor of belief-fulfilling actions depend on the precision of these beliefs (19). Computational treatments of psychosomatic disorders are rare (but see (98, 99)). This may be due to the (perceived) lack of a comprehensive framework that formalizes interoception and homeostatic/allostatic control and makes them measurable in the individual patient. In the following, we consider one concrete problem of differential diagnosis and describe how the conceptual grid described above may guide the search for the locus of the primary (initial) abnormality.
Petzschner et al.
13
ACCEPTED MANUSCRIPT
Example: Depression and somatic symptoms Depression includes many patients with somatic abnormalities, including cardiac
RI PT
(100), immunological (101), and metabolic disturbances (102). One long-standing explanation of this association highlights maladaptive beliefs. For example, false high-level beliefs about volatility of the world could cause prolonged allostatic
SC
responses, with persistent sympathetic activation and ensuing damage to cardiovascular, immunological and metabolic health (“allostatic load”; (89, 103)). In
M AN U
our framework, the influence of high-level beliefs could be mediated via projections from allostatic control regions (e.g., AI, ACC) on sympathetic effector regions (e.g., hypothalamus, amygdala or PAG) where they elicit autonomic actions by altering homeostatic setpoints. Notably, HBMs can infer fluctuations in beliefs about
TE D
environmental volatility from behavioral and peripheral physiological measurements (32, 58, 75, 98). These belief trajectories could be integrated into physiological models (e.g., dynamic causal models; (96, 104)) of the above connection strengths
EP
and, by comparing models with/without modulatory effects of these beliefs, identify patients in whom bodily symptoms are possible consequences of beliefs. One might
AC C
also hypothesize that these connection strengths correlate with peripheral indices of sympathetic activation (105). An opposite interpretation views depression as “reactive” to initial somatic disease. In our framework this can be formalized as a metacognitive response to (real or perceived) chronic dyshomeostasis. One implementation of metacognition in HBMs is through a top-level layer which holds beliefs about the performance of the inference-control loop. In this “allostatic self-efficacy” (19) concept, persistently Petzschner et al.
14
ACCEPTED MANUSCRIPT elevated PEs decrease one’s beliefs of mastery over bodily states; this metacognitive “diagnosis” of lack of control may lead to depression as a form of learned helplessness. This proposal could be tested by correlating model-based indices of interoceptive PE signaling with questionnaire measures of self-efficacy
RI PT
and helplessness.
Critically, our framework emphasizes that dyshomeostasis could be real or
A real bodily source of dyshomeostasis (that evades cerebral attempts of regulation).
(ii)
Sensations: altered bodily receptors (“broken sensor”); e.g. visceral hypersensitivity (106)).
Inference: illusionary dyshomeostasis, due to impairments of the afferent
TE D
(iii)
M AN U
(i)
SC
perceived, and could exist independently from the brain or be caused by it:
branch of the inference-control loop; for example, atrophic (107) or inflammatory (108) processes within the insula, or functional pathologies of
EP
NMDARs and/or neuromodulators that alter the signaling of pwPEs (for reviews, (62, 109)). For example, abnormally high precision of beliefs
AC C
about bodily states could render unremarkable events, such as normal sensory noise, meaningful; an interoceptive analog to “aberrant salience” (110) in schizophrenia.
(iv)
Control: inadequate deployment of autonomic, endocrine, immunological actions; for example, due to inflammatory changes in allostatic control regions (AI, ACC; (108)), regions implementing homeostatic reflex arcs (e.g., hypothalamus; (111)), or their projections (112); or due to
Petzschner et al.
15
ACCEPTED MANUSCRIPT inadequately shifted setpoints as a result of false beliefs/forecasts (see above). Distinguishing these options is hard: the closed-loop nature of the inference-control
RI PT
cycle means that any primary disturbance will cause compensatory changes downstream. Inflammation-sensitive imaging (108, 113) could help, but only covers a few possible causes. Instead, we propose that model-based inference (from
SC
behavior and fMRI data (96); Supplementary Material) on pwPE signaling in brainstem-hypothalamic-insular-cingulate circuitry could help identify a primary
M AN U
dysfunction. For example, under experimentally controlled perturbations of a (yet undisturbed) bodily state, pwPE signals in posterior/mid insula to predictable and unpredictable interosensations should differ, depending on whether the pathology is
TE D
located at inference or control levels.
Computational Psychiatry
EP
Bayesian perspectives of inference-control impairments feature frequently in computational concepts of depression (19, 87), autism (20, 114-117), schizophrenia
AC C
(12, 21, 60, 61, 118, 119), and anxiety (58, 120, 121). Here, we briefly discuss one application of this framework to distinguish disease mechanisms in autism spectrum disorder (ASD).
Example: Autism Spectrum Disorder
Petzschner et al.
16
ACCEPTED MANUSCRIPT Hierarchical Bayesian theories of ASD revisit longstanding observations of perceptual anomalies in patients, including the excessive processing of irrelevant details and concomitant difficulties of abstraction. They suggest two competing explanations (114-117): sensory inputs of overwhelming precision, or higher-order
RI PT
beliefs that are too imprecise for providing generalizable predictions. In either case, a child with ASD would incessantly experience large PEs during perception (see equation in Figure 3A). Typical symptoms, such as repetitive behaviors and
SC
avoidance of complex and volatile situations (e.g., social interactions), can then be
M AN U
interpreted as coping mechanisms to reduce PEs; see (20) for discussion. Additionally, ASD individuals show various interoceptive disturbances (122, 123) that may equally result from an increase in sensory precision from the viscera or a failure to attenuate it (7). Viscerosensory precision-weighting has been linked to oxytocin; associated disturbances during development might compromise the construction of
(7, 124).
TE D
generative models that attribute self vs. other agency to interoceptive experiences
EP
The competing explanations of high sensory vs. low belief precision (116) could be disambiguated by psychophysical experiments in combination with Bayesian models
AC C
of perception. These have previously been used to assess individual sensory processing (50, 125, 126) in healthy volunteers; as have been EEG-based circuit models of precision-weighting in auditory cortex (127). These models could be used in ASD to detect (sub)groups with exaggerated precision estimates of sensory inputs and insufficiently precise predictions, respectively (20).
CHALLENGES AND OPPORTUNITIES Petzschner et al.
17
ACCEPTED MANUSCRIPT Assessing the computational anatomy of circuit dysfunctions follows principles of homeostatic thinking, as is commonplace in medicine, and holds great diagnostic potential. However, its clinical translation faces nontrivial challenges – particularly in
RI PT
application to psychosomatics. Chicken and egg problems
The inference-control loop represents the conceptual heart of theories of
SC
homeostasis, allostasis, and cybernetics (Figure 1). Its closed-loop nature means that a dysfunction in one domain typically invokes a cascade of changes throughout
M AN U
the circuit, making it difficult to differentiate cause from consequence. However, different primary disturbances induce distinct patterns of change that might be discriminable statistically – as commonly done in fields familiar with compensatory changes throughout dyshomeostatic systems, such as internal
TE D
medicine (cf. differential diagnosis of hypothalamic/pituitary/glandular disturbances in endocrinology). Computational psychiatry/psychosomatics could finesse this by
EP
statistical comparison of models embodying alternative disease processes (95). Additionally, in medicine, challenge (perturbation) approaches are often crucial for
AC C
diagnosis. Combining designed perturbations with model selection and prospective assessments of disease trajectories (10, 96) represents a promising approach to resolve ambiguity created by circular causality. One central challenge for computational psychosomatics concerns availability of somatic perturbation techniques. In contrast to computational psychiatry, where we can
adopt
methods
psychology/psychophysics,
for
manipulating
manipulating
beliefs the
and
precisions
somato-cerebral
branch
from of
psychosomatics only has access to few techniques. These include cardiac Petzschner et al.
18
ACCEPTED MANUSCRIPT challenges with short-acting sympathomimetics (128); manipulating inspiratory breathing load or air composition (129, 130); transcutaneous vagus nerve stimulation (131); baroreceptor stimulation (132); acute induction of inflammation by vaccination (133); or C-fiber stimulation under capsaicin (134). Developing further challenges
RI PT
that are non-invasive and provide temporal control should become a priority topic for computational psychosomatics.
SC
Universality versus specificity
The HBM framework suggests pwPEs as a central computational quantity for
M AN U
inference, forecasting, action and metacognition alike. This generalizing view has pros and cons. On the one hand, it suggests a conceptual grid for differential diagnosis and implies that computational differentiation of pwPE abnormalities could find broad diagnostic application. On the other hand, one may be concerned that we
TE D
portray cortex as a “non-specific hierarchical Bayesian machine” (as put by one reviewer) without neuroanatomical specificity. We do not wish to convey this impression. The inference problems the brain faces vary, for example, depending on
environmental
EP
the sensory channels involved and the depth of hierarchical coupling among states.
Different
tasks
require
different
types
of
(cortically
AC C
represented) generative models and thus distinct circuits; compare proposed circuits for interoception/allostasis (19, 87) and vision/oculomotor control (42, 72)). Empirically, in tasks using the same sensory modality but requiring inference on concrete vs. abstract social quantities, pwPEs were reflected by activity in partially overlapping and partially distinct circuits (75, 135). Second,
we
do
not
claim
that
the
framework
presented
covers
all
psychiatric/psychosomatic phenomena under the sun. Not all symptoms relate to Petzschner et al.
19
ACCEPTED MANUSCRIPT perception, forecasting, action or metacognition as the core components of our framework. However, where this relation exists, our framework may provide useful guidance in establishing analogous schemes for differential diagnostics in computational psychiatry and computational psychosomatics. Combined with models
RI PT
that can infer pwPE signaling in cortical hierarchies from neuroimaging or electrophysiological data (Supplementary Material), this could allow for non-invasive readouts of circuit function that may support differentiation of potential failure loci.
SC
The promise and limitations of this approach requires prospective patient studies that
AC C
EP
TE D
M AN U
evaluate its predictive validity.
Petzschner et al.
20
ACCEPTED MANUSCRIPT ACKNOWLEDGMENTS We acknowledge support by the René and Susanne Braginsky Foundation, the University of Zurich, the Deutsche Forschungsgemeinschaft (TR-SFB 134), and the
RI PT
UZH Clinical Research Priority Programs “Multiple Sclerosis” (CRPP MS) and
AC C
EP
TE D
M AN U
SC
“Molecular Imaging” (MINZ).
Petzschner et al.
21
ACCEPTED MANUSCRIPT FINANCIAL DISCLOSURES
AC C
EP
TE D
M AN U
SC
RI PT
All authors report no biomedical financial interests or potential conflicts of interest.
Petzschner et al.
22
ACCEPTED MANUSCRIPT FIGURE LEGENDS Figure 1. (A) Simple example of a homeostatic reflex arc as described by classical cybernetics. Sensory inputs (sensations) about an environmental quantity X (e.g.,
RI PT
current body temperature) are compared to a pre-defined setpoint (e.g., ideal body temperature). Corrective actions occur as a function of the mismatch between input and setpoint, such that X is moving closer to the setpoint (e.g. heating or cooling the
SC
body). (B) Extension to an inference-control loop, where perception (inference of environmental states) under an individual’s generative model of the world update
M AN U
beliefs that change the reflex arc’s setpoint (e.g., allostatic control of bodily states); in other cases, actions might be chosen based on the perception rather than the sensation (not shown here). (C) Further extension of the inference-control loop to include forecasting and metacognition. We wish to emphasize that this plot is highly schematic and provide a core summary of different types of inference-control loops;
TE D
it should not be misunderstood as a detailed circuit proposal.
EP
Figure 2. Two examples of perceptual inference. From left to right: prior belief, sensory data, resulting perception (posterior). (A) A classical example of a visual
AC C
illusion: We perceive the surrounding objects in the image as concave and the center object as convex, even though the sensory data stem from a 2D gray scale image. The reason is that humans (likely due to experience) hold an implicit belief that light comes from above. If light comes from above, the shadow of a concave object should be located that the top, while the shadow of a convex object should be located at the bottom. The resulting percept is thus a re-interpretation of current sensory input based on an implicit a priori belief about lights and shadows. (B) Example of the placebo effect: Treatment with drugs that contain no therapeutic Petzschner et al.
23
ACCEPTED MANUSCRIPT ingredient can alter the perception of a physical condition (e.g. reduce physical pain) and elicit autonomic reactions (e.g. an immune response). Again, the change in perception depends on a prior belief – here, that the treatment will be effective. Notably, the placebo effect scales with the predicted efficacy of the intervention (for
RI PT
example, syringes are typically considered more potent than pills).
SC
Figure 3. Schematic of inference-control in a Bayesian framework. (A) Upper panel: Illustration of Bayes’ rule using Gaussian distributions as an example. Bayes’ rule
M AN U
describes how different information sources – prior beliefs (predictions based on a model the environment and the body within) and new sensory data (likelihood) – are combined to update the belief (posterior). The amount of belief update is proportional to the prediction error (PE) – the difference between predicted (prior) and actual
TE D
(sensory) data – weighted by a precision ratio (π, inverse variance) of prior beliefs and sensory inputs (likelihood), respectively. Simply speaking, precise prior beliefs diminish and precise sensory data increase the impact of prediction errors on belief
EP
updates. Lower panel: Illustration of the concept of a generative model. A generative model infers hidden states of the world (environment or body) by inverting a
AC C
probabilistic forward model from those states to possible sensory data (likelihood), under prior beliefs about the values of the hidden states. Inverting a generative model thus corresponds to the application of Bayes’ rule. Notably, the mapping from states to data can be mechanistically interpretable (e.g., biophysical models of neuronal responses) or descriptive, such as noisy fluctuations around a constant value or a periodic function (cf. circadian rhythms of bodily states). (B) Example of an inference-control loop that is cast as a hierarchical Bayesian Model. This figure is not
Petzschner et al.
24
ACCEPTED MANUSCRIPT meant to provide a detailed description, nor does it claim to represent the only possible layout. In all brevity, the key premise here is that the brain represents and updates generative models (“model of the body/world”), with hierarchically structured beliefs. A low-level belief about a bodily/environmental state x (“prior”) is displayed
RI PT
separately from the rest of the model. The expected sensory inputs (under this prior) can be compared against actual sensations to yield a PE; this PE can be sent up the inference hierarchy and update the model. Switching from perception to action
SC
requires (temporarily) abolishing sensory precision; see (19, 42) for details. Actions can then be implemented in two main ways. Homeostatic (reactive) control unfolds
M AN U
as a direct function of PE and serves to fulfil beliefs about sensory input (as encoded by the prior; this can be seen as a probabilistic setpoint; (19)). Allostatic (predictive) control prospectively shifts this probabilistic setpoint to elicit actions; this requires predicting future states as a function of actions and bodily/environmental dynamics
TE D
(“forecasting”). Finally, metacognition could be implemented as an additional layer in the model that holds (and updates) expectations with regard to the amount of PE at
EP
the top of the inference hierarchy; see (19).
AC C
Figure 4. Highly schematic illustration of the inference-control loop for interoception and exteroception.
Exteroception: Exterosensations (sensory inputs caused by states of the external environment) originate from receptors (e.g., mechanoreceptors, proprioceptors, photoreceptors) and are transmitted via the classical sensory channels (vision, audition, touch, taste, smell) to reach the brain’s primary sensory areas. From the perspective of perception as inference, exterosensations are combined with a priori Petzschner et al.
25
ACCEPTED MANUSCRIPT beliefs, based on a model of the environment, resulting in a perception of the environment that is referred to as exteroception. Interoception: Interosensations (sensory inputs caused by bodily states) originate
RI PT
from various bodily receptors (baroreceptors, chemoreceptors, thermoreceptors, etc.). Interosensations carry information about bodily states such as temperature, pain, itch, blood oxygenation, intestinal tension, heartrate, hormonal concentration
SC
etc. and reach the brain via two major afferent pathways: small diameter modalityspecific afferent fibers in lamina 1 of the spinal cord that project to specific thalamo-
M AN U
cortical nuclei and the vagus and glossopharyngeal nerves projecting to the nucleus of the solidary tract. Both pathways converge on the posterior insula cortex. From the perspective of perception as inference, interosensations are combined with a priori beliefs, based on a model of the body, resulting in a perception of the body that is referred to as interoception. Interoception and exteroception combined yield the
TE D
percept of the body within its environment that informs action selection. both with
AC C
EP
regard to internally directed (autonomic) and externally directed (motor) actions.
Petzschner et al.
26
ACCEPTED MANUSCRIPT REFERENCES
1. Casey BJ, Craddock N, Cuthbert BN, Hyman SE, Lee FS, Ressler KJ (2013): DSM-5 and RDoC: progress in psychiatry research? Nat Rev Neurosci. 14:810-814.
RI PT
2. Stephan KE, Bach DR, Fletcher PC, Flint J, Frank MJ, Friston KJ, et al. (2016): Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis. Lancet Psychiat. 3:77-83. 3. Kapur S, Phillips AG, Insel TR (2012): Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol Psychiatry. 17:1174-1179.
SC
4. Maia TV, Frank MJ (2011): From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 14:154-162.
M AN U
5. Montague PR, Dolan RJ, Friston KJ, Dayan P (2012): Computational psychiatry. Trends Cogn Sci. 16:72-80. 6. Stephan KE, Mathys C (2014): Computational approaches to psychiatry. Curr Opin Neurobiol. 25:85-92. 7. Friston KJ, Stephan KE, Montague R, Dolan RJ (2014): Computational psychiatry: the brain as a phantastic organ. The Lancet Psychiatry. 1:148-158. 8.
Wang XJ, Krystal JH (2014): Computational psychiatry. Neuron. 84:638-654.
TE D
9. Huys QJ, Maia TV, Frank MJ (2016): Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 19:404-413. 10. Paulus MP, Huys QJ, Maia TV (2016): A Roadmap for the Development of Applied Computational Psychiatry. Biol Psychiatry Cogn Neurosci Neuroimaging. 1:386-392.
EP
11. Corlett PR, Fletcher PC (2014): Computational psychiatry:a a Rosetta Stone linking the brain to mental illness. Lancet Psychiat. 1:399-402.
AC C
12. Stephan KE, Friston KJ, Frith CD (2009): Dysconnection in schizophrenia: from abnormal synaptic plasticity to failures of self-monitoring. Schizophr Bull. 35:509-527. 13. Huys QJ, Guitart-Masip M, Dolan RJ, Dayan P (2014): Decision-theoretic psychiatry. Clin Psychol Sci. in press. 14. Mathys C (2016): How could we get nosology from computation? In: Redish AD, Gordon JA, editors. Computational Psychiatry: What Can Theoretical Neuroscience and Psychiatry Teach Each Other? Cambridge, MA: MIT Press. 15.
Wiener N (1948): Cybernetics. New York: Wiley.
16.
Ashby WR (1956): An introduction to cybernetics. London: Chapman & Hall.
17.
Powers WT (1973): Feedback: beyond behaviorism. Science. 179:351-356.
18. Conant RC, Ashby WR (1970): Every good regulator of a system must be a model of that system. International Journal of Systems Science. 1:89-97.
Petzschner et al.
27
ACCEPTED MANUSCRIPT 19. Stephan KE, Manjaly ZM, Mathys C, Weber LAE, Paliwal S, Gard T, et al. (2016): Allostatic Self-Efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression. Front Hum Neurosci. 10:550. 20. Haker H, Schneebeli M, Stephan KE (2016): Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? Front Psychiatry. 7:107.
RI PT
21. Stephan KE, Baldeweg T, Friston KJ (2006): Synaptic plasticity and dysconnection in schizophrenia. Biol Psychiatry. 59:929-939. 22. Cannon WB (1929): Organization for Physiological Homeostasis. Physiological Reviews. 9:399-431. 23. Modell H, Cliff W, Michael J, McFarland J, Wenderoth MP, Wright A (2015): A physiologist's view of homeostasis. Adv Physiol Educ. 39:259-266.
Friston K (2003): Learning and inference in the brain. Neural Netw. 16:1325-1352.
M AN U
25.
SC
24. Kersten D, Mamassian P, Yuille A (2004): Object perception as Bayesian inference. Annu Rev Psychol. 55:271-304.
26. Helmholtz H (1860/1962): Handbuch der physiologischen Optik (English trans., Southall JPC, Ed.). New York: Dover. 27. Knill D, Richards W (1996): Perception as Bayesian inference. Cambridge: Cambridge University Press.
TE D
28. Friston K (2005): A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci. 360:815-836. 29. Dayan P, Hinton GE, Neal RM, Zemel RS (1995): The Helmholtz machine. Neural Comput. 7:889-904. 30. Adams WJ, Graf EW, Ernst MO (2004): Experience can change the 'light-fromabove' prior. Nat Neurosci. 7:1057-1058. Geisler WS, Kersten D (2002): Illusions, perception and Bayes. Nat Neurosci. 5:508-
EP
31. 510.
AC C
32. Diaconescu AO, Mathys C, Weber LA, Daunizeau J, Kasper L, Lomakina EI, et al. (2014): Inferring on the intentions of others by hierarchical Bayesian learning. PLoS Comput Biol. 10:e1003810. 33. Devaine M, Hollard G, Daunizeau J (2014): The social Bayesian brain: does mentalizing make a difference when we learn? PLoS Comput Biol. 10:e1003992. 34. Daw ND, Dayan P (2014): The algorithmic anatomy of model-based evaluation. Philos Trans R Soc Lond B Biol Sci. 369. 35. Friston K, Schwartenbeck P, Fitzgerald T, Moutoussis M, Behrens T, Dolan RJ (2013): The anatomy of choice: active inference and agency. Front Hum Neurosci. 7:598. 36. Penny W, Stephan KE (2014): A Dynamic Bayesian Model of Homeostatic Control. Lecture Notes in Computer Science. 8779:60-69.
Petzschner et al.
28
ACCEPTED MANUSCRIPT 37. Chambon V, Sidarus N, Haggard P (2014): From action intentions to action effects: how does the sense of agency come about? Front Hum Neurosci. 8:320. 38. Fleming SM, Daw ND (2017): Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation. Psychol Rev. 124:91-114. 39. Ortega PA, Braun DA (2013): Thermodynamics as a theory of decision-making with information-processing costs. P Roy Soc A Math Phy. 469:20120683.
RI PT
40. Todorov E (2009): Efficient computation of optimal actions. Proc Natl Acad Sci U S A. 106:11478-11483. 41. Kording KP, Wolpert DM (2004): The loss function of sensorimotor learning. Proc Natl Acad Sci U S A. 101:9839-9842.
SC
42. Friston KJ, Daunizeau J, Kilner J, Kiebel SJ (2010): Action and behavior: a freeenergy formulation. Biol Cybern. 102:227-260.
M AN U
43. Huys QJ, Lally N, Faulkner P, Eshel N, Seifritz E, Gershman SJ, et al. (2015): Interplay of approximate planning strategies. Proc Natl Acad Sci U S A. 112:3098-3103. 44. Kaelbling LP, Littman ML, Cassandra AR (1998): Planning and acting in partially observable stochastic domains. Artificial Intelligence Journal. 101:99-134. 45. Keramati M, Gutkin B (2014): Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife. 3.
TE D
46. Doya K, Ishii S, Pouget A, Rao RP (2011): Bayesian Brain: Probabilistic Approaches to Neural Coding. Cambridge, MA: MIT Press. 47. Friston K (2010): The free-energy principle: a unified brain theory? Nat Rev Neurosci. 11:127-138. 48. Lee TS, Mumford D (2003): Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am A Opt Image Sci Vis. 20:1434-1448.
EP
49. Rao RP, Ballard DH (1999): Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci. 2:79-87.
AC C
50. Ernst MO, Banks MS (2002): Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 415:429-433. 51. Angelaki DE, Gu Y, DeAngelis GC (2009): Multisensory integration: psychophysics, neurophysiology, and computation. Curr Opin Neurobiol. 19:452-458. 52. Petzschner FH, Glasauer S, Stephan KE (2015): A Bayesian perspective on magnitude estimation. Trends Cogn Sci. 19:285-293. 53. Jazayeri M, Shadlen MN (2010): Temporal context calibrates interval timing. Nat Neurosci. 13:1020-1026. 54. Mathys CD, Lomakina EI, Daunizeau J, Iglesias S, Brodersen KH, Friston KJ, et al. (2014): Uncertainty in perception and the Hierarchical Gaussian Filter. Front Hum Neurosci. 8:825.
Petzschner et al.
29
ACCEPTED MANUSCRIPT 55. Felleman DJ, Van Essen DC (1991): Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex. 1:1-47. 56. Mumford D (1992): On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybern. 66:241-251. 57. Hilgetag CC, Medalla M, Beul SF, Barbas H (2016): The primate connectome in context: Principles of connections of the cortical visual system. Neuroimage. 134:685-702.
RI PT
58. Browning M, Behrens TE, Jocham G, O'Reilly JX, Bishop SJ (2015): Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat Neurosci. 18:590-596.
59. McCormick DA, Wang Z, Huguenard J (1993): Neurotransmitter control of neocortical neuronal activity and excitability. Cereb Cortex. 3:387-398.
SC
60. Corlett PR, Taylor JR, Wang XJ, Fletcher PC, Krystal JH (2010): Toward a neurobiology of delusions. Prog Neurobiol. 92:345-369.
M AN U
61. Adams RA, Stephan KE, Brown HR, Frith CD, Friston KJ (2013): The computational anatomy of psychosis. Front Psychiatry. 4:47. 62. Corlett PR, Honey GD, Krystal JH, Fletcher PC (2011): Glutamatergic model psychoses: prediction error, learning, and inference. Neuropsychopharmacology. 36:294315. 63. Bastos AM, Usrey WM, Adams RA, Mangun GR, Fries P, Friston KJ (2012): Canonical microcircuits for predictive coding. Neuron. 76:695-711.
TE D
64. Friston K, Kiebel S (2009): Predictive coding under the free-energy principle. Philos Trans R Soc Lond B Biol Sci. 364:1211-1221. 65. Friston K (2009): The free-energy principle: a rough guide to the brain? Trends Cogn Sci. 13:293-301.
EP
66. Adams RA, Shipp S, Friston KJ (2013): Predictions not commands: active inference in the motor system. Brain Struct Funct. 218:611-643.
AC C
67. Brown H, Adams RA, Parees I, Edwards M, Friston K (2013): Active inference, sensory attenuation and illusions. Cogn Process. 68. Voss M, Ingram JN, Wolpert DM, Haggard P (2008): Mere expectation to move causes attenuation of sensory signals. PLoS One. 3:e2866. 69. Friston K, Samothrakis S, Montague R (2012): Active inference and agency: optimal control without cost functions. Biol Cybern. 106:523-541. 70. Donnarumma F, Maisto D, Pezzulo G (2016): Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi. PLoS Comput Biol. 12:e1004864. 71. Kiebel SJ, Daunizeau J, Friston KJ (2008): A hierarchy of time-scales and the brain. PLoS Comput Biol. 4:e1000209. 72. Perrinet LU, Adams RA, Friston KJ (2014): Active inference, eye movements and oculomotor delays. Biol Cybern. 108:777-801.
Petzschner et al.
30
ACCEPTED MANUSCRIPT 73. Daunizeau J, den Ouden HE, Pessiglione M, Kiebel SJ, Friston KJ, Stephan KE (2010): Observing the observer (II): deciding when to decide. PLoS One. 5:e15555. 74. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007): Learning the value of information in an uncertain world. Nat Neurosci. 10:1214-1221.
RI PT
75. Iglesias S, Mathys C, Brodersen KH, Kasper L, Piccirelli M, den Ouden HE, et al. (2013): Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning. Neuron. 80:519-530. 76. Seth AK, Suzuki K, Critchley HD (2011): An interoceptive predictive coding model of conscious presence. Front Psychol. 2:395. 77. Seth AK (2013): Interoceptive inference, emotion, and the embodied self. Trends Cogn Sci. 17:565-573.
SC
78. Gu X, Hof PR, Friston KJ, Fan J (2013): Anterior insular cortex and emotional awareness. J Comp Neurol. 521:3371-3388.
M AN U
79. Craig AD (2002): How do you feel? Interoception: the sense of the physiological condition of the body. Nat Rev Neurosci. 3:655-666. 80. Craig AD (2003): Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol. 13:500-505. 81. Critchley HD, Harrison NA (2013): Visceral influences on brain and behavior. Neuron. 77:624-638.
TE D
82. Cechetto DF, Saper CB (1987): Evidence for a viscerotopic sensory representation in the cortex and thalamus in the rat. J Comp Neurol. 262:27-45. 83. Evrard HC, Logothetis NK, Craig AD (2014): Modular architectonic organization of the insula in the macaque monkey. J Comp Neurol. 522:64-97.
EP
84. Wiech K (2016): Deconstructing the sensation of pain: The influence of cognitive processes on pain perception. Science. 354:584-587.
AC C
85. Buchel C, Geuter S, Sprenger C, Eippert F (2014): Placebo analgesia: a predictive coding perspective. Neuron. 81:1223-1239. 86. Wager TD, Atlas LY (2015): The neuroscience of placebo effects: connecting context, learning and health. Nat Rev Neurosci. 16:403-418. 87. Feldman-Barrett LF, Simmons WK (2015): Interoceptive predictions in the brain. Nat Rev Neurosci. 16:419-429. 88. Sterling P (2014): Homeostasis vs allostasis: implications for brain function and mental disorders. JAMA Psychiatry. 71:1192-1193. 89. 15.
Sterling P (2012): Allostasis: a model of predictive regulation. Physiol Behav. 106:5-
90. Simmons WK, Avery JA, Barcalow JC, Bodurka J, Drevets WC, Bellgowan P (2013): Keeping the body in mind: insula functional organization and functional connectivity integrate interoceptive, exteroceptive, and emotional awareness. Hum Brain Mapp. 34:2944-2958.
Petzschner et al.
31
ACCEPTED MANUSCRIPT 91. Nguyen VT, Breakspear M, Hu X, Guo CC (2016): The integration of the internal and external milieu in the insula during dynamic emotional experiences. Neuroimage. 124:455463. 92. Allen M, Frank D, Schwarzkopf DS, Fardo F, Winston JS, Hauser TU, et al. (2016): Unexpected arousal modulates the influence of sensory noise on confidence. Elife. 5.
RI PT
93. Garfinkel SN, Minati L, Gray MA, Seth AK, Dolan RJ, Critchley HD (2014): Fear from the heart: sensitivity to fear stimuli depends on individual heartbeats. J Neurosci. 34:65736582. 94. Salomon R, Ronchi R, Donz J, Bello-Ruiz J, Herbelin B, Martet R, et al. (2016): The Insula Mediates Access to Awareness of Visual Stimuli Presented Synchronously to the Heartbeat. J Neurosci. 36:5115-5127.
SC
95. Stephan KE, Schlagenhauf F, Huys QJ, Raman S, Aponte EA, Brodersen KH, et al. (2017): Computational neuroimaging strategies for single patient predictions. Neuroimage. 145:180-199.
97.
M AN U
96. Stephan KE, Iglesias S, Heinzle J, Diaconescu AO (2015): Translational Perspectives for Computational Neuroimaging. Neuron. 87:716-732. Fava GA, Sonino N (2010): Psychosomatic medicine. Int J Clin Pract. 64:1155-1161.
98. de Berker AO, Rutledge RB, Mathys C, Marshall L, Cross GF, Dolan RJ, et al. (2016): Computations of uncertainty mediate acute stress responses in humans. Nat Commun. 7:10996.
TE D
99. Fineberg SK, Steinfeld M, Brewer JA, Corlett PR (2014): A Computational Account of Borderline Personality Disorder: Impaired Predictive Learning about Self and Others Through Bodily Simulation. Front Psychiatry. 5:111. 100. Joynt KE, Whellan DJ, O'Connor CM (2003): Depression and cardiovascular disease: mechanisms of interaction. Biol Psychiatry. 54:248-261.
EP
101. Dowlati Y, Herrmann N, Swardfager W, Liu H, Sham L, Reim EK, et al. (2010): A meta-analysis of cytokines in major depression. Biol Psychiatry. 67:446-457.
AC C
102. Renn BN, Feliciano L, Segal DL (2011): The bidirectional relationship of depression and diabetes: a systematic review. Clin Psychol Rev. 31:1239-1246. 103. McEwen BS (1998): Stress, adaptation, and disease. Allostasis and allostatic load. Ann N Y Acad Sci. 840:33-44. 104. Friston KJ, Harrison L, Penny W (2003): Dynamic causal modelling. Neuroimage. 19:1273-1302. 105. Freeman R, Chapleau MW (2013): Testing the autonomic nervous system. Handb Clin Neurol. 115:115-136. 106. Holzer P (2015): Acid-sensing ion channels in gastrointestinal function. Neuropharmacology. 94:72-79. 107. Chatterjee SS, Mitra S (2015): "I do not exist"-Cotard syndrome in insular cortex atrophy. Biol Psychiatry. 77:e52-53.
Petzschner et al.
32
ACCEPTED MANUSCRIPT 108. Setiawan E, Wilson AA, Mizrahi R, Rusjan PM, Miler L, Rajkowska G, et al. (2015): Role of translocator protein density, a marker of neuroinflammation, in the brain during major depressive episodes. JAMA Psychiatry. 72:268-275. 109. Stephan KE, Diaconescu AO, Iglesias S (2016): Bayesian inference, dysconnectivity and neuromodulation in schizophrenia. Brain. 139:1874-1876.
RI PT
110. Kapur S (2003): Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiatry. 160:13-23. 111. Huitinga I, Erkut ZA, van Beurden D, Swaab DF (2004): Impaired hypothalamuspituitary-adrenal axis activity and more severe multiple sclerosis with hypothalamic lesions. Ann Neurol. 55:37-45.
SC
112. Hanken K, Eling P, Kastrup A, Klein J, Hildebrandt H (2015): Integrity of hypothalamic fibers and cognitive fatigue in multiple sclerosis. Mult Scler Relat Disord. 4:3946.
M AN U
113. Harrison NA, Cooper E, Dowell NG, Keramida G, Voon V, Critchley HD, et al. (2015): Quantitative Magnetization Transfer Imaging as a Biomarker for Effects of Systemic Inflammation on the Brain. Biol Psychiatry. 78:49-57. 114. Lawson RP, Rees G, Friston KJ (2014): An aberrant precision account of autism. Front Hum Neurosci. 8:302. 115. Pellicano E, Burr D (2012): When the world becomes 'too real': a Bayesian explanation of autistic perception. Trends Cogn Sci. 16:504-510.
TE D
116. Van de Cruys S, de-Wit L, Evers K, Boets B, Wagemans J (2013): Weak priors versus overfitting of predictions in autism: Reply to Pellicano and Burr (TICS, 2012). Iperception. 4:95-97. 117. Austerweil JL (2015): Contradictory "heuristic" theories of autism spectrum disorders: the case for theoretical precision using computational models. Autism. 19:367-368.
EP
118. Fletcher PC, Frith CD (2009): Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat Rev Neurosci. 10:48-58.
120.
AC C
119. Jardri R, Duverne S, Litvinova AS, Deneve S (2017): Experimental evidence for circular inference in schizophrenia. Nat Commun. 8:14218. Paulus MP, Stein MB (2006): An insular view of anxiety. Biol Psychiatry. 60:383-387.
121. Paulus MP, Stein MB (2010): Interoception in anxiety and depression. Brain Struct Funct. 214:451-463. 122. Garfinkel SN, Tiley C, O'Keeffe S, Harrison NA, Seth AK, Critchley HD (2016): Discrepancies between dimensions of interoception in autism: Implications for emotion and anxiety. Biol Psychol. 114:117-126. 123. Gu X, Eilam-Stock T, Zhou T, Anagnostou E, Kolevzon A, Soorya L, et al. (2015): Autonomic and brain responses associated with empathy deficits in autism spectrum disorder. Hum Brain Mapp. 36:3323-3338. 124. Quattrocki E, Friston K (2014): Autism, oxytocin and interoception. Neurosci Biobehav Rev. 47:410-430.
Petzschner et al.
33
ACCEPTED MANUSCRIPT 125. Berniker M, Voss M, Kording K (2010): Learning priors for Bayesian computations in the nervous system. PLoS One. 5. 126. Vilares I, Howard JD, Fernandes HL, Gottfried JA, Kording KP (2012): Differential representations of prior and likelihood uncertainty in the human brain. Curr Biol. 22:16411648.
RI PT
127. Moran RJ, Campo P, Symmonds M, Stephan KE, Dolan RJ, Friston KJ (2013): Free energy, precision and learning: the role of cholinergic neuromodulation. J Neurosci. 33:82278236. 128. Khalsa SS, Craske MG, Li W, Vangala S, Strober M, Feusner JD (2015): Altered interoceptive awareness in anorexia nervosa: Effects of meal anticipation, consumption and bodily arousal. Int J Eat Disord. 48:889-897.
SC
129. Pappens M, Van den Bergh O, Vansteenwegen D, Ceunen E, De Peuter S, Van Diest I (2013): Learning to fear obstructed breathing: comparing interoceptive and exteroceptive cues. Biol Psychol. 92:36-42.
M AN U
130. Paulus MP, Flagan T, Simmons AN, Gillis K, Kotturi S, Thom N, et al. (2012): Subjecting elite athletes to inspiratory breathing load reveals behavioral and neural signatures of optimal performers in extreme environments. PLoS One. 7:e29394. 131. Brock C, Brock B, Aziz Q, Moller HJ, Pfeiffer Jensen M, Drewes AM, et al. (2016): Transcutaneous cervical vagal nerve stimulation modulates cardiac vagal tone and tumor necrosis factor-alpha. Neurogastroenterol Motil.
TE D
132. Makovac E, Garfinkel SN, Bassi A, Basile B, Macaluso E, Cercignani M, et al. (2015): Effect of parasympathetic stimulation on brain activity during appraisal of fearful expressions. Neuropsychopharmacology. 40:1649-1658. 133. Harrison NA, Brydon L, Walker C, Gray MA, Steptoe A, Dolan RJ, et al. (2009): Neural origins of human sickness in interoceptive responses to inflammation. Biol Psychiatry. 66:415-422.
EP
134. Baumann TK, Simone DA, Shain CN, LaMotte RH (1991): Neurogenic hyperalgesia: the search for the primary cutaneous afferent fibers that contribute to capsaicin-induced pain and hyperalgesia. J Neurophysiol. 66:212-227.
AC C
135. Diaconescu AO, Mathys C, Weber LA, Kasper L, Mauer J, Stephan KE (2017): Hierarchical prediction errors in midbrain and septum during social learning. Soc Cogn Affect Neurosci.
Petzschner et al.
34
RI PT SC M AN U perception
EP
setpoint
comparator
action
sensation
X
C
model of the world
metacogn
forecasting
perception
B
TE D
A
AC C
internal external
ACCEPTED MANUSCRIPT
prior
comparator
action
prior
compara
sensation
X
model of world
action
X
SC
RI PT
ACCEPTED MANUSCRIPT
M AN U
A
TE D
Light comes from above.
B
AC C
EP
The sourrounding objects are concave.
Drug X is an effective pain killer.
Placebo
The pill reduced my pain.
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
TE D
A
Bayes‘ Rule posterior~ likelihood · prior p(x|y,m)~ p(y|x,m)p(x|m)
forec
likelihood
EP
prior posterior
B
µprior
AC C
µposterior
µlikelihood
πprior prior
Belief update ∆belief ~precision · PE µposterior = µprior +
πlikelihood
πlikelihood · PE πlikelihood + πprior
posterior likelihood
p(x|y,m) p(y|x,m)p(x|m)
likelihood: p(y|x,m) prior: p(x|m) posterior: p(x|y,m) x = state of world & body y = sensory data m = model
internal external
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
TE D
world
changing the world through exteroactions
EP
exterosensations from the world
combined percept of the body in the world
AC C
exteroception interoception
changing the body through interoactions
interosensations from the body
body
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT Computational Psychosomatics and Computational Psychiatry: Towards A Joint Framework for Differential Diagnosis Supplemental Information
RI PT
COMPUTATIONAL ASSAYS: MODEL-BASED INFERENCE ON INDIVIDUAL PATHOPHYSIOLOGY
The framework presented in the main text implies that disruptions could arise from the computation and/or signaling of a limited set of quantities: bottom-up signals (sensory input or PEs), top-down signals (expectations or predictions), and their precision. How could generative
SC
models help determine, in an individual patient, where the primary problem is located? In psychophysical experiments, Bayesian models can distinguish different causes of
M AN U
perceptual abnormalities, e.g., by manipulating sensory and belief precisions (1). Crucially, the dependence of belief updates on a precision ratio (Figure 3) means that the same abnormality in belief updating could result from altered precision of bottom-up signals (sensory inputs or PEs), or from inversely altered precision of prior beliefs. This ambiguity is at the core of several pathophysiological debates (e.g., in autism; see main text).
TE D
In behavioral experiments, belief updating can be induced by changing the individual experience (2, 3), frames and suggestions (4, 5), or dynamically via cues (6-9). This may not only concern low-level sensory priors, but also abstract beliefs, including volatility (7, 10, 11), optimism (12), intentions of others (8), and metacognitive beliefs (e.g., learned helplessness
EP
(13)). HBMs provide trajectories of PEs, precisions and predictions at multiple, hierarchically coupled levels and in individual subjects (14, 15). Additionally, these computational trajectories
AC C
can be used as regressors in fMRI analyses (7, 10). The application of generative models to behavioral data has so far been restricted to the exteroceptive domain, while interoception has relied more on questionnaires (16) and behavioral assessments such as heartbeat detection tasks (17), but also on neurophysiological studies. For example, the functional connectivity of interoceptive circuits has been studied in health (18) and depression (19, 20). However, functional connectivity is descriptive and specifies undirected coupling estimates. In order to reveal more specific circuit mechanisms, such as the directionality of message passing in cortical hierarchies, generative models of fMRI and MEG/EEG data are required. Here, we focus on dynamic causal models (DCMs). These can not only estimate directed connection strengths, but can also assess putative biological mechanisms of precision-weighting and prediction/PE signaling.
1
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT While the neurobiology of precision-weighted PE signaling in cortex is not understood fully, a general picture is emerging. The evidence so far suggests that predictions are signaled via NMDA receptors (NMDARs) at descending connections (mainly originating from infragranular layers) whereas PEs are likely conveyed via AMPA (and possibly also NMDARs) at ascending connections (primarily originating from supragranular layers) (21). Finally, precision-weighting depends on postsynaptic gain which is determined by neuromodulators (e.g., dopamine,
RI PT
acetylcholine; (22)) and local GABAergic inhibition (for reviews, see (23-27)). It is worth noting that both NMDAR function and synthesis of neuromodulatory transmitters can be altered by bodily factors, such as peripherally produced pro-inflammatory cytokines, hormones, and metabolites (e.g., kynurenines); for reviews, see (28) and (29).
SC
Some of these mechanisms are already, in principle, amenable to estimation in individual subjects. For example, DCM of high-resolution fMRI may help to characterize connections of
M AN U
ascending (PEs) and descending (predictions) nature by distinguishing hemodynamic and neuronal contributions to layer-wise fMRI signals (30). DCMs for EEG/MEG can distinguish the contributions of ionotropic receptors with different time constants (AMPA, NMDA, and GABAA) (31-33) and recognize induced changes in postsynaptic gain (34, 35). The feasibility of physiologically fine-grained inference has been demonstrated by several proof-of-concept studies in animals and humans (32, 35-37). DCMs can also be combined with generative models of behavior in order to test, for example, how specific connections change dynamically
TE D
as a function of prediction of PEs (38, 39). Together with Bayesian model comparison techniques for evaluating the relative plausibility of alternative disease mechanisms, a combined generative modeling approach to behavioral and neurophysiological data may provide a foundation for the development of tests for differential diagnosis (40). For
EP
computational psychosomatics, additional extensions of the generative models are required that (i) incorporate peripheral (autonomic) physiological data, and (ii) integrate cerebral and
AC C
bodily processes at different time scales. While models of the first type have been presented recently, e.g. (41), the latter pose an interesting challenge for the future (for discussion, see (42-44)).
Finally, it is worth mentioning that to achieve clinical utility, a computational assays would not necessarily need to probe each and every component of the general inference-control loop we discuss in the main text. Even a binary distinction between two competing explanations could already be extremely useful, as demonstrated by the example on autism. It is also worth remembering that in internal medicine, comprehensive differential diagnosis is not always achievable by a single diagnostic test but frequently requires multiple tests of different nature.
2
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT
7.
8.
9. 10.
11. 12. 13. 14. 15.
16. 17.
18.
19.
20.
21.
RI PT
6.
SC
5.
M AN U
4.
TE D
3.
EP
2.
Vilares I, Howard JD, Fernandes HL, Gottfried JA, Kording KP (2012): Differential representations of prior and likelihood uncertainty in the human brain. Curr Biol. 22:16411648. Adams WJ, Graf EW, Ernst MO (2004): Experience can change the 'light-from-above' prior. Nat Neurosci. 7:1057-1058. Petzschner FH, Glasauer S (2011): Iterative Bayesian estimation as an explanation for range and regression effects: a study on human path integration. J Neurosci. 31:1722017229. Buchel C, Geuter S, Sprenger C, Eippert F (2014): Placebo analgesia: a predictive coding perspective. Neuron. 81:1223-1239. Wager TD, Atlas LY (2015): The neuroscience of placebo effects: connecting context, learning and health. Nat Rev Neurosci. 16:403-418. Vossel S, Mathys C, Daunizeau J, Bauer M, Driver J, Friston KJ, et al. (2014): Spatial attention, precision, and Bayesian inference: a study of saccadic response speed. Cereb Cortex. 24:1436-1450. Iglesias S, Mathys C, Brodersen KH, Kasper L, Piccirelli M, den Ouden HE, et al. (2013): Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning. Neuron. 80:519-530. Diaconescu AO, Mathys C, Weber LA, Daunizeau J, Kasper L, Lomakina EI, et al. (2014): Inferring on the intentions of others by hierarchical Bayesian learning. PLoS Comput Biol. 10:e1003810. Petzschner FH, Maier P, Glasauer S (2012): Combining symbolic cues with sensory input and prior experience in an iterative bayesian framework. Front Integr Neurosci. 6:58. de Berker AO, Rutledge RB, Mathys C, Marshall L, Cross GF, Dolan RJ, et al. (2016): Computations of uncertainty mediate acute stress responses in humans. Nat Commun. 7:10996. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007): Learning the value of information in an uncertain world. Nat Neurosci. 10:1214-1221. Stankevicius A, Huys QJ, Kalra A, Series P (2014): Optimism as a prior belief about the probability of future reward. PLoS Comput Biol. 10:e1003605. Lieder F, Goodman ND, Huys QJM (2013): Learned helplessness and generalization. Proceedings of the 35th Annual Conference of the Cognitive Science Society. In press. Mathys C, Daunizeau J, Friston KJ, Stephan KE (2011): A Bayesian foundation for individual learning under uncertainty. Front Hum Neurosci. 5:39. Mathys CD, Lomakina EI, Daunizeau J, Iglesias S, Brodersen KH, Friston KJ, et al. (2014): Uncertainty in perception and the Hierarchical Gaussian Filter. Front Hum Neurosci. 8:825. Mehling WE, Price C, Daubenmier JJ, Acree M, Bartmess E, Stewart A (2012): The Multidimensional Assessment of Interoceptive Awareness (MAIA). PLoS One. 7:e48230. Garfinkel SN, Seth AK, Barrett AB, Suzuki K, Critchley HD (2015): Knowing your own heart: distinguishing interoceptive accuracy from interoceptive awareness. Biol Psychol. 104:65-74. Simmons WK, Avery JA, Barcalow JC, Bodurka J, Drevets WC, Bellgowan P (2013): Keeping the body in mind: insula functional organization and functional connectivity integrate interoceptive, exteroceptive, and emotional awareness. Hum Brain Mapp. 34:2944-2958. Avery JA, Drevets WC, Moseman SE, Bodurka J, Barcalow JC, Simmons WK (2014): Major depressive disorder is associated with abnormal interoceptive activity and functional connectivity in the insula. Biol Psychiatry. 76:258-266. Guo CC, Hyett MP, Nguyen VT, Parker GB, Breakspear MJ (2016): Distinct neurobiological signatures of brain connectivity in depression subtypes during natural viewing of emotionally salient films. Psychol Med. 46:1535-1545. Self MW, Kooijmans RN, Super H, Lamme VA, Roelfsema PR (2012): Different glutamate receptors convey feedforward and recurrent processing in macaque V1. Proc Natl Acad Sci U S A. 109:11031-11036.
AC C
1.
3
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
22. McCormick DA, Wang Z, Huguenard J (1993): Neurotransmitter control of neocortical neuronal activity and excitability. Cereb Cortex. 3:387-398. 23. Corlett PR, Taylor JR, Wang XJ, Fletcher PC, Krystal JH (2010): Toward a neurobiology of delusions. Prog Neurobiol. 92:345-369. 24. Adams RA, Stephan KE, Brown HR, Frith CD, Friston KJ (2013): The computational anatomy of psychosis. Front Psychiatry. 4:47. 25. Corlett PR, Honey GD, Krystal JH, Fletcher PC (2011): Glutamatergic model psychoses: prediction error, learning, and inference. Neuropsychopharmacology. 36:294-315. 26. Bastos AM, Usrey WM, Adams RA, Mangun GR, Fries P, Friston KJ (2012): Canonical microcircuits for predictive coding. Neuron. 76:695-711. 27. Friston K, Kiebel S (2009): Predictive coding under the free-energy principle. Philos Trans R Soc Lond B Biol Sci. 364:1211-1221. 28. Dantzer R, Heijnen CJ, Kavelaars A, Laye S, Capuron L (2014): The neuroimmune basis of fatigue. Trends Neurosci. 37:39-46. 29. Stephan KE, Friston KJ, Frith CD (2009): Dysconnection in schizophrenia: from abnormal synaptic plasticity to failures of self-monitoring. Schizophr Bull. 35:509-527. 30. Heinzle J, Koopmans PJ, den Ouden HE, Raman S, Stephan KE (2016): A hemodynamic model for layered BOLD signals. Neuroimage. 125:556-570. 31. Marreiros AC, Kiebel SJ, Friston KJ (2010): A dynamic causal model study of neuronal population dynamics. Neuroimage. 51:91-101. 32. Moran RJ, Symmonds M, Stephan KE, Friston KJ, Dolan RJ (2011): An in vivo assay of synaptic function mediating human cognition. Curr Biol. 21:1320-1325. 33. Gilbert JR, Symmonds M, Hanna MG, Dolan RJ, Friston KJ, Moran RJ (2016): Profiling neuronal ion channelopathies with non-invasive brain imaging and dynamic causal models: Case studies of single gene mutations. Neuroimage. 124:43-53. 34. Moran RJ, Stephan KE, Kiebel SJ, Rombach N, O'Connor WT, Murphy KJ, et al. (2008): Bayesian estimation of synaptic physiology from the spectral responses of neural masses. Neuroimage. 42:272-284. 35. Moran RJ, Campo P, Symmonds M, Stephan KE, Dolan RJ, Friston KJ (2013): Free energy, precision and learning: the role of cholinergic neuromodulation. J Neurosci. 33:8227-8236. 36. Schmidt A, Diaconescu AO, Kometer M, Friston KJ, Stephan KE, Vollenweider FX (2013): Modeling ketamine effects on synaptic plasticity during the mismatch negativity. Cereb Cortex. 23:2394-2406. 37. Moran RJ, Jung F, Kumagai T, Endepols H, Graf R, Dolan RJ, et al. (2011): Dynamic causal models and physiological inference: a validation study using isoflurane anaesthesia in rodents. PLoS One. 6:e22790. 38. Roy M, Shohamy D, Daw N, Jepma M, Wimmer GE, Wager TD (2014): Representation of aversive prediction errors in the human periaqueductal gray. Nat Neurosci. 17:1607-1612. 39. den Ouden HE, Daunizeau J, Roiser J, Friston KJ, Stephan KE (2010): Striatal prediction error modulates cortical coupling. J Neurosci. 30:3210-3219. 40. Stephan KE, Schlagenhauf F, Huys QJ, Raman S, Aponte EA, Brodersen KH, et al. (2017): Computational neuroimaging strategies for single patient predictions. Neuroimage. 145:180-199. 41. Staib M, Castegnetti G, Bach DR (2015): Optimising a model-based approach to inferring fear learning from skin conductance responses. J Neurosci Methods. 255:131-138. 42. Stephan KE, Iglesias S, Heinzle J, Diaconescu AO (2015): Translational Perspectives for Computational Neuroimaging. Neuron. 87:716-732. 43. Stephan KE, Manjaly ZM, Mathys C, Weber LAE, Paliwal S, Gard T, et al. (2016): Allostatic Self-Efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression. Front Hum Neurosci. 10:550. 44. Freund P, Friston K, Thompson AJ, Stephan KE, Ashburner J, Bach DR, et al. (2016): Embodied neurology: an integrative framework for neurological disorders. Brain. 139:1855-1861.
4
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT GLOSSARY The active inference framework shares the core ideas of predictive coding but generalizes them to action selection. In particular, active inference postulates that belief updating in response to prediction errors is only one way of achieving a long-term goal of surprise minimization. Alternatively, the brain can reduce surprise or prediction error by eliciting actions that lead to sensory inputs that are in accordance with the brain's expectations. Predictions (prior expectations) about sensory inputs therefore define preferences or goals that engender behaviour; in other words, control is directed towards sensory input, not motor output. Importantly, the choice between reducing prediction error through changing predictions (updates of the generative model) or through action depends on precision.
M AN U
SC
RI PT
Active Inference
Allostasis refers to the process of achieving stability (homeostasis) through (behavioral or physiological) change. Allostatic regulation occurs prior to a homeostatic perturbation and serves to avoid dyshomeostatic future states, guided by predictions of a model (forecasting). In contrast to the reactive (error-driven) control of classical homeostasis, allostasis reflects a prospective (anticipatory) form of control.
Bayes' rule
)=
( | , ) ( | ) ( | )
AC C
( | ,
EP
TE D
Allostasis
Cybernetics
Bayes theorem allows one to compute a posterior belief, given some sensory input (likelihood, p(y|x,m)) and a prediction (prior, p(x|m)). The posterior is a “compromise” between likelihood and prior, weighted by their relative precisions, and represents the optimal integration of prior knowledge with new observations. The model evidence p(y|m) in the denominator of Bayes’ theorem is a normalization constant that forms the basis for Bayesian model comparison. Cybernetics is a transdisciplinary approach that is concerned with the possible mechanisms of feedback-based regulation within a system. In the classical negative feedback loop of cybernetics, a sensor signal (reflecting the state
5
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT of some internal variable) is compared against an internal reference (set point), and regulatory action is chosen as a function of the mismatch between the two, with the goal of keeping the sensor signal close to the reference signal. Exteroception refers to the perception of the external world. Understanding perception as inference, exteroception is the result of inferring on external states of the world by combining exterosensations (i.e., sensory signals originating from receptors of classical sensory channels like vision and audition) with exteroceptive priors (i.e., a-priori knowledge about the external states) to form a posterior belief about these states.
Forecasting
Forecasting means predicting the world’s future states taking into account both the influence of possible actions and the world’s endogenous dynamics, and can be used to inform action selection. In the context of allostasis, predictions concern the degree to which the action considered will keep bodily states close to a homeostatic set point over time.
M AN U
SC
RI PT
Exteroception
A generative model m specifies how sensory data y are generated from hidden states x by combining a likelihood function p(y|x, m) with a prior p(x|m). In the context of perception, this corresponds to a probabilistic mapping from hidden states in the world to the sensory inputs the agent receives (likelihood), together with an a-priori distribution of the world's possible states.
TE D
AC C
EP
Generative Model
Hierarchical Bayesian Model
Hierarchical Bayesian models are generative models of sensory inputs that reflect the hierarchical structure of processes in the physical and social environment. Each level of the hierarchy provides a prediction (prior) for the state of the level below; the mismatch (prediction error) between this prediction and the actual state (likelihood) is sent upwards and serves to update the prior. Importantly, under broad assumptions (i.e., for all distributions from the exponential family), hierarchical Bayesian belief updates have a generic form in that they are
6
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT proportional to the prediction error and the precision of the prior belief. Homeostasis refers to the property of a biological agent to actively regulate its bodily states (such as temperature or salt concentration) such that they stay within a healthy normal range with respect to pre-defined set points. The goal is the constancy of the internal environment (or milieu intérieur) in which the cells of the body live and survive. In classical homeostasis, regulatory actions ensue as a function of the mismatch between the actual bodily sate (as signaled by sensory input from the respective organ) and the expected state (i.e., the set point).
Inference
In most cases, sensory organs provide only limited and noisy information about an agent's environment. Therefore, the true state of the world may be hidden from the observing agent. By employing a generative model of how environmental states cause the sensory input, an agent may infer on the respective unknown states. Inference corresponds to the inversion of the generative model, that is, given the observed data (sensory inputs) and prior knowledge, the agent can compute the probability of the hidden states (the posterior) according to Bayes' rule.
AC C
EP
Interoception
TE D
M AN U
SC
RI PT
Homeostasis
Likelihood, Prior, Posterior
Interoception refers to the sense of the physiological condition of the body and can be understood as inference on internal (bodily) states, where interosensations (i.e., sensory signals originating from internal sensors like thermoreceptors or baroreceptors) are combined with prior knowledge (expectations) to form posterior beliefs about states of the body such as temperature or heart rate.
The likelihood function p(y|x,m) describes how any given state of the world x (e.g., a light switch) causes a sensory input y (e.g., light on the retina of the agent) with a certain probability. The prior p(x|m) expresses the range of values environmental states inhabit a priori (e.g., is it light or dark more often?) and thus encodes learned environmental statistics. The posterior p(x|y,m) is the result of the combination of
7
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT likelihood and prior according to Bayes' rule and represents the inference on a hidden state (e.g., 'the light switch is on'), given prior knowledge and a new observation. Metacognition, or cognition about cognition (thinking about thinking), implies both knowledge about one's own cognition and control of one's own cognition. In the sense of self-monitoring or self-evaluation, it refers to a set of beliefs held by the brain about its own functional capacity.
Predictive Coding
Predictive coding postulates that the brain infers the most likely causes (environmental states) underlying sensory input by inverting a hierarchical generative model which reflects the hierarchical structure of the environment. At any given level of the model, prediction errors (deviations of the actual input from the expected input) signal that the model needs to be updated and thus drive inference and learning. Inspired by the remarkably hierarchical structure of sensory processing streams in cortex, the idea is that backward connections signal predictions to the next lower level of the hierarchy, while forward connections transmit the prediction errors the higher level cortical regions to update the prediction. Predictive coding attempts to describe perception, but does not directly speak to action selection and control.
EP
AC C
Psychosomatics
TE D
M AN U
SC
RI PT
Meta-cognition
Sensation
In psychosomatic disorders, mental processes such as beliefs and expectations are believed to play a significant role in the development, expression, or resolution of a physical illness. Psychosomatic medicine more generally explores the influence of social, psychological and behavioral factors on bodily processes. Sensory inputs (sensations) reach the brain through afferent pathways both from the external senses – vision, audition, touch, taste and smell – as well as internal sensors (baroreceptors, chemoreceptors, thermoreceptors, etc.). In frameworks like Predictive Coding, the actual sensations will, on any level of the cortical hierarchy, be compared against the expected sensory inputs (under some prior) to yield a
8
Petzschner et al.
Supplement
ACCEPTED MANUSCRIPT
RI PT
prediction error, which then serves as the (sensory) input to next higher level. The term "sensation" is not meant to imply a necessarily conscious event. Instead, any conscious experience is more likely to reflect the result of an inference process, i.e., the combination of a sensory input (a sensation) with a prior belief (an expectation) to form a posterior belief. Surprise refers to the unlikeliness of (sensory) events (given a model), or, in other words, how far sensory events deviate from prior expectations. Mathematically, the (Shannon) surprise S of an event y is equal to the negative logarithm of the probability of that event, given the model m:
SC
Surprise
M AN U
= −log( ( | ))
AC C
EP
TE D
This information-theoretic concept of surprise is equivalent to the negative log model evidence; a good model is thus one that minimises surprise about the data. Practically, prediction error (PE), i.e., the difference between predicted and actual input, is often used as a proxy for surprise.
9
