A computational psychiatry approach identifies how alpha-. 2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque.
Supplementary Information for
A computational psychiatry approach identifies how alpha2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque Authors:
S.A. Hassani1*, M. Oemisch1*, M. Balcarras1, S. Westendorff1, S. Ardid2, M.A. van der Meer3, P. Tiesinga4 and T. Womelsdorf1
Authors Institutions: 1 Department of Biology, Centre for Vision Research, York University, Toronto, Ontario M6J 1P3, Canada; 2 Department of Mathematics, Boston University, Boston, MA 02215, USA; 3Department
of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755,USA.
4Department
of Neuroinformatics, Donders Centre for Neuroscience, Radboud University Nijmegen,
6525 AJ, Nijmegen, The Netherlands * These
authors contributed equally to this work.
Supplementary Results 1: Early error commissions, attentional lapses and perseverations were unaffected by Guanfacine. Premature fixation breaks are errors committed during covert attention deployment to one stimulus (i.e. after color onset) and before the change event (the dimming). During the 19 week testing period we found that on average 8.10 % (SE: 0.98) and 13.05 % (SE: 1.35) of trials were premature fixation breaks in the Control and Guanfacine sessions respectively, which is significantly different (Wilcoxon rank sum test, p < 0.01), similar to what we had found for higher doses in the dose identification phase of our experiment (see above). We next asked whether erroneous saccadic responses during the time of the stimulus change varied between
drug conditions. The task design included trials in which the rewarded stimulus dimmed before, at the same time, and after the unrewarded stimulus. The pilot dose identification testing suggested that Guanfacine reduces these errors which would be a strong indication that Guanfacine acts by reducing interference from salient but irrelevant stimulus events as changes of the unrewarded stimulus when it changed before, or at the same time as the rewarded stimulus. However, we found that outside the pilot dose testing phase, the proportion of errors committed in Control and Guanfacine conditions was on average not different between those trials in which dimming occurred (1) in the unrewarded stimulus before the rewarded stimulus (Control: 0.26 (SE: 0.01); Guanfacine: 0.24 (SE: 0.01)), (2) at the same time in rewarded and unrewarded stimuli (Control: 0.30 (SE: 0.01); Guanfacine: 0.30 (SE: 0.01)), or (3) in the rewarded stimulus before the unrewarded stimulus (Control: 0.17 (SE 0.02); Guanfacine: 0.16 (SE 0.02)). To ensure that we did not miss an effect that occurred only in subsets of trials, we performed this error-saccade analysis also within sliding windows of 5 trials from trial 1 to 30 since the reversal, but did not find differences between Control and Guanfacine conditions for any trial window (data not shown). We next confirmed the negative results from the error type analysis by comparing the performance accuracy (rather than the proportion of error subtype) in trials when rewarded and non-rewarded dimming occurred at the same time and found that Control and Guanfacine days were in fact not different (overall accuracy on same-time dimming trials: Control: 64.89% (SE: 1.97); Guanfacine: 62.32% (SE: 1.44), Wilcoxon rank sum test, p = 0.5019). Likewise, there was no difference in accuracy on trials when the unrewarded stimulus dimmed before the rewarded stimulus (Control: 71.06% (SE: 1.76); Guanfacine: 69.62% (SE: 2.32), Wilcoxon rank sum test, p = 0.9302), and neither when the rewarded stimulus dimmed before the unrewarded stimulus
(Control: 60.46% (SE: 01.84); Guanfacine: 58.68% (SE: 01.49), Wilcoxon rank sum test, p = 0.4137). We analyzed accuracy also with a sliding window of 5 trials from trial 1 to 30 since the reversal and did not found apparent differences between Control and Guanfacine conditions for any trial window (data not shown). The 19 week testing period provided sufficient data to test for variations of rare behavioral errors such as perseveration errors. The monkey showed perseveration of unrewarded choices following an unrewarded trial resulting from the wrong color choice in 12.6-14.4% of all unrewarded choices (sequences of successive error trials such as CEE, CEEE, …, CEEn). These color-based perseveration errors did not differ between Control (12.6%, SE 3.1) and Guanfacine (14.4%, SE 2.9) days (Wilcoxon rank sum test, p = 0.066). To test whether the animal persevered on features other than color we calculated the same percent perseverations (successively unrewarded choices) on the motion direction (e.g. successive unrewarded downward saccades to the dimming) and the stimulus location (e.g. successive unrewarded choice on the motion direction of the stimulus in the right visual field), and on combinations of all features (e.g. successive erroneous choice on the stimulus with the same color on the right side moving downward). There was no significant difference in the percentage of perseveration on motion direction (Control: 9.38% (SE: 3.10); Guanfacine: 10.75% (SE: 2.20), Wilcoxon rank sum test, p = 0.0798), stimulus location (Control: 10.56% (SE: 03.08); Guanfacine: 11.78% (SE 3.09); Wilcoxon rank sum test, p = 0.2201), or conjunctions of stimulus features between Control and Guanfacine sessions (all tests for differences, p > 0.05).
Supplementary Results 2: Consistency of learning benefit with Guanfacine across blocks in the experimental session.
This result of enhanced learning success during the actual learning period of the task could be robust across all blocks of a session on Guanfacine days. In another scenario, it could emerge particularly at later stages of a training session where sustained attention and motivation may benefit most from enhanced noradrenergic action. Alternatively, it may be evident only during early blocks in which the brain concentration of Guanfacine action will be relatively higher than late in the session (Supplementary Fig. 2A, for pharmacokinetic results of Guanfacine, see Supplementary Results 3). We tested these alternatives by calculating the average learning trials for sets of 4 adjacent blocks relative to the first block of the day with a sliding window until block eight (which is the average number of performed blocks, see above). We then took the reverse approach and calculated the average learning trials for blocks relative to the last block of the day (see Methods). This procedure ensured that a maximal number of blocks contributed to the estimated learning success across the day. We found that relative to the first block of the day, seven of eight block sets showed an average faster learning on Guanfacine days than on Control days (Supplementary Fig. 2B). In contrast, we found that only four of seven block sets since the last block of a day’s session showed faster average learning in Guanfacine than Control sessions. To test whether the learning effect is still robust across the behavioral sessions we used permutation statistics) (see Supplementary Methods), finding that the likelihood to observe faster learning in Guanfacine versus Control block sets in 11 of 15 possible block sets is significantly larger than chance (permutation statistics, p < 0.001) (Supplementary Fig. 2B).
Supplementary Results 3: Characterization of Guanfacine’s pharmacokinetics using High Performance Liquid Chromatography (HPLC) and Mass Spectrometry (MS).
We characterized the pharmacokinetics of Guanfacine at the dose that we identified to be the behaviourally efficient dose (0.75mg/kg) in the dose testing phase of the experiment. In order to quantify Guanfacine’s metabolism and degradation rate within the macaque model we devised a protocol in which blood samples were taken every 40 minutes for 4 hours after Guanfacine injection and the blood concentration of Guanfacine was measured using High Performance Liquid Chromatography (HPLC) and Mass Spectrometry (MS) similar to previous studies in humans1. Using this method, we acquired a resolution capable of detecting as little as 30 femtomoles of drug. The procedure started with the placement of a catheder for later blood sample extraction using light anaesthesia (Dexdomitor and Ketamine) reversed with Antisedan. The awake animal was then seated in a custom primate chair and engaged in watching a movie while 300 μl blood samples were taken every 40 minutes for 4 hours (0, 40, 80, 120, 160, 200, and 240min). This time frame was well within the range of all recording sessions relative to injection. The blood samples were left in room temperature until clotting was observed, typically 30-60min, and then transported to a 4 °C fridge. Upon the final sample extraction, all blood samples were transported to a centrifuge where they were spun at 2000 rpm for 40 minutes in order to separate the serum. The serum was then aliquoted and spin filtered (3kDa molecular weight cut off) and had acid added to the sample to help with preservation. The samples were then frozen at -80 °C until the HPLC protocol was applied. Each sample provided triplicate results (technical replicate) and was loaded into the HPLC into a c18 reversed phase column where unbound protein and molecules were washed out for 15 min with 5% aceto-nitrile. Then a quick ramp up to 80% aceto-nitrile (20 min process with a period of 5 min with 80% acetonitrile) released the bound compounds in the HPLC column. Then the washed solution was subjected to a multi reaction monitoring protocol using a MS causing the breaking of Guanfacine
into two component fragments (control experiments with drug only samples were already done in order to quantify MS peaks expected by Guanfacine) that were used to identify and quantify Guanfacine blood concentrations. The results of this protocol yielded an expected half life of Guanfacine of 43.23 min with a plateau of Guanfacine concentrations 2 hours after injection at
