Supplementary material: Orbitofrontal and striatal circuits dynamically encode the shift. between goal-directed and habitual actions

Transcription

1 Supplementary material: Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions Christina M. Gremel 1 & Rui M. Costa 1,2 1 Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda, USA 2 Champalimaud Neuroscience Programme, Champalimaud Institute for the Unknown, Lisbon, Portugal

2 Gremel et al., supplementary material 2 Supplementary Figure S1. Lever-press related behaviors under RI and RR schedule training. (a-d) Average lever presses (a), rewards earned (b), reward rate (c), and head entries by C57BL/6J

3 Gremel et al., supplementary material 3 mice (n = 10) during concurrent RI and RR schedule training. (e, f) Average lever presses (e) and head entries (f) performed in previously RI and RR training contexts during outcome revaluation testing. (g, h) Example of the micro-structure of lever-press related behaviors in a mouse on the last day of RI (g) and RR (h) schedule training. Represented are executed lever-presses (red ticks) and head entries (blue ticks), as well as reinforcer deliveries (black triangles) across time within each schedule, with a probability of reinforcement set at p = 0.1; or 10%). Inserts within each panel show a representative distribution of behaviors across time leading up to a reinforcer delivery. As previously reported (DeRusso et al., 2011), there is no evidence of RI-schedule induced scalloped responding as seen for fixed-interval schedules of reinforcement. (i, j) Distribution of inter-reward intervals on the last day of schedule training in the RI (i) and (j) RR contexts, with similar distributions (unpaired t-tests t 206 < 0.66, p > 0.05). Error bars indicate s.e.m. * = Repeated measures ANOVA, Bonferroni corrected p < See Supplementary Methods for statistical analysis details. Statistical Analysis (a) Lever presses repeated measures ANOVA: Interaction: [F(8, 160) = 3.21, p <0.01], Training Day: [F(8, 160) = 47.58, p < 0.001], Training Schedule: [F(1, 160) = 12.15, p < 0.01] (b) Rewards earned repeated measures ANOVA: Interaction: [F(8, 160) = 0037, p > 0.05], Training day: [F(8,160) = 49.42, p < 0.001], Schedule: [F(1,160) = 0.34, p > 0.05] (c) Reward rate/min repeated measures ANOVA: Interaction: [F(8, 160) = 0.23, p > 0.05], Training day: [F(8,160) = 9.72, p < 0.001], Schedule: [F(1,160) = 1.86, p > 0.05] (d) Head entries repeated measures ANOVA: Interaction: [F(8, 160) = 0.37, p > 0.05], Training Day: [F(8,160) = 2.4, p < 0.05], Schedule: [F(1,160) = 0.82 p > 0.05] (e) Lever presses Repeated measures ANOVA Interaction (Schedule x Revaluation state: [F(1,19) = 4.51, p < 0.05] (VRI vs. DVRI: Bonferroni corrected p > 0.05; VRR vs. DVRR: Bonferroni corrected p > 0.05), Schedule [F(1,19) = 1.12, p < 0.05]; Revaluation State [F(1,19) = 17.07, p < 0.001] (f) Head entries Repeated measures ANOVA Interaction (Schedule x Revaluation state: [F(1,19) = 0.66, p > 0.05] Schedule [F(1,19) =.17, p > 0.05]; Revaluation State [F(1,19) = 6.19, p < 0.05]

4 Gremel et al., supplementary material 4 Supplementary Figure S2. Histological analyses of DMS, DLS, and OFC lesions and projections. (a) Schematic adapted from Paxinos and Franklin mouse atlas of DMS and DLS NMDA-induced lesions, with lesion inclusion area in black, and a representative lesion in grey. (b) Schematic of OFC ibotenic acid-induced lesions, with lesion inclusion area in black, and a representative lesion in grey.

5 Gremel et al., supplementary material 5 Supplementary Figure S3. Effect of dorsal medial and dorsal lateral striatum lesions on within-subject shifting between goal-directed and habitual actions. (a) Example schematic of within-subject training and outcome revaluation testing. Mice were trained to press the same lever for the same reinforcer under RI and RR training schedules. Using sensory-specific satiation, mice were given a outcome revaluation test where responding in previously RI and RR trained contexts was probed in valued and devalued states. (b-d) Average lever-presses performed (b), rewards earned (c), and head entries made (d) by Sham (n = 9), DMS (n = 5), and DLS (n = 7) lesioned across CRF and RI and RR schedule training. (e) Average consumption of pellets and sucrose

6 Gremel et al., supplementary material 6 (counterbalanced across earned reinforcer and home-cage reward) during the outcome revaluation test for Sham and DMS or DLS lesioned mice. (f-h) Average lever-presses (f), head-entries (g), and normalized (test day) head entries (h) performed in valued and devalued states in previously RI and RR schedule trained contexts in Sham, DMS, and DLS lesioned mice. Error bars indicate s.e.m. * = Repeated measures ANOVA, Bonferroni corrected p < See Supplementary Methods for statistical analysis details. Statistical Analysis (b) Lever-presses RI schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.34, p > 0.05], Training day: [F(8, 128) = 29.12, p < ], Lesion group: [F(2, 128) = 0.58, p > 0.05] RR schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 1.27, p > 0.05], Training day: [F(8, 128) = 31.56, p < ], Lesion group: [F(2, 128) = 0.93, p > 0.05] (c) Rewards earned RI schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.67, p > 0.05], Training day: [F(8, 128) = 26.06, p < ], Lesion group: [F(2, 128) = 2.78, p > 0.05] RR schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.89, p > 0.05], Training day: [F(8, 128) = 39.95, p < ], Lesion group: [F(2, 128) = 2.51, p > 0.05] (d) Reward rate/min RI schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 2.03, p < 0.05], Training day: [F(8, 128) = 9.248, p < ] (Day 2 of CRF training CTLRI vs. DLSRI Bonferroni corrected p < 0.001, Day 3 of CRF training CtlRI vs. DLSRI or DMSRI Bonferroni corrected p < 0.001), Lesion group: [F(2, 128) = 5.404, p < 0.05] RR schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.54, p > 0.05], Training day: [F(8, 128) = 8.94, p < ], Lesion group: [F(2, 128) = 2.37, p > 0.05] (e) Head Entries RI schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.71, p > 0.05], Training day: [F(8, 128) = 1.55, p > 0.05], Lesion group: [F(2, 128) = 0.17, p > 0.05] RR schedule Repeated measures ANOVA: Interaction: [F(16, 128) = 0.69, p > 0.05], Training day: [F(8, 128) = 1.17, p > 0.05], Lesion group: [F(2, 128) = 0.31, p > 0.05] (f) Consumption: Interaction: [F(2, 34) = 0.50, p > 0.05], Outcome: [F(2, 34) = 2.4, p > 0.05], Lesion group: [F(1, 34) = 0.05, p > 0.05] (g) Lever Presses Repeated measures ANOVA: Sham mice: Interaction: [F(1, 14) = 2.55, p > 0.05], Schedule: [F(1, 14) = 0.52, p > 0.05], Revaluation state: [F(1, 14) = 16.18, p < 0.05] DMS lesioned mice: Interaction: [F(1, 11) = 0.26, p > 0.05], Schedule: [F(1, 11) = 1.35, p > 0.05], Revaluation state: [F(1, 11) = 2.156, p > 0.05]

7 Gremel et al., supplementary material 7 DLS lesioned mice: Interaction: [F(1, 12) =.21, p > 0.05], Schedule: [F(1, 12) = 0.28, p > 0.05], Revaluation state: [F(1, 12) = 6.97, p < 0.05] (h) Head Entries Repeated measures ANOVA: Sham mice: Interaction: [F(1, 14) = 0.22, p > 0.05], Schedule: [F(1, 14) = 0.15, p > 0.05], Revaluation state: [F(1, 14) = 7.54, p < 0.05] DMS lesioned mice: Interaction: [F(1, 11) = 0.26, p > 0.05], Schedule: [F(1, 11) 5.91, p < 0.05], Revaluation state: [F(1, 11) =0.515, p > 0.05] DLS lesioned mice: Interaction: [F(1, 12) = 1.07, p > 0.05], Schedule: [F(1, 12) = 0.02, p > 0.05], Revaluation state: [F(1, 12) = 3.374, p > 0.05] (i) Normalized Head Entries Repeated measures ANOVA: Sham mice: Interaction: [F(1, 14) = 0.72, p > 0.05], Schedule: [F(1, 14) = 5, p = 0.05], Revaluation state: [F(1, 14) = 7.47, p < 0.05] DMS lesioned mice: Interaction: [F(1, 11) = 0.00, p > 0.05], Schedule: [F(1, 11) = 0.43, p > 0.05], Revaluation state: [F(1, 11) =0.35, p > 0.05] DLS lesioned mice: Interaction: [F(1, 12) = 2.63, p > 0.05], Schedule: [F(1, 12) = 0.00, p > 0.05], Revaluation state: [F(1, 12) = 9.25, p < 0.05]

8 Gremel et al., supplementary material 8

9 Gremel et al., supplementary material 9 Supplementary Figure S4. Effect of OFC lesions on shifting between goal-directed and habitual actions. (a) Example schematic of within-subject training and outcome revaluation testing. Mice were trained to press the same lever for the same reinforcer under RI and RR training schedules. Using sensory-specific satiation, mice were given a outcome revaluation test where responding in previously RI and RR trained contexts was probed in valued and devalued states. (be) Average lever-presses performed (b), head entries made (c), rewards earned (d), and reward rate (e) by Sham (n =7) and OFC-lesioned (n = 5) mice across CRF and RI and RR schedule training in the within-subject procedure. (f) Average lever presses and (g) head-entries and normalized (test day) head entries performed in previously RI and RR trained contexts across valued and devalued states by Sham and OFC lesioned mice. (h) Average consumption of pellets and sucrose between Sham and OFC lesioned mice during outcome revaluation testing. (i, p) Example schematic of RI only (i) or RR only (p) schedule training and outcome revaluation testing. Mice were trained to press the lever under an RI or RR schedule, and during the subsequent outcome revaluation test responding was probed in valued and devalued states. (j-l, and q-s) Average response rate and lever-presses performed (j, q), head-entries made (k, r), and rewards earned (l, s), under RI (j-l) and RR (q-s) schedules of reinforcement by Sham (RI n = 10, RR n = 8) and OFC-lesioned (RI n = 10, RR n = 11) mice. (m-o and t-v) Average normalized (test day) lever presses and lever presses performed (m, t), normalized (test day) head entries and head entries performed (n, s), and average consumption of pellets and sucrose (o, v) by Sham and OFC lesioned mice in previously RI or RR trained contexts across valued and devalued days. Error bars indicate s.e.m. * = Repeated measures, one-way, two- ANOVA, Bonferroni corrected p < See Supplementary Methods for statistical analysis details. Statistical Analysis b) Lever presses Repeated measures ANOVA RI schedule: Interaction: [F(8, 56) = 0.47, p > 0.05], Training day: [F(8, 56) = 10.14, p < ], Lesion group: [F(1, 7) = 0.04, p > 0.05]

10 Gremel et al., supplementary material 10 RR schedule Repeated measures ANOVA: Interaction: [F(8, 56) = 1.89, p = 0.08] (post hoc analyses revealed Day 5, 6 Bonferroni corrected ps < 0.05, Training day: [F(8, 56) = , p < ]), Lesion group: [F(1, 7) = 7.80, p < 0.05] c) Head entries Repeated measures ANOVA RI schedule: Interaction: [F(8, 56) = 0.67, p > 0.05], Training day: [F(8, 56) = 26.06, p < ], Lesion group: [F(1,7) = 2.78, p > 0.05] RR schedule Repeated measures ANOVA: Interaction: [F(8, 56) = 0.50, p > 0.05], RR Training day: [F(8, 56) = 1.98 p = 0.06], RR Lesion group: [F(1, 7) = 0.40, p > 0.05] d) Rewards earned Repeated measures ANOVA RI schedule: Interaction: [F(8, 56) = 1.67, p > 0.05], Training day: [F(8, 156) = 28.87, p < ], Lesion group: [F(1, 7) = 0.15, p > 0.05] RR schedule: Interaction: [F(8, 56) = 0.65, p > 0.05], Training day: [F(8, 56) = 9.81, p < ], Lesion group: [F(1, 7) = 1.82 p > 0.05] e) Reward rate/min Repeated measures ANOVA RI schedule: Interaction: [F(8, 56) = 0.48, p > 0.05], Training day: [F(8, 56) = 3.92, p < 0.01], Lesion group: [F(1, 7) = 0.09, p > 0.05] RR schedule: Interaction: [F(8, 56) = 0.25, p > 0.05], Training day: [F(8, 56) = 7.85, p < ], Lesion group: [F(1, 7) = 0.21, p > 0.05] f) Lever presses Repeated measures ANOVA Sham Interaction: [F(1,12) = 3.9, p > 0.05], Schedule: [F(1, 12) = 0.25, p > 0.05], Revaluation state: [F(1,12) = 4.19, p = 0.09] OFC lesion Interaction: [F(1,8) = 0.43, p > 0.05], Schedule: [F(1, 8) = 0.47, p > 0.05], Revaluation state: [F(1,8) = 0.25, p > 0.05] g) Head entries Repeated measures ANOVA; Sham Interaction: [F(1,12) = 0.16, p > 0.05], Schedule: [F(1, 12) = 1.3, p > 0.05], Revaluation state: [F(1,12) = 0.0, p > 0.05] OFC lesion Interaction: [F(1,8) = 0.07, p > 0.05], Schedule: [F(1, 8) = 0.17, p > 0.05], Revaluation state: [F(1,8) = 0.23, p > 0.05] h) Consumption; Interaction: [F(1, 22) = 0.03, p > 0.05], outcome: [F(1, 22 = 20.6, p < 0.001], Lesion group: [F(1, 22) = 1.45, p > 0.05] RI schedule Repeated measures ANOVA j) Lever presses/min: Interaction: [F(8, 144) = 0.55, p > 0.05], Training day: [F(8, 144) = 13.84, p < ], Lesion group: [F(1, 29) = 0.15, p > 0.05]. Lever presses: Interaction: [F(8, 144) = 0.97, p > 0.05], Training day: [F(8, 144) = 13.34, p < ], Lesion group: [F(1, 29) = 1.08, p > 0.05] k) Head entries RI schedule: Interaction: [F(8, 144) = 0.19, p > 0.05], Training day: [F(8, 144) = 21.68, p < ], Lesion group: [F(1, 29) = 0.10, p > 0.05] l) Rewards earned RI schedule: Interaction: [F(8, 144) = 0.19, p > 0.05], Training day: [F(8, 144) = 21.68, p < ], Lesion group: [F(1, 29) = 0.10, p > 0.05] m) RI schedule Revaluation test Repeated measures ANOVA

11 Gremel et al., supplementary material 11 Normalized Lever presses RI schedule: Interaction [F(1,18) = 0.06 p > 0.05], Revaluation state [F(1,18) =0.07, p > 0.05], Lesion Group [F(1,18) = 0.00, p > 0.05]. Lever presses RI schedule: Interaction [F(1,18) = 1.42 p > 0.05], Revaluation state [F(1,18) =1.29, p > 0.05], Lesion group [F(1,18) = 0.14, p > 0.05] n) Head entries RI schedule: Interaction [F(1,36) = 0.004, p > 0.05], Day [F(1,36) =0.92, p > 0.05], Lesion group [F(1,36) = 0.13, p > 0.05]. Normalized (test day) Head entries RI schedule: Interaction [F(1,18) = 1.16, p > 0.05], Revaluation state [F(1,18) =0.27, p > 0.05], Lesion group [F(1,18) = 0.00, p > 0.05] o) Consumption RI schedule; Interaction: [F(1, 38) = 0.30, p > 0.05], outcome: [F(1, 38 = 21.64, p < 0.001], Lesion group: [F(1, 38) = 0.28, p > 0.05] q) RR schedule only Repeated measures ANOVA Lever presses/min: [F(8, 136) = 2.37, p = 0.02], Training day [F(8,136 ) =17.95 p < 0.001], Lesion group [F(1,17) = 4.90, p=0.04]. Lever presses RR: Interaction [F(8,136) = 0.96, p > 0.05], Training day [F(8,136) =31.73, p < 0.001], Lesion group [F(1,17) = 0.32, p < 0.001] r) Head entries RR schedule: Interaction [F(8,136) = 0.63, p > 0.05], Training day [F(8,136) =4.05, p < 0.001], Lesion group [F(1,17) = 2.70, p> 0.05] s) Rewards earned RR schedule: Interaction [F(8,136) = 0.29, p > 0.05], Training day [F(8,136) =1.14, p < 0.001], Lesion group [F(1,17) = 0.85, p< 0.05] t) RR schedule only Revaluation test Normalized (test day) Lever presses (RR schedule): Interaction [F(1,17) = 3.61, p =0.06] (Sham V vs. DV Bonferroni corrected p < 0.05; Wilcoxon W = -23, p = 0.06), Revaluation state [F(1,17) =3.08 p = 0.09], Lesion group [F(1,17) = 3.4, p = 0.08]. Lever presses (RR schedule): Interaction [F(1,17) = 1.77, p > 0.05], Revaluation state [F(1,17) =5.01, p < 0.05], Lesion group [F(1,17) = 2.81, p > 0.05] u) Normalized (test day) Head entries RR schedule: Interaction [F(1,17) = 0.52 p >0.05], Revaluation state [F(1,17) =0.00, p >0.05], Lesion group [F(1,17) = 0.00, p > 0.05]. Head entries RR schedule: Interaction [F(1,17) = 1.18 p =0.06], Revaluation state [F(1,17) =0.02, p < 0.02], Lesion group [F(1,17) = 0.07, p > 0.05]. v) Consumption; Interaction: [F(1, 28) = 0.13, p > 0.05], outcome: [F(1, 28 = 31.55, p < 0.001], Lesion group: [F(1, 28) = 0.12, p > 0.05]

12 Gremel et al., supplementary material 12 Supplementary Figure S5. In vivo simultaneous recordings of OFC, DMS and DLS during learning and execution of goal-directed actions and habits. (a) Average lever presses, (b) rewards earned, and (c) head entries across each of the six days of RI and RR schedule training in mice used in the in vivo recording experiments using the within-schedule design. (d) Average lever

13 Gremel et al., supplementary material 13 presses, (e) head entries, and (f) normalized (test day) head entries made during devaluation testing (valued and devalued test days) in the previous RI and RR training contexts. (g) Example of waveform, cluster separation from noise on the basis of principle component analyses, and interspike interval histogram for a single unit recorded in OFC. (h) Verified electrode placements in OFC and dorsal striatum (black squares represent electrode placement in the OFC, while blue and red squares represent electrode location in the DMS and DLS respectively). (i-k) Histogram showing the distribution of time for the beginning of significant firing rate modulation relative to lever-pressing for up-modulated and down-modulated DMS (i), DLS (j), and OFC (k). Error bars indicate s.e.m. See Supplementary Methods for statistical analysis details. Statistical analysis a) Lever presses: [F(8,112) = 0.76, p > 0.05], Day [F(8,112) =25.6, p < 0.001], Schedule [F(1,112) = 0.05, p< 0.05] b) Rewards earned: [F(7,98) = 0.94, p > 0.05], Day [F(7,98) =1.6, p >0.05], Schedule [F(1,98) = 2.08, p > 0.05] c) Head entries: [F(8,112) = 0.76, p > 0.05], Day [F(8,112) =25.6, p < 0.001], Schedule [F(1,112) = 0.05, p< 0.05] d) Lever presses: [F(1,16) = 2.43, p > 0.05], Day [F(1,16) =.31, p >0.05], Schedule [F(1,28) = 0.01, p > 0.05] e) Head entries: [F(1,16) = 0.31, p > 0.05], Day [F(1,16) =0.03, p >0.05], Schedule [F(1,16) = 0.56, p > 0.05] f) Normalized (test day) Head entries: [F(1,16) = 1.77, p > 0.05], Day [F(1,16) =0.76, p >0.05], Schedule [F(1,16) = 0.00, p > 0.05]

14 Gremel et al., supplementary material 14 Supplementary Figure S6. Scatter-plots of baseline frequency in RI (x-axis) versus RR (y-axis) contexts across training for all recorded units within DMS (a), DLS (f), and OFC (k). Summary bar graphs showing average baseline frequency of all units recorded in RI and RR contexts across training in DMS (b), DLS (g), and OFC (l). Absolute modulation rate for all lever-press related neurons in DMS (c), DLS (h), and OFC (m). Absolute modulation rate for Both lever-press related neurons in DMS (d), DLS (i), and OFC (n). Absolute modulation rate for Specific lever-press

15 Gremel et al., supplementary material 15 related neurons in DMS (e), DLS (j), and OFC (o). Error bars indicate s.e.m. * = Repeated measures ANOVA, Bonferroni corrected p < Statistical analysis b) DMS mean baseline firing rate: paired t-test (t 82 = 0.18, p > 0.05). c) DMS Both neuron mean modulation rate: repeated measures ANOVA (Schedule x Training Day) [F(1,30) = 0.004, p > 0.05]. Schedule [F(1,30) = 0.14, p >0.05], Training Day [F(1,30) = 5.37, p < 0.05]. d) DMS Specific neuron mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,25) = 3.66, p = 0.06]. Schedule [F(1,25) = 1.17, p >0.05], Training Day [F(1,25) = 0.000, p > 0.05]. f) DLS mean baseline firing rate: paired t-test (t 89 = 0.46, p > 0.05). g) DLS Both neuron mean modulation rate: repeated measures ANOVA (Schedule x Training Day) [F(1,26) = 0.26, p > 0.05]. Schedule [F(1,26) = 0.62, p >0.05], Training Day [F(1,26) = 0.47, p > 0.05]. h) DLS Specific neuron mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,39) = 0.42, p >0.05]. Schedule [F(1,39) = 0.43, p >0.05], Training Day [F(1,39) = 0.67, p > 0.05]. j) OFC mean baseline firing rate: paired t-test (t 309 = 0.71, p > 0.05). k) OFC Both neuron mean modulation rate: repeated measures ANOVA (Schedule x Training Day) [F(1,37) = 2.25, p > 0.05]. Schedule [F(1,37) = 1.63, p >0.05], Training Day [F(1,37) = 1.15, p > 0.05]. l) OFC Specific neuron mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,70) = 0.38, p >0.05]. Schedule [F(1,70) = 0.61, p >0.05], Training Day [F(1,70) = 5.36, p < 0.05].

16 Gremel et al., supplementary material 16 Supplementary Figure S7. Modulation direction of lever-press related neurons. Summary plots showing the percentage of neurons recorded (DMS n = 134; DLS n = 173; OFC n = 470) across training) in both RI and RR contexts that were up or down-modulated across training and on Valued and Devalued days during outcome revaluation testing under RI schedule for DMS (a), DLS (b), and OFC (c), and under RR schedules of reinforcement for DMS (d), DLS (e), and OFC (f). Error bars indicate s.e.m. Statistical analysis a) DMS direction of lever-press related activity for RI vs. RR context: Up modulated Chi square = 3.53, p >0.05. b) DLS direction of lever-press related activity for RI vs. RR context: Up modulated Chi square = 7.47, p = 0.06

17 Gremel et al., supplementary material 17 c) OFC direction of lever-press related activity for RI vs. RR context: Up modulated Chi square = 0.77, p > d) DMS direction of lever-press related activity for RI vs. RR context: Down modulated Chi square = 12.82, p < 0.05 e) DLS direction of lever-press related activity for RI vs. RR context: Down modulated Chi square = 4.69, p > 0.05 f) OFC direction of lever-press related activity for RI vs. RR context: Down modulated Chi square = 16.29, p < 0.05 g) DMS mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,89) = 0.89, p > 0.05]. Schedule [F(1,74) = 1.89, p > 0.05], Training Day [F(1,89) = 3.87, p = 0.052]. h) DLS mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,95) = 0.24, p > 0.05]. Schedule [F(1,95) = 0.00, p > 0.05], Training Day [F(1,95) = 1.37, p > 0.05]. i) OFC mean modulation rate: 2-way ANOVA (Schedule x Training Day) [F(1,148) = 1.88, p > 0.05]. Schedule [F(1,148) = 1.29, p > 0.05], Training Day [F(1,148) = 0.00, p > 0.05].

18 Gremel et al., supplementary material 18 Supplementary Figure S8. Lever-press related activity during outcome revaluation testing. Scatter-plot showing baseline firing rate of recorded neurons in RI (x-axis) and RR (y-axis) training contexts in DMS (a), DLS (e), and OFC (i). Averaged baseline firing rate for RI and RR training contexts for DMS (b), DLS (f), and OFC (j). Percentage of recorded neurons per mouse showing lever-press related changes in activity in RI and RR training contexts across Valued and Devalued states of outcome revaluation testing in DMS (c), DLS (g), and OFC (k). The percentage of leverpress related neurons that modulated firing rate during lever-press behavior in only one previous

19 Gremel et al., supplementary material 19 schedule trained context (Specific) or modulated firing rate during lever-press behavior in both previously schedule trained contexts (Both) for DMS (d), DLS (h), and OFC (l). Error bars indicate s.e.m. * = Chi square analyses p < Statitistical analysis b) DMS mean baseline firing rate: paired t-test (t 82 = 0.18, p > 0.05). c) DMS %lever-press related activity for RI vs. RR context: Chi square = 8.89, p < 0.05 d) DMS %lever-press neurons for Both vs. Specific for RI vs. RR context: Chi square = 21.59, p < f) DLS mean baseline firing rate: paired t-test (t 83 = 0.51, p > 0.05). g) DLS %lever-press related activity for RI vs. RR context: Chi square = 0.50, p >0.05 h) DLS %lever-press neurons for Both vs. Specific for RI vs. RR context: Chi square = 4.66, p < 0.05 j) OFC mean baseline firing rate: paired t-test (t 181 = 0.50, p > 0.05). k) OFC %lever-press related activity for RI vs. RR context: Chi square = 0.46, p >0.05 l) OFC %lever-press neurons for Both vs. Specific for RI vs. RR context: Chi square = 17.26, p < 0.001

20 Gremel et al., supplementary material 20 Supplementary Figure S9. Context specific lever-press related neural activity modulation. Average modulation rate for neurons that modulated firing rate only in RI or RR training contexts (Specific neurons) in the devalued state for DMS (a), DLS (b), and OFC (c). Scatter-plots of the devaluation index for the RI training context (x-axis) versus the modulation rate of Specific neurons in the RI training context (y-axis) in the Devalued state during outcome revaluation testing for DMS (d), DLS (e), and OFC (f). Scatter-plots of the devaluation index for the RR training context (x-axis) versus the modulation rate of Specific neurons in the RR training context (y-axis) in the Devalued

21 Gremel et al., supplementary material 21 state during outcome revaluation testing for DMS (g), DLS (h), and OFC (i). Error bars indicate s.e.m. r = Pearson correlation analyses. Statistical analysis a) DMS mean Specific neuron modulation rate for RI and RR contexts: unpaired t-test (t 11 = 0.32, p > 0.05). b) DLS mean Specific neuron modulation rate for RI and RR contexts: unpaired t-test (t 12 = 0.60, p > 0.05). c) OFC mean Specific neuron modulation rate for RI and RR contexts: unpaired t-test (t 33 = 0.94, p > 0.05).

22 Gremel et al., supplementary material 22 Supplementary Figure S10. Valued to devalued state-induced change in neural modulation. (a, b, c) For each mouse (n = 6), the z-score of the net modulation for DMS (a), DLS (b), and OFC (c) in the Valued and Devalued states in the RR context. (d, e, f) For each mouse, the z-score of the net modulation for DMS (a), DLS (b), and OFC (c) One mouse was removed from each DMS and DLS in RI and RR contexts due to the lack of significant modulation in recorded neurons during one or the other valuation states.

23 Gremel et al., supplementary material 23 Supplementary Figure S11. Chemicogenetic inhibition of OFC projection neurons via hm4d i receptor activation during outcome revaluation testing. (a,b) Fluorescence images showing representative spread of AAV2/9.CamKII.HI.GFP-Cre expression (green) (a) and AAV9.hSyn-DIOhM4D i -mcherry (red) expression (b) following OFC injection (100 nl/each/per side) (*note- each injection contained an additional 100 nl of AAV2/9.DIO.ChR2-YFP, for a total injection volume of

24 Gremel et al., supplementary material nl). Of note, hm4di receptor expression is going to be limited to cells that also express crerecombinase. Note the lesser degree of DREADD expression, further limiting any effect of activation to OFC projection neurons (red) (scale bar (mm) is in lower right corner) (c, d) An electrode array was implanted at the injection site, and the effect of hm4d i receptor agonist clozapine-n-oxide (CNO) systemic administration (1 mg/kg) (10ml/kg) on neural activity was examined in awakebehaving mice in a separate trial. OFC neural activity was recorded for 30 min prior to CNO administration, and the effect of CNO on OFC activity was examined for ~1 h and 45 min in mice expressing hm4d receptors in the OFC. CNO led to a decrease neural activity within 1 h, as shown in examples (c) and (d) from 2 different mice. Given the temporal limitations associated with this method and with extracellular recordings, we cannot say whether recorded neurons expressed hm4d i receptors. However, we did see a significant reduction in mean firing rate (t-test pre CNO to post CNO treatment ps < 0.05) in ~60% percent of recorded units suggesting that at a circuit level the firing rate of OFC neurons was reduced. (e) Effect of CNO on lever presses during outcome revaluation testing for Ctl mice previously co-injected with a GFP virus and the conditional virus expressing DREADD receptors (resulting in no expression of DREADD receptors), and hm4d i mice co-injected with the Cre virus and the conditional virus expressing DREADD receptors. During outcome revaluation testing, on valued and devalued days, all mice were given a 1 h pretreatment of CNO (1 mg/kg) (10ml/kg), and the effect on lever-press behavior in previously RI and RR trained contexts was examined. (F) Schematic of spread area in OFC following Cre or GFP virus injections (green), as well as spread of DREADD expression (red). Error bars indicate s.e.m. Statistical analysis e) Lever presses, repeated measures ANOVA: Ctl interaction (Schedule x Revaluation State) [F(1, 20) = 2.041, p > 0.05], Revaluation State [F1,20) = 11.37, p < 0.001], Schedule [1, 20] = 0.64, p > 0.05]; intra-ofc hm4d i interaction (Schedule x Revaluation State) [F(1, 18) = 0.15, p > 0.05], Revaluation State [F1,18) = 0.02, p > 0.05], Schedule [1, 18] = 0.00, p > 0.05]

25 Gremel et al., supplementary material 25

26 Gremel et al., supplementary material 26 Supplementary Figure S12. Optogenetic activation of OFC during outcome revaluation testing. (a-d) Fluorescence images showing spread of AAV2/9.CamKII.ChR2-YFP.SV40 expression (green) following (a) OFC injection (300 nl) (Stanford-Deisseroth Lab) to (b) dorsal striatum (CPu), (c) basolateral amygdala (BLA), and (d) substantia nigra (SN) and ventral tegmental area (VTA) (scale bar (mm) is in lower right or left corners). (e) Schematic of spread area in OFC following virus injections (green), as well site of bilateral stimulation (solid black dot = left hemisphere, open black dot = right hemisphere. An electrode array was lowered into the injection site in a subset of mice to examine the effect of photostimulation on OFC neural activity. (f) In an anesthetized mouse, photostimulation with 5 msec light pulse at a frequency of 10 Hz reliably evoked a ~1:1 ratio of spike to light pulse. Note that the waveforms did not change between stimulation and non-stimulation. (g) In an awake behaving mice, 5 msec light pulse at 10 Hz did not evoke a clear 1:1 spike to light pulse ratio, but (h) did lead to an overall increase in firing rate. (i, j) Lever presses (i) and normalized (test day) lever presses (j) during outcome revaluation testing. On valued and devalued days, following pre-feeding mice were lightly anesthetized and optical fibers were connected to bilateral indwelling optical fiber ferrules. After 15 min, mice were place into the operant chambers, and lever pressing in the absence of photostimulation (5 min) and in the presence of photostimulation with 473 nm wavelength light, < 5 mw, at 10 hz, 5 ms pulse (5 min) was examined. (k) Effect of photostimulation on the revaluation index for each mouse in the RI and RR training contexts. Photostimulation had no effect on revaluation indices in the RI context (paired t-test: t 5 = 1.62, p > 0.17), but did drastically affect revaluation indices in the RR context (biasing devalued conditions towards valuation, paired t-test: t 5 = 4.35, p < 0.02, the only animal not affected was an animal that did not show devaluation without light). Error bars indicate s.e.m. * = Repeated measures ANOVA, Bonferroni corrected p < Statistical analysis i) Lever presses, repeated measures ANOVA: No photostimulation interaction (Schedule x Revaluation state) [F(1, 10) = 2.17, p > 0.05], Day [F1,10) = 2.58, p > 0.05], Schedule [1, 10] = 0.95,

27 Gremel et al., supplementary material 27 p > 0.05]; 10 hz 473 nm wavelength light interaction (Schedule x Revaluation state) [F(1, 10) = 1.74, p > 0.05], Day [F1,10) = 0.19, p > 0.05], Schedule [1, 10] = 3.2, p > 0.05] j) Normalized (test day) Lever presses, repeated measures ANOVA: No photostimulation interaction (Schedule x Revaluation state) [F(1, 10) = 8.10, p < 0.05], Day [F1,10) = 1.75, p > 0.05], Schedule [1, 10] = 0.00, p > 0.05]; 10 hz 473 nm wavelength light interaction (Schedule x Revaluation state) [F(1, 10) = 3.68, p = 0.08], Day [F1,10) = 0.28, p > 0.05], Schedule [1, 10] = 6.67, p < 0.05]