Part 2:A Dopamine Gradient Controls Access To Distributed Working Memory in The Large-scale Monkey Cortex
Mar 20, 2022
Contact: Audrey Hu Whatsapp/hp: 0086 13880143964 Email: audrey.hu@wecistanche.com
Dopamine shifts between activity-silent and persistent activity modes of working memory
Recent experimental and modeling results show that some delay tasks can be solved with little or no persistent activity (Mongillo et al., 2008; Rose et al., 2016; Watanabe and Funahashi, 2014; Wolff et al., 2017). This has spurred a debate about whether persistent activity or ‘‘activity-silent’’ mechanisms underlie working memory (Constantinidis et al., 2018; Lundqvist et al., 2018). Is dopamine modulation throughout the cortex relevant to this debate? We endowed the model with short-term plasticity to assess the possibility of activity-silent working memory in the large-scale network. Short-term plasticity was implemented at all synapses between excitatory cells (using the same parameters Mongillo et al., 2008) and from excitatory to CB/SST cells. We investigated activity-silent representations by ‘‘pinging’’ the system with a neutral stimulus and reading out the activity generated in response, similar to the experimental protocol in Wolff et al. (2017) (Figure 4A, i). For optimal mid-levels of dopamine release (Figure 4A, ii), the model generated persistent activity that was very similar to the network without short-term plasticity. The strong and distributed activation of the frontal and parietal cortex is reminiscent of the ignition response to consciously observed stimuli (van Vugt et al., 2018).
For low and high levels of dopamine release, there was no persistent activity (Figure 4A, iii). However, when we pinged the system with a neutral stimulus, activity relating to the target cue was generated transiently throughout the frontoparietal network (Figure 4A, iii), suggesting that a memory of the target stimulus was stored internally. During the delay period, the synaptic efficacy increased at connections between neurons coding for the target stimulus. Previous models of activity-silent short-term memory have focused on local synaptic changes in the pre-frontal cortex (Mongillo et al., 2008). In our model, most of the increase in synaptic efficacy was in synaptic connections from neurons in sensory areas (Figure 4A, iii). We then restricted short-term synaptic plasticity to presynaptic neurons outside of the frontoparietal network. Pinging this system again resulted in activation of the target-related activity throughout the frontoparietal network (Figure S6). Next, we performed the opposite manipulation and restricted short-term synaptic plasticity to pre-synaptic neurons in the frontoparietal network. Pinging that system did not lead to activation of the frontoparietal network (Figure S6). This suggests that synaptic plasticity at connections from (presynaptic) prefrontal cortical neurons is not required for activity-silent memory. Finally, we restricted short-term plasticity to local connections. In that network, activity-silent memory recall also failed (Figure S6). This suggests that short-term facilitation in inter-areal feedforward connections from early sensory areas to the frontal and parietal cortex is a potential substrate for ‘‘activity-silent’’ memory in the absence of a strong initial pre- frontal response to the stimulus.
Why does the brain have two parallel systems for holding items in short-term memory? To explore this question, we simulated the model using a ping protocol (Wolff et al., 2017) with a distractor. After a behaviorally relevant cue and during the delay period, we introduced a distractor that should be filtered out by the network, followed by a neutral ping stimulus (Figure 4B, i). For mid-level dopamine release, persistent activity coding for the target stimulus is engaged and maintained through the distractor and ping (Figure 4B, ii). The distractor is represented transiently in the inferior temporal (IT) and lateral intraparietal cortex (LIP) (thus replicating the experimental results in Suzuki and Gottlieb, 2013) but does not reach most of the frontoparietal network. In the low- and high-dopamine cases, during the ping, the activity-silent mechanism regenerates activity related to the last encoded stimulus, the distractor, in the frontal and parietal cortex (Figure 4B, iii). Thus, pinging from the activity-silent state scenario always re-calls the latest item but cannot ignore a distractor. Therefore, dopamine release may serve to encode salient items in working memory and protect them from distraction.
Dopamine increases distractor resistance by shifting the subcellular target of inhibition
How does dopamine protect working memory from distraction? To examine this question, we analyzed activity within CR/VIP and CB/SST neurons during a working memory task with a dis- tractor (Figure 5A). CB/SST and CR/VIP neurons are in competition because they mutually inhibit each other. When CB/SST cell firing is higher, pyramidal cell dendrites are relatively inhibited. Conversely, when CR/VIP cell firing is higher, pyramidal cell dendrites are disinhibited. Each cortical area in the model contains two selective populations of pyramidal, CB/SST, and CR/VIP cells. We first analyzed trials in which the model successfully ignores the distractor. In the target-selective populations, CR/VIP neurons fire at a much higher rate than CB/SST neurons (Figures 5B and 5C). Thus, the dendrites of the target-selective pyramidal cells are disinhibited, allowing inter-areal target-related activity to flow between cortical areas. In the distractor-selective populations, throughout the frontoparietal network, CB/SST neurons fire at a slightly higher rate than CR/VIP cells. Thus, activity from other cortical areas is blocked from entering the dendrites of dis- tractor-selective pyramidal cells in the frontal and parietal cortex. To test the importance of this effect, we transiently inhibited CB/SST2 cells in the frontoparietal network during the presentation of the distractor (CB/SST2; Figure 5D). This transient inhibition of CB/SST2 cells was sufficient to switch the network to a distractible state, with the distractor stimulus held in working memory until the end of the trial (Figure 5D).
Because dopamine increases the strength of inhibition to dendrites and decreases inhibition to somata, it is possible that this aspect of dopamine modulation enhances distractor resistance of the system. We removed this effect of dopamine modulation while leaving dopamine’s effects on NMDA and adaptation currents as before (Figure 5E). We repeated the working memory task in the presence of the distractor with a mid-level of dopamine, which normally results in distractor-resistant working memory. Without the dopamine-dependent shift of inhibition from the soma to the dendrite, the system becomes distractible (Figures 5F and 5G). Previous modeling work has shown that persistent activity can depend on local recurrent excitatory connections or a combination of local and inter-areal loops (Mejias and Wang, 2021; Murray et al., 2017). We searched the parameter space for the strength of local and inter-areal excitatory- to-excitatory connections and found that when a subset of local cortical areas was endowed with sufficient recurrent excitation to generate persistent activity in isolation (e.g., gE(se)E(LF) = 0:33nA, mE E = 1:25), high somatic inhibition and low dendritic inhibition
were generally associated with distractibility (Figure 5H; Figure S7). Low somatic and high dendritic inhibition were associated with distractor-resistant behavior (Figure 5H; Figure S7). Therefore, the action of dopamine in shifting inhibition from the soma to the dendrite (Gao et al., 2003), via its strong effect on CB/SST cells (Mueller et al., 2020), prevents the distractor-related activity from sensory areas from disrupting ongoing persistent activity in the frontoparietal network.
Learning to optimally time dopamine release through reinforcement
In real life, we experience a constant flow of sensory inputs, and our working memory system must be flexible in determining the timing of relevant versus irrelevant information. Dopamine neurons fire in response to task-relevant stimuli (Schultz et al., 1993) but should not fire in response to task-irrelevant distracting stimuli, regardless of timing. We hypothesized that the correct timing of dopamine release could be learned by simple reward-learning mechanisms.
We created a simplified model of the ventral tegmental area (VTA) with GABAergic and dopaminergic neurons and connected this to our large-scale cortical model (Figure 6A) (cf. Braver and Cohen, 2000). Cortical pyramidal cells target GABAergic and dopaminergic cells in the VTA (Soden et al.,2020; Watabe-Uchida et al., 2012). Dopaminergic cells are also strongly inhibited by local VTA GABAergic cells (Soden et al., 2020). Dopamine in the model is released in the cortex in response to VTA dopaminergic neuron firing, and cortical dopamine levels slowly return to baseline following cessation of dopaminergic neuron firing (Muller et al., 2014). In the model, the synaptic strengths of cortical inputs from the selected populations to VTA populations are increased following a reward and weakened following an incorrect response (Harnett et al., 2009; Soltani and Wang, 2006).
We tested the model on a variant of the target-distractor-ping task introduced earlier(Figures 4B, i,and6B). For the first30trials, the first stimulus (cue 1, red) was rewarded (rule 1). For the following 30trials, the second stimulus (cue2, blue) was rewarded (rule2).Forthefinal30trials,weswitchedbacktorule1(Figure6B). Bytheseventhtrialofthefirstblock,distractor-resistant persistent activity emerged, and the first cue was remembered correctly. This behavior persisted until the next block. Following a few trials of the second block, dopamine release in response to the first stimulus was reduced, and neural populations throughout the cortex only transiently represented the first (now irrelevant) stimulus. However, the dopamine response to the second stimulus increased so that persistent activity representing the second stimulus was engaged. Following the second rule switch, the system again switched back to engaging persistent activity in response to the first cue. Additionally, the number of trials to engage appropriate persistent activity decreased gradually with each switch. We further tested the model on a version of the task in which the relevant red cue could be shown first or second within a block before the blue cue became relevant in the second block. The model was also able to learn this task, although it took more trials (10–15) to learn the switch(for the firstfew blocks). Thus, by means of simple reward-learning mechanisms, the optimal timing of dopamine release can be learned, allowing flexible engagement of distributed persistent activity in working memory.
DISCUSSION
We uncovered a macroscopic gradient of dopamine D1 receptor density along the cortical hierarchy. By building a novel anatomically constrained model of the monkey cortex, we showed how dopamine can engage distributed persistent activity mechanisms and protect memories of behaviorally relevant stimuli from distraction. This work leads to new predictions that would not have been possible with local circuit models. For example, the model shows that dopamine’s enhancement of inhibition from CB/SST-expressing cells to the dendrites of pyramidal cells blocks distracting sensory information from entering the frontoparietal working memory network. Second, when an initial stimulus fails to robustly activate the prefrontal cortex, we found that the memory of the original stimulus can be recalled through an activity-silent synaptic mechanism in inter-areal connections from the sensory to the frontoparietal cortex. Last, our model predicts that dopamine can switch between activity-silent and distributed persistent activity mechanisms, and the timing of dopamine release could be learned through reinforcement. This suggests that distributed persistent activity may be engaged for behaviorally relevant stimuli that need to be remembered and protected from distractors.
A gradient of D1 receptors along the cortical hierarchy
We used quantitatively in vitro receptor autoradiography to create a high-resolution, high-fidelity map of cortical dopamine receptor architecture. The dopamine system can also be imaged in vivo using positron emission tomography (PET) and single-photon emission computed tomography (SPECT) scans. These scans can provide information regarding individual and group differences but are limited in spatial resolution and signal-to-noise ratio (Abi-Dargham et al., 2002; Froudist-Walsh et al., 2017a; Roffman et al., 2016; Slifstein et al., 2015) and are often unreliable for cortical measurements (Egerton et al., 2010; Farde et al., 1988). It is now possible to map the expression of genes coding for dopamine receptors across the brain. Gene expression methods have certain advantages, especially RNA sequencing, which can provide cell-specific data. However, mRNA expression is not always closely related to or even positively correlated with the receptor density at the cell membrane (Arnatkeviciute et al., 2019; Beliveau et al., 2017). Receptor density at the membrane is the functionally important quantity and is measured here directly. The map of D1 receptor density here greatly expands previous descriptions of D1 receptor densities (Goldman-Rakic et al., 1990; Impieri et al., 2019; Lidow et al., 1991; Niu et al., 2020; Richfield et al., 1989). We show that D1 receptor density increases along the cortical hierarchy, peaking in the prefrontal and posterior parietal cortex. A previous study of 12 cortical areas suggested a posterior-anterior gradient of D1 receptor expression (Lidow et al., 1991). Here we assess D1 receptor density in 109 cortical areas, take into account variation in neuron density across the cortex and show that the D1 receptor gradient more closely follows the cortical hierarchy than a strict posterior-anterior gradient. The distinction is clear, with higher levels of D1 receptor density per neuron in areas of the posterior parietal cortex than the somatosensory and primary motor cortex. Future work is required to test the degree to which gradients of gene expression accurately capture the receptor gradient (Beliveau et al., 2017; Hurd et al., 2001). The gradient of dopamine D1 receptors is similar to gradients of other anatomical and functional properties described across the cortex, many of which increase or decrease along the hierarchy (Burt et al., 2018; Fulcher et al., 2019; Goulas et al., 2018; Margulies et al., 2016; Sanides 1962; Shafiei et al., 2020; Wang 2020). We observed some interesting patterns of D1R density per neuron (Figure 1F), such as a gradual caudorostral increase within the prefrontal cortex, which resembles previously reported gradients of plasticity, laminar connectivity, and abstraction (Badre and D’Esposito 2009; Riley et al., 2018; Vezoli et al., 2021). Because of the small number of animals and relatively similar D1R expression levels in several areas of the frontal and parietal cortex, the comparison of D1R density between pairs of areas is difficult. As shown originally in Markov et al. (2014a), the hierarchy itself is steep for early sensory areas and becomes shallower for higher-association areas. Therefore, the exact positions of areas like LIP or 10 are not as robustly distinguishable as those of V1, V2, and V4. Nonetheless, we expect the general pattern of an increase in D1R density per neuron along the cortical hierarchy to hold. Although the D1R labeling per neuron, as well as synaptic excitation and inhibition display a smooth gradient, quantitative variations of circuit properties, can give rise to a non-smooth pattern of persistent activity along the cortical hierarchy through a phenomenon akin to bifurcations described by the theory of nonlinear dynamical systems (Mejias and Wang, 2021; Wang, 2020). Such a sudden transition was observed in a monkey experiment where elevated persistent activity associated with working memory was absent in the middle temporal area (MT) but significantly presented one synapse away in the nearby medial superior temporal area (MST) (Men-doza-Halliday et al., 2014). Simultaneous recording from many parcellated areas using new tools, such as Neuropixels (Jun et al., 2017), from behaving animals could systematically test our model prediction in future experiments. This increasing gradient of dopamine receptors along the cortical hierarchy is a major anatomical basis by which dopamine can modulate higher cognitive processing.
An inverted U relationship between dopamine and distributed working memory activity
Previous experimental and modeling studies have shown an inverted U relationship between D1 receptor stimulation and persistent activity in the prefrontal cortex in monkeys performing working memory tasks (Brunel and Wang, 2001; Vijayraghavan et al., 2007; Wang et al., 2019). Dopamine activity in the VTA is relatively low during the delay period but still has an inverted U shape relationship with short-term memory performance in the rat (Choi et al., 2020). In our model, this may be interpreted as the VTA continuing to provide low-level dopamine to the cortex to maintain cortical dopamine levels within the appropriate bounds for distributed persistent activity. We found dense D1 and D2 receptor labeling in the striatum. However, we focused our working memory modeling on the cortex and VTA. Notably, optogenetic manipulation of substantia nigra pars compacta dopamine neurons (which principally target the striatum) does not have specific short-term memory effects (Choi et al., 2020). This suggests that cortical rather than striatal dopamine release is likely to be more important to short-term memory. By constructing a novel large-scale model based on the D1 receptor map and tract-tracing data, we found that the inverted U relationship between D1 receptor stimulation and persistent activity held across the frontal and parietal cortex during working memory. The working memory activity pattern was strikingly similar to that seen experimentally, according to a meta-analysis of 90 electrophysiology studies of delay period activity in the monkey cortex (Leavitt et al., 2017). Analyzing the model showed that the pattern of inter-areal connections was the strongest determinant of the pattern of working memory activity.
Noudoost and Moore (2011) found that injecting a D1 antagonist into FEF led to an increase in firing rates in V4. Similarly, in our model, when cortical dopamine levels are close to the optimal range for working memory (i.e., the peak of the inverted U), then reducing D1 receptor stimulation via an antagonist would lead to an increase in V4 activity during the second peak of the response to visual stimulation (Figure S3). However, our model focused on distributed working memory in a large-scale cortical system and was not built to uncover mechanisms of attention or decision-making. Recent electrophysiology and modeling studies of non-human primate attention have suggested that the dominant net effect of attention on neural activity in the sensory cortex is inhibition (Huang et al., 2019; Yoo et al., 2021). This may be consistent with subtle enhancement of firing for neurons whose receptive field is in the focus of attention, combined with greater inhibition of neurons with nearby receptive fields. We showed that somatosensory and visuospatial working memory tasks engage largely overlapping higher cortical areas during the delay period. It is likely that, at a neural level, these networks may overlap only partially. To simulate these mixed inhibitory and excitatory effects of attention and identify the degree to which different types of working memory engage the same neurons, future models will require more neural populations per area, perhaps with structured connectivity, such as a ring (Ardid et al., 2007). Local circuit modeling has shown previously that a circuit designed for working memory is suitable for decision-making (Wang 2002). Our model may also be suitable for considering decision processes distributed across cortical areas.
Prefrontal and parietal contributions to distributed working memory
It is increasingly feasible to uncover the circuit mechanisms underlying distributed cognitive functions because of advances in recording technology (Jun et al., 2017) and large-scale cortical models (Cabral et al., 2011; Chaudhuri et al., 2015; Honey et al., 2007; Joglekar et al., 2018; Mejias et al., 2016; Mejias and Wang, 2021; Schmidt et al., 2018; Shine et al., 2018). Most previous large-scale cortical models have focused on replicating resting-state functional connectivity (Cabral et al., 2011; Chaudhuri et al., 2015; Honey et al., 2007) or propagation of neural activity along the hierarchy (Chaudhuri et al., 2015; Joglekar et al., 2018; Schmidt et al., 2018), with the notable exception of one recent model that simulated distributed working memory in a network of 30 cortical areas (Mejias and Wang, 2021). Compared with previous efforts, our model additionally includes (1) a D1 receptor gradient; (2) multiple inhibitory cell types and distinct pyramidal cell compartments; (3) at least 33% more cortical areas connected via quantitative graded and directed connectivity data, and, for some figures, (4) short-term synaptic plasticity; and (5) a VTA module with reinforcement learning mechanisms. The large-scale nature of the model enabled us to investigate the contributions of different brain regions to distributed working memory activity.
Some experimental studies have aimed to dissociate the contribution of the prefrontal and parietal cortex to working memory via temporary inactivations. For example, Chafee and Goldman-Rakic (2000) examined the effects of reversibly cooling the prefrontal or parietal cortex on activity in the other area and behavior during a visuospatial working memory task without a distractor. Cooling affected the FEF (area 8) and nearby prefrontal cortex, including the principal sulcus (areas 46 and 9). Cooling of the parietal cortex included LIP as well as parts of areas DP (dorsal prelate gyrus), 7A, and 5. Cooling the parietal cortex led to a substantial reduction in prefrontal firing rates with only a minor effect on performance. Cooling the prefrontal cortex led to a substantial reduction in parietal firing rates and a large increase in behavioral errors (Chafee and Goldman-Rakic 2000). This is consistent with our simulation results showing that prefrontal and parietal inactivation can have a robust effect on reducing mnemonic delay activity but that prefrontal inactivation has much larger effects on performance (Figures 3E and 3F). Suzuki and Gottlieb (2013) inactivated areas LIP and dorsolateral prefrontal cortex (dlPFC) using the GABA-A receptor agonist muscimol and assessed performance on a similar visuospatial working memory task with and without distractor stimuli. In these experiments, neither LIP nor dlPFC inactivation caused errors in trials without distractors (Suzuki and Gottlieb, 2013). However, inactivation of dlPFC, but not LIP, led to a dramatic increase in errors on trials with distractors (Suzuki and Gottlieb, 2013). This is consistent with our simulation results showing that precise lesions to dlPFC affect behavior on challenging working memory trials with distractor stimuli, but larger lesions are required to disrupt performance in simple working memory trials without distractors, and lesions to LIP have only subtle effects on performance. This agrees with recent models of distributed working memory that suggest that the prefrontal cortex may have a particularly important role in maintaining distributed persistent activity (Mejias and Wang, 2021; Murray et al., 2017). The effects of lesions on model performance are consistent with recent reports showing that there is a distinction between areas that are active during normal behavior and those that are essential for a computation (Pinto et al., 2019; Zatka- Haas et al., 2021) and that cortical lesions have greater effects on performance in more challenging tasks (Pinto et al., 2019).
Lesions to areas with a high D1 receptor density disrupt working memory
Working memory activity was most disrupted by lesions to areas with a high D1 receptor density, a prediction that can be tested experimentally. Humans with traumatic brain injury often have working memory deficits (Dunning et al., 2016). Pharmacological treatment of these deficits, including with dopaminergic drugs, has had mixed success (Froudist-Walsh et al., 2017b). Our model simulations suggest that D1 agonists or antagonists could be effective at restoring normal working memory functioning following lesions to particular cortical areas, but the correct treatment may depend on the baseline cortical dopamine levels of the individual. Dopaminergic drugs have also been suggested as treatments for individuals with schizophrenia with working memory deficits (Yang and Chen 2005). In individuals with schizophrenia, PV and SST gene expression is reduced across multiple areas of the cortical working memory network (Tsubo- moto et al., 2019). Disruption of these inhibitory neurons is likely to contribute to working memory deficits. Future adaptations of our model could allow simulation of working memory deficits and motivate potential treatments for individuals based on their particular anatomy, gene expression, and patterns of cortical dopamine release or receptor density (Abi-Dargham et al., 2002; Slifstein et al., 2015).
A dopamine switch between the activity-silent state and persistent activity
For very low or high levels of D1 receptor stimulation, it was possible to maintain stimulus information in the absence of persistent activity via synaptic mechanisms. This pattern of successful memory recall without frontoparietal delay period activity is reminiscent of a passive short-term memory trace thought to rely on ‘‘activity-silent’’ synaptic mechanisms (Rose et al., 2016; Trbutschek et al., 2017; Wolff et al., 2017) that could occur without ignition of the frontoparietal cortex (Trbutschek et al., 2017, 2019). Previous models with short-term synaptic plasticity have focused on local activity in the prefrontal cortex (Mongillo et al., 2008) and, thus, implicitly imply that the initial stimulus must significantly engage prefrontal neural activity and store the memory trace via short-term plasticity in local pre- frontal connections. However, some stimuli may be remembered without a strong initial prefrontal response. We found that short-term synaptic plasticity in inter-areal connections from sensory to frontoparietal areas was most important for maintaining the silent memory trace. In particular, this is a potential mechanism for activity-silent short-term memory in the absence of a strong initial prefrontal response to the stimulus. It has been proposed that nonspecific excitatory or inhibitory currents could account for switches between active and silent states (Barbosa et al., 2020). Our model suggests that dopamine could, in fact, account for the switch from the silent to the active state. Indeed, because of the inverted U relationship between dopamine and persistent firing, a dopamine response to the reward at the end of a trial could also terminate the persistent activity. Another recent proposal suggests that activity-silent short-term memory could be under-taken via hippocampal-prefrontal episodic memory mechanisms, perhaps in combination with short-term synaptic changes in the cortex (Beukers et al., 2021). Future studies should aim to disentangle the contributions of rapid synaptic changes within the prefrontal cortex (Mongillo et al., 2008), at inter-areal connections from sensory areas (this paper), or in the hippocampus (Beukers et al., 2021) to activity-silent short-term memory in the primate. We found that, in the activity-silent state, the most recently encoded stimulus was always encoded most strongly, even when it was a distractor. This may reflect the involuntary encoding of irrelevant stimuli in a short-term synaptic memory trace (Barbosa et al., 2021, 2020). This prediction should hold as the number of distractors increases. The activity-silent system may still be able to recall earlier stimuli for a limited time when another input biases the network toward the activity pattern used during encoding of the earlier stimulus to trigger pattern completion and recall of the memory (Manohar et al., 2019) or through active forgetting of the distracting stimuli (Wolff et al., 2021). Alternatively, multiple competing memories may be represented in neural activity (Barbosa et al., 2021; Panichello and Buschman, 2021), which would rely on an unspecified selection mechanism and may occur in parallel with short-term synaptic changes. In our model, stimuli stored in persistent activity (and thus dependent on mid-level dopamine release) were more robust against distraction, consistent with drug studies in humans (Fallon et al., 2017a, 2017b). Thus, dopamine release may engage distributed persistent activity to protect memories of important stimuli from distraction.
Dopamine increases distractor resistance by shifting the subcellular target of inhibition
The resilience of the active working memory state in the model depended on CB/SST cells blocking distracting inputs from sensory areas to the dendrites of pyramidal cells in the frontal and parietal cortex. Previous modeling work on local cortical circuits has suggested that greater dendritic and less somatic inhibition could increase distractor resistance (Wang et al., 2004a) and that selective disinhibition of the dendrite (through CR/VIP cells) could allow specific information to be passed through the network (Yang et al., 2016). In our large-scale model, CR/VIP cells selectively disinhibited the dendrites of target-selective cells, allowing target-related activity to flow through the cortical network. D1 receptors in the monkey cortex are more strongly expressed on CB/SST neurons than other interneuron types (Mueller et al., 2020). In agreement with these anatomical findings, the application of dopamine to a frontal cortex slice increases inhibition to the dendrites and decreases inhibition to the somata of pyramidal cells (Gao et al., 2003). We found that, as long as local cortical areas (or potentially cortico-subcortical loops) are capable of maintaining persistent activity, then shifting the balance of inhibition from the soma to the dendrite can allow maintenance of an active representation of a stimulus in persistent activity while shielding it from distracting input from sensory areas. The ability of cortical areas to maintain persistent activity itself depends on dopaminergic enhancement of NMDA-dependent excitation. In mice, inhibition of SST neurons in the medial prefrontal cortex during the sample period of a spatial working memory task impairs performance and increases representation of irrelevant information in prefrontal activity (Abbas et al., 2018). Consistent with our model, this suggests that SST neurons gate entry of information into working memory and that inhibition of SST neurons in frontoparietal areas allow distracting information to enter.Learning to engage distributed persistent activity through reinforcement
Distractor resistance in response to all stimuli could render the working memory system inflexible and unresponsive to new, potentially important inputs. Previous studies have shown that lesioning the prefrontal cortex impairs the ability to switch atten- tion between stimuli across trials (Rossi et al., 2007). Our model predicts that the prefrontal cortex is more crucial for persistent activity than activity-silent short-term memory, which can rely on short-term synaptic changes outside of the prefrontal cortex. We show that by using a simple reward-based learning mechanism, a cortical VTA model (cf. Braver and Cohen, 2000; Frank 2005) can successfully perform a task with reversals between the memory cue and distractor stimuli across trials. In our model, the timing of dopamine release in the cortex can be learned to engage distributed persistent activity throughout the frontoparietal network only in response to reward-predicting cues. Dopa- mine neurons burst about 130–150 ms after reward-predicting stimuli, coinciding with a rise in activity in frontal cortical neurons (de Lafuente and Romo, 2012). Because of the slow dynamics of cortical dopamine (Muller et al., 2014), we suggest that a transient increase in dopamine release in response to the target stimulus (Choi et al., 2020; Schultz et al., 1993) may be sufficient to maintain distributed persistent activity for several seconds. This mechanism may thus be reserved for behaviorally important stimuli that must be protected from distraction even when the behaviorally relevant stimuli change from trial to trial. In contrast, irrelevant or less salient stimuli result in lower dopamine release and may be remembered via silent mechanisms or forgotten. We investigated model performance on a reversal-learning task with identical repeated trials within a block. In natural life, no two situations are exactly the same. It is likely that the brain generalizes across similar situations to enable reinforcement learning to be used in practice. This ability to generalize may arise from dopa- mine-dependent plasticity in the prefrontal cortex (Wang et al., 2018). The classic reward-prediction-error hypothesis treats dopamine as a global scalar reward prediction error signal that is spatiotemporally uniform (Schultz 1998). Here we aim to high- light one form of spatial heterogeneity and suggest that broad dopamine release will affect each cortical area according to the D1 receptor density per neuron. Recent work suggests that there is temporal heterogeneity in dopamine release, which is released in waves in the mouse striatum (Hamid et al., 2021). Whether such dopamine waves also occur in the cortex or in pri- mates remain to be seen. Even if dopamine is released in waves across the cortex, its effect on cortical areas will be dependent on the D1 receptor gradient presented here.
Roles of other neuromodulatory and subcortical systems
In addition to dopamine, other neuromodulators, such as acetyl- choline (Croxson et al., 2011; Sun et al., 2017; Yang et al., 2013) and noradrenaline (Arnsten et al., 2012), affect prefrontal delay period firing and performance on visuospatial working memory tasks. Cholinergic mechanisms may complement dopaminergic mechanisms.For example, nicotinic alpha-7receptorsdepolarize pyramidal cells enable NMDA receptors to be engaged via the removal of the magnesium block (Yang et al., 2013). This may compensate for the reduction in presynaptic glutamate release in response to D1 stimulation and enable dopamine’s permissive effects on NMDA transmission (Seamanset al.,2001). Muscarinic M1 receptor activation closes KCNQ channels, which contribute to the hyperpolarizing effect of high levels of D1 stimulation (Arns- ten et al., 2012; Galvin et al., 2020). Thus M1 stimulation may enable persistent activity over a larger range of dopamine release. The effects of noradrenaline on working memory circuits depend on the targeted adrenergic receptors. Moderate release of noradrenaline engages adrenergic a2A receptors, which may counteract the hyperpolarizing effects of hyperpolarization-activated cyclic nucleotide-gated (HCN) channels (Arnsten, 2000; Arnsten et al., 2012; Li and Mei, 1994; Robbins and Arnsten, 2009)andkeeptheD1effectsincheckbydecreasingcalcium-cyclic AMP (cAMP) signaling. Greater noradrenergic levels engage a1 and b1 receptors, which promote calcium-cAMP signaling and, at high levels, provide negative feedback via KCNQ and HCN channels (Arnsten et al., 2020). Studies linking neuromodulators to working memory have focused on the dorsolateral prefrontal cortex. Much less is known about the influence of these and other neuromodulators on the distributed network activity that underlies working memory outside of the prefrontal cortex. Future work should focus on the interaction of distinct neuromodulators and how the release of different combinations of neuromodulators may affect distributed activity patterns and behavior, taking into account the different distributions of these receptors across the cortex (Froudist-Walsh et al., 2021). Subcortical struc- tures, such as the thalamus, may play a significant role in working memory (Fuster and Alexander, 1971; Guo et al., 2017; Jaramillo, et al., 2019; Watanabe and Funahashi, 2012). Future experiments and computational modeling studies should aim to disentangle the contribution of the thalamus to sensory working memory and motor preparation (Guo et al., 2017; Watanabe and
Funahashi, 2012) and clarify the degree to which such mechanisms are shared across species. When appropriate weighted and directed connectivity data become available, future large-scale cortical models should also integrate further structures, such as the thalamus (Jaramillo et al., 2019), basal ganglia (Wei and Wang, 2016), the claustrum, and the cerebellum to identify their contributions to working memory.
Conclusion
We experimentally found a macroscopic gradient of dopamine D1 receptor density along the cortical hierarchy. By building a novel connectome-based biophysical model of the monkey cortex, endowed with multiple types of inhibitory cells, we show how dopamine can engage robustly distributed persistent activity mechanisms across connected higher cortical areas and protect memories of salient stimuli from distraction. Because distributed persistent activity is necessary for internal manipulation of information in working memory (Masse et al., 2019; Takeda and Funahashi, 2004; Trbutschek et al., 2019), dopamine release in the cortex may be a key step toward higher cognition and thought.
