Part 1:Identification Of Dopaminergic Neurons That Can Both Establish Associative Memory And Acutely Terminate Its Behavioral Expression
Mar 19, 2022
Contact: joanna.jia@wecistanche.com / WhatsApp: 008618081934791
Michael Schleyer,1* Alice Weiglein,1 Juliane Thoener,1 Martin Strauch,2 Volker Hartenstein,3
Melisa Kantar Weigelt,1 Sarah Schuller,1 Timo Saumweber,1 Katharina Eichler,4,5,6 Astrid Rohwedder,4,7 Dorit Merhof,2 Marta Zlatic,5,7 Andreas S. Thum,4,8 and Bertram Gerber1,9,10*
1Leibniz Institute for Neurobiology, Department Genetics of Learning and Memory, 39118 Magdeburg, Germany, 2Institute of Imaging & Computer Vision, RWTH Aachen University, 52056 Aachen, Germany, 3Department of Molecular, Cell and Developmental Biology, University of California, Los Angeles, California 90095-1606, 4University of Konstanz, Institute for Biology, 78464 Konstanz, Germany, 5HHMI Janelia Research Campus, Ashburn, Virginia 20147, 6Institute of Neurobiology, University of Puerto Rico Medical Science Campus, Old San Juan, Puerto Rico 00901, 7Department of Zoology, University of Cambridge, Cambridge, CB2 3EJ, United Kingdom, 8University Leipzig, Institute for Biology, 04103 Leipzig, Germany, 9Centre for Behavioural Brain Sciences, 39108 Magdeburg, Germany, and 10Institute for Biology, Otto von Guericke University Magdeburg, 39106 Magdeburg, Germany
An adaptive transition from exploring the environment in search of vital resources to exploiting these resources once the search was successful is important to all animals. Here we study the neuronal circuitry that allows larval Drosophila melanogaster of either sex to negotiate this exploration-exploitation transition. We do so by combining Pavlovian conditioning with high-resolution behavioral tracking, optogenetic manipulation of individually identified neurons, and EM data-based analyses of synaptic organization. We find that optogenetic activation of the dopaminergic neuron DAN-i1 can both establish memory during training and acutely terminate learned search behavior in a subsequent recall test. Its activation leaves innate behavior unaffected, however. Specifically, DAN-i1 activation can establish associative memories of opposite valence after paired and unpaired training with odor, and its activation during the recall test can terminate the search behavior resulting from either of these memories. Our results further suggest that in its behavioral significance DAN-i1 activation resembles, but does not equal sugar reward. Dendrogram analyses of all the synaptic connections between DAN-i1 and its two main targets, the Kenyon cells, and the mushroom body output neuron MBON-i1, further suggest that the DAN-i1 signals during training and during the recall test could be delivered to the Kenyon cells and to MBON-i1, respectively, within previously unrecognized, locally confined branching structures. This would provide an elegant circuit motif to terminate the search on its successful completion.
Key words: dopamine; exploration-exploitation; mushroom body; optogenetics; reinforcement; search
Significance Statement
In the struggle for survival, animals have to explore their environment in search of food. Once the food is found, however, it is adaptive to prioritize exploiting it over continuing a search that would now be as pointless as searching for the glasses you are wearing. This exploration-exploitation trade-off is important for animals and humans, as well as for technical search devices. We investigate which of the only 10,000 neurons of a fruit fly larva can tip the balance in this trade-off, and identify a single dopamine neuron called DAN-i1 that can do so. Given the similarities in dopamine neuron function across the animal king-
dom, this may reflect a general principle of how search is terminated once it is successful.
Introduction
An animal exploring its environment may use past experience to guide its search for food. When food is found, however, it is adaptive to terminate the search and instead exploit the food source. Here we study the circuits that allow larval Drosophila melanogaster to negotiate this exploration-exploitation transition. The numerical simplicity of the larval brain, combined with cell-specific transgene expression and knowledge of its synaptic connectivity (Pfeiffer et al., 2010; Li et al., 2014; Eichler et al., 2017; Saumweber et al., 2018; Eschbach et al., 2020), allows this to be investigated at the level of individually identified neurons.
D. melanogaster larvae readily associate odors with a food reward (Scherer et al., 2003; Gerber and Hendel, 2006; Schipanski et al., 2008; Rohwedder et al., 2012; Apostolopoulou et al., 2014; Kudow et al., 2017, 2019). Critically, learned behavior based on these odor-food associations ceases if during a recall test the reward is present (Fig. 1A). Learned behavior can therefore be viewed as a search, which is adaptively terminated by the sought-for outcome (i.e., by the reward) (Schleyer et al., 2011, 2013, 2015a). Innate olfactory behavior is not likewise affected (Fig. 1B), suggesting that such innate behavior is organized in a reflex-like manner (Schleyer et al., 2011, 2013, 2015a,b). Thus, natural rewards can induce two effects: during the training, they provide a reward signal that can be associated with cues that subsequently guide reward search; and during the recall test, they can acutely terminate this learned search, preventing the animals from drifting away from a resource once it is found. This transition thus reflects a switch in the strategy for obtaining a reward, not a change in motivation (i.e., not a loss of interest in the reward).
Invertebrates and invertebrates alike, dopaminergic neurons (DANs) provide reinforcing signals for associative learning (Waddell, 2013; Schultz, 2015; Kaun and Rothenfluh, 2017). In D. melanogaster, distinct sets of DANs convey appetitive and aversive reinforcement, respectively (larvae: Schroll et al., 2006; Rohwedder et al., 2016; Saumweber et al., 2018; Eschbach et al., 2020; adult flies: Schwaerzel et al., 2003; Claridge-Chang et al., 2009; Liu et al., 2012) (a similar scenario may be emerging invertebrates, too: Lammel et al., 2012; Groessl et al., 2018; Menegas et al., 2018). Establishing a compartmental structure, these DANs intersect the parallel fibers of Kenyon cells (KCs), the intrinsic neurons of the highest-order olfactory center of the insects, called the mushroom body. Respecting this same compartmental structure, mushroom body output neurons (MBONs) collect in-formation across the KCs and send it toward efferent circuitry (see Fig. 2A–D) (larvae: Selcho et al., 2009; Pauls et al., 2010; Eichler et al., 2017; Saumweber et al., 2018; Eschbach et al., 2020; adults: Sejourne et al., 2011; Places et al., 2013; Aso et al., 2014a,b; Oswald and Waddell, 2015; Takemura et al., 2017). Notably, a recent electron microscope reconstruction revealed that DANs have two main targets: the KCs and the MBONs (see Fig. 2B)(larvae: Eichler et al., 2017; adults: Takemura et al., 2017).
In larval D. melanogaster, activation of DANs from the pPAM cluster can exert a rewarding effect during training (Rohwedder et al., 2016). From the four days of this cluster, two can individually confer such a rewarding effect (DAN-h1 and DAN-i1) (Saumweber et al., 2018). Here, we chose to focus
Figure 1. Sugar can confer reward and search termination signals. A Larvae were trained such that an odor was presented either paired or unpaired with sugar. In a subsequent recall test, they were assayed for their odor preference either in the absence or in the presence of the sugar (sketch toward the top). Testing in the absence of sugar revealed associative olfactory memory, quantified as a positive PI. The behavioral expression of this memory was terminated in the presence of sugar. Thus, sugar can confer two kinds of signal: during the training, it provides a reward signal that can be associated with the odor to guide the animals later on in their reward search; and during the recall test, the sugar as the sought-for- reward provides a signal to terminate this learned search. Sample sizes: N=29, N =28. B, Larvae were tested for innate odor preference, either in the absence or in the presence of sugar (sketch toward the top). Sugar did not affect innate olfactory behavior, suggesting its reflex-like organization. Sample sizes: N=20 each. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p, 0.05, corrected according to Bonferroni-Holm). *Significance from zero (one-sample sign test, p, 0.05, corrected according to Bonferroni-Holm). ns, not significant. All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1. For the odor preference values underlying the PI scores of this experiment, see Figure 9A.
Figure 2. The circuitry of the DAN-KC-MBON matrix. The medial lobe chiasm. Odor information is processed to induce an innate approach, as well as toward the calyx region of the mushroom body KCs (purple; displayed fully is one example KC on the right side). Taste reward information is likewise processed to guide innate behavior, as well as via mostly aminergic modulatory neurons (displayed is DAN-i1, turquoise/black) toward individual compartments of the mushroom body lobes (on the left side, compartments are indicated). Each of the four compartments of the medial lobe (h-k) receives input from a single DAN that also innervates the same compartment in the contralateral hemisphere. The MBONs of the medial lobe draw from single compartments, typically of both hemispheres (displayed is MBON-i1, magenta/yellow), and deliver their output, in configurations that differ between compartments, across the hemispheric divide. Collectively, the MBONs are responsible for memory-based search behavior. Data are based on Eichler et al. (2017) and Saumweber et al. (2018). B, The DAN-KC-MBON matrix in the i-compartment. DAN-i1 and most KCs establish mutual chemical synapses, and both DAN-i1 and KCs provide output to MBON-i1. For simplicity, only synapses with the left DAN-i1 and MBON-i1 neurons are shown in detail. The same connectivity is found in most,if not all, other compartments (Eichler et al., 2017; Saumweber et al., 2018). C, DAN-i1 (turquoise) receives input from outside the mushroom body (dark gray) ipsilateral to its cell body (region labeled 1) and then innervates the i-compartment in both hemispheres (labeled 2 and 3). MBON-i1 (magenta) crosses the midline twice; it receives input in the i-compartment of both hemispheres and provides output in regions adjacent to the mushroom body of both hemispheres. D,Same as in C, but in a slightly tilted view, showing skeleton reconstructions of the left DAN-i1 and MBON-i1 neurons and one example KC (black). Data are based on Eichler et al. (2017). E, Dendrogram of the left DAN-i1 neuron. Colored dots represent output synapses from DAN-i1. Triangles represent input synapses to DAN-i1. Colors represent the partner neuron of the respective synapse. All other synapses within the mushroom body are labeled “Other.” Not displayed are synapses that DAN-i1 forms with neurons outside the mushroom body, in particular in region 1 (Eschbach et al., 2020). Stippled lines indicate regions where the neuron crosses the midline of the brain. F, Same as in E, but for the left MBON-i1 neuron. Not displayed are synapses that MBON-i1 forms with neurons outside the mushroom body. For high-resolution versions of the DAN-i1 and MBON-i1 neurons of both hemispheres, as well as all mature KCs, see Extended Data Figures 2-1, 2-2, and 2-3, respectively.
on DAN-i1 because it has previously been analyzed in more detail (Saumweber et al., 2018) and because it yields a more robust rewarding effect than DAN-h1 (Saumweber et al., 2018; unpublished data). We first provide a detailed account of the spatial arrangement of all synapses between DAN-i1 and both its major output partners, MBON-i1 and the KCs. Then, we ask whether optogenetic activation of DAN-i1 can mediate not only a reward signal during training but also a signal to acutely terminate learned to search for the reward during the recall test.
cistanche bodybuilding
Materials and Methods
Experimental model and subject details. Third-instar feeding-stage larvae of both sexes (D. melanogaster), aged 5d after egg-laying, were used throughout. Flies were maintained on standard medium, in the mass culture at 25°C, 60%-70% relative humidity, and a 12/12 h light/dark cycle. We took a spoonful of food medium from a food vial, randomly selected the desired number of larvae, briefly rinsed them in tap water, and started the experiment.
We used transgenic larvae to express the ChR2-XXL light-gated ion channel in DANs. To this end, the effector strain UAS-ChR2-XXL (Dawydow et al., 2014) (kindly provided by R. Kittel, University Leipzig), was crossed to one of two driver strains: either 58E02-Gal4 (Pfeiffer et al., 2008; Liu et al., 2012; Rohwedder et al., 2016) (Bloomington Stock Center no. 41347) or the split-Gal4 driver strain SS00864 (Eichler et al., 2017; Saumweber et al., 2018) to obtain double- heterozygous offspring. A driver controls the driver strains were crossed to a local copy of w1118 (Bloomington Stock Center #3605, #5905, #6326). As effector controls a strain carrying the landing sites used for the Gal4 (attP2) or the split-Gal4 (attP40/attP2), yet without a Gal4 domain inserted (“empty”) (Pfeiffer et al., 2010), was crossed to UAS- ChR2-XXL. We confirmed the expression pattern of58E02 and SS00864 by crossing them to pJFRC-10xUAS-IVS-mCD8::GFP (Pfeiffer et al., 2010) (Bloomington Stock Center #32185). Because ChR2-XXL is sensitive enough to be activated by daylight (not shown), flies were raised in vials constantly darkened by black cardboard wrapping. For the experiment presented in Figure 8E, UAS-ChR2 (Schroll et al., 2006) was crossed to 58E02-Gal4.
Experimental setup. For behavioral experiments, larvae were trained in Petri dishes of 9 cm inner diameter and tested in Petri dishes of either 9 or 15cm inner diameter (both from Sarstedt) as mentioned in the figure legends, in all cases filled with 1% agarose (electrophoresis grade; Roth). Like the odor, we used n-amyl acetate diluted 1:20 in paraffin oil (AM; CAS: 628-63-7; Merck), and in some cases additionally undiluted 1-octanol (OCT; CAS: 111-87-5; Merck).
Experiments were performed inside a 43 43 73cm surrounding box equipped with a custom-made light table featuring a 24 12 LED array (470nm; Solarox) and a 6-mm-thick diffusion plate of frosted Plexiglas on top to ensure uniform blue light for ChR2-XXL activation (120mW/cm2). Petri dishes were placed directly on top of the diffusion plate. The Petri dishes were surrounded by a polyethylene diffusion ring; behind the diffusion ring, 30 infrared LEDs (850nm; Solarox) were mounted to provide illumination that was invisible to the larvae yet allowed the recording and tracking of their behavior for offline analysis. To this end, a camera (Basler acA204090umNIR; Basler) equipped with an infrared-pass filter was placed above the Petri dish. For additional details regarding the experimental setup, see Saumweber et al. (2018).
cistanche bodybuilding
Associative odor-sugar learning. Learning experiments followed established protocols (Michels et al., 2017). Odor containers were pre-pared by adding 10ml of odor substance to custom-made Teflon containers (5 mm inner diameter with a lid perforated with 7 holes of 0.5 mm diameter each). Petri dishes were covered with modified lids perforated in the center by 15 holes of 1 mm diameter each to improve aeration.
For odor-sugar training, 20 larvae were placed in the middle of a Petri dish filled with agarose that contained 2mol/L D-fructose (CAS: 57-48-7; Roth) as a reward (1) and equipped with two odor containers on opposite sides, both filled with AM (AM1). After 2.5min, the larvae were displaced onto a fresh Petri dish with plain, tasteless agarose, equipped with two empty containers (EM), where they also spent 2.5min. Three such “paired” training cycles were performed, in each case using fresh Petri dishes. In half of the cases, training started with reward-containing Petri dishes as indicated (AM1/EM), whereas in the other half of the cases the sequence was reversed (EM/AM1). For each cohort of larvae trained AM1/EM (or EM/AM1), a second cohort was trained reciprocally, that is, by “unpaired” presentations of odor and reward (AM/EM1, or EM1/AM, respectively).
Following training, the larvae were transferred to the middle of a test Petri dish and tested for their odor preference. Importantly, the recall test was conducted either on a plain Petri dish to assess memory-based search behavior or on a reward-containing Petri dish to determine whether the presence of reward terminated the memory-based search. In either case, one side of the test Petri dish was equipped with an AM container and the other side with an EM container. The number of larvae on either side was determined from the camera image at 3min after the test had started. From these numbers, an odor preference was calculated as follows:
Thus, PI values can range from 1 to 1, with positive values indicating that the larvae preferred the odor more after paired than unpaired training. Positive PI values accordingly indicate an appetitive associative memory that is best interpreted as memory-based search behavior. Negative PI values, by contrast, indicate aversive associative memory.
Associative learning by optogenetic DAN activation. For optogenetic experiments, training and test were performed analogously. However, all Petri dishes were filled with plain agarose (i.e., no rewarding tastants were used). Instead, for paired training, AM was paired with continuous blue light stimulation to activate either 58E02-DAN or 864- DAN for 2.5min, followed by 2.5min of darkness without an odor (AM1/EM). For unpaired training, odor presentation and light stimulation occurred separately (AM/EM1). For the experiment presented in Figure 8E, odor and light were presented for 5min each; that experiment otherwise followed the procedures described by Rohwedder et al. (2016).
After three such training cycles, the recall test was conducted, and odor preference and PI were calculated as described. Critically, the recall test was conducted either without blue light activation to determine whether the DANs mediate a reward signal during training or with blue light activation to determine whether, in addition, the DANs can mediate a signal to terminate the search.
Variations of this protocol are indicated in the figure legends.
Innate olfactory behavior. Odor containers and Petri dishes were prepared as described above. Cohorts of;20 larvae were collected from the vial, briefly washed in tap water, and placed onto a Petri dish with an AM container on one side and an empty container on the other side. After 3min, the innate odor preference was determined according to Equation 1. This preference test was conducted either in the absence or in the presence of sugar, or in the absence or in the presence of blue light activation, to determine whether these test conditions alter innate olfactory behavior.
Analysis of behavioral modulations after training with 864-DAN activation. Larval behavior was video-tracked and analyzed as described in detail by Paisios et al. (2017). In brief, two aspects of larval chemotaxis were analyzed. First, the modulation of head castrate (HC per second) (HC/s) was calculated as follows:
In this measure, the heading angle (HA) describes the orientation of the animal’s head relative to the odor, with absolute heading angles (abs (HA)) of 0° or 180°, for example, indicating that the odor is to the front or the rear of the larvae, respectively. This measure thus yields positive scores for the attraction (i.e., when the HC directs the larvae toward rather than away from the odor target), whereas it yields negative scores for aversion.
Immunohistochemistry. Larval brains were dissected in Ca21-free saline solution and fixed in Bouin’s solution diluted 1:2 with Ca21-free saline solution for 7min. Three washing steps (each 10min) in PBST (0.3% Triton-X 100 in PBS) were followed by incubation with the primary antibodies overnight at 4°C. After three washes (each 10min) in PBS, the tissue was incubated with the secondary antibodies for 1 h at room temperature. After three final washing steps in PBS (each 10min), samples were mounted in Vectashield (Vector Laboratories). Preparations were examined under a DM 6000 CS confocal microscope (Leica Microsystems).
cistanche bodybuilding
As antibodies, we either used monoclonal anti-FASII mouse antibody (DSHB, diluted 1:50 in blocking solution containing 4% normal goat serum) and anti-GFP rabbit antibody (A11122, Invitrogen, diluted 1:1000 in blocking solution containing 4% normal goat serum) as primary antibodies and Cy3 anti-mouse antibody (715-165-150, Jackson ImmunoResearch Laboratories) and Alexa-488 anti-rabbit antibody (A11034, Invitrogen), both diluted 1:200 in PBS, as secondary antibodies; or we used monoclonal anti-ChR2 mouse antibody (ProGen Biotechnik) diluted 1:100 in 0.3% PBST as primary antibody, Cy3 anti-mouse antibody (715-165-150, Jackson ImmunoResearch Laboratories), diluted 1:300 in 0.3% PBST, as the secondary antibody, and Alexa-488 anti- HRP (Jackson ImmunoResearch Laboratories), diluted 1:300 in 0.3% PBST, for counterstaining.
