for more information:Ali.ma@wecistanche.com

Pls click here to Part 2

Pls click here to Part 3

Deciphering Neural Codes of Memory during Sleep Zhe Chen1," and Matthew A. Wilson2,"

1Department of Psychiatry, Department of Neuroscience & Physiology, New York University School of Medicine, New York, NY 10016, USA

2Department of Brain and Cognitive Sciences, Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA

Abstract

Memories of experiences are stored in the cerebral cortex. Sleep is critical for consolidating hippocampal memory of wake experiences into the neocortex. Understanding representations of neural codes of hippocampal-neocortical networks during sleep would reveal important circuit mechanisms on memory consolidation, and provide novel insights into memory and dreams. Although sleep-associated ensemble spike activity has been investigated, identifying the content of memory in sleep remains challenging. Here, we revisit important experimental findings on sleep-associated memory(i.e., neural activity patterns in sleep that reflect memory processing) and review computational approaches for analyzing sleep-associated neural codes(SANC). We focus on two analysis paradigms for sleep-associated memory and propose a new unsupervised learning framework ("memory first, meaning later") for the unbiased assessment of SANC.

Keywords

sleep-associated memory; memory consolidation; memory replay; neural representation;population decoding; functional imaging

Click to Cistanhe stem and Cistanche products for  improving memory

Memory, Sleep, and Neural Codes

Memory is referred to the capacity of an organism to encode, store, retain and retrieve information. It can be viewed as a lasting trace of past experiences that influences current or future behavior. Memory uniquely defines a sense of self-identity and includes all information of 'who', 'what', 'when', and 'whereof our life experiences in the past and present, remote or recent. The period over which information in memory remains available varies from seconds (short-term memory)to years (long-term memory). Long-term memory is often divided into two types: explicit or declarative memory("knowing what)and implicit or procedural memory("knowing how"). Declarative memory also includes episodic memory (see Glossary), semantic memory (knowledge), and autobiographical memory.

Episodic memory stores details of specific events in space and time, each associated with unique multimodal, multi-dimensional information content. The hippocampus plays a pivotal role in spatial and episodic memory [1]. Sleep is important for learning and memory [2–6]. On average, human being spends about one-third of their life during sleep, whereas rodents sleep 12–14 hours a day. Memory consolidation occurs in sleep, during which a short-term memory can be transformed into long-term memory. Sleep deprivation deteriorates performance in memory tests and negatively affects attention, learning, and many other cognitive functions [6,7]. A fundamental task in the study of memory is to understand the representation of sleep-associated neural codes (SANC) that support memory processing. Simply put, how can be about memory during sleep? Since sleep-associated memory is influenced by WAKE experiences, how do we identify and interpret memory-related neural presentations during sleep in an unbiased way?

To address these questions, neuroscientists record neuronal ensemble activity from the hippocampus and neocortex in sleep sessions before and after a behavioral session. In animal studies, “neural codes” are acquired by implanting multielectrode arrays to record in vivo extracellular neuronal ensemble spike activity [8– 12]. In human studies, measurements of brain signals are acquired through non-invasive EEG or fMRI recordings [13– 16]. For this article, we will review important work in both research areas, with more focus on rodent studies.

At the neuronal ensemble level, the computational task of identifying memory-related neural representations of population codes (i.e., neural activity patterns that reflect memory processing) in sleep remains challenging for several important reasons: First, although local field potentials (LFPs) reveal important information of circuits at a macroscopic scale, they lack the cellular resolution to reveal sleep memory content. Second, sleep-associated ensemble spike activities are sparse (low occurrence) and fragmental in time. Third, the magnitude of neural population synchrony, measured as the spiking fraction of all recorded neurons during each network burst, follows a lognormal distribution: strongly synchronized events are interspersed irregularly among many medium and small-sized events [17]. Finally, the lack of ground truth makes the interpretation and assessment of memory-related neural representations difficult. In the past two decades, although several systematic studies have examined memory content in SLEEP compared to WAKE, many memory-related research questions remained elusive. In the next section, we review some experimental and computational strategies to answer these questions.

Hippocampal-Neocortical Circuits in Sleep

During sleep, the brain is switched into an “off-line” state that is distinct from wakefulness at both microscopic (spike timing) and macroscopic (e.g., neocortical EEG oscillations) levels. In different stages of sleep, such as slow-wave sleep (SWS) and rapid eye movement (REM) sleep, brain activity varies and the cerebral cortex exhibits a wide range of oscillatory activities (Box 1) [18]. During SWS, the neocortex is known to oscillate between UP and DOWN states [19]. During neocortical UP states, increased population synchrony of pyramidal cells in hippocampal-neocortical networks is accompanied by hippocampal sharp-wave ripples (SWRs, Box 1, Figure 1b) [20,21]. Most animal studies on memory and sleep use the rodent model. A widely adopted spatial memory paradigm is to let rodents freely forage in a closed environment. During active exploration, many hippocampal pyramidal neurons show localized spatial tuning or place receptive fields (RFs) [22]. Notably, many hippocampal pyramidal neurons are also responsible for non-spatial sequence coding [23,24], as well as conjunctive coding of both spatial and non-spatial memories [25]. During sleep, in the absence of external sensory input or cues, the hippocampal network is switched to a state that is mainly driven by internal computations.

In a seminal study, Pavlides and Winson [8] first reported that the activity of rat hippocampal place cells in the awake state influenced the firing characteristic (e.g., firing rate and burst rate) in subsequent sleep episodes. Wilson and McNaughton [9] extended the first-order to second-order statistical analysis and demonstrated that rat hippocampal place cells that were co-active during spatial navigation exhibited an increased tendency to fire together during subsequent sleep, whereas neurons that were active but had non-overlapping place RFs did not show such increase. This effect declined gradually during each post-RUN sleep session. Kudrimoti et al. [11] and Nádasdy et al. [12] further studied spike patterns involving multi- neuron patterns (e.g., triplet) during sleep. These studies revealed the temporal relationship between hippocampal replays and SWRs [12], as well as the memory trace decay time [11]. Additional studies also revealed that rodent hippocampal spatiotemporal patterns in SWS reflected the activation patterns or temporal order in which the neurons fired during spatial navigation [10, 12,26,27]. Specifically, subsets of hippocampal neurons fire in an orderly manner at a faster timescale within SWRs, with either the same or reverse order as in active navigation. In a linear track environment, such population burst events, depending on their contents, can be categorized as “forward” or “reverse” replay—referred to as reactivated hippocampal sequences of the run trajectory (Figure 1c). Such hippocampal replay events are prevalent in SWS [26], quiet wakefulness [28,29], and “local sleep” (also known as “microsleep” —a phenomenon that neurons go offline in one cortical area but not others in an awake yet sleep-like state) [30], although the functional roles in each of those states are most likely to be different. The engagement of the replay process, the frequency of activation, and the time during which replay occurs can affect subsequent performance on behavioral tasks or learned skills. In a series of studies [26,31,32], researchers have found that following RUN experiences, hippocampal place cells reactivated in a temporally precise order repeatedly in SWS and REM sleep. Unlike SWS, the firing-rate correlation in REM sleep was not related to the preceding familiar RUN experience (possibly due to the trace decay during the interleaving SWS) [11], and the memory replays occurred more frequently for remote yet repeated RUN experiences [31]. These findings suggest that reactivated hippocampal sequences in post-RUN sleep consolidate memory of experiences and that SWR-associated hippocampal activity may contribute to this process.

A central hypothesis of memory consolidation is that the hippocampus and neocortex interact with each other through the temporal coordination of neuronal activity in the form of slow oscillations, SWRs, and sleep spindles [33–39]. While memory reactivation during sleep has been mainly reported in rodents, including the rat primary visual cortex (V1) [36], the barrel cortex [40], the posterior parietal cortex [41], the medial prefrontal cortex (mPFC) [42,43], the primary motor cortex (M1) [44,45] and the medial entorhinal cortex (MEC) [46]; general phenomena of neocortical memory reactivation were also reported in the other species, such as in the songbird during sleep [47] and in the macaque monkey during rest [48]. The assumption of hippocampal-neocortical interactions during sleep would naturally suggest examining the interactions of simultaneously recorded hippocampal-neocortical ensembles [36,38,41,46]. Comparing the spatiotemporal neural patterns in each area during both WAKE and SLEEP would leverage our knowledge of hippocampal spatial coding and further our understanding of the role of hippocampal-neocortical memory processing during sleep. In one study of rodent hippocampal-visual circuits [36], researchers found that memory reactivation in the V1 was temporally coordinated with memory reactivation in the hippocampus during SWS (Figure 2a,b). In another study [37], researchers found that auditory cues associated with neural activity during learning enhanced replay of the same neural patterns if the same auditory cues were presented during sleep. Although the auditory stimuli did not affect the number of replay events, the replay content was biased by the respective sounds (Figure 2c), suggesting mechanisms of selective memory enhancement in sleep. In another recent report on a similar study [38], researchers simultaneously recorded ensemble spikes from the rat auditory cortex and hippocampus while presenting task-related sounds during sleep (Figure 2d), and found that the patterned activation in the auditory cortex preceded and predicted the subsequent content of hippocampal activity during SWRs (Figure 2e), while hippocampal patterns during SWRs also predicted subsequent auditory cortical activity. Consistently, delivering sounds during sleep biased the auditory cortical activity patterns, and sound-based auditory cortical patterns predicted subsequent hippocampal activity. Among many neocortical structures, the MEC is an important neocortical circuit that sends input to the hippocampus and plays an important role in spatial navigation and memory processing. Two recent rodent experimental findings have shown that there was coordinated replay between hippocampal (CA1) place cells and grid cells at deep MEC layers (L4/5) during rest [49]; however, the cell assemblies at superficial MEC layers replayed trajectories independently of the hippocampal reactivation rest or sleep, suggesting that the superficial MEC can trigger its replay events and initiate recall and consolidation processes independent of hippocampal SWRs, whereas deep MEC layers are directly influenced by hippocampal replay [46].

Overall, these findings suggest that the neocortex communicates with the hippocampus about “when” and “what” to reactivate memory during sleep, and the activation of specific cortical representations during sleep influences the consolidated memory contents. Nearly all reported findings are correlation-based observations. The first direct causal evidence of hippocampal-cortical coupling in memory consolidation during sleep was demonstrated physiologically and behaviorally in [39]. Importantly, it was found that reinforcing the endogenous coordination between hippocampal SWRs, cortical delta waves, and spindles by timed electrical stimulations resulted in a reorganization of the mPFC network, along with subsequently increased prefrontal task responsivity and high recall post-sleep performance [39].

In addition to considering the specific ensembles that participate in reactivated memory patterns, the temporal structure of memory patterns can also vary by brain state [25]. The reactivated patterns during SWRs closely resembled the compressed structure of encoded memory observed within individual cycles of the theta rhythm during awake behavior in the hippocampus [12,50]. During SWS, the hippocampal-neocortical memory reactivation occurred at a faster time scale, with reported time compression factors of 9–10 in the rodent hippocampus [26], and compression factor of 6–7 in the rodent mPFC [42], although there was also inconsistent report on no evidence of time compression or expansion in other rodent brain regions [40]. In REM sleep, the speed of hippocampal replay is close to or slightly faster than the actual run speed [31]. Notably, spatial memory was impaired by selective suppression or disruption of SWRs by electrical or optogenetic stimulations [51– 53], suggesting the causal role of SWRs for hippocampal replays during the off-line state.

In contrast to animal research (almost exclusively in rodents), human studies have provided more limited access to the content of sleep-associated memory at the neuronal ensemble level. Nevertheless, memory study of human subjects, such as H.M. [54], provides a unique and valuable perspective far beyond rodent studies. For healthy or diseased human subjects, semi-invasive ECoG recording or non-invasive EEG/MEG recordings and fMRI imaging have been widely used in sleep studies [13– 16]. However, none of them directly measure single neuronal activity, which therefore poses great challenges in studying sleep’s memory content. When single units are available, different cortical areas display distinct yet localized spatiotemporal spike and LFP patterns [55]. In a remarkable study, researchers used fMRI and machine learning tools to decode (or more precisely, “classify”) visual imagery of brain patterns in the visual cortex (V1, V2, and V3 areas) during REM sleep, as compared to spatiotemporal brain patterns of fMRI imaging during wakeful state [56]. This provided the first clue about the content of human dreams (Figure 3). In a sleep study on epilepsy patients, it was reported that single-unit spike activity in the MTL was modulated around REM onsets, which was similar in REM sleep, wakefulness, and controlled visual stimulations, suggesting that REM during sleep rearranged discrete epochs of visual-like processing as during awake vision [57].

Despite rapid progress in experimental investigations and growing knowledge of hippocampal-neocortical circuit mechanisms, answers to many research questions remain completely or partially unknown. Since most “content” questions are driven by statistical analyses of SANC, it is imperative to develop computational paradigms to investigate the representation of sleep-associated memory.

Computational and Statistical Methods: Strengths and Limitations

In WAKE, how do we interpret the representation (“meaning”) of neural codes? This is formally established by the neural encoding problem. Given the measured sensory input or motor behavior associated with neural responses, we can identify the meaning of neural spike patterns in a supervised manner. In SLEEP, the essential computational question is: what and how much information can be about from memory-related neural representations during sleep? Since the representation of an experience is sparse, the answer to this question is nontrivial. To date, several computational methods (Box 2) have been developed to analyze SANC derived from hippocampal-neocortical circuits. However, most of the methods cannot identify the “meaning” (content) of memory other than merely establishing significant “similarity” (by correlation or matching) of spike activities between WAKE and SLEEP. In other words, they can reveal the presence of memory replay, but not necessarily the content of replay. As a general principle of deciphering sleep-associated memory content, it is critical to developing statistical methods that allow studying memory without first having to establish how brain activity encodes behavioral variables such as spatial locations or movement kinematics. During sleep, the brain is normally disconnected from the external sensory world, although sensory stimulation may induce physiological changes in sleep-associated memory [37,38,70]. The content of sleep memory lacks behavioral readout; therefore it is preferred to use computational methods that do not require behavioral measurements apriori.

Here we would like to discuss two quantitative approaches for the analysis of SANC. In the first approach, the principal component analysis (PCA) method [43,58] (Box 2, Figure Ia) does not explicitly define the neuronal RF. Instead, it computes the correlation matrix of cell assemblies in a TEMPLATE epoch and then further compares it with another spatiotemporal population spike matrix from the MATCH epoch—moving the population spike vector in time would allow us to assess the time-varying reactivation strength. The basic statistical assumption is that the spatiotemporal patterns of a specific behavior can be well characterized by the correlation matrix of ensemble spiking. Conceptually, the choice of TEMPLATE and MATCH is arbitrary and this analysis can be applied to both directions (WAKE➔ASLEEP or SLEEP➔WAKE). However, the limitation of linear subspace methods, including both PCA and independent component analysis (ICA) [59,53], is that they assume a stationary correlation statistic during the complete TEMPLATE or MATCH period, which is untrue in the presence of distinct or complex behaviors that drive the state-dependent neuronal responses. Furthermore, the derived reactivation strength from these methods does not identify the “meaning” of memory; instead, it is positively correlated with the quadratic power of temporal firing rate in the neuronal ensemble.

The second approach is a population-decoding method. Unlike the traditional supervised or RF-based decoding methods [64,65], an unsupervised population-decoding method [66–69] has been developed for recovering hippocampal spatial memory with the assumption of place RFs (Box 2, Figure Ib). This is achieved by associating spatiotemporal spiking patterns with unique latent states without defining the meanings of those states apriori. Such an approach is conceptually appealing since it requires no assumption of explicit behavioral measures. In the case of rodent navigation example, the latent states may represent an animal’s spatial locations. Statistically, the latent states are assumed to follow a Markovian or semi- Markovian transition dynamics. Trajectories across spatial locations (“states”) are associated with consistent hippocampal ensemble spike patterns. In other non-spatial tasks, the latent states may also accommodate non-spatial features of experiences or distinct behavioral patterns that cannot be measured directly. The connection between latent states and spatiotemporal spiking patterns can be established from statistical inference, hypothesis testing, and Monte Carlo shuffled statistics [66–68]. Furthermore, additional features (such as spiking synchrony or LFP features in terms of power or instantaneous phase) can be incorporated into the statistical model for further disassociating distinct latent states. Since this model-based approach is built upon a generative model, model fitting is therefore strongly dependent on the probability distributions that describe the data generation process. If there is a model mismatch, this approach may yield poor performance.

The standard paradigm for memory is to first figure out how the brain encodes information during WAKE, and then determine if those coded patterns appear later, during either SLEEP or subsequent behavioral memory testing — thereby “meaning first, memory later”. In contrast, the new framework allows us to shift the paradigm and look at memory first (by decoding intrinsic structure in neural codes), and then determine the meaning later (i.e., how that structure might correlate with subsequent behavior), thereby “memory first, meaning later”[69]. The main differences between these two paradigms are their assumptions and analysis order (independent of the chronological order). The unsupervised approach is unbiased in that it avoids predefining neural activity patterns in WAKE associated with a specific task or behavior, and also enables us to seek structures that are either not explicitly defined or simply indefinable. Therefore, this unbiased approach may potentially provide us opportunities to discover hidden structures in brain activity, which may represent well-defined WAKE experiences or may reflect some undefined processes (e.g., creative thoughts and imagination). More importantly, this approach may suggest outstanding research questions for experimental investigations. For instance, how can we distinguish the memory in sleep related to previous navigating experiences in two or more distinct spatial environments? How can we decipher non-spatial hippocampal episodic memory [23,71–74] in sleep?

From a data analysis perspective, several technical challenges are worth consideration. First, the sleep episodes have short epochs, sparse and sporadic firing (reduced firing rate compared to wake), and compressed timescale. Dealing with these issues often involves unsubstantiated assumptions (e.g., temporal independence, homogeneity) in data analysis. Second, our empirical studies using synthetic sleep spike data [69] have demonstrated that the number of active hippocampal pyramidal cells is critical for reliable representation of the space as well as detection of spatiotemporal reactivated patterns in SWS. Since only a small fraction (~10–15%) of hippocampal neurons that are active during WAKE is reactivated at any given time during SWS, a reliable investigation of sleep-associated population codes would require simultaneous recording of hundreds of neurons in WAKE. Third, there is large diversity among hippocampal pyramidal neurons for their contributions to the sequence replay [75]. Furthermore, a small percentage of hippocampal pyramidal neurons have no significant spatial tuning but may still fire during sleep. It is unclear whether their firing activities represent other non-spatial episodic memory components in the memory space, and how we can identify their statistical significance. Similar challenges would also apply to the neocortex [76,77].

Future Directions

Neural population recording

Recent advances in neural recordings have greatly expanded our capability to investigate neuronal population codes [78–80]. According to the newest technology in multi-electrode

recording (personal communication, Professor M. Roukes at Caltech), it is predicted that by

TrendsNeurosci. Author manuscript; available in PMC 2018 May 01.

In the year 2020 neuroscientists would be able to simultaneously record 10,000–100,000 hippocampal neurons from rats (based on new development of stacked nanoprobes [81]). As a result, the statistical power of SANC analysis would increase significantly by ~100 fold. In addition, calcium imaging is another emerging technique for measuring the large-scale activity of neuronal populations, which has been successfully used for chronic recordings from the rodent hippocampus [82–85] and cortex [86]. Since calcium signals are merely indirect measurements of neuronal spiking, the precise relationship between calcium signals and spiking is not fully identifiable and is also susceptible to biophysical variations. Therefore, improving the temporal resolution (>500 Hz) and light sensitivity for fluorescence images would potentially enable us to examine large-scale population codes at faster timescales. Combining electrophysiology and cell-type-specific imaging techniques would be an important future direction due to their complementary strengths. In human/non-human primate studies, a new tool that integrates electrophysiological and fMRI (known as neural- event-triggered fMRI) recordings [87] has proven valuable for examining the spatial mapping of apriori defined local brain patterns. Developing wireless multi-electrode recording techniques [88] is also crucial for chronic neural recording from non-human primates in a naturalistic sleep environment.

REM sleep

While NREM sleep has been strongly implicated in the reactivation and consolidation of memory traces, the exact function ofREM sleep remains elusive [89,90]. Unlike NREM sleep, in REM sleep there is no UP state or population synchrony associated with hippocampal SWRs, resulting in a decrease in neuronal firing and an increase in synchrony, both of which are correlated with the power of theta oscillations [91]. This implies that the ensemble spike activity is even more sparse and unstructured. Moreover, there is some experimental evidence that in REM sleep rat hippocampal neurons exhibit a gradual phase shift from the novel (theta peak) to the familiar (theta trough) firing-phase pattern [92]. Such experience-dependent phase reversal suggests that hippocampal circuits may be selectively restructured during REM sleep by selectively strengthening recently acquired memories and weakening remote ones—an idea consistent with the original Crick-Mitchison’s hypothesis of “reversal learning” in REM sleep [93]. Experimentally, the total REM sleep duration is much shorter than the NREM sleep duration for rodents and human adults. Most animal experiments have primarily targeted waking behaviors, thereby limiting the recording period of sleep. To increase the length of sleep or the probability of transition into REM from NREM sleep, optogenetic manipulations of specific neural circuits have been considered in rodents [94–96]. Alternatively, one can investigate rodent infants or other specifies that have longer REM sleep episodes. Recent single-unit recordings in human MTL suggested that eye movements during REM sleep might reflect a change ofthe visual imagery in dreams [57]. With ever-accumulating “BIG neural data”, an ultimate goal is to decipher the animal’s dreams during REM sleep about WAKE experiences— a demanding task still requiring extensive experimental and computational investigations.