Multiple And Dissociable Effects Of Sensory History On Working-Memory Performance Part 3

Task dependence of attractive performance bias between trials

Next, we repeated the same between-trial analyses but investigated the role of task relevance and cue type in the previous trial. This allowed us to test for and compare behavioral biases elicited by the probed (and reported) orientation and by the unreported orientation in the previous trial. 

We also tested whether the cue type in the previous trial affected bias in the current trial. We only looked at trials in which two orientations were presented in the previous trial.

Experiments are very closely related to memory because the purpose of experiments is to verify and prove some theories and hypotheses, and these theories and hypotheses require experimentation and analysis through our memory.

When conducting experiments, we need to remember important information such as the purpose of the experiment, experimental procedures, experimental conditions, and experimental results. Only when we can clearly remember this information can we accurately analyze and interpret the experimental results and draw correct conclusions.

At the same time, experimentation can also help us improve our memory. Through repeated experiments, memorization, and summary, we can consolidate our memory, deepen our understanding and mastery of experimental knowledge, and thereby improve our memory ability.

In addition, experiments can also stimulate our memory through the stimulation of multiple senses, such as vision, hearing, and touch. Through the involvement of multiple senses, we can better remember and understand knowledge, thus having an advantage in memory.

In short, experimentation and memory are inextricably linked. By actively participating in experiments, we can not only exercise our practical abilities but also improve our memory and understanding. Let us make full use of the experimental learning platform and strive to improve our learning abilities! It can be seen that we need to improve memory, and Cistanche deserticola can significantly improve memory because Cistanche deserticola is a traditional Chinese medicinal material that has many unique effects, one of which is to improve memory. The efficacy of minced meat comes from its various active ingredients, including acid, polysaccharides, flavonoids, etc. These ingredients can promote brain health in a variety of ways.

Click know ways to improve your memory

A repeated-measures ANOVA on the sum of angular distances of the performance bias for task relevance and cue type confirmed an effect of task relevance (F(1,19) = 14.684, p = 0.001; Fig. 3D) but showed no effect of cue type (F(1,19) = 1.423, p = 0.248) or interaction (F(1,19) = 1.633, p = 0.216). Task-relevant orientations in trials of either cue type resulted in a significant bias (the previous trial was reported first and contained two items: t(19) = 3.524, p = 0.002; the previous trial was reported second and contained two items: t(19) = 4.476, p, 0.001). Unprobed orientations did not lead to a significant bias (both p . 0.2). 

No reliable difference was observed between the strength of the bias between report second two-items or report first two-items conditions in previous trials (t(19) = 1.691, p = 0.107).
Following up, we assessed the performance bias as a function of the angular distance between the current target orientation and previously presented orientations. Cluster-based permutation testing showed an attractive bias toward the task-relevant orientation on the previous trial when a report first cue (p = 0.001; 7–58°; Fig. 3C) or a report second cue (p, 0.001; 4–77°) was presented in the previous trial. 

For unreported orientations, no bias was observed (no candidate clusters for report first or report second cue). Together, this pattern of results shows an attractive between-trial bias, but only about items that were relevant in the previous trial.

Neural classification of presented orientations

For our classification analysis, grating orientations were binned into 10 equally spaced bins. We applied linear discriminant analysis (LDA) on spatial and temporal features from all 306 MEG sensors ranging from 400 ms before stimulus onset up to 900 ms poststimulus onset (see Materials and Methods). 

If orientation information was present, LDA likelihood estimations gave rise to representational similarity curves centered on the presented orientation that could be convolved with a cosine function to result in a single evidence estimation per time point. LDA classification reflected significant evidence for the presented orientation after the visual onset of the grating (Fig. 4). 

This revealed significant decoding of both the first grating orientation (100– 615 ms; p, 0.001) and the second grating orientation (95– 590 ms; p, 0.001). A nonsignificant trend was observed for the evidence of the first grating orientation following the presentation of the second grating (250–600 ms; t(19) = 1.971; p = 0.063; see also cross-decoding analyses below). There was no difference in classifier evidence after a report first or report second cue (250–600 ms; t(19) = 0.798; p = 0.435).

Within-trial classification bias away from the previous stimulus

Representational similarity curves were used to estimate the direction and magnitude of neural biases in orientation representation. By training the classifier on all stimuli, orientation history biases should cancel out, allowing testing on trials with specific prior orientations (clockwise vs counterclockwise to the current stimulus) to reveal any neural biases. 

To investigate how information from the second grating was modulated by information from the first grating, we separately assessed trials with report first and report second cues. In doing so, we evaluated the classification evidence in the MEG data in the epoch following the presentation of the second grating. For these analyses, we again only selected trials with both gratings were presented. 

We separated trials in which the first grating was clockwise versus counterclockwise relative to the second grating (angular distance of 10–50°, based on behavioral results, see Fig. 2A, B). 

We considered the average of time points between 250– 600 ms for all future analyses, since in this time window stimulus orientation could be decoded with reliable accuracy in this time window (see Fig. 4; see also Fig. 5, gray shaded area). Echoing the performance biases, no significant bias occurred in the report's first trials (Z = 0.645; p = 0.516; Fig. 5A, B). 

However, on report second trial, we observed a repulsive effect, away from the previously encoded grating orientation (Z = 2.743; p = 0.006; Fig. 5C, D). There was no correlation, across participants, between the magnitude of the bias in the behavioral and neural data on report first trials (r = 0.010; p = 0.968) or report second trials (r = 0.328; p = 0.158).

Repulsive neural biases between trials away from sensory history

To probe for neural between-trial biases, we used the same approach as for within-trial neural biases. We tested the LDA evidence derived during the stimulus encoding period for systematic deviations in likelihood estimations as a function of the angular distance between the current orientation and the probed orientation in the previous trial. 

If the behavioral between-trial bias reflects neural modulation during the encoding of sensory features, we would expect to see an increase in the likelihood of orientations presented in the previous trial, in line with the attractive performance bias. We trained the classifier on data following the presentation of both the first and second grating orientation combined and included all cue types and several items presented. 

Informed by our behavioral analyses on between trial performance biases, for the test set we selected trials where the previous probe angle had a relative difference of 0–60° (derived from significant angular differences; Fig. 3A, B) positive or negative from the presented grating orientation. The results were qualitatively the same and remained significant when other angular ranges were selected.

Contrary to our expectations, classifier evidence was significantly shifted away from the target orientation on the previous trial (250–600 ms postgrating onset; Z = 3.228, p = 0.001; Fig. 6A–D) rather than mirroring the attractive behavioral bias. In practice, this would mean that if the cued orientation on the previous trial was CW, classifier evidence for CCW bins increased, and vice versa. 

The repulsive bias away from the target on the previous trial was significant for the first (Z = 2.020, p = 0.043) and the second grating (Z = 2.690, p = 0.007) in the current trial when considered separately (Fig. 6C). The between-trial repulsive bias during stimulus-two processing was present if no orientation was presented in the first interval (Z = 2.298, p = 0.021) but not when the first grating was also presented (Z = 1.714, p = 0.087). Interestingly, topographies in Figure 6D, G were highly similar (r = 0.563; p, 0.001), and both topographies correlated negatively with the stimulus-decoding topography [r = 0.558, p, 0.001 (Fig. 6D); r = 0.763, p, 0.001 (Fig. 6G)].

Next, we tested whether this repulsive neural bias was affected by the task relevance of the grating and by the cueing condition of the previous trial. We quantified this bias for task-relevant and task-irrelevant grating orientations in the previous trial. Only trials where the previous trial contained two items were included in this analysis. 

When averaging the neural bias over 250–600 ms, the task-relevant orientation showed a repulsive neural bias (Z = 2.565, p = 0.010), but we found no neural bias for task-irrelevant orientations (Z = 0.174, p = 0.861), though this difference did not reach significance (t(19) = 1.80, p = 0.088). Yet, cluster-based permutation testing indicated a significant cluster indicating that task-relevant orientations exerted a significantly stronger repulsive bias than task-irrelevant orientations (500–550 ms; p = 0.039).

Cross-decoding evidence for previously presented stimuli

If information about the previously presented orientation is still partially present in the visual system during and after the presentation of the current grating orientation, it could interact with encoding, possibly leading to the observed repulsive bias. One possibility is that the lingering representation is in an orthogonal representational format, which is different from the sensory coding of features (Libby and Buschman, 2021). 

In this case, there would be little to no overlap between the activation pattern elicited during sensory input and the pattern related to the lingering representation of that past grating orientation. A classifier trained to separate perceptual information would therefore not cross-generalize if tested on the memory code. 

Alternatively, the lingering code could be present in a stable representation that shares similarities with the representation of incoming sensory information. If this were the case, a classifier, trained on the data from the current grating orientation, would cross-generalize and identify information about the past grating (or suppression of information expressed in negative evidence).

We adapted cross-decoding to detect lingering orientation-selective activity from the previous trial (also see Wan et al., 2020). After training the classifier for the presented orientation and testing for evidence of the previous trial's target orientation, we observed significant negative classifier evidence in the period of 250–600 after grating onset (concatenating over grating one and grating two: t(19) = 3.078, p = 0.006; grating one alone, t(19) = 1.376, p = 0.185; grating two, t(19) = 2.951, p = 0.008; Fig. 6F).

Negative classifier evidence indicates that, while information is still present about the previous orientation, orientation-selective patterns may be sign-reversed relative to stimulus encoding.

This suppression of evidence for the previous trial's orientation could have been the cause of the apparent repulsive bias in the decoding of the current trial's orientation. If this was the case, we would expect the two measures to be positively correlated: stronger suppression of the previous orientation (negative classifier evidence) should lead to a more negative bias for the current orientation. We tested this using Pearson correlations across participants and found a significant correlation when assessing all grating presentations together (r = 0.832, p, 0.001; grating one alone, r = 0.685, p, 0.001; grating two alone, r = 0.751, p, 0.001; Fig. 6E).

No correlation was observed between the attractive behavioral performance bias and the magnitude of the negative shift in the neural data (r = 0.186, p = 0.432) nor with the magnitude of negative decoding (r = 0.144, p = 0.544).

Discussion

The present study investigated biases from previously perceived and memorized information on neural coding and behavioral responses. We observed both neural and behavioral biases that demonstrate interactions between past and present sensory processing. Orientations presented in the same trial exerted repulsive performance biases on currently perceived orientations. 

In contrast, task-relevant orientations in the previous trial exerted an opposite, attractive performance bias, altogether providing evidence for two counteracting biasing processes acting in tandem. Interestingly, multivariate decoding of neural data indicated that both stimuli from the same and previous trial generate a repulsive neural bias, suggesting that the two types of performance biases may arise from modulations acting on different stages of stimulus processing. 

While repulsive biases could reflect mechanisms akin to visual adaptation that promote visual discriminability, the attractive performance bias observed across trials was not observed at the level of the sensory representation. Since a neural repulsive bias occurred instead, we speculate that postperceptual modulatory mechanisms may override any early repulsive sensory modulation and lead to attractive performance biases.

The present behavioral results provide evidence of two types of performance biases: a repulsive within-trial and an attractive between-trial performance bias (cf. Bae and Luck, 2020; Fischer et al., 2020). Both biases were modulated by the task relevance of the inducing stimulus. First, we identified a repulsive performance bias away from the first orientation in a trial, but no significant retrospective repulsive bias from the second orientation, although the second grating was presented closer in time to the probe. 

We speculate that this may have happened because the relevance cue appeared together with this second stimulus, indicating its irrelevance in probe-first trials. Similarly, participants' responses in the current trial were biased only toward task-relevant orientations in the previous trial. At the neural level, biases were also dependent on task relevance. A repulsive neural bias was only present when the presented orientation was cued as task-relevant and therefore encoded into working memory. 

By contrast, task-irrelevant gratings that were not encoded into working memory could be decoded with similar precision but did not exhibit a significant bias. Gratings from previous trials that were associated with an attractive performance bias also led to repulsive neural biases during working memory encoding. Only task-relevant orientations led to a repulsive bias.

Adaptation could partly explain the repulsive neural bias observed here. Visual adaptation has been proposed as the cause of repulsive neural biases (Jazayeri and Movshon, 2006, 2007; Kohn, 2007; Stocker and Simoncelli, 2007; Webster, 2015). Since adaptation reduces firing in recently active neurons (Clifford et al., 2000; Wainwright, 1999), it is an efficient use of finite neural resources when the environment is autocorrelated (Stocker and Simoncelli, 2007; Webster, 2015) because neurons can code for a larger range of stimuli when their responses are not saturated. Curiously, we observed a repulsive neural bias on report second trials only when the presented orientation was task-relevant and encoded into working memory. The interaction with task relevance suggests that stimulus processing is only biased when it is primed for use in upcoming behavior.

This task-dependent modulation was unlikely the result of reduced processing of task-irrelevant stimuli, as overall orientation decoding was not affected by task relevance. In line with recent studies, it is possible that a context-sensitive repulsive bias, possibly occurring at a post-perceptual stage (Zamboni et al., 2016; Fritsche and de Lange, 2019), exists alongside an early perceptual bias based on visual adaptation (Fritsche et al., 2017). The task-dependency of the neural bias could be the basis of recently observed repulsive biases in behavior (Bae and Luck, 2017; Czoschke et al., 2019, 2020; Chunharas et al., 2022), which may help individuate concurrently maintained stimuli (Wei et al., 2012). Under this explanation, only attended and encoded features would be subject to interactions with previous features.

The bias imposed by the orientation from the previous trial was attractive in behavior but repulsive in the neural data at early processing stages. This contrast is ostensibly at odds with previous behavioral studies that have assigned an early perceptual origin to attractive between-trial biases (Fischer and Whitney, 2014; Cicchini et al., 2017, 2018). There is still little direct neural evidence confirming this. EEG studies have shown that previous trial information can be decoded during the encoding phase of the current trial (Bae and Luck, 2019) or immediately before the current trial (Barbosa et al., 2020), and visually evoked neural responses in numerosity judgment tasks are modulated by stimulus history (Fornaciai and Park, 2018, 2020). However, the mere presence of prior stimulus information does not imply an attractive bias on the current stimulus. One study did observe an attractive behavioral bias and a neural bias in early visual areas using fMRI (St. John-Saaltink et al., 2016), but the study used only two stimulus orientations (45° and 135°) with an offset too large to produce a reliable behavioral bias, meaning that our findings may reflect a different biasing phenomenon.

Contradicting the early sensory origin of serial attractive bias, a recent fMRI study (Sheehan and Serences, 2022) found evidence consistent with the present results. Sheehan and Serences observed repulsive neural biases relative to the orientation on the previous trial across the visual cortex, despite an attractive behavioral bias, and found that models incorporating early visual adaptation and a postperceptual origin of attractive biases could explain both effects. These results are broadly in line with our findings. We found that the repulsive neural bias of the classified orientation emerged relatively early in the trial, consistent with the early visual cortex as the source of the bias. 

Our primary analysis approach did not allow us to infer anatomic sources of the bias (since we included data from all sensors as features in the classifier and their anatomic interpretability was further reduced by the dimensionality reduction step). We therefore conducted a searchlight analysis to identify which sensors contributed most to stimulus classification and the repulsive bias. These spatially resolved results were in line with the visual cortical site of the repulsive bias observed by Sheehan and Serences. 

Together, the observations make a case that prior stimuli lead to repulsion at the encoding stage and that attractive performance biases may arise from a different type of bias that did not systematically affect orientation decoding. Our findings further indicate that the link between neural adaptation and behavior can be context-dependent since repulsive neural biases could lead to both repulsive (within-trial) and attractive (between-trial) behavioral biases. Therefore, future models linking visual adaptation to behavior may need to incorporate context dependence. Additionally, the high temporal resolution of MEG allowed us to show that neural biases arise within 500ms of stimulus onset, have a posterior origin, and that they occur simultaneously relative to multiple prior stimuli (from the same trial and the previous trial).

While Sheehan and colleagues (Sheehan and Serences, 2022) argued that past stimuli were stored in a nonsensory code, they did not directly examine whether the representation of past stimuli occurred in a shared neural subspace with the representation of current stimuli. Here, we addressed this issue by showing that neural suppression of recently active neural populations could account for the observed repulsive bias. We tested this using cross-decoding analyses, training a classifier on the presented orientation, and predicting previous orientations. Consistent with the suppression of recent stimulus-specific activity, cross-decoding yielded below-chance decoding of the previous orientation. In turn, this may have shifted the neural tuning curve for the current orientation away from the previous orientation, generating a repulsive bias. This relationship was confirmed by the robust correlation between cross-decoding and repulsive bias magnitude.

Another recent study observed a repulsive neural bias in single-unit recordings from frontal eye fields (FEF) paired with an attractive behavioral bias in a delayed-saccade task (Papadimitriou et al., 2017). The authors suggested that lingering attention to the previous target location could warp the representation of current target locations (Zirnsak et al., 2014). Since we observed neural biases primarily in posterior sensors, our results are more in line with the sensory origin of the repulsive bias, but this may interact with attentional biases originating in the frontal cortex (Moore and Armstrong, 2003; Taylor et al., 2007). Attentional modulation could be one explanation for the context-dependency of biases observed here.

The current study, along with the study by Sheehan and Serences (2022) provides evidence against an early sensory origin of the attractive serial bias, but it does not provide direct evidence for how it arises. We speculate that the attractive bias does not manifest itself in orientation decoding or is not strong enough to overcome the repulsive bias during encoding. The attractive bias could originate postdecoding but since we were not able to identify stimulus-specific activity patterns later in the maintenance delay, we cannot say how this occurs. One possibility is that serial biases in behavior are not caused by biased neural representations at all. Akrami et al. (2018; Boboeva et al., 2023) have suggested that prior stimulus information is stored in parietal neural activity, separately from the maintenance of the current memory stimulus. Its influence on behavior only emerges when a response is made that draws on both neural signals, thus creating an attractive bias. Bayesian accounts of serial dependence (Fritsche et al., 2020), while conceptually distinct, could equally rely on the separate maintenance and decision-stage integration of the prior and current stimuli. Such models could explain why there is so little positive evidence for an attractive neural bias at any processing stage.

Altogether, we demonstrate a consistent repulsive shift in neural evidence during working memory encoding. Our results imply that perceptual adaptation, along with context-sensitive factors, contributes to feature-selective down weighting to exert a repulsive bias away from recent stimulus features. Interestingly, no evidence of an attractive neural bias acting directly on sensory aspects of encoding was observed. Neural data thereby provide indirect evidence for the postperceptual account of attractive between-trial biases, rather than modulating encoding stages (Bliss et al., 2017; Fritsche et al., 2017; Pascucci et al., 2019; Bae and Luck, 2020; Kim et al., 2020). We speculate that the attractive between-trial bias instead arises through postperceptual processing stages involving memory (Bliss et al., 2017; Fritsche et al., 2017), perceptual decision-making, or motor planning (Boettcher et al., 2021; de Azevedo Neto and Bartels, 2021).

The source of the attractive between-trial bias, whatever its neural mechanism, may be strong enough to override the repulsive bias during the perceptual/ encoding stage. Together, these co-existing biases may help guide efficient coding for nuanced perceptual discriminations and visual stability across our environment.

References

Akrami A, Kopec CD, Diamond ME, Brody CD (2018) Posterior parietal cortex represents sensory history and mediates its effects on behavior. Nature 554:368–372.

Bae GY, Luck SJ (2017) Interactions between visual working memory representations. Atten Percept Psychophys 79:2376–2395.

Bae GY, Luck SJ (2019) Reactivation of previous experiences in a working memory task. Psychol Sci 30:587–595.

Bae GY, Luck SJ (2020) Serial dependence in vision: merely encoding the previous-trial target is not enough. Psychon Bull Rev 27:293–300.

Barbosa J, Stein H, Martinez RL, Galan-Gadea A, Li S, Dalmau J, Adam KCS, Valls-Solé J, Constantinidis C, Compte A (2020) Interplay between persistent activity and activity-silent dynamics in the prefrontal cortex underlies serial biases in working memory. Nat Neurosci 23:1016–1024.

Bays PM, Catalao RFG, Husain M (2009) The precision of visual working memory is set by allocation of a shared resource. J Vis 9:7.1–11.

Bliss DP, Sun JJ, D'Esposito M (2017) Serial dependence is absent at the time of perception but increases in visual working memory. Sci Rep 7:14739.

Boboeva V, Pezzotta A, Clopath C, Akrami A (2023) From recency to central tendency biases in working memory: a unifying network model. bioRxiv 491352. https://doi.org/10.1101/2022.05.16.491352.

Boettcher SE, Gresch D, Nobre AC, van Ede F (2021) Output planning at the input stage in visual working memory. Sci Adv 7:eabe8212.

Born RT, Tootell RBH (1992) Middle temporal visual area. Nature 357:497– 499.

For more information:1950477648nn@gmail.com