for more information:ali.ma@wecistanche.com

pls click here to Part 2

Benjamin Kowialiewski1,2,3 & Benoît Lemaire2 & Steve Majerus2,4 & Sophie Portrat4

Accepted: 12 February 2021 / Published online: 25 March 2021

# The Author(s) 2021

Click Cistanche UK for memory

Abstract

The maintenance of serial order information is a core component of working memory (WM). Many theoretical models assume the existence of specific serial order mechanisms. Those are considered to be independent of the linguistic system supporting the maintenance of item information. This is based on studies showing that psycholinguistic factors strongly affect the ability to maintain item information while leaving order to recall relatively unaffected. Recent language-based accounts suggest, however, that the linguistic system could provide mechanisms that are sufficient for serial order maintenance. A strong version of these accounts postulates serial order maintenance as emerging from the pattern of activation occurring in the linguistic system. In the present study, we tested this assumption via a computational modeling approach by implementing a purely activation-based architecture. We tested this architecture against several experiments involving the manipulation of semantic relatedness, a psycholinguistic variable that has been shown to interact with serial order processing in a complex manner. We show that this activation-based architecture struggles to account for interactions between semantic knowledge and serial order processing. This study fails to support activated long-term memory as an exclusive mechanism supporting serial order maintenance.

Keywords: Working memory. Serial order. Computational modeling. Semantic knowledge

Introduction

The ability to maintain serial order information is a core component of verbal working memory (WM). Mechanisms involved in the maintenance of serial order (i.e., the sequential order ofthe to-be-remembered items) have been considered to be independent of those involved in the maintenance of item information (i.e., the linguistic content of the to-be-remembered items). This assumption is supported by different lines of research. Studies examining the impact of psycholinguistic factors, such as lexicality, on verbal WM performance, typically observe effects on item recall, with minimal effects on serial order recall (Allen & Hulme, 2006; Hulme, 2003;

Department of Psychology, University of Zurich, Binzmühlestrasse 14, 8050, Zürich, Switzerland

University of Liège, Liège, Belgium

Laboratoire de Psychologie et NeuroCognition (LPNC), Université Grenoble Alpes, Bâtiment Michel Dubois prev. BSHM, 1251 Avenue Centrale, 38400 Saint-Martin-d’Hères, France

Fund for Scientific Research – F.R.S.-FNRS, Brussels, Belgium

Romani, Mcalpine, & Martin, 2008; Roodenrys, Hulme, Lethbridge, Hinton, & Nimmo, 2002; Saint-Aubin & Ouellette, 2005; Walker & Hulme, 1999). In addition, serial order-recall performance is more strongly affected by rhythmic and articulatory interfering tasks than is the maintenance of item information (Gorin, Kowialiewski, & Majerus, 2016; Henson, Hartley, Burgess, Hitch, & Flude, 2003). Neuropsychological studies have also reported the existence of double dissociations between serial order and item-recall performance in several brain-injured patients and populations affected by neurodevelopmental disorders (Brock & Jarrold, 2005; Majerus, Attout, Artielle, & Kaa, 2015; Martinez Perez, Poncelet, Salmon,& Majerus,2015). Finally, the maintenance of item and serial order information is supported by different neural substrates, as reported by neurostimulation and neuroimaging studies (Attout, Fias, Salmon, & Majerus, 2014; Guidali, Pisoni, Bolognini, & Papagno, 2019; Kalm & Norris, 2014; Majerus et al., 2010; Papagno et al., 2017).

At the same time, other studies suggest that serial order recall can also interact with linguistic knowledge. Although lexical knowledge strongly enhances recall of item information, it also constrains phoneme migration errors within and between items (Jefferies, Frankish, & Lambon Ralph, 2006). Likewise, nonwords, even if more poorly recalled as compared to words at the item level, can show a relative advantage regarding serial order recall (Fallon, Mak, Tehan, & Daly, 2005; Kowialiewski & Majerus, 2018; Saint-Aubin & Poirier, 1999). Recently, Kalm and Norris (2014)showed that the serial order of nonwords could be decoded on the basis of neural patterns elicited within the dorsal language pathways supporting encoding and maintenance of verbal information. Similarly, Papagno et al. (2017) showed that serial order-recall performance decreases, as compared to item-recall performance when the posterior part ofthe dorsal language pathway is stimulated using direct electric stimulation in neurosurgical patients.

At a theoretical level, it has been claimed that the temporary maintenance of serial order information could be performed without the need for a specific item and serial order representational levels (Acheson & MacDonald, 2009; Jones & Macken, 2018; Schwering & MacDonald, 2020). A strong version of such an account considers that serial order information is exclusively maintained via the pattern of activations occurring within the linguistic system (Acheson, MacDonald, & Postle, 2011; Martin & Saffran, 1997; Poirier, Saint-Aubin, Mair, Tehan, & Tolan, 2015). For in- stance, according to Martin & Saffran (1997, p. 672):

“In principle, interactive activation processes could also play a role in maintaining serial order. The word node representing the first word in a sequence is primed first and therefore has more time to gain support from activated phonological and semantic representations compared to nodes that are primed later in a sequence. Thus, word nodes should show a gradient of activation levels across serial positions. [ … ] Recency effects in supraspan recall reflect the increased phonological support that is due to the fact that at the time of recall, the activation levels of the terminal items have been less affected by the decay function inherent in the activation model.”

Cistanche can improve memory

Likewise, Acheson et al. (2011, pp. 45–46) suggested that serial ordering errors could occur directly via an item’s rela- tive level of activation in a language network:

“These interactive activation frameworks provide a potential explanation as to how semantic representation might influence the order of lexical-level utterance plans. When someone hears a word or a sequence of words, activation from that input simultaneously feeds forward to phonological representations and feeds back to semantic representations as well. After initial encoding, lexical activation is determined by repeated interaction with semantic and phonological representations. Serial ordering errors occur when the relative ac- tivation levels ofthe lexical items change because ofthis interaction.”

Based on this idea, Poirier et al. (2015) developed a more elaborate description of such models, termed the ANet ac- count. According to this account, items in a to-be-remembered list are sequentially encoded in the linguistic long-term memory system with decreasing strength following an activation gradient,1 as displayed in Fig. 1. Serial order information is maintained via this activation gradient. Serial recall is performed by selecting the most strongly activated item at each recall attempt. Due to the selection mechanism being noisy, serial order errors eventually occur. An important prediction from this model is that modifying an item’slevel of activation within the linguistic system should also affect the patternofserialorderingerrorsinWM (Achesonetal., 2011).

Recent evidence appears to support this theoretical position. Poirier et al.(2015) manipulated semanticrelatedness by presenting triplets of semantically related items in the first half of to-be-remembered lists. The subsequent items of the lists were semantically unrelated in the control condition (e.g., officer–badge– siren– music – tourist – yellow). In the experimental condition, the fifth item was semantically related to the triple tinthefirsthalf of the list. Compared to the control condition, the authors ob- served an increase of migration errors of the fifth item toward earlier serial positions, that is, towards the semantically related tripletsofwords.Theauthorsassumedthatsincethesemantically related triplets were pre-activated, the semantically related target (i.e., viaspreadingofactivationwithinasemanticnetwork),this target should have a higher activation level in the experimental condi- tion (Fig. 1c) as compared to the control condition (Fig. 1b). Since recall of serial order information is performed by selecting the most activated item, a gradient of activation in long-term memory could theoretically predict more migrations of the se- mantically related target toward earlier serial positions. As such, the manipulation of semantic relatedness is a critical and direct test of activation-based models, because it is supposed to modify the relative pattern of activation occurring within the linguistic system. This relative activation should in turn affect the process- ing of serial order information (Acheson et al., 2011), which the data ofPoirier and colleagues appear to support. This was indeed a core prediction from their ANet account:

“In Experiment 1,we manipulated the level ofactivation of a target item to test the prediction that this would increase order errors for that item, making it likely that the CQ [Competitive Queuing] mechanism would select this item earlier because ofits heightened activation; this early selection would mean that activation affected the order in which items were recalled.” (Poirier et al., 2015, p. 492).

Fig. 1 Illustration of the activation gradient (a) in a semantically unrelated condition, (b) in a condition in which items A, B, and C are semantically related, and (c) in a condition in which items A, B, C, and E are semantically related. Semantically related items are marked with an asterisk. As can be seen, the presence of semantic relatedness boosts the item's activation level for the related items

Given that this theoretical account is in striking contrast with the majority of computational models of WM positing distinct item and serial order processing levels, the aim of the present study was to test the computational plausibility of a purely activation-based linguistic account for representing serial order information in a WM context. Most computational models of WM indeed explicitly assume the existence of serial order mechanisms that are distinct from those involved in item infor- mation. This is for instance the case as regards the TBRS* and SOB-CS architectures (Oberauer & Lewandowsky, 2011; Oberauer, Lewandowsky, Farrell, Jarrold, & Greaves, 2012), but also the computational models ofBurgess and Hitch (1999, 2006) and Brown, Hulme, and Preece (2000). These types of architectures consider that serial order information is maintained via the creation of item-position associations, the positions being represented by specific representational mechanisms. These models, although strongly differing on the nature of the serial position representations, reliably reproduce important serial order phenomena, including primacy and recency effects and transposition error patterns.

which is a critical psycholinguistic factor to test the plausibility of a purely activation-based architecture. To overview the computational architecture, we first assumed serial order in-formation as being maintained via a Primacy gradient of acti- vation in long-term memory (Martin & Saffran, 1997;Page & Norris, 1998; Poirier et al., 2015). We then adapted this architecture by adding lateral excitatory connections to model semantic effects.

Cistanche can improve memory

Computational modeling

Architecture

The architecture we used is a connectionist model composed ofa single layer. When encoded, an item becomes active. This activation is supposed to occur directly in the long-term mem- ory knowledge base. Semantically related items are connected via direct bidirectional excitatory connections, whose plausi- bility to model semantic effects in WM has already been dem- onstrated in three independent models (Haarmann & Usher, 2001; Kowialiewski & Majerus, 2020; Kowialiewski, Portrat, & Lemaire, 2021). The items are successively activated with decreasing strength using an activation gradient. Each encoded item automatically spreads activation towards the other semantically related items. Recall is performed by suc- cessively retrieving each item according to their activation value. For simplicity, we used the last implementation of the Primacy model, which was made available by Norris, Kalm, and Hall (2020). Our Julia implementation of the architecture we propose is freely available on the Open Science Framework (OSF): https://osf.io/9e4hu/.

Encoding In the original Primacy model, encoding follows an activation gradient, which we denote V. This is defined by a peak value, γ, and a step value, α. The value γ is a free parameter and represents the startingvalue with which the first item is associated. The α value represents the amount of de- pletion from the γ value at each encoding stage. This param- eter is fixed to 1. For instance, given a γ value of 20, the activation gradient is [20, 19, 18, 17, 16, 15] for a six-item list. Note that rehearsal is never explicitly modeled in the Primacy model. This includes the last implementation by Norris and colleagues. Activation within the model is simply derived from what would be expected ifrehearsal theoretically occurs.

Spreading activation During encoding, activation spreads to- ward semantically related nodes. This is modeled by including bidirectional excitatory connections. The strength of these connections is a free parameter, λ. At each encoding stage, items are activated using the activation gradient V. Activation then spreads bidirectionally within the network:

where Ai represents the final activation value associated to item i, andAj is the activation coming from each semantically related item, j, scaled by the connection weight, λ. The sub- script t represents the timestamp.

It is important to note that we do not intend to explicitly represent semantic knowledge. What we intend to represent through this spreading activation principle is the fact that se- mantically related items reactivate each other. In turn, this reactivation is supposed to modify item relative activation and therefore the pattern of serial order errors (Acheson et al., 2011). In other words, modifying the items’ relative level of activation in the semantic network also changes the model’s internal representation of their serial order.

Recall After all items have been encoded, the model has to retrieve the items. This is made using a competitive queuing mechanism.2 Recall is a two-step process.

2 Poirier and colleagues suggested the competitive queueing mechanism as being modeled using an accumulator model, following Hurlstone and Hitch (2015). We implemented such a competitive queueing mechanism based on accumulator principles (available on OSF). This did not provide any enhancement of the model whatsoever, with the exception that the accumulator model provides the further opportunity to make predictions on recall latencies, which is beyond the purpose of this study. Therefore, we simply stuck with the last available implementation of the Primacy model.

First, an item is selected as a potential candidate. This process is subject to noise:

This is modeled by adding temporary zero-centered random Gaussian noise to each item’s activation, with a standard deviation of σ, a free parameter. The most activated item is then selected. Response suppression (Duncan & Lewandowsky, 2005) already occurs at this stage, by setting the recalled item to a very low value (i.e., -999). This prevents the model from recalling an item twice. Second, the activation value of the selected item is compared to an omission threshold. This threshold is drawn from a random Gaussian distribution N(θ, σ′), where θ and σ′ are two free parameters. If the selected item’s activation value (without the noise added during the first step) is above the retrieval threshold, the item is correctly recalled. Otherwise, an omission is produced. It must be pointed out that this implementation assumes response suppression as being always applied during the first step of retrieval, regardless of whether an omission had been produced during the second step. This choice of implementation by Norris et al. (2020) is unlikely to be plausible. But from the experience we gained by running the model many times, this is the only way the Primacy model can produce omission errors while modeling realistic serial position curves. Note that it is possible to produce realistic serial position curves while avoiding this implementation problem without affecting the model’s core assumptions. We, however, preferred to stick with the original implementation for simplicity. By the time of each successive recall attempt, all items have decayed:

where D is a free parameter, ranging from 0 to 1. Due to this decay parameter, items recalled later in the lists are more sub- ject to noise, because activation values converge towards an asymptote. All the parameters of the model are listed in Table

1. Method

Datasets The validity of this model was tested on three different sets of data: two datasets (Kowialiewski et al., 2021; Kowialiewski & Majerus, 2020) that include semantic and neutral conditions (i.e., the neutral condition being a semanti- cally unrelated condition), and the data from Poirier et al. (2015), which we already described in the Introduction. The model is based on several parameters that partially depend on the task. Parameters were therefore estimated independently for each set of data. First, the parameters that do not depend on the semantic relatedness were estimated based on the neutral

condition, in order to obtain a baseline model that would be able to reproduce standard serial-recall performance. Second, the semantic condition was used to estimate the λ parameter, which controls the level of semantic relatedness between items.

Overall scoring procedure Serial position curves are plotted using a strict serial-recall criterion, in which an item is scored as correct only if it is recalled at the correct serial position. For instance, given the target sequence “Item1 – Item2 – Item3 – Item4 – Item5 – Item6” and the recall output “Item1 – Item2 – blank – Item3 – Item4 – Item6”, only Items 1, 2, and 6 would be scored as correct. To fit the experimental data, we also used an item-recall criterion, in which an item is scored as correct if correctly recalled, independently of its serial position. In the example mentioned above, items 1, 2, 3, 4, and 6 would be scored as correct. To assess the overall impact of semantic relatedness on order-recall performance, we computed an order-recall score for each experimental condition. This was done by dividing the number oftime items have been recalled in the correct position (i.e., strict serial-recall criterion) by the number of times items have been recalled, regardless of their serial position (i.e., item-recall criterion).

Transposition rate The pattern of transposition errors in the Poirier et al. (2015) study was plotted using transposition rates. We computed the number of transposition errors that occurred for item 5 (which is semantically relatedor not to items 1,2,and 3), and for each position towards which item 5 could migrate. We then divided these numbers of transposition errors by the total number of times item 5 was recalled. This was computed separately for each experimental condition.

Parameter estimation Estimation of the model’s basic parameters was performed using a simulated annealing algorithm

(French & Kus, 2008; Kirkpatrick, Gelatt, & Vecchi, 1983) to find the lowest root mean square error (RMSE) between experimental and simulated serial position recall scores, on both strict and item-recall criteria. The RMSE was therefore always com- puted over 12 data points: six data points for the strict serial- recall criterion, and six data points for the item-recall criterion. The lower and upper boundaries of each free parameter are reported in Table 1. Estimation of the semantic parameter λ was much simpler and only required a grid search in [0,0.1] with a step of 0.0001. Importantly, λ was always estimated while keeping the model’s basic parameters constant. The value of λ that produced the smallest mean difference between the neutral condition and the experimental condition to the empir- ical data was then used. The idea was to select the value of λ that produces a difference between neutral and experimental scores similar to the human one. This was operationalized by minimizing the gap between the human mean difference and the model mean difference. We now present the three datasets as well as the simulations of these corresponding experiments. A summary ofthe different experimental conditions with study list examples is provided in Table 2.

Model assessment

Dataset #1: Kowialiewski and Majerus (2020)

Data This dataset was used to assess the model’s ability to reproduce the overall impact of semantic relatedness on serial-recall performance and order-recall performance. It is well established that semantic relatedness strongly enhances recall performance at the item level (see Kowialiewski & Majerus, 2020, for a meta-analysis). Semantic relatedness also has a small deleterious impact on the ability to recall serial order information, even though the effect is subtle (see also

Ishiguro & Saito, 2020). Accordingly, we expect the architec- ture to have little or no impact on order-recall performance. We used the data reported in Kowialiewski and Majerus (2020), where they manipulated the semantic relatedness on six-item lists under interfering conditions or under immediate serial-recall tasks. Only the results from the latter condition were reported.