The Limits Of Automatic Sensorimotor Processing During Word Processing: Investigations With Repeated Linguistic Experience, Memory Consolidation During Sleep, And Rich Linguistic Learning Contexts Part 3

Experiment 3

To address these issues, in Experiments 3 and 4, we moved from investigating novel words towards vertically associated familiar words whose referents participants never experienced directly (such as Hades or Pterosaur). 

It is important to be familiar with the relationship between words and memory because we use language skills and memory every day to communicate and learn. When we are familiar with some vocabulary, we can use them more fluently, which not only improves our language skills but also allows us to express ourselves more accurately and clearly.

At the same time, familiarity with vocabulary is also of great help to our memory. When we learn new vocabulary, we can remember it more easily if we can consolidate it through repeated repetition and use. This is because our memory will gradually form "neural pathways" after experiencing many repeated stimuli. These neural pathways can help us better retain memories.

In addition, familiarity with vocabulary can also help us understand and remember new knowledge more easily. When we encounter some new concepts or knowledge, it will be easier for us to understand and remember the knowledge if we can connect it with already familiar vocabulary.

In short, being familiar with vocabulary and paying attention to memory play a very important role in our language ability and learning ability. We should constantly strive to learn and consolidate our vocabulary and strengthen our memory through constant repetition and use. Only in this way can we express ourselves more fluently and achieve greater learning results. It can be seen that we need to improve memory, and Cistanche deserticola can significantly improve memory because Cistanche deserticola is a traditional Chinese medicinal material that has many unique effects, one of which is to improve memory. The efficacy of minced meat comes from the various active ingredients it contains, including acid, polysaccharides, flavonoids, etc. These ingredients can promote brain health in various ways.

Click know 10 ways to improve memory

Thus, we employed words that are established lexicon entries and were not just recently learned in an artificial lab setting. We can therefore assume that participants have repeatedly encountered and used these words in natural communication settings, and to have a clear meaning representation of the described entities (including their vertical location).

Methods

Participants

Following the results of the power analysis, we tested 45 participants (all right-handed; 36 female, 9 male; MAge = 23.5 years, SDAge = 6.30 years.). Data from one additional participant was excluded due to high error rates.

Materials and procedures

To create the item material, we collected rating data from 25 participants who did not participate in the actual study. For a set of 62 words, participants indicated on 5-point scales (a) the typical vertical locations associated with the word's referents (from very low to very high), and (b) how much direct experience they made with the word's referents during their lifetime (from no experience to very much experience). 

The 62 items were selected because we expected them to cover the entire range for all collected variables, including some filler items; the entire item list can be found at https://osf.io/vxrhn. Participants were explicitly instructed that depictions of the referents, for example in pictures and movies, can be counted as direct experience.

We selected eight words that were associated with a vertical location, but with which participants indicated very little or no direct sensorimotor experience (see Table 2). Apart from the word material, the procedure of Experiment 3 was identical to the Test Phase of Experiments 1 and 2.

Results

The data were analyzed using the procedure described for the Test Phase analysis of Experiment 1. Error trials (3.3 %) and one overly fast trial were excluded from the analysis. Since we used real words, the factor learning context was replaced with the implied location. Mean reaction times by implied location and response direction are displayed in Fig. 3 (left panel).

We performed the same model comparison as described in the previous analyses, except that the factor "learned direction" was now replaced by "implied direction". The model including the fixed effect for the two-way interaction between response direction and implied location did not perform better than the model without this interaction, as indicated by a likelihood-ratio test (휒2(1) = 1.80, p = 0.180). 

We obtained a BIC-approximated Bayes factor of BF = 0.0233, indicating that the data are about 43 times more likely under the baseline model (strong evidence in favor of the baseline model). The model parameters for the model including the interaction are reported in Table 3.

Discussion

Even though we employed real words as item material, we again observed no action-congruency effect. Thus, the factors discussed as potentially leading to the absence of this effect for novel words in Experiments 1 and 2-the limited learning experience with these novel words, the fact that participants never used them as a cue to retrieve sensorimotor experience, or that they never used or encountered them in natural contexts-do not offer a sufficient explanation for this absence.

Interestingly, even some limited experience with the word's referents seems insufficient to elicit action congruency effects, as participants in the rating did not equivocally indicate that they have no experience with the referents-the ratings differ slightly from the minimum value. This raises the possibility that the action-congruency effects observed by Öttl et al. (2017) after exposing participants to the word referents were partly due to the high salience and recency of the sensorimotor experience.

Experiment 4

At this point, there is a fairly simple alternative explanation for the results in Experiment 3: Some of the words were not particularly frequent (for example Hades or supernova), and (some) participants could have simply not known the words. In this case, one could not reasonably expect any congruency effects.

We thus replicated Experiment 3, while ensuring that participants indeed knew the words presented to them. 

In this context, we also considerably extended the rating study and explicitly differentiated between direct and "indirect" experience (for example, in pictures and movies), to obtain a maximally adequate item set for Experiment 4.

Method

Participants

In this experiment, we tested 44 native German-speaking participants (one fewer than required due to technical problems, 41 right-handed; 35 female, 9 male; MAge = 23.6 years, SDAge = 4.24 years). Data from five additional participants was excluded due to high error rates (see the previous experiments)

Materials and procedures

The material for Experiment 4 was obtained in a web-based rating study, using the jsPsych software (de Leeuw, 2015).

We assembled a list of 348 items and instructed participants (who did not participate in the actual experiment) to indicate, on a 5-point scale, the vertical location of the described object, the amount of direct sensorimotor experience with the described object, and the amount of "indirect" sensorimotor experience (for example in pictures or movies). They were further allowed to indicate that they didn't know a word. 

The item material was selected to cover the whole range of value combinations of vertical location and amount of experience, and rating results indicated that this manipulation was successful. The entire item list can be found at https://osf.io/vxrhn.

The questionnaire was administered to 203 participants. Each participant was presented with 30 randomly selected items, resulting in between 10 and 34 ratings per word. As the up-words (down-words) for our experimental material, we selected four words that, on average, (a) received very high/very low location ratings, (b) received very low direct experience ratings, (c) received low indirect experience ratings, and (d) were known to most participants. 

Thus, the words and their referents were very familiar to participants (prevalence-the number of speakers knowing a word-is strongly correlated with familiarity and word frequency; Brysbaert et al., 2019), but participants had little to no sensorimotor experience with the word referents. The selected items are displayed in Table 2. Apart from the word material, the material and procedure of the Experiment 4 test phase were identical to that of Experiment 3.

After the experiment, participants were handed a questionnaire and instructed to indicate for each of the eight words in the item material whether they knew the word or not, and if they did, the vertical location associated with the described object (up vs. down).

Results

The data were analyzed as described in Experiment 3. Again, error trials (3.3 %) and one overly fast trial were excluded from the analysis. Mean reaction times by implied location and response direction are displayed in Fig. 3 (right panel). Again, the model including a two-way interaction fixed effect between response direction and implied location did not outperform the model without this interaction (휒2(1) = 0.14, p = 0.711). 

We obtained a BIC-approximated Bayes factor of BF = 0.0103, indicating that the data are about 97 times more likely under the baseline model (strong evidence in favor of the baseline model). The model parameters for the model including the interaction are reported in Table 3. 

The pattern of results stays unchanged if we exclude from the analysis trials including words for which participants gave no answers or incorrect location judgments, or indicated that they did not know the word, in the post-test phase questionnaire (7.2% of the data).

Discussion

The results of Experiment 3 were replicated in Experiment 4, demonstrating that (a) we again find no evidence for an action-congruency effect for items participants have no direct sensorimotor experience with, and (b) that the absence of this effect is not due to participants not knowing the words presented to them. Interestingly, the results in Experiment 4 also indicate that low to moderate levels of indirect sensorimotor experience with the described objects are not sufficient to elicit an action-congruency effect. 

This can potentially be attributed to the fact that "depicted" objects are usually not experienced in the same vertical location as their "real" counterparts (movies and pictures are usually encountered directly in front of the observer, or on a display they are holding in their hands).

These findings demonstrate that just controlling for the associated vertical location (see Goodhew & Kidd, 2016) when selecting the item material for studies on congruency effects is insufficient: In such studies, the absence of congruency effects could also simply result from a lack of direct experience, and not necessarily the absence of sensorimotor activation in their respective experimental paradigms.

General Discussion

In four experiments, we tested whether speakers automatically activate sensorimotor experience in word processing when they have no direct experience available with the word's referents. To this end, we employed words associated with a vertical location and employed an experimental paradigm in which previous studies observed automatic motor congruency effects during word processing (Lachmair et al., 2011; Öttl et al., 2017; Thornton et al., 2013). 

In Experiments 1 and 2, participants learned novel words and their associated vertical locations in several separate learning phases, with an intervening memory consolidation via night's sleep. 

In Experiment 3 and 4, we employed familiar, vertically-associated words whose referents participants did not experience directly. In line with previous results by Günther et al. (2018), we did not observe automatic action-congruency effects for words learned from language alone in any of the experiments, even though participants in the present set of studies had far more experience with the words in richer linguistic learning contexts, and even though they had the opportunity for memory consolidation during sleep.

Discussion of alternative explanations

This pattern of results is not a consequence of using only a limited number of eight items: In a pilot experiment, Günther et al. (2018) observed an action-congruency effect for a set of eight real words for which direct experience available (cloud or basement), the same number as items in the present experiments. It is also not a consequence of using novel words per se: Öttl et al. (2017) observed the effect for their set of eight novel words for which direct experience with the referents was available. 

Furthermore, Günther et al. (2020) also observed a congruency effect using a set of eight words, both for real words (Experiment 1) as well as novel words (Experiment 2). 

By following the results of an earlier power analysis (see Günther et al., 2018), it is also unlikely that the absence of an effect in four different experiments results from insufficient statistical power (even if the power estimate of 0.90 was an extreme over-estimation and the true power of each of our experiments was only at 0.50, the probability of not finding an existing effect would only be at (1 − 0.50) 4 = 0.0625 across all four studies). 

Finally, the absence of an effect can also not be ascribed to participants not understanding the words or not associating them with a vertical dimension, as indicated by the explicit judgment task in Experiments 1 and 2 and the rating results in Experiments 3 and 4.

In principle, there is also the possibility that we did not find a congruency effect as a result of the specific properties of the item material and the required response: In all experiments, participants had to react with upwards or downward hand movements, while interactions with the referents of the presented words would not necessarily involve such vertical hand movements (take, for example, an underground city or artificial sun in Experiments 1 and 2, or earth's core or Jupiter in Experiments 3 and 4). 

However, previous studies have shown that this word-level congruency effect is also consistently found for items such as plateau, planet, sky, cloud, or skyscraper on the one hand, and swamp, submarine, basement, or underground on the other hand (Lachmair et al., 2011)-all entities that we arguably don't interact with more using vertical hand movements than the concepts in question. 

We thus don't consider it likely that specifically, this property of the item material used in the present studies caused the absence of the effect. However, it might still be the case that automatic congruency effects are more likely for non-experienced concepts that would inherently afford such vertical movements, which can be investigated in future studies.

In the context of this argument, it is also important to note that the congruency effect investigated here is not the classical action-sentence congruency effect (ACE; Glenberg & Kaschak, 2002) found for sentences describing certain actions and specific movements; instead, it is a pure word-level effect. This is especially relevant as the reliability of the classical ACE has recently been called into question (Papesh, 2015), especially since a large multi-lab collaboration has failed to replicate it (Morey et al., in press). 

However, this debate on the ACE did not yet consider word-level effects, which have been reliably observed across many different studies (Dudschig et al., 2012, 2014a, b; Dudschig & Kaup, 2017; Lachmair et al., 2011; Öttl et al., 2017; Thornton et al., 2013; Vogt et al., 2019; see also the pilot study in Günther et al., 2018). In the studies where this word-level effect was not observed, this can either be attributed to missing saliency of the vertical dimension in both the stimulus and response set (Dudschig & Kaup, 2017) or, as in the studies presented here, to the specific word material (novel word labels for non-experienced referents; compare Günther et al., 2018). 

Given the apparent robustness of this word-level effect,4 we don't consider it likely that the null results of the present study are the result of a general non-replicability.5

Theoretical implications

Our results are in line with accounts postulating that language processing does not always entail the automatic activation of sensorimotor experience (see Lebois et al., 2015). This is often explained in terms of task demands, in that we only engage in sensory and motor processing when required by the task (Günther et al., 2020; Ostarek & Huettig, 2019). However, we need to extend upon this explanation: As demonstrated by previous studies, such automatic action congruency effects also emerge when direct experience with word referents is available, irrespective of whether they are well-known familiar words (Lachmair et al., 2011) or newly learned novel words (Öttl et al., 2017). 

On the other hand, when that direct experience is missing, we observe no such effects, neither for novels (Experiments 1 and 2) nor for familiar words (Experiments 3 and 4). Thus, the sensorimotor experience can be automatically activated even when not required by the task, but only if the direct experience with the referent is available and sufficiently strong links to the linguistic stimulus are established.

Taken together, we can thus identify factors leading to the activation of sensorimotor experience during language processing. Previous research has shown that concepts can be linked to sensorimotor experience (Lachmair et al., 2011) and that these connections can be established directly through experience with the referent (Öttl et al., 2017), but also indirectly via language (Günther et al., 2020). 

Whether this information is activated in a given context then depends on to what extent this "sensorimotor schema" is made salient. In cases where the connection to sensorimotor experience is strongly established, which can for example result from direct referent experience, this information is salient by default and will thus be easily activated even if not required by the task (as in the original Stroop task; Stroop, 1935). However, even then the task (i.e., the context in which language processing takes place) has to make this "sensorimotor schema" at least minimally salient: When reducing the saliency of the vertical dimension in the item set (by including words not related to a vertical dimension) as well as the response set (by including horizontal in addition to vertical answers), the action-congruency effect observed by Lachmair et al. (2011) disappears (Dudschig & Kaup, 2017). 

On the other hand, even in cases where the connection to sensorimotor experience is weaker-for example when direct referent experience is missing-it can still be made salient depending on the task at hand and the level of processing: Günther et al. (2020) observe action-congruency words for novel words learned from language alone in a plausibility judgment task for sentences, which necessarily requires meaning access and a simulation of the sentence contents. However, as demonstrated in the present study, indirect connections between words and sensorimotor experience provided via language are not by themselves salient enough to be spontaneously activated.

In principle, it might of course still be the case that the availability of direct referent experience is not the deciding factor at play here. For example, one could assume that sensorimotor simulations can play the role of actual direct experience: If participants consistently had to simulate actions including the newly-learned words, such as You scratch your mende when they learned that a mende is a bionic foot, these simulations could be sufficient to establish strong connections between the words and sensorimotor information. 

Judging the plausibility of such sentences was the test phase employed by Günther et al. (2020). If we imagine an experimental setup where such plausibility judgments form the learning phase instead of the test phase, one might expect to observe the automatic congruency effects that were absent in the present study. We leave such investigations to future research. 

Nevertheless, from a present study, we can conclude that neither (a) a memory consolidation phase via sleep nor (b) a rich linguistic learning context results in any measurable automatic activation of the sensorimotor meaning aspect of words learned from purely linguistic experience.

Acknowledgments We thank Emmanuel Vrachimis for his contribution to the construction of the item material, and our student assistants for their help in the data collection. All data, analysis scripts, and material are available at https://osf.io/vxrhn.

Funding Open Access funding enabled and organized by Projekt DEAL.

Declarations

Conflict of interest All authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical approval This study was funded by a Research Fellowship (no. 392225719) from the German Research Foundation (DFG), awarded to Fritz Günther, and by the Collaborative Research Centre 833 (SFB 833) "The Construction of Meaning"/Z2 project appointed to Sigrid Beck and Barbara Kaup by the DFG. All procedures performed in studies involving human participants were by the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards, and approved by the department's ethical committee (Kommission für Ethik in der psychologischen Forschung), Az: Kaup_2018_0907_135. Informed consent was obtained from all individual participants included in the study.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. 

The images or other third-party material in this article are included in the article's Creative Commons license unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

References

Baayen, R. H., & Milin, P. (2010). Analyzing reaction times. International Journal of Psychological Research, 3(2), 12–28.

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278.

Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 637–660.

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.

Borghi, A. M., Glenberg, A. M., & Kaschak, M. P. (2004). Putting words in perspective. Memory & Cognition, 32, 863–873.

Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436.

Brysbaert, M., Mandera, P., McCormick, S. F., & Keuleers, E. (2019). Word prevalence norms for 62,000 English lemmas. Behavior Research Methods, 51, 467–479.

Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How many words do we know? Practical estimates of vocabulary size are dependent on word definition, the degree of language input, and the participant's age. Frontiers in Psychology, 7, 1116.

Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82, 207–428.

de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47, 1–12.

Dudschig, C., de la Vega, I., De Filippis, M., & Kaup, B. (2014). Language and vertical space: On the automaticity of language action interconnections. Cortex, 58, 151–160.

Dudschig, C., de la Vega, I., & Kaup, B. (2014). Embodiment and second language: Automatic activation of motor responses during processing spatially associated L2 words and emotional L2 words in a vertical Stroop paradigm. Brain and Language, 132, 14–21.

For more information:1950477648nn@gmail.com