A Resource-rational Model Of Human Processing Of Recursive Linguistic Structure Part 1

A major goal of psycholinguistic theory is to account for the cognitive constraints limiting the speed and ease of language comprehension and production. 

There is indeed a certain relationship between limiting language and memory, but it does not mean that language can strictly limit our memory. The main impact of language limitations is on the expressive power of language itself, not on our memory.

Language is a tool for expressing thoughts and feelings through which we communicate and communicate. Different languages have different vocabulary, grammar, and expressions, and these characteristics together shape our knowledge and understanding of the world. Therefore, the expressive ability of language is directly related to our way of thinking, and they jointly participate in the formation and development of human thinking.

However, our memory is not limited by language. Memory ability is a powerful tool that our brain has, and we can improve our memory through different ways and methods. For example, using different memory techniques, repeated training and practice, using multiple sensory stimulations, etc., these methods can stimulate our brains and improve our memory.

Some studies show that speaking multiple languages can improve our memory. Learning a new language is a challenging learning process that stimulates our brains to become more flexible and adaptable. Being proficient in multiple languages can also promote our multicultural vision and awareness.

In short, some people may think that language has limitations on our thinking and memory, but this concept is wrong. We should actively explore our potential and improve our thinking and memory abilities through continuous innovation, learning, and practice. Using multiple languages is a useful exploration and attempt, which can promote our thinking progress and personal growth. It can be seen that we need to improve memory, and Cistanche deserticola can significantly improve memory because Cistanche deserticola is a traditional Chinese medicinal material that has many unique effects, one of which is to improve memory. The efficacy of minced meat comes from the various active ingredients it contains, including acid, polysaccharides, flavonoids, etc. These ingredients can promote brain health in various ways.

Click know 10 ways to improve memory

Wide-ranging evidence demonstrates a key role for linguistic expectations: A word's predictability, as measured by the information-theoretic quantity of surprise, is a major determinant of processing difficulty. 

But surprisingly, standard theories, fail to predict the difficulty profile of an important class of linguistic patterns: the nested hierarchical structures made possible by recursion in human language. 

These nested structures are better accounted for by psycholinguistic theories of constrained working memory capacity. However, progress in theory unifying expectation-based and memory-based accounts has been limited. 

Here we present a unified theory of a rational trade-off between precision of memory representations with ease of prediction, a scaled-up computational implementation using contemporary machine learning methods, and experimental evidence in support of the theory's distinctive predictions. 

We show that the theory makes nuanced and distinctive predictions for difficulty patterns in nested recursive structures predicted by neither expectation-based nor memory-based theories alone. 

These predictions are confirmed 1) in two language comprehension experiments in English, and 2) in sentence completions in English, Spanish, and German. 

More generally, our framework offers computationally explicit theory and methods for understanding how memory constraints and prediction interact in human language comprehension and production.

Language expresses recursive thoughts via linear strings of words (1). Therefore, a central part of language comprehension is recovering a hierarchical structure from a linear sequence. 

While we do this seemingly without effort, it has long been observed that humans' ability to do so can run against the limitations of short-term memory (2). Such limitations are of central importance to understanding the nature of human language processing and have been an important subject of study (3–8). 

Human processing limitations often give rise to measurable, localized differences in comprehension difficulty between otherwise similar sentences, and modeling those difficulty differences has been a key aim of psycholinguistic research (3–8). 

However, it has proven challenging to develop a unified account of what makes different sentences easier or harder for humans to comprehend.

Research has identified two seemingly disparate perspectives on what makes sentences hard to comprehend. Expectation-based models (9, 10) describe how context generates expectations about likely future input. 

According to such models, words are harder to process when they are harder to anticipate from the preceding context. In contrast, memory-based models hold that the difficulty of processing stems from limits on the ability to store representations of preceding context and to retrieve and integrate them with new input (4, 5, 7). 

Both perspectives are supported by substantial bodies of empirical evidence (11– 16), and it has remained an open question how they can be theoretically and empirically reconciled (14–16).

Here, we develop a theory and implemented model reconciling expectation-based and memory-based theories, building on recent research that has proposed a key role for noise and uncertainty in modeling human mental representations of linguistic input (17–19). 

Whereas traditional models of language processing generally assume veridical context and input representations-the problem of sentence-level comprehension is cast as one of analyzing a known sequence of words to determine its structure and meaning and to predict future input-"noisy channel" language processing theory treats these representations as uncertain, and hypothesizes that analysis and prediction in human language processing approximates normative principles of Bayesian inference given these uncertain representations (20, 21). 

These ideas have led to the proposal of unifying expectation-based and memory-based theories of processing difficulty through lossy-context surprisal (22). Lossy-context surprisal posits that human processing difficulty is determined by expectations derived not from veridical context but from probabilistic inference over imperfect memory representations of the context. 

In principle, this approach could account for the predictions of both expectation-based and memory-based models: Words are easy to process when they are easy to anticipate-as predicted by expectation-based models- but if the relevant contextual information is poorly represented in memory, upcoming words may be difficult to anticipate correctly, yielding processing difficulty as predicted by traditional memory-based theories. 

However, to date, many parts of this theory remain to be specified. The theory lacks an implemented specification of which aspects of the preceding context are prone to memory loss, which is key to deriving testable predictions. Ideally, this specification should be based on deeper theoretical principles. 

Furthermore, no scaled implementation of noisy channel processing, which is necessary to make fine-grained predictions on the difficulty profiles of specific sentences, has been available.

In this work, we present theoretical and empirical advances that address these limitations. On a theoretical level, we propose a resource-rational model (23) of fine-grained memory representations, based on the hypothesis that memory representations are optimized to minimize expected downstream processing effort given cognitive resource constraints. 

Combining this idea with lossy-context surprisal as a processing difficulty metric leads to wide-ranging empirical predictions. 

To evaluate those predictions and understand them in detail, we implement the proposed model using contemporary neural network modeling and fit it on large-scale text data, enabling the theory to make detailed predictions regarding human comprehension behavior for arbitrary natural language input.

Our theory derives predictions for difficulty patterns in human processing of recursive structures that neither expectation-based nor memory-based theories individually could account for. Recursive structures, in particular, cases of center embedding where sentences are nested inside one another, are crucial for psycholinguistic theory because they reveal human limitations in processing the hierarchical structures of language (2–4, 24–26). Consider Fig. 1A. 

In these sentences, varying numbers of sentences are embedded within each other. More center embedding leads to structures that are more difficult to process: Whereas items 1 and 2 in Fig. 1A are readily understood, item 3 is considerably harder. 

Adding further levels of embedding would increase difficulty to the point of incomprehensibility. More levels of center embedding are rarer in language use (27), so purely expectation-based theories correctly predict that they are difficult overall, but fail to predict where this difficulty manifests in human processing: when exiting the embedding, at the word "was" in the examples of Fig. 1A. 

If context were veridically represented and used to predict upcoming input, then exiting the embedding at this point should be exactly what is expected, and easy for human language processing.

Some memory-based theories predict that exiting the embedding is difficult, on the basis that the complexity of the preceding context makes retrieval of the correct site for structural integration challenging (4, 5, 7).
Here, however, we present experimental work showing that this difficulty is modulated by fine-grained differences in the context: for example, changing report to fact in the sentences of Fig. 1A turns out to make exiting the center embedding easier. This phenomenon is not predicted by existing memory-based theories.

Our model is capable, in principle, of accounting for all these patterns. When memory representations are imperfect, rational comprehenders should reconstruct the context based on their knowledge of the statistics of the language. 

Comprehenders' structural expectations of inputs should thus be biased toward contexts with high a priori probability that are similar in form to the true contexts. In a resource-rational model, this will particularly affect variants that differ in words that are normally easy to reconstruct from other parts of the context, such as high-frequency function words. 

For instance, we expect that a context such as "the report that the doctor annoyed the patient... " will compete with variants such as "the report by the doctor annoyed the patient...," where "annoyed" is the verb belonging to the initial noun "report." 

For such a nonveridical variant, no third verb is expected. Rational comprehenders with imperfect memory should thus be more likely to expect the final verb when such nonveridical versions with lower embedding depth have a lower a priori probability. 

In contrast, when nonveridical variants have high a priori probability, comprehenders should not expect the final verb, and comprehension will be disrupted when it is encountered.

For more information：1950477648nn@gmail.com