Intra-individual Variability in Task Performance After Cognitive Training Is Associated With Long-term Outcomes in Children Part 2

2.7.2 Adolescent Brain and Cognitive Development (ABCD) Study

The ABCD study (https://abcdstudy.org, data release version 3.0) is a longitudinal, multicentre study of children’s cognitive and neurobiological development starting from age 10, with data on 11,878 children. This study includes a wide range of standardized questionnaires and interviews covering both general well-being and clinical measures. In addition, participants perform several cognitive tasks, including an Nback task (Casey et al., 2018). We tested if intra-individual variability, once again quantified as the coefficient of variation in response times, during the N-Back task was associated with scores on the Child Behavioural Checklist (CBCL, Achenbach, 1991). 

The N-Back task is a task that is widely used to test and train working memory. In the N-Back task, participants need to remember the previous stimulus to answer the question correctly in the subsequent task. In recent years, many studies have shown a close relationship between such tasks and memory function.

The study found that after N-Back task training, participants' working memory ability will improve. Working memory plays a very important role in daily life. It can help us process information and make decisions better, and at the same time help us better understand complex information and solve problems better.

Additionally, the N-Back task helps improve attention and cognitive flexibility. By training working memory, participants' brains can better adapt to changing environments and tasks, and better cope with complex problems and stress.

In general, the relationship between the N-Back task and memory is very close. Just as we need to exercise our bodies to maintain good health, we also need to train our brains to maintain excellent cognitive and memory functions. By actively engaging in the N-Back task, we can enhance our working memory, improve our attention and flexibility, and thereby improve our cognitive abilities and thinking skills. Therefore, we should actively engage in N-Back tasks to maintain a healthy brain and excellent cognitive function. It can be seen that we need to improve our memory. Cistanche deserticola can significantly improve memory, because Cistanche deserticola can also regulate the balance of neurotransmitters, such as increasing the levels of acetylcholine and growth factors. These substances are very important for memory and learning. In addition, meat can also improve blood flow and promote oxygen delivery, which can ensure that the brain receives sufficient nutrients and energy, thus improving brain vitality and endurance.

Click know 10 ways to improve memory

The CBCL is a measure of current behavioral and psychopathological symptoms with high correspondence to the SDQ when both scales are applied to the same individuals. The ABCD study includes the CBCL, but not SDQ. We also tested for potential relationships between intra-individual variability and body mass index (BMI) scores. We chose BMI as an additional translational measure because it is robustly associated with physical, cognitive, and socioeconomic well-being. We used only those participants whose performance in the N-Back task was deemed adequate by the ABCD study’s established QA procedure (overall response accuracy for 0-back or 2-back >60%) at all three currently available time points, and whose BMI fell between the 1st and 99th percentile (BMI percentiles = 13.3 and 35.0, respectively). We restricted the range for BMI because there was a small minority (<2%) of children with extremely low or high BMI values; however, robustness checks including all BMI values yielded similar results. The final sample size for our analyses was 8,522 children.

3 RESULTS

3.1 Behavioural results

Children in the WMT did not differ from those in the CMP at baseline. Before the start of any training program, all participants were assessed using a battery of tests that included general intelligence (a modified version of the Raven Matrices), working memory (visual, spatial), inhibition (Go-NoGo task), school performance (including reading, arithmetic, geometry, etc) and psychological well-being screening measures (SDQ). Statistical comparisons between the two groups show that they did not differ in any of these baseline measures (Table S1). While the two groups showed no differences in any measure at baseline, including cognitive tasks measuring working memory and attention skills, we do not have baseline measures on the N-Back and Flanker tasks reported in the following section.

3.2 Accuracy and response-time variability findings

Overall, the group of children randomly assigned to undergo adaptive WMT performed more accurately and with less trial-to-trial variability in response times during the N-Back and Flanker tasks than those in the CMP (see Supplementary Tables S2 and S3 for the full set of descriptive statistics and results). The WMT group responded more accurately in the Flanker task across both the congruent and incongruent trials. In the N-Back task, children in the WMT group were more accurate than those in the CMP on low working memory trials (0-1 back), but the two groups did not significantly differ on high working memory trials (2-3 back).

In addition to better accuracy, children who received adaptive WMT also showed less intra-individual variability in response times than children in the CMP group (Supplementary Tables S2 and S3). We computed the intra-ICV as the intra-individual RT standard deviation/intraindividual RT mean. Children in the WMT group used external mice during the training intervention rather than the MRI-compatible button-box used during both the Flanker and N-back tasks, and thus familiarity with the response device cannot be a source for differences in ICV across groups.

3.3 Diffusion Decision Model analyses

We fit a DDM model to the children’s behavior in the Flanker task to determine the mechanisms leading to differences in response time variability between the treatment groups. Using the DDM, we can determine if response time variability is driven by (1) differences in the non-decision time; (2) differences in the boundary or threshold determining when there is sufficient evidence to make one response versus the other (often interpreted as response caution); and/or (3) the drift rates (i.e., how quickly and robustly evidence is accumulated). We separated the drift rate into two components to measure children’s sensitivity to the relevant information from the target compared to the irrelevant information from the flankers. The fits are summarized in Table 3 and show that children in the WMT group were more sensitive to the information carried by the target fish (i.e., its direction) relative to the distractor fish (posterior probability = 0.992) and utilized a higher response threshold (posterior probability = 0.952) than children in the CMP. These DDM results are consistent with better attention to task-relevant features following WMT.

We also simulated responses from the fitted diffusion decision model to test if it could reproduce the patterns of response time variability observed in the Flanker task. To generate simulated responses in the Flanker task, we used each participant’s best-fitting DDM parameters. We then compared the simulated response times across groups and found that the RTs were less variable for simulated agents using parameters from the WMT participants than for simulated agents based on CMP children’s parameters (Table S4). Thus, the fitted DDM can both explain and generate different levels of response time variability in the two groups.

In addition to fitting the DDM, we also conducted a post-hoc test fitting the ex-gaussian distribution to each child’s response times. Although the ex-gaussian model does not allow for the same type of mechanistic inferences as the DDM, we fit and report it to facilitate comparison with previously published papers using this method to quantify intra-individual variability in RT. These ex-gaussian results are consistent with the ICV and DDM results and indicated that the standard deviation (sigma) and exponential (tau) parameters differed between the working memory trained and CMP, but there was no significant difference in the means (mu) of the response time distributions (Table S5). In other words, more variable individuals were not reliably faster or slower to respond overall. Rather they were more inconsistent in the way they executed their responses. Intra-individual variability was highly correlated across the N-Back and Flanker tasks (r = 0.65, p = 0.0008, 95% CI [0.32, 0.84]).

Together, the pattern of behavioral results across both cognitive tasks and several complementary forms of the analysis suggest that the adaptive WMT intervention may have increased children’s ability to engage and maintain attention on task-relevant information in a domain-general manner soon after the five weeks of training were complete.

3.4 fMRI results

Along with better accuracy, the WMT group showed increased activation compared to the CMP in brain regions that are part of attention and control networks during the low working memory trials. These included portions of front-striatal-thalamic systems such as the right caudate, putamen, pallidum, thalamus, inferior middle and superior frontal gyri, the dorsal anterior cingulate, and the supplementary motor cortex (Table S6, Figure 1). Consistent with the behavioral findings of similar accuracy in the high working memory condition, there were no significant differences in the BOLD signal across groups during the high working memory trials. We did not detect any significant differences in activity as a function of WMT during the Flanker task.

In addition, we found that task-related BOLD signal levels in regions that showed greater activity in the group (see Figure S2) also correlated with the intra-individual coefficients of variation and/or accuracy on the N-back task across all participants (Figure 1, bottom row). Some relation to accuracy and the intra-ICV in these regions is to be expected given that there are group differences in intra-individual variability. However, activity in the dorsal striatal functional ROI, encompassing dorsal caudate and putamen, was significantly associated with intra-individual variability even after accounting for the effects of the WMT condition (coef = −0.25, p = 0.004; Table 4). There were similar, though not significant, trends in the dorsolateral prefrontal cortex (dlPFC) for intra-individual variability, and in the anterior cingulate cortex/supplementary motor area for accuracy (Table 4).

3.5 Associations between post-training cognitive task performance and follow-up measures in the fMRI sample

Response time variability explained significant additional variance in future SDQ scores (standardized coef. = 0.32 ± 0.14), geometry (standardized coef. = −0.66 ± 0.23), and reading (standardized coef. = −0.32 ± 0.14), after accounting for baseline scores in those measures and IQ (Table S7). Thus, response time variability after 5 weeks of training was associated with future improvement in SDQ scores and academic skills that were not yet evident in direct tests of those skills at the same time point. In contrast, no post-training cognitive task performance or baseline measures were significantly associated with future arithmetic scores. Thus, in our sample, the intra-individual variability in response times measured right after the intervention correlated with future performance in the same academic domains that Berger and colleagues previously found to be improved 1 year after WMT in an independent sample.

3.6 Conceptual replication of the association between post-training ICV and follow-up measures in the BFHSW sample

Consistent with our results in the fMRI sample, changes in ICV (post versus pre-intervention) during the Go/Nogo task were also associated with improvement in geometry and reading skills 1 year after WMT in the independent BFHSW sample (Tables 1 and 2), although the effect for reading does not survive Bonferroni correction for multiple comparisons. Note that our regression specification includes regressors for both baseline (W1) and post-training (W2) Go/Nogo ICV, and in this specification, the coefficient for W2 Go/Nogo ICV represents the effect of the change in performance between W1 and W2. The same holds for the use of future academic skill scores as the dependent variable when including the baseline score as an independent variable in the regression.

Interestingly, while N-back ICV was significantly different between WMT and control groups just after training, significant differences in Go/Nogo ICV did not appear until 12 months after training (Table 2). This delayed emergence of significant improvements in Go/Nogo ICV is consistent with the delayed emergence of Go/Nogo accuracy improvements reported by Berger et al. (2020). Even though they did not yet significantly differ across treatments, changes in Go/Nogo ICV from baseline to post-training still predicted future improvements in performance. A relevant question for future research is to determine which types of cognitive tasks (e.g., working memory, response inhibition, etc.) are best suited to evaluate intra-individual variability in response time or accuracy to forecast the emergence of far transfer benefits following working memory or other training regimes.

3.7 The association between ICV and measures of well-being generalizes to the ABCD study

We used data from the first three waves of the longitudinal ABCD study to test whether the relationship we observed in our fMRI sample between response-time variability during the N-back task and measures of well-being was generalized to an independent and larger set of children. We used the data on BMI and scores on the Child Behavioral Checklist (CBCL) as our measures of well-being in the ABCD study. Unlike our fMRI and the BFHSW studies, the ABCD study does not include a WMT intervention. Therefore, we used the ABCD data to test if there was a significant association between N-back ICV and current well-being and, if so, whether this relationship holds over the first- and second-year follow-ups in this longitudinal study.

We fit and compared Hierarchical Bayesian regression models that assumed the association between N-back ICV at baseline and BMI, or CBCL, at baseline, 1- and 2-year follow-up was either stable or decreased over time. Concretely, we tested whether regression models allowing for an interaction between baseline N-back ICV values and assessment wave (i.e., the explanatory power of ICV could decrease or increase) were better than models assuming a fixed association between baseline N-back ICV values and well-being at all waves. The baseline coefficients were the same in both the fixed and interaction models (Table 5) and indicated that greater variability in response times during the N-back at baseline was associated with decreased well-being in terms of both baseline BMI (standardized coefficient = 0.02, posterior probability >0.999) and CBCL (standardized coefficient = 0.03, posterior probability >0.999) scores, consistent with the findings in the fMRI sample.

We compared the fixed and interaction models using leave-one-out cross-validation with Pareto-smoothed importance sampling (PSISLOO, Vehtari et al., 2017). The model comparisons modestly favored the simpler fixed model without follow-up wave interactions when explaining both BMI (difference in expected log pointwise predictive density (elpd) for the interaction model = −2.3, standard error (SE) of the difference = 1.6) and CBCL (elpd difference = −2.6, SE = 1.9). Moreover, neither interaction model suggested a decrease in the explanatory power of baseline N-back ICV for well-being at 1- or 2-year follow-up visits relative to baseline. If anything, there was a slight increase in the regression coefficient for ICV between baseline and year 2 when seeking to explain BMI. Thus, the results from the ABCD data show that the relationship between N-back ICV and children’s well-being generalizes across measures of well-being (SDQ, CBCL, BMI), and the explanatory power of N-back ICV persists through at least 2 years of experience and development in the absence of any cognitive training intervention.

4 DISCUSSION

The present study examined how the neurocognitive mechanisms underlying the short-term impact of adaptive WMT in primary school children relate to training benefits that emerge months or years after training. Overall, our results suggest that in addition to working memory itself, there may be concurrent benefits to selective and sustained attention during or directly after five weeks of training. We show that intra-individual variability in response times during several different cognitive tasks can be used to detect short-term training effects in children and that such measures may be indicative of the persistence and/or emergence of far-transfer benefits months to years after training is completed.

Our findings indicate that better attention is among the immediate results of adaptive WMT. Working memory and attention processes are thought to be closely linked and interdependent (Astle & Scerif, 2011; D’Esposito & Postle, 2015; Engle, 2018; Eriksson et al., 2015; Gazzaley & Nobre, 2012; Unsworth & Robison, 2017; Wass et al., 2012). Although they have different primary targets, the Flanker, Go/Nogo, and N-Back tasks require the ability to maintain attentional focus throughout the task (sustained attention), and to identify the target stimuli and filter out or inhibit responses to non-target stimuli (selective attention). At the neural level, differences between the WMT and CMP groups were found in the striatum as well as the lateral and medial prefrontal cortices, which are brain regions that, among other things, support selective and sustained attention functions (Frank et al., 2001; Mcnab & Klingberg, 2008; Zanto et al., 2011). 

These neural differences were accompanied by better signal detection performance (i.e., higher d-prime), reduced intra-individual variability in response times, and more efficient accumulation of relevant information (i.e., higher DDM drift rates) in children who received adaptive WMT. All of these behavioral measures are related to and dependent on attention. Therefore, taken together, our neural and behavioral results suggest that the benefits of the program used in this study are at least partially mediated by more effective attention processes leading to consistent and effective responses to task-relevant information and reduced processing of irrelevant, distracting stimuli.

These results lend further support to theories of the mechanisms underlying training benefits. A meta-analysis of previous training studies concluded that the Cogmed-RM adaptive WMT program has effects on attention in daily life (Spencer-Smith & Klingberg, 2015). The more effective attention processes we detected at the end of the training are consistent with previous results and theories about the basis of far-transfer effects following cognitive training as well (Dahlin et al., 2008; Greenwood & Parasuraman, 2016; Morrison & Chein, 2011). Specifically, these far-transfer benefits occur when the trained and transfer skills share common fundamental cognitive processes. Given the important role of attention as a prerequisite to many cognitive processes, it could serve as a basis for far-transfer effects following WMT.

Recently, benefits of the adaptive WMT in school-age children, relative to standard classroom instruction, have been shown to emerge over 6–12 months (Berger et al., 2020). Initial improvements in attention may serve as a scaffold for later changes in higher cognitive processes that facilitate better school performance. Our current results suggest that attention functions might be among the first to improve from this type of training and that later emerging benefits to academic skills and general well-being are associated with immediate improvements in attention processes. It is not surprising that WMT would also influence attention control (e.g., selective attention, sustained attention, or goal-directed attention reallocation) given that these processes are postulated to be pre-requisites for the successful implementation of working memory (Astle & Scerif, 2011; D’Esposito & Postle, 2015; Eriksson et al., 2015; Gazzaley & Nobre, 2012; Unsworth & Robison, 2017; Wass et al., 2012). 

There is also evidence that the associations between working memory capacity and various cognitive and academic skills are partially mediated by a common reliance on attention control (Engle, 2018; Fukuda & Vogel, 2011; Unsworth & Robison, 2017). Given the apparent role of attention processes in mediating the far transfer of training effects, it is important to measure these processes when assessing the efficacy of ofWMT and other forms of cognitive training. It will also be important for future studies to directly compare adaptive WMT versus interventions that use adaptive mechanisms to train attention, inhibition, or other cognitive abilities. Our current results indicate that metrics quantifying intra-individual variability in response time will be useful in determining the relative short and long-term efficacy of different training regimes.

The ability of intra-individual variability metrics to detect individual differences in attention control could explain the association we find between ICV and the future emergence of benefits to academic skills and general well-being after WMT. Intra-individual response time variability metrics are sensitive and reliable measures of individual differences in attention control processes (MacDonald et al., 2009; Saville et al., 2011). They are often used as an index of an individual’s attention allocation efficiency or degree of fluctuation in attention control during task performance (Bellgrove et al., 2004; Isbell et al., 2018; Kelly et al., 2008; Stuss et al., 2003; Unsworth, 2015). Intra-individual variability has been linked with cognitive control measures in healthy children and adults, and the variability in response times measured in one task is correlated with working and long-term memory or intelligence measured in separate tasks (Bellgrove et al., 2004; Isbell et al., 2018; Larson & Saccuzzo, 1989; Montez et al., 2017; van Belle et al., 2015). It also differs between healthy individuals and those with attention deficit hyperactivity disorder (ADHD) (Castellanos et al., 2005; Geurts et al., 2008; Karalunas et al., 2014; Kofler et al., 2013; van Belle et al., 2015). However, increased response time variability is not unique to ADHD and is seen in various psychiatric and neurological disorders (e.g., traumatic brain injury, dementia, and schizophrenia), in which attention deficits may play an important, though less prominent, role (Geurts et al., 2008; Haynes et al., 2017; Ilg et al., 2018; Kofler et al., 2013; MacDonald et al., 2006). 

Increased intra-individual variability is commonly observed in non-affected relatives as well as patients, indicating that it may capture shared genetic or environmental risk factors for current and future psychopathologies (Adleman et al., 2014; Ilg et al., 2018; Karalunas et al., 2014; Kuntsi et al., 2010; Stuss et al., 2003). A recent review by Haynes et al. highlights several longitudinal studies in older adults that have shown that the intra-individual variability in response times is associated with future levels of cognitive impairment and mortality (Haynes et al., 2017). Thus, our current results, together with the existing body of work indicate that intra-individual variability measures are sensitive not only to current cognitive and neurological function, but also associated with the future stability, improvement, or decline of those functions.

We found that intra-individual variability metrics can detect short-term efficacy and are indicative of the emergence of longer-term benefits of working memory interventions aimed at improving cognitive skills and academic performance in children. We could detect significant differences between trained and untrained groups in intraindividual response time variability during cognitive tasks probing working memory and attention (N-Back and Flanker) directly after five weeks of WMT, while significant improvements in variability during a response inhibition task (Go/Nogo) did not emerge until months later. Nevertheless, consistent with their ability to forecast cognitive decline in the elderly, we found that measures of the intra-individual variability in the N-back and Go/Nogo tasks computed at the end of training were associated with improvements in academic skills and general well-being in children up to 1 year after training. Across both tasks, lower post-training variability was related to better future scores on tests of academic skills and strengths/weaknesses in the classroom and social behavior. The results from the ABCD data are also consistent with the idea that measures of performance variability are associated with both current and future well-being, specifically behavioral problems and BMI.

Our results suggest that measures of intra-individual variability are useful in evaluating intervention efficacy. However, several important questions still need to be addressed. For example, can we use intra-individual variability metrics to determine when an individual has received a sufficient dose of the training intervention? If so, then we could tailor the amount of training to each person to improve the cost-benefit trade-offs inherent in any training program. Another key question our findings raise is what types of tasks (e.g., those targeting working memory, attention, task-switching, etc.) and measures of intra-individual variability are best suited to assessing the short and long-term outcomes of cognitive training. Previous work has quantified intra-individual variability in response times in several different ways (Geurts et al., 2008; Karalunas et al., 2014; van Ravenzwaaij et al., 2011). We found significant differences in response time variability between training groups in selective attention (Flanker), working memory (N-Back) tasks, and response inhibition tasks (Go/Nogo) using several complementary measures of variability. However, there may be differences in how well the different measures of variability and/or task designs predict the emergence of benefits to specific areas of academic performance or general well-being in the longer term. This question will be important to address in future studies that collect and compute multiple longitudinal measures in large samples of participants.

We note a few potential limitations of this work. One potential limitation is that familiarity with the use of computer devices may underlie the differences in response time variability between control and training groups. However, we think this is unlikely given that in the fMRI study, the training group implemented the responses during the training with a mouse whereas inside the scanner children responded using an MRI-adapted button box. Moreover, significant improvements in performance variability in the Go/Nogo task did not emerge until months after training suggesting the difference was due to further development in cognitive skills rather than simple action familiarity or motor skills. Secondly, our initial fMRI study did not include the NBack or Flanker at baseline so we could not control for baseline performance in those exact tasks. 

The lack of between-group differences in any other pre-intervention measures of working memory or attention suggests that the probability of randomization failures leading to training-independent differences in working memory or attention is very low. The fact that we can replicate our results from the fMRI sample in the BFHSW sample using a Go/Nogo task measured at baseline and post-intervention further indicates that randomization failure is an unlikely cause for our original results. Lastly, an important limitation is that our current data cannot tell us whether these effects are specific to adaptive WMT per se or if other forms of cognitive training might lead to similar benefits. Our results on RT variability suggest that some of the initial training benefits are mediated by improvements in attention control. While attention control and working memory are interrelated, it should be possible to train attention control using cognitive tasks that make limited demands on working memory to better distinguish between the two skills. Determining the best types and forms of cognitive training, and potentially how to customize the training for individuals of different ages or abilities is an important goal for future research.

5 CONCLUSION

Effective means of enhancing cognitive abilities have been a longstanding goal in many disciplines. Our current work adds to the existing evidence that adaptiveWMT can significantly benefit school-aged children (Berger et al., 2020; Jones et al., 2020; Karbach et al., 2015; Titz & Karbach, 2014; Wass et al., 2012). Moreover, it provides additional insights into the mechanisms underlying these benefits. Together with the recent findings of Berger et al. (2020), it also highlights the importance of including long-term follow-ups in any evaluation of training efficacy. In addition to long-term follow-up data, we demonstrate the utility of using response time variability metrics as an immediate indicator of intervention success. The practical relevance of such an immediate assessment tool should not be overlooked, as it could potentially allow for tailoring training interventions in terms of duration or content without needing to wait for years for follow-up data to determine whether long-term benefits will emerge.

REFERENCES

1. Achenbach, T.M. (1991) Manual for the child behavior checklist/4-18 and 1991 profile. Department of Psychiatry, University of Vermont.

2.Adleman, N. E., Yi, J. Y., Deveney, C. M., Guyer, A. E., Leibenluft, E., & Brotman, M. A. (2014). Increased intrasubject variability in response time in unaffected preschoolers at familial risk for bipolar disorder. Psychiatry Research, 30, 219(3), 687–689.

3. Ali, S., Macoun, S. J., Bedir, B., & MacDonald, S. W. S. (2019). Intraindividual variability in children is related to informant ratings of attention and executive function. Journal of Clinical and Experimental Neuropsychology, 41(7), 740–748.

4. Astle, D. E., Barnes, J. J., Baker, K., Colclough, G. L., &Woolrich, M.W. (2015). Cognitive training enhances intrinsic brain connectivity in childhood. The Journal of Neuroscience: The official journal of the Society for Neuroscience, 35(16), 6277–6283.

5. Astle, D. E., & Scerif, G. (2011). Interactions between attention and visual short-term memory (VSTM): What can be learned from individual and developmental differences? Neuropsychologia, 49(6), 1435–1445.

6.Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: A meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377.

6.Baddeley, A. (1996). The fractionation of working memory. Proceedings of the National Academy of Sciences of the United States of America, 93(24), 13468–13472.

7.Baddeley, A. (2010). Working memory. Current Biology, 20(4), R136–R140.

8.Bellgrove, M. A., Hester, R., & Garavan, H. (2004). The functional neuroanatomical correlates of response variability: Evidence from a response inhibition task. Neuropsychologia, 42(14), 1910–1916.

9.Berger, E. M., Fehr, E., Hermes, H., Schunk, D., & Winkel, K. The impact of working memory training on children’s cognitive and noncognitive skills. SSRN Electronic Journal, https://doi.org/10.2139/ssrn.3622985

10.Bogg, T., & Lasecki, L. (2014). Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence. Frontiers in Psychology, 5, 1589.

For more information:1950477648nn@gmail.com