Predicting Alzheimer’s Disease Progression Using Multi-modal Deep Learning Approach

Garam Lee^1,2, Kwangsik Nho^3,4 et al

Alzheimer’s disease (AD) is a progressive neurodegenerative condition marked by a decline in cognitive functions with no validated disease-modifying treatment. It is critical for timely treatment to detect AD in its earlier stage before clinical manifestation. Mild cognitive impairment (MCI) is an intermediate stage between cognitively normal older adults and AD. To predict conversion from MCI to probable AD, we applied a deep learning approach, a multimodal recurrent neural network. We developed an integrative framework that combines not only cross-sectional neuroimaging biomarkers at baseline but also longitudinal cerebrospinal fluid (CSF) and cognitive performance biomarkers obtained from the Alzheimer’s Disease Neuroimaging Initiative cohort (ADNI). The proposed framework integrated longitudinal multi-domain data. Our results showed that 1) our prediction model for MCI conversion to AD yielded up to 75% accuracy (area under the curve (AUC)=0.83) when using the only single modality of data separately; and 2) our prediction model achieved the best performance with 81% accuracy (AUC=0.86) when incorporating longitudinal multi-domain data. A multi-modal deep learning approach has the potential to identify persons at risk of developing AD who might benefit most from a clinical trial or as a stratification approach within clinical trials.

Contact: joanna.jia@wecistanche.com / WhatsApp: 008618081934791

cistanche extract powder pdf for AD

Alzheimer’s disease (AD) is an irreversible, progressive neurodegenerative disorder characterized by abnormal accumulation of amyloid plaques and neurofibrillary tangles in the brain, causing problems with memory, thinking, and behavior. AD is the most common form of dementia with no validated disease-modifying treatment. An estimated 5.7 million Americans are living with AD in 2018. By 2050, this number is projected to rise to nearly 14 million1. Currently available treatments decelerate only the progression of AD and no treatment developed so far can cure a patient who is already in AD. Thus, it is of fundamental importance for timely treatment and progression delay to develop strategies for detection of AD at early stages before clinical manifestation. As a result, the concept of mild cognitive impairment (MCI) was introduced. MCI, a prodromal form of AD, is defined to describe people who have mild symptoms of brain malfunction but can still perform everyday tasks. Patients in the phase of MCI have an increased risk of progressing to dementia1–4. Some patients in their MCI stages are converted to AD within a limit of the time window after baseline, while some are not. It has been reported that MCI patients progress to AD at a rate of 10% to 15% per year and 80% of these MCI patients will have converted to AD after approximately six years of follow-up5,6. It is an ongoing topic among AD-related researchers to identify biomarkers that classify patients with MCI who later progress to AD (MCI converter) from those with MCI who do not progress to AD (MCI non-converter).

Various machine learning methods have been applied to identify biomarkers for MCI conversion prediction and improve their performances. Support vector machine (SVM) is one of the methods frequently used for solving classification problems. A lot of studies applied SVM for MCI conversion prediction7–12. Multi-task learning along with SVM was used to identify AD-relevant features, showing 73.9% accuracy, 68.6% sensitivity, and 73.6% specifcity7. For the use of additional subjects, a domain transfer learning method to use auxiliary samples such as AD and cognitively normal older adults (CN) subjects, as well as MCI subjects, showed 79.4% accuracy, 84.5% sensitivity, and 72.7% specifcity8. A linear discriminant analysis (LDA) was used based on cortex thickness data showing 63% sensitivity and 76% specifcity13. Furthermore, the integration of multi-modality data improves the performance for MCI conversion prediction by extracting complementary AD-related biomarkers from each modality. Cerebrospinal fluid (CSF), MRI, and cognitive performance biomarkers were combined, resulting in 68.5% accuracy 53.4% sensitivity, and 77% specifcity14,15. Along with MRI and CSF biomarkers, APOE ε4 status was integrated16.

In this study, in order to predict MCI to AD conversion, we proposed a multimodal recurrent neural network method, a deep learning approach, based on the integration of demographic information, longitudinal CSF biomarkers, longitudinal cognitive performance, and cross-sectional neuroimaging biomarkers at baseline obtained from the Alzheimer’s Disease Neuroimaging Initiative cohort (ADNI). Our proposed deep learning method can incorporate longitudinal multiple domain data and take variable-length longitudinal data to capture temporal features at multiple time points. In particular, non-overlapping samples, as well as overlapping samples from each data, can be used to build a prediction model.

Results

Study participants.

All individuals used in the analysis were participants of the Alzheimer’s Disease Neuroimaging Initiative (ADNI)17,18. The overall goal of ADNI is to test whether serial magnetic resonance imaging (MRI), position emission tomography (PET), other biological markers, and clinical and neuropsychological assessment could be combined to measure the progression of MCI and early AD. Demographic information, raw neuroimaging scan data, APOE genotype, CSF measurements, neuropsychological test scores, and diagnostic information are publicly available from the ADNI data repository (http://adni.loni.usc.edu). Informed consent was obtained for all subjects, and the study was approved by the relevant institutional review board at each data acquisition site (for up-to-date information, see http://adni.loni.usc.edu/wp-content/themes/freshnews-dev-v2/ documents/policy/ADNI_Acknowledgement_List%205-29-18.pdf). All methods were performed in accordance with the relevant guidelines and regulations. In this study, a total of 1,618 ADNI participants aged 55 to 91 were used, which include 415 cognitively normal older adult controls (CN), 865 MCI (307 MCI converter, and 558 MCI non-converter), and 338 AD patients (Table 1).

We used four different types, or modalities of data: demographic information, neuroimaging phenotypes measured by MRI, cognitive performance, and CSF measurements. Demographic information includes age, sex, years of education, and APOE ε4 status. Cognitive performance includes composite scores for executive functioning (ADNI-EF) and memory (ADNI-MEM) derived from the ADNI neuropsychological battery using item response theory as described in detail elsewhere19. CSF biomarkers for AD include amyloid-β 1–42 peptide (Aβ1–42), total tau (t-tau), and tau phosphorylated at the threonine 181 (p-tau). AD-related neuroimaging biomarkers measured by MRI include hippocampal volume and entorhinal cortical thickness.

cistanche GINSENG tubulosa

Experimental setting.

To evaluate the performance and effectiveness of our proposed longitudinal multi-modal deep learning method, we used three schemes and compared their performances (Table 2). In the experiment named “baseline”, 4 modalities data at the baseline visit (cognitive performance, CSF, demographic information, and MRI) were incorporated. In “single modal”, only longitudinal cognitive performance data was used for the predictor (we tried all other single modality, and the performance with cognitive scores was the best). Finally, the four modalities of longitudinal data were combined and used for training the classifier in the experiment marked as “proposed”. Table 3 shows summary statistics of each modality of data and hyperparameters used for training GRUs.

For training our models, subjects in CN and AD groups are used as well as MCI-C and MCI-NC. This approach is motivated by8,10,11,20–22. Tey uses CN and AD subjects for training a classifier such as SVM23 or locally linear embedding (LLE)24, and then the classifier is used for the classification of MCI-C and MCI-NC. In our experiment, CN and AD are used as an auxiliary dataset to pre-train the classifier, and then MCI-C and MCI-NC are also used for training.

We tested the classifier on MCI patients to predict the conversion after Δt from baseline (6, 12, 18, and 24 months) as shown in Fig. 1. Due to the nature of our data the sample size available for training varies over Δt (Fig. 2). For example, if AD occurs early from the baseline visit, then we have relatively fewer training samples because we have a smaller data window to predict on. At each prediction time (Δt), we ran 5-fold cross-validation 10 times in which every fold has the same ratio of MCI-C and MCI-NC subjects. MCI samples were partitioned into 5 subsets, and one subset was selected for testing, while the remaining subsets were used for training.

Comparison of prediction of MCI to AD conversion using cross-sectional data at baseline and longitudinal data. To evaluate the advantage of using longitudinal data, we first compared the performances of two schemes: “baseline” and “proposed” (Figs 3 and 4). Intuitively, data from multiple time points has more information than data at a single time point. Tus, the GRU analyzes the temporal changes in cognitive performance and CSF to extract features (which are not contained in baseline visit data) for the correct MCI conversion prediction. As shown in Table 4(a,b), the prediction model based on longitudinal data shows better performance than the model using only cross-sectional data at baseline. In particular, sensitivity is an important measure for the prediction task in which identifying the true positive rate is crucial25. In the prediction of MCI conversion, a classifier with a higher true positive rate is more applicable for timely treatment.

Comparison of prediction of MCI to AD conversion using single modal and multimodal data. For evaluating the effectiveness of multimodal data integration, we compared the performances of “proposed” and “single modal” experiments. Figure 3 shows the accuracies of “proposed” and models with a single modality of data. We removed the accuracy from the model with demographic data because the prediction performance was too low. The model using cognitive performance was observed to be the most accurate among models that use every single modality of data. Even though the sample size for neuroimaging data was larger than those of cognitive performance and CSF biomarkers (Fig. 3), the model with neuroimaging data showed less accuracy. This is because cognitive performance is longitudinal data which takes advantage of giving relatively closer data records to MCI conversion points. However, a model with cognitive performance shows an extremely high variance of sensitivity for predicting 18 and 24 months. It is observed that model only with cognitive performance not a stable predictor for a long period of prediction while integrating other biomarkers can alleviate the high variance is proposed.

Discussion

We proposed an integrative approach for the prediction of MCI to AD conversion using a deep learning approach, more specifically, a multi-modal recurrent neural network. Our method takes advantage of the longitudinal and multi-modal nature of available data to discover nonlinear patterns associated with MCI progression. To evaluate the advantages of our proposed method, we compared performance outputs from three schemes: “baseline”, “single modal”, and “proposed”. As observed in Fig. 4, “baseline” and “single modal” with cognitive test biomarkers show similar performances over prediction periods. Using longitudinal data or combining multimodal data are effective ways for increasing predictive power thus, it seems natural for combining longitudinal multimodal data (“proposed”) to show the best performance. In Table 5, as predicted further periods, the reliability of performance improvement is lower due to the lack of positive samples. However, specificities of the proposed model showed enhanced performance over competing methods consistently. In addition, the prediction results of our model were compared to those of previous studies with machine learning approaches (Table 6). Our method showed comparable prediction ability even though we had a highly unbalanced ratio of positive and negative samples. Specifically, the sensitivity of our model shows higher performance while specificity is lower. Moreover, Te balanced accuracy26, which is a measure of accuracy considering the sensitivity and specificity shows 0.82 for our model and 0.81 for27.

The biggest advantage of our approach is that irregular longitudinal data can be used. One of the major problems when dealing with longitudinal data is that a preprocessing step is required for handling variable-length of sequential data and missing values. In previous studies, the fixed length of time points was collected by taking data that fell within a certain time window. Additionally, an additional feature extraction phase is required to produce a fixed-size feature representation. In the first training step, separate GRU components make an encoding process, where longitudinal data are transformed into a vector containing AD-sensitive features. Tanks to the structure of GRU, our approach is capable of accepting any irregular length of data as an input without preprocessing.

In addition, our method can make full use of available subjects from each modality for training our classifier. This is a huge advantage in the face of data scarcity. As seen in Fig. 2, the number of subjects with CSF data is the smallest in the overlapping sample. Traditional approaches can use only the overlapping samples while non-overlapping samples were abandoned. In our case, non-overlapping samples contribute to training the individual GRU component it belongs to for better representation learning. Furthermore, additional modality data are easily integrated into the model. Contrary to the kernel-based integration, the concatenation-based integration method can incorporate other domains of data such as multi-modal neuroimaging and genomic data without any prior knowledge. Thus, next, we will integrate multi-modal neuroimaging and genomics data for learning features that might be useful in predicting early MCI to AD conversion.

Although there are some strengths as described above, our approach has some limitations. In the first training step, the input of each modality was transformed into a feature vector that is optimized for MCI conversion prediction only by a single modality. Tus, features that are irrelevant to AD progression with respect to the single modality will be filtered out. However, if there are features that cannot be extracted by a single modality but only can be explained by a combination of multi-modality of data then those are also likely to be filtered out. This is because parameters in GRUs are not updated against the final prediction result. In other words, parameter optimization for the second training step does not affect the parameters in each GRU for feature extraction, thus each GRU cannot learn from the final prediction based on the combined features. To solve this problem, we will link GRUs to logistic regressions at the second step so that GRU learns feature representation from multi-modality as well as a single modality. In addition, we plan to modify the structure of our model making it possible for individual GRU components to extract integrative features. We are currently investigating this possibility as a sequel to this work.

Methods

Recurrent Neural Network.

Recurrent Neural Network (RNN) is a class of deep learning architecture used when sequential data can be considered. In natural language processing (NLP), speech recognition, and anomaly detection in time series, RNN is popularly used for analyzing the sequence of words and time series data28. The advantage of applying RNN is that variable-length sequences can be processed to exploit temporal patterns hidden

in the given sequence. In the sentiment analysis task, for example, the goal is to classify the sentiment (good or bad) of a given sentence. The classifier needs to take a sentence (a sequence of words) as an input, understand the context in it, and return a correct sentiment as an output29. For the prediction task that detects initial diagnosis of heart failure in30, RNN takes time series of electronic health records (EHRs) using a 12 to the 18-month observation window. In these cases where variable-length input should be dealt with RNN is an appropriate candidate to use.

An RNN processes one element of an input sequence at a time and updates its memory state that implicitly contains information about the history of all the past elements of the sequence31. The hidden state is represented as a Euclidean vector (i.e., a sequence of real numbers) and is updated recursively from the input at the given step and the previous value of the hidden state (Fig. 5). Suppose we have N number of subjects, each of which has a sequence … … x xxx { ,,,,, } n n in Tn 1 2 where xin is a data record of n-th sample and the t-the element in a sequence and T is the length of the sequence. The corresponding sequence of output is recursively computed as:

In our model, the last output sequence provided by the RNN is treated as the probability vector for classification, and the cross-entropy loss function (equation (3)) is used to quantify how “far away” our n-th prediction is from the n-th ground truth label in. That is, we choose the optimal parameters ⁎ ⁎ ⁎ W, W, W h x y that minimize the cross-entropy loss of the given data (equation (4)). The algorithm we use to optimize the parameters is Backpropagation Through Time (BPTT)32, which updates the weights in the RNN to minimize the given loss function. However, when the task requires long sequences of input to be processed, training an RNN is difcult33. This is called the long-term dependency problem. Variants of RNN such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) have been developed and practically used to solve this problem34,35. In the proposed model, we use GRU for each modality of data to process multiple time points of the input. The detailed structure of GRU is described in the supplementary.

Multi-modal GRU for MCI conversion prediction.

Our problem can be considered as a sequential data classifcation. The classification objective is to predict whether an individual with MCI at baseline is converted to AD or not using sequence data, which consists of four modalities including cognitive performance, CSF, and MRI biomarkers as well as demographic information. Even though demographic data and MRI biomarkers are not longitudinal data we will consider them as length-one sequential data.

To apply a GRU-based classification algorithm to our problem, we need to design a model that can incorporate the four modalities of data. The main idea of our model is to separately build GRU feature extractors for each modality and integrate the extracted four feature vectors at the end. Our model is comprised of two training steps: (1) learning a single GRU for each modality of data, and (2) learning the integrative feature representation to make the final prediction. At the first training step, a single GRU is trained separately for each modality in which the classification objective is to predict conversion to AD from MCI. Using GRUs is essential to take longitudinal data and transform them into a fixed-size vector. This is quite similar to the approach proposed in36 that maps the input sequence into fixed-length representation. In the second step, MCI conversion is predicted based on the four vectors produced from each GRU component. For merging four-vectors, we select concatenation-based data integration, which is conceptually the simplest method to integrate multiple sources of data into a single vector37. For the final prediction, l1-regularized logistic regression38 is used for the classification between MCI-C and MCI-NC. The overview of our proposed method is illustrated in Fig. 6.

Conclusion

Here, we proposed a multi-modal deep learning approach to study the prediction of MCI to AD conversion using longitudinal cognitive performance and CSF biomarkers as well as cross-sectional neuroimaging and demographic data at baseline. We applied multiple GRUs to use longitudinal multi-domain data and all subjects with each modality data. Our results showed that we achieved the better prediction accuracy of MCI to AD conversion by incorporating longitudinal multi-domain data. A multi-modal deep learning approach has the potential to identify persons at risk of developing AD who might benefit most from a clinical trial or as a stratification approach within clinical trials.

where can i buy cistanche supplement to prevent and treat Alzheimer's disease, click here for a sample

References

1. Alzheimer’s, A. 2015 Alzheimer’s disease facts and figures. Alzheimer's Dement 11, 332–384 (2015).

2. Albert, M. S. et al. Te diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer's Dement 7, 270–279

3. Jack, C. R. Jr. et al. Introduction to the recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer's Dement 7, 257–262,

4. Sperling, R. A. et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer's Dement 7, 280–292.

5. Petersen, R. C. et al. Mild cognitive impairment: clinical characterization and outcome. Archives of neurology 56, 303–308 (1999).

6. Tábuas-Pereira, M. et al. Prognosis of Early-Onset vs. Late-Onset Mild Cognitive Impairment: Comparison of Conversion Rates and Its Predictors. Geriatrics 1, 11 (2016).

7. Zhang, D., Shen, D. & Alzheimer’s Disease Neuroimaging, I. Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. Neuroimage 59, 895–907,

8. Cheng, B., Liu, M., Zhang, D., Munsell, B. C. & Shen, D. Domain Transfer Learning for MCI Conversion Prediction. IEEE Trans Biomed Eng 62, 1805–1817.

9. Zhang, D., Shen, D. & Initiative, A. S. D. N. Predicting future clinical changes of MCI patients using longitudinal and multimodal biomarkers. PloS one 7, e33182 (2012).

10. Nho, K. et al. Automatic Prediction of Conversion from Mild Cognitive Impairment to Probable Alzheimer’s Disease using Structural Magnetic Resonance Imaging. AMIA Annu Symp Proc 2010, 542–546 (2010).

11. Wee, C. Y., Yap, P. T., Shen, D. & Alzheimer’s Disease Neuroimaging, I. Prediction of Alzheimer’s disease and mild cognitive impairment using cortical morphological patterns. Hum Brain Mapp 34, 3411–3425,

12. Wolz, R. et al. Multi-method analysis of MRI images in early diagnostics of Alzheimer’s disease. PLoS One 6, e25446

13. Cho, Y., Seong, J. K., Jeong, Y., Shin, S. Y. & Alzheimer’s Disease Neuroimaging, I. Individual subject classification for Alzheimer’s disease based on incremental learning using a spatial frequency representation of cortical thickness data. Neuroimage 59, 2217–2230, https://doi.org/10.1016/j.neuroimage.2011.09.085 (2012).

14. Kim, D. et al. A Graph-Based Integration of Multimodal Brain Imaging Data for the Detection of Early Mild Cognitive Impairment (E-MCI). Multimodal Brain Image Anal (2013) 8159, 159–169,

15. Ewers, M. et al. Prediction of conversion from mild cognitive impairment to Alzheimer’s disease dementia based upon biomarkers and neuropsychological test performance. Neurobiol Aging 33, 1203–1214,

(2012).

16. Heister, D. et al. Predicting MCI outcome with clinically available MRI and CSF biomarkers. Neurology 77, 1619–1628

17. Saykin, A. J. et al. Genetic studies of quantitative MCI and AD phenotypes in ADNI: Progress, opportunities, and plans. Alzheimer’s & dementia: the journal of the Alzheimer’s Association 11, 792–814 (2015).

18. Saykin, A. J. et al. Alzheimer’s Disease Neuroimaging Initiative biomarkers as quantitative phenotypes: Genetics core aims, progress, and plans. Alzheimer’s & dementia: the journal of the Alzheimer’s Association 6, 265–273 (2010).

19. Nho, K. et al. Voxel and surface-based topography of memory and executive deficits in mild cognitive impairment and Alzheimer’s disease. Brain imaging and behavior 6, 551–567 (2012).

20. Falahati, F., Westman, E. & Simmons, A. Multivariate data analysis and machine learning in Alzheimer’s disease with a focus on structural magnetic resonance imaging. J Alzheimers Dis 41, 685–708.