The Application Of Deep Learning For Automated Segmentation Of Kidney Histologic Primitives

Contact: Audrey Hu (Whatsapp:008613880143964) Email: audrey.hu@wecistanche.com

PART Ⅱ: Development and evaluation of deep learning based segmentation of histologic structures in the kidney cortex with multiple histologic stains

Catherine P.Jayapandian, Yiiang Chen,Andrew R. Janowczyk & et al.

The application of deep learning for automated segmentation (delineation of boundaries) of histologic primitives (structures)from whole slide images can facilitate the establishment of novel protocols for kidney biopsy assessment. Here, we developed and validated deep learning networks for the segmentation of histologic structures on kidney biopsies and nephrectomies. For development, we examined 125 biopsies for Minimal Change Disease collected across 29 NEPTUNE enrolling centers along with 459 whole slide images stained with Hematoxylin & Eosin(125), Periodic Acid Schiff(125), Silver (102), and Trichrome(107) divided into training, validation and testing sets (ratio 6:1:3). Histologic structures were manually segmented (30048 total annotations) by five nephropathologists. Twenty deep learning models were trained with optimal digital magnification across the structures and stains. Periodic Acid Schiff-stained whole slide images yielded the best concordance between pathologists and deep learning segmentation acrossall structures (F-scores:0.93 for glomerular tufts,0.94 for glomerular tuft plus Bowman's capsule, 0.91 for proximal tubules,0.93 for distal tubular segments,0.81 for peritubular capillaries, and 0.85 for arteries and afferent arterioles).Optimal digital magnifications were 5X for glomerular tuft/tuft plus Bowman's capsule, 10X for proximal/distal tubule, arteries and afferent arterioles, and 40X for peritubular capillries. Silver stained whole slide images yielded the worst deep learning performance.Thus,this largest study to date adapted deep learning for the segmentation of kidney histologic structures across multiple stains and pathology laboratories. All data used for training and testing and a detailed online tutorial will be publicly available.

real function improvement herb: cistanche

CLICK HERE TO PART Ⅰ

DISCUSSION

The assessment of renal biopsy is unique compared with other surgical pathology specimens because of the variety of stains routinely used. Morphologic assessment relies on the quality of the preparations, the pathologists'expertise in detecting the individual structures and associated changes, and quantitative or semiquantitative metrics used to capture the extent of tissue damage. Visual histologic quantitative assessment such as counting, distribution,and morphometry of certain histologic primitives are known to be robust predictors of outcome for various kidney diseases.10,17-23 However, quantitative analysis remains a challenge for the human eye. Some of these primitives(e.g, peritubular capillaries)cannot be measured visually or manully and warrant the aid of computational algorithms. Recent studies have suggested that computer vision tools can serve as triage and decision support tools for disease diagnosis with digital pathology.

24~27

Thus, automated image analysis tools need to be implemented and integrated into the pathology workflow for efficient and reliable segmentation of histologic primitives across multiple types of stains. DL segmentation tools could greatly facilitate derivation of not only the visual but also subvisual histomorphometric features(e.g,shape, textural, and graph features) for correlation with diagnosis and outcome.280 This study attempts to address the challenges of computational renal pathology for large-scale tissue interrogation by providing DL algorithms for thorough annotation of6 histologic primitives on renal parenchyma of minimal change disease (MCD),using whole slide images (WSIs)of 4 stains and generated across 29 NEPTUNE enrolling centers. In the past few years, several studies have demonstrated the utility of DL networks for low level image analyses (i.e,detection, segmentation, and classification of histologic primitives)and highlevel complex prognosis and prediction tasks.1-35 Our study is the largest,comprehensive DL study of kidney biopsies, presenting algorithms that were developed on different stains and using a large number of annotated images, compared with those previously published. The primary conclusions and signifificant fifindings from our work are described next.

Comparison with current literature

The differences between previous studies36–44 and our contributions are summarized in the Supplementary Figure S6.

Previously published studies focus on a single histologic primitive and a single stain. For example, Marsh et al. evaluated CNNs for detection of global glomerulosclerosis in transplant kidney frozen sections stained with H&E36; Kanna et al. evaluated CNNs to discriminate normal, segmentally and globally sclerosed glomeruli from trichrome stained formalin fifixed and paraffifin embedded kidney sections37; Gallego et al. applied DL to detect glomeruli on PAS-stained sections; Bel

Previously published studies focus on a single histologic primitive and a single stain. For example, Marsh et al. evaluated CNNs for detection of global glomerulosclerosis in transplant kidney frozen sections stained with H&E36; Kanna et al. evaluated CNNs to discriminate normal, segmentally and globally sclerosed glomeruli from trichrome stained formalin fifixed and paraffifin embedded kidney sections37; Gallego et al. applied DL to detect glomeruli on PAS-stained sections; Bel et al. demonstrated segmentation of normal and pathologic histologic structures using PAS stained WSIs of nephrectomy cortex tissue.Temerinac Ott et al. demonstrate a DL approach to improve glomerular detection on 1 staining using results from differently stained sections of same tissue.1 Our DL networks on all 4 stains represent a first step for future clinical deployment allowing for the detection,segmentation, and ultimately quantification of several normal histologic primitives in all stains routinely used for diagnostic purposes.

Another critical element that needs to be taken into consideration before their use in large-scale DL networks is how they can be applied to heterogeneous datasets. Our DL models were trained and tested on a very heterogeneous set of WSIs with preanalytic variations in tissue acquisition, processing, and slide preparation using 4 stains, thus facilitating the rigorous evaluation of the applicability of the DL approach in a multisite setting.

Different DL approaches have been used for the segmentation of histologic primitives, such as Gadermayr et al's application of generative adversarial deep networks for stain independent glomerular segmentation. Bel et al. employed cycle consistent generative adversarial networks (cycle-GANs)in DL applications for multicenter stain transformation. Hermsen et al. has demonstrated U-Net based segmentation of 7 tissue classes using 40 transplant biopsies on PAS stain.2Our approach, in this study, was to develop multiple U-Net based DL networks using optimal digital magnification and varying number of annotations across primitives and stains.

All previous works have used relatively smaller number of WSIs of renal biopsies/nephrectomies compared with our study(Table 2).The use of a large WSI dataset allowed us to provide insights to pathologists for generating well annotated training exemplars for each primitive and stain, as well as the number of training exemplars required for best network performance using U-Net CNNs(Figure 8).

Specificity of the segmentation of the individual histologic primitives and their pathologic variation is critical for the deployment of DL models into clinical practice.12,43 The DL networks generated in this work are specific to structurally normal histologic primitives, such as those seen in MCD or nephrectomies, and can be applied to both adult and pediatric renal biopsies. When the DL networks were tested on patches of renal parenchyma from nephrectomy specimens, the specificity for the structurally normal histologic primitives was maintained. The DL framework presented in this study will also enable architecting of networks in the future that are specifically focused on automated segmentation and assessment of structurally abnormal histologic primitives and their correlation with clinical outcomes.

kidney function improvemet: cistanche tubulosa extract

DL-based ranking of different stains

Our study suggests that the PAS stain is best suited for identification of structurally normal histologic primitives using the U-Net model. This may be because PAS appears to be consistently more homogeneous across pathology laboratories compared with TRI or SIL. PAS-stained WSIs highlight the basement membranes of different structures, which in turn provides superior definition of the boundary of each single primitive to be segmented. For this reason, PAS was the only stain used for segmentation of peritubular capillaries. On the basis of our results, PAS and H&E stains showed better performance for glomerular tuft and unit segmentation, PAS and TRI for arteries/arterioles, PAS and SIL for tubular segments, and PAS for peritubular capillaries. Optimal digital magnification for DL models

Our results suggest that with a unified patch size of 256 ×256, optimal magnification for the DL models was 5× for glomeruli,10× for tubules and vessels, and 40× for capillaries(Figure 1).Interestingly, most of the optimal magnifications were concordant with the magnifications that pathologists tend to use when annotating the individual primitives, except for glomeruli where the pathologists used 15× to 20×.Larger structures such as glomeruli, tubules,and vessels were more precisely segmented by the network at 5×to 10× magnification regardless of the stain. For smaller structures such as peritubular capillaries, larger digital magnification (40×) was required for accurate DL segmentation.

DL segmentation performance across sites and artifacts Heterogeneity of tissue preparation and lack of standardization of the analytics is particularly relevant for multicenter studies, where the pathology material is collected from several laboratories.As expected, heterogeneity in tissue presentation and glass, tissue, and scanning artifacts was observed, each with variable contribution to the DL performance. For example, although in general tissue artifacts had limited impact on the DL networks, the thickness of the section appeared to affect performance. The impact of individual artifacts was also relative to the histologic primitive; for example,glass artifacts showed a slight negative impact on DL performance for arteries/arterioles and proximal tubules.

Additionally, there was variability in DL performance across sites, and this variability appeared to be histologic primitive dependent (Supplementary Figure S4).

DL performance as a function of number of training exemplars

Our quantitative data validated the intuitive assumption that more exemplars are needed for those primitives that are more difficult to identify visually (i.e., tangentially cut arteries/arterioles or primitives at the edge of the region of interest [ROI])(Figure 8).For those primitives that were too small or il defined (i.e. peritubular capillaries), curation and iterative annotation was necessary to improve segmentation accuracy. For segmentation of glomerular tufts, the network converged to maximum accuracy with a small number (60-183)of training exemplars; performance did not improve with inclusion of additional exemplars. For tubules and arteries/arterioles segmentation, the corresponding networks showed marginal to intermediate performance improvement with an increasing number of exemplars. In contrast, a significant increase in F-score and DSC(0.27-0.81)was observed with a 2.5 fold increase in the number of peritubular capillary exemplars, a linear scope of F-score increase indicating even better accuracy with more exemplars.

Interpreting segmentation results

Few false positives were observed in regions of interest with artifacts (ie. tssue folds, uneven staining), suggesting the need for digital quality assessment of the slide images prior to invocation of the computational models (Supplementary Figure S4).In a few ROIs, the DL appeared to outperform the pathologists—for example, when a small portion of an artery/arteriole was at the edge of the ROIs and was not manually annotated as ground truth by the pathologist because they were visually difficult to detect. This can be explained by the protocol used for segmentation of arteries, where pathologists included only arteries where the wall (tunica media and intima) and lumen were visible and segmented the outer boundary of the tunica media. Thus, the models, trained to detect the tunica media and intima of the arteries correctly identified small fragments of tunica media (arterial/arteriolar wall tangentially cut) as arteries/arterioles despite the lack of a lumen (Figure 9).

Additionally, tubules in renal biopsy sections are more often seen in transverse than longitudinal sections. The initial classifier missed some longitudinally sectioned tubules, mostly on H&E stained images, because the tubule boundaries were less sharp, and longitudinally sectioned tubules were underrepresented in the initial training set. To facilitate and improve the process of annotation and the network, the false negative errors associated with the U-Net segmentation of the tubules were visually identified and manually refined by the pathologist, and the updated annotations were returned to the network. A few small arterioles were also incorrectly identified as distal tubules by the DL algorithm (false positives)during the first iteration. These false positive annotations were removed by the pathologist upon review of the initial classifier output and corrected images were returned to the network for retraining without changing the experimental setup or the network parameters to eliminate false positives and negative errors of the DL algorithm.45

In line with current sharing guidelines, with this report, we are making all of our data and accompanying ground truth annotations publicly available for the community. Online supplemental material released as part of this work is anticipated to advance the field of computational renal pathology and provide best practices for generating annotations, augmentations,′magnifications and recommended stains to perform segmentation tasks optimally.

In conclusion, this study represents a solid foundation toward invoking machine learning classifiers to aid large-scale tissue quantification efforts and the implementation of machine human interactive protocols in clinical and pathology workflows. DL segmentation of histologic primitives enables computational derivation of histomorphometric features for enabling biopsy interpretation. Additionally, the framework presented in this work will also pave the way for development of new DL networks in the future that are specifically geared toward(i) abnormal or pathologic histologic primitives(i.e.,global and segmental sclerosis, glomerular proliferative features, collecting ducts, veins and peripheral nerves, tubular atrophy, interstitial fibrosis, and arteriosclerosis),(ii)renal cortex and medullary compartments, and (ii) a wider spectrum of diseases. Further, these novel approaches could pave the way for the development of machine learning tools that provide disease prognosis or and even facilitate discovery predicting treatment response4 of clinically actionable, nondestructive computational pathology based imaging diagnostic biomarkers for kidney 25,27,48 diseases.2

Cistanche effective for renal disease

Note: cistanche is a tobic herb that grows in the deserts. it is also called the dragon herb and the desert ginseng. cistanche has many effects which are great for human health according to the parmocological studies, such as improve immunity, anti-fatigue and improve kidney function etc. Chengdu wecistanhce are focusing on supplying premium quality cistanche products.