Showing 1–50 of 84 results for author: Duncan, J S

AI-powered multimodal modeling of personalized hemodynamics in aortic stenosis

Authors: Caglar Ozturk, Daniel H. Pak, Luca Rosalia, Debkalpa Goswami, Mary E. Robakowski, Raymond McKay, Christopher T. Nguyen, James S. Duncan, Ellen T. Roche

Abstract: Aortic stenosis (AS) is the most common valvular heart disease in developed countries. High-fidelity preclinical models can improve AS management by enabling therapeutic innovation, early diagnosis, and tailored treatment planning. However, their use is currently limited by complex workflows necessitating lengthy expert-driven manual operations. Here, we propose an AI-powered computational framewo… ▽ More Aortic stenosis (AS) is the most common valvular heart disease in developed countries. High-fidelity preclinical models can improve AS management by enabling therapeutic innovation, early diagnosis, and tailored treatment planning. However, their use is currently limited by complex workflows necessitating lengthy expert-driven manual operations. Here, we propose an AI-powered computational framework for accelerated and democratized patient-specific modeling of AS hemodynamics from computed tomography. First, we demonstrate that our automated meshing algorithms can generate task-ready geometries for both computational and benchtop simulations with higher accuracy and 100 times faster than existing approaches. Then, we show that our approach can be integrated with fluid-structure interaction and soft robotics models to accurately recapitulate a broad spectrum of clinical hemodynamic measurements of diverse AS patients. The efficiency and reliability of these algorithms make them an ideal complementary tool for personalized high-fidelity modeling of AS biomechanics, hemodynamics, and treatment planning. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Comments: CO and DHP contributed equally to this work. JSD and ETR are corresponding authors

Spectral Brain Graph Neural Network for Prediction of Anxiety in Children with Autism Spectrum Disorder

Authors: Peiyu Duan, Nicha C. Dvornek, Jiyao Wang, Jeffrey Eilbott, Yuexi Du, Denis G. Sukhodolsky, James S. Duncan

Abstract: Children with Autism Spectrum Disorder (ASD) frequently exhibit comorbid anxiety, which contributes to impairment and requires treatment. Therefore, it is critical to investigate co-occurring autism and anxiety with functional imaging tools to understand the brain mechanisms of this comorbidity. Multidimensional Anxiety Scale for Children, 2nd edition (MASC-2) score is a common tool to evaluate th… ▽ More Children with Autism Spectrum Disorder (ASD) frequently exhibit comorbid anxiety, which contributes to impairment and requires treatment. Therefore, it is critical to investigate co-occurring autism and anxiety with functional imaging tools to understand the brain mechanisms of this comorbidity. Multidimensional Anxiety Scale for Children, 2nd edition (MASC-2) score is a common tool to evaluate the daily anxiety level in autistic children. Predicting MASC-2 score with Functional Magnetic Resonance Imaging (fMRI) data will help gain more insights into the brain functional networks of children with ASD complicated by anxiety. However, most of the current graph neural network (GNN) studies using fMRI only focus on graph operations but ignore the spectral features. In this paper, we explored the feasibility of using spectral features to predict the MASC-2 total scores. We proposed SpectBGNN, a graph-based network, which uses spectral features and integrates graph spectral filtering layers to extract hidden information. We experimented with multiple spectral analysis algorithms and compared the performance of the SpectBGNN model with CPM, GAT, and BrainGNN on a dataset consisting of 26 typically developing and 70 ASD children with 5-fold cross-validation. We showed that among all spectral analysis algorithms tested, using the Fast Fourier Transform (FFT) or Welch's Power Spectrum Density (PSD) as node features performs significantly better than correlation features, and adding the graph spectral filtering layer significantly increases the network's performance. △ Less

Submitted 23 April, 2024; originally announced July 2024.

Comments: ISBI 2024 Oral

Assessment of Clonal Hematopoiesis of Indeterminate Potential from Cardiac Magnetic Resonance Imaging using Deep Learning in a Cardio-oncology Population

Authors: Sangeon Ryu, Shawn Ahn, Jeacy Espinoza, Alokkumar Jha, Stephanie Halene, James S. Duncan, Jennifer M Kwan, Nicha C. Dvornek

Abstract: Background: We propose a novel method to identify who may likely have clonal hematopoiesis of indeterminate potential (CHIP), a condition characterized by the presence of somatic mutations in hematopoietic stem cells without detectable hematologic malignancy, using deep learning techniques. Methods: We developed a convolutional neural network (CNN) to predict CHIP status using 4 different views fr… ▽ More Background: We propose a novel method to identify who may likely have clonal hematopoiesis of indeterminate potential (CHIP), a condition characterized by the presence of somatic mutations in hematopoietic stem cells without detectable hematologic malignancy, using deep learning techniques. Methods: We developed a convolutional neural network (CNN) to predict CHIP status using 4 different views from standard delayed gadolinium-enhanced cardiac magnetic resonance imaging (CMR). We used 5-fold cross validation on 82 cardio-oncology patients to assess the performance of our model. Different algorithms were compared to find the optimal patient-level prediction method using the image-level CNN predictions. Results: We found that the best model had an area under the receiver operating characteristic curve of 0.85 and an accuracy of 82%. Conclusions: We conclude that a deep learning-based diagnostic approach for CHIP using CMR is promising. △ Less

Submitted 26 June, 2024; originally announced June 2024.

STNAGNN: Spatiotemporal Node Attention Graph Neural Network for Task-based fMRI Analysis

Authors: Jiyao Wang, Nicha C. Dvornek, Peiyu Duan, Lawrence H. Staib, Pamela Ventola, James S. Duncan

Abstract: Task-based fMRI uses actions or stimuli to trigger task-specific brain responses and measures them using BOLD contrast. Despite the significant task-induced spatiotemporal brain activation fluctuations, most studies on task-based fMRI ignore the task context information aligned with fMRI and consider task-based fMRI a coherent sequence. In this paper, we show that using the task structures as data… ▽ More Task-based fMRI uses actions or stimuli to trigger task-specific brain responses and measures them using BOLD contrast. Despite the significant task-induced spatiotemporal brain activation fluctuations, most studies on task-based fMRI ignore the task context information aligned with fMRI and consider task-based fMRI a coherent sequence. In this paper, we show that using the task structures as data-driven guidance is effective for spatiotemporal analysis. We propose STNAGNN, a GNN-based spatiotemporal architecture, and validate its performance in an autism classification task. The trained model is also interpreted for identifying autism-related spatiotemporal brain biomarkers. △ Less

Submitted 17 June, 2024; originally announced June 2024.

2.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation: Application to Low-count PET Reconstruction with CT-less Attenuation Correction

Authors: Tianqi Chen, Jun Hou, Yinchi Zhou, Huidong Xie, Xiongchao Chen, Qiong Liu, Xueqi Guo, Menghua Xia, James S. Duncan, Chi Liu, Bo Zhou

Abstract: Positron Emission Tomography (PET) is an important clinical imaging tool but inevitably introduces radiation hazards to patients and healthcare providers. Reducing the tracer injection dose and eliminating the CT acquisition for attenuation correction can reduce the overall radiation dose, but often results in PET with high noise and bias. Thus, it is desirable to develop 3D methods to translate t… ▽ More Positron Emission Tomography (PET) is an important clinical imaging tool but inevitably introduces radiation hazards to patients and healthcare providers. Reducing the tracer injection dose and eliminating the CT acquisition for attenuation correction can reduce the overall radiation dose, but often results in PET with high noise and bias. Thus, it is desirable to develop 3D methods to translate the non-attenuation-corrected low-dose PET (NAC-LDPET) into attenuation-corrected standard-dose PET (AC-SDPET). Recently, diffusion models have emerged as a new state-of-the-art deep learning method for image-to-image translation, better than traditional CNN-based methods. However, due to the high computation cost and memory burden, it is largely limited to 2D applications. To address these challenges, we developed a novel 2.5D Multi-view Averaging Diffusion Model (MADM) for 3D image-to-image translation with application on NAC-LDPET to AC-SDPET translation. Specifically, MADM employs separate diffusion models for axial, coronal, and sagittal views, whose outputs are averaged in each sampling step to ensure the 3D generation quality from multiple views. To accelerate the 3D sampling process, we also proposed a strategy to use the CNN-based 3D generation as a prior for the diffusion model. Our experimental results on human patient studies suggested that MADM can generate high-quality 3D translation images, outperforming previous CNN-based and Diffusion-based baseline methods. △ Less

Submitted 15 June, 2024; v1 submitted 12 June, 2024; originally announced June 2024.

Comments: 15 pages, 7 figures

The Imaging Database for Epilepsy And Surgery (IDEAS)

Authors: Peter N. Taylor, Yujiang Wang, Callum Simpson, Vytene Janiukstyte, Jonathan Horsley, Karoline Leiberg, Beth Little, Harry Clifford, Sophie Adler, Sjoerd B. Vos, Gavin P Winston, Andrew W McEvoy, Anna Miserocchi, Jane de Tisi, John S Duncan

Abstract: Magnetic resonance imaging (MRI) is a crucial tool to identify brain abnormalities in a wide range of neurological disorders. In focal epilepsy MRI is used to identify structural cerebral abnormalities. For covert lesions, machine learning and artificial intelligence algorithms may improve lesion detection if abnormalities are not evident on visual inspection. The success of this approach depends… ▽ More Magnetic resonance imaging (MRI) is a crucial tool to identify brain abnormalities in a wide range of neurological disorders. In focal epilepsy MRI is used to identify structural cerebral abnormalities. For covert lesions, machine learning and artificial intelligence algorithms may improve lesion detection if abnormalities are not evident on visual inspection. The success of this approach depends on the volume and quality of training data. Herein, we release an open-source dataset of preprocessed MRI scans from 442 individuals with drug-refractory focal epilepsy who had neurosurgical resections, and detailed demographic information. The MRI scan data includes the preoperative 3D T1 and where available 3D FLAIR, as well as a manually inspected complete surface reconstruction and volumetric parcellations. Demographic information includes age, sex, age of onset of epilepsy, location of surgery, histopathology of resected specimen, occurrence and frequency of focal seizures with and without impairment of awareness, focal to bilateral tonic-clonic seizures, number of anti-seizure medications (ASMs) at time of surgery, and a total of 1764 patient years of post-surgical follow up. Crucially, we also include resection masks delineated from post-surgical imaging. To demonstrate the veracity of our data, we successfully replicated previous studies showing long-term outcomes of seizure freedom in the range of around 50%. Our imaging data replicates findings of group level atrophy in patients compared to controls. Resection locations in the cohort were predominantly in the temporal and frontal lobes. We envisage our dataset, shared openly with the community, will catalyse the development and application of computational methods in clinical neurology. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Brain Morphology Normative modelling platform for abnormality and Centile estimation: Brain MoNoCle

Authors: Bethany Little, Nida Alyas, Alexander Surtees, Gavin P Winston, John S Duncan, David A Cousins, John-Paul Taylor, Peter Taylor, Karoline Leiberg, Yujiang Wang

Abstract: Normative models of brain structure estimate the effects of covariates such as age and sex using large samples of healthy controls. These models can then be applied to smaller clinical cohorts to distinguish disease effects from other covariates. However, these advanced statistical modelling approaches can be difficult to access, and processing large healthy cohorts is computationally demanding. T… ▽ More Normative models of brain structure estimate the effects of covariates such as age and sex using large samples of healthy controls. These models can then be applied to smaller clinical cohorts to distinguish disease effects from other covariates. However, these advanced statistical modelling approaches can be difficult to access, and processing large healthy cohorts is computationally demanding. Thus, accessible platforms with pre-trained normative models are needed. We present such a platform for brain morphology analysis as an open-source web application https://cnnplab.shinyapps.io/normativemodelshiny/, with six key features: (i) user-friendly web interface, (ii) individual and group outputs, (iii) multi-site analysis, (iv) regional and whole-brain analysis, (v) integration with existing tools, and (vi) featuring multiple morphology metrics. Using a diverse sample of 3,276 healthy controls across 21 sites, we pre-trained normative models on various metrics. We validated the models with a small clinical sample of individuals with bipolar disorder, showing outputs that aligned closely with existing literature only after applying our normative modelling. Further validation with a cohort of temporal lobe epilepsy showed agreement with previous group-level findings and individual-level seizure lateralisation. Finally, with the ability to investigate multiple morphology measures in the same framework, we found that biological covariates are better explained in specific morphology measures, and for clinical applications, only some measures are sensitive to the disease process. Our platform offers a comprehensive framework to analyse brain morphology in clinical and research settings. Validations confirm the superiority of normative models and the advantage of investigating a range of brain morphology metrics together. △ Less

Submitted 26 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Cascaded Multi-path Shortcut Diffusion Model for Medical Image Translation

Authors: Yinchi Zhou, Tianqi Chen, Jun Hou, Huidong Xie, Nicha C. Dvornek, S. Kevin Zhou, David L. Wilson, James S. Duncan, Chi Liu, Bo Zhou

Abstract: Image-to-image translation is a vital component in medical imaging processing, with many uses in a wide range of imaging modalities and clinical scenarios. Previous methods include Generative Adversarial Networks (GANs) and Diffusion Models (DMs), which offer realism but suffer from instability and lack uncertainty estimation. Even though both GAN and DM methods have individually exhibited their c… ▽ More Image-to-image translation is a vital component in medical imaging processing, with many uses in a wide range of imaging modalities and clinical scenarios. Previous methods include Generative Adversarial Networks (GANs) and Diffusion Models (DMs), which offer realism but suffer from instability and lack uncertainty estimation. Even though both GAN and DM methods have individually exhibited their capability in medical image translation tasks, the potential of combining a GAN and DM to further improve translation performance and to enable uncertainty estimation remains largely unexplored. In this work, we address these challenges by proposing a Cascade Multi-path Shortcut Diffusion Model (CMDM) for high-quality medical image translation and uncertainty estimation. To reduce the required number of iterations and ensure robust performance, our method first obtains a conditional GAN-generated prior image that will be used for the efficient reverse translation with a DM in the subsequent step. Additionally, a multi-path shortcut diffusion strategy is employed to refine translation results and estimate uncertainty. A cascaded pipeline further enhances translation quality, incorporating residual averaging between cascades. We collected three different medical image datasets with two sub-tasks for each dataset to test the generalizability of our approach. Our experimental results found that CMDM can produce high-quality translations comparable to state-of-the-art methods while providing reasonable uncertainty estimations that correlate well with the translation error. △ Less

Submitted 5 April, 2024; originally announced May 2024.

Comments: 15 pages, 5 figures

Anti-seizure medication tapering is associated with delta band power reduction in a dose, region and time-dependent manner

Authors: Guillermo M. Besne, Nathan Evans, Mariella Panagiotopoulou, Billy Smith, Fahmida A Chowdhury, Beate Diehl, John S Duncan, Andrew W McEvoy, Anna Miserocchi, Jane de Tisi, Mathew Walker, Peter N. Taylor, Chris Thornton, Yujiang Wang

Abstract: Anti-seizure medications (ASMs) are the primary treatment for epilepsy, yet medication tapering effects have not been investigated in a dose, region, and time-dependent manner, despite their potential impact on research and clinical practice. We examined over 3000 hours of intracranial EEG recordings in 32 subjects during long-term monitoring, of which 22 underwent concurrent ASM tapering. We es… ▽ More Anti-seizure medications (ASMs) are the primary treatment for epilepsy, yet medication tapering effects have not been investigated in a dose, region, and time-dependent manner, despite their potential impact on research and clinical practice. We examined over 3000 hours of intracranial EEG recordings in 32 subjects during long-term monitoring, of which 22 underwent concurrent ASM tapering. We estimated ASM plasma levels based on known pharmaco-kinetics of all the major ASM types. We found an overall decrease in the power of delta band activity around the period of maximum medication withdrawal in most (80%) subjects, independent of their epilepsy type or medication combination. The degree of withdrawal correlated positively with the magnitude of delta power decrease. This dose-dependent effect was strongly seen across all recorded cortical regions during daytime; but not in sub-cortical regions, or during night time. We found no evidence of differential effect in seizure onset, spiking, or pathological brain regions. The finding of decreased delta band power during ASM tapering agrees with previous literature. Our observed dose-dependent effect indicates that monitoring ASM levels in cortical regions may be feasible for applications such as medication reminder systems, or closed-loop ASM delivery systems. ASMs are also used in other neurological and psychiatric conditions, making our findings relevant to a general neuroscience and neurology audience. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Multi-centre normative brain mapping of intracranial EEG lifespan patterns in the human brain

Authors: Heather Woodhouse, Gerard Hall, Callum Simpson, Csaba Kozma, Frances Turner, Gabrielle M. Schroeder, Beate Diehl, John S. Duncan, Jiajie Mo, Kai Zhang, Aswin Chari, Martin Tisdall, Friederike Moeller, Chris Petkov, Matthew A. Howard, George M. Ibrahim, Elizabeth Donner, Nebras M. Warsi, Raheel Ahmed, Peter N. Taylor, Yujiang Wang

Abstract: Background: Understanding healthy human brain function is crucial to identify and map pathological tissue within it. Whilst previous studies have mapped intracranial EEG (icEEG) from non-epileptogenic brain regions, these maps do not consider the effects of age and sex. Further, most existing work on icEEG has often suffered from a small sample size due to this modality's invasive nature. Here, we… ▽ More Background: Understanding healthy human brain function is crucial to identify and map pathological tissue within it. Whilst previous studies have mapped intracranial EEG (icEEG) from non-epileptogenic brain regions, these maps do not consider the effects of age and sex. Further, most existing work on icEEG has often suffered from a small sample size due to this modality's invasive nature. Here, we substantially increase the subject sample size compared to existing literature, to create a multi-centre, normative map of brain activity which additionally considers the effects of age, sex and recording site. Methods: Using interictal icEEG recordings from n = 513 subjects originating from 15 centres, we constructed a normative map of non-pathological brain activity by regressing age and sex on relative band power in five frequency bands, whilst accounting for the site effect. Results: Recording site significantly impacted normative icEEG maps in all frequency bands, and age was a more influential predictor of band power than sex. The age effect varied by frequency band, but no spatial patterns were observed at the region-specific level. Certainty about regression coefficients was also frequency band specific and moderately impacted by sample size. Conclusion: The concept of a normative map is well-established in neuroscience research and particularly relevant to the icEEG modality, which does not allow healthy control baselines. Our key results regarding the site and age effect guide future work utilising normative maps in icEEG. △ Less

Submitted 27 April, 2024; originally announced April 2024.

Alpha rhythm slowing in temporal epilepsy across Scalp EEG and MEG

Authors: Vytene Janiukstyte, Csaba Kozma, Thomas W. Owen, Umair J Chaudhury, Beate Diehl, Louis Lemieux, John S Duncan, Fergus Rugg-Gunn, Jane de Tisi, Yujiang Wang, Peter N. Taylor

Abstract: EEG slowing is reported in various neurological disorders including Alzheimer's, Parkinson's and Epilepsy. Here, we investigate alpha rhythm slowing in individuals with refractory temporal lobe epilepsy (TLE), compared to healthy controls, using scalp electroencephalography (EEG) and magnetoencephalography (MEG). We retrospectively analysed data from 17,(46) healthy controls and 22,(24) individu… ▽ More EEG slowing is reported in various neurological disorders including Alzheimer's, Parkinson's and Epilepsy. Here, we investigate alpha rhythm slowing in individuals with refractory temporal lobe epilepsy (TLE), compared to healthy controls, using scalp electroencephalography (EEG) and magnetoencephalography (MEG). We retrospectively analysed data from 17,(46) healthy controls and 22,(24) individuals with TLE who underwent scalp EEG and (MEG) recordings as part of presurgical evaluation. Resting-state, eyes-closed recordings were source reconstructed using the standardized low-resolution brain electrographic tomography (sLORETA) method. We extracted low (slow) 6-9 Hz and high (fast) 10-11 Hz alpha relative band power and calculated the alpha power ratio by dividing low (slow) alpha by high (fast) alpha. This ratio was computed for all brain regions in all individuals. Alpha oscillations were slower in individuals with TLE than controls (p<0.05). This effect was present in both the ipsilateral and contralateral hemispheres, and across widespread brain regions. Alpha slowing in TLE was found in both EEG and MEG recordings. We interpret greater low (slow)-alpha as greater deviation from health. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations

Authors: Chenyu You, Yifei Min, Weicheng Dai, Jasjeet S. Sekhon, Lawrence Staib, James S. Duncan

Abstract: Fine-tuning pre-trained vision-language models, like CLIP, has yielded success on diverse downstream tasks. However, several pain points persist for this paradigm: (i) directly tuning entire pre-trained models becomes both time-intensive and computationally costly. Additionally, these tuned models tend to become highly specialized, limiting their practicality for real-world deployment; (ii) recent… ▽ More Fine-tuning pre-trained vision-language models, like CLIP, has yielded success on diverse downstream tasks. However, several pain points persist for this paradigm: (i) directly tuning entire pre-trained models becomes both time-intensive and computationally costly. Additionally, these tuned models tend to become highly specialized, limiting their practicality for real-world deployment; (ii) recent studies indicate that pre-trained vision-language classifiers may overly depend on spurious features -- patterns that correlate with the target in training data, but are not related to the true labeling function; and (iii) existing studies on mitigating the reliance on spurious features, largely based on the assumption that we can identify such features, does not provide definitive assurance for real-world applications. As a piloting study, this work focuses on exploring mitigating the reliance on spurious features for CLIP without using any group annotation. To this end, we systematically study the existence of spurious correlation on CLIP and CILP+ERM. We first, following recent work on Deep Feature Reweighting (DFR), verify that last-layer retraining can greatly improve group robustness on pretrained CLIP. In view of them, we advocate a lightweight representation calibration method for fine-tuning CLIP, by first generating a calibration set using the pretrained CLIP, and then calibrating representations of samples within this set through contrastive learning, all without the need for group labels. Extensive experiments and in-depth visualizations on several benchmarks validate the effectiveness of our proposals, largely reducing reliance and significantly boosting the model generalization. △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: Accepted by CVPR 2024

Robust automated calcification meshing for biomechanical cardiac digital twins

Authors: Daniel H. Pak, Minliang Liu, Theodore Kim, Caglar Ozturk, Raymond McKay, Ellen T. Roche, Rudolph Gleason, James S. Duncan

Abstract: Calcification has significant influence over cardiovascular diseases and interventions. Detailed characterization of calcification is thus desired for predictive modeling, but calcified heart meshes for physics-driven simulations are still often reconstructed using manual operations. This poses a major bottleneck for large-scale adoption of computational simulations for research or clinical use. T… ▽ More Calcification has significant influence over cardiovascular diseases and interventions. Detailed characterization of calcification is thus desired for predictive modeling, but calcified heart meshes for physics-driven simulations are still often reconstructed using manual operations. This poses a major bottleneck for large-scale adoption of computational simulations for research or clinical use. To address this, we propose an end-to-end automated meshing algorithm that enables robust incorporation of patient-specific calcification onto a given heart mesh. The algorithm provides a substantial speed-up from several hours of manual meshing to $\sim$1 minute of automated computation, and it solves an important problem that cannot be addressed with recent template registration-based heart meshing techniques. We validated our final calcified heart meshes with extensive simulations, demonstrating our ability to accurately model patient-specific aortic stenosis and Transcatheter Aortic Valve Replacement. Our method may serve as an important tool for accelerating the development and usage of physics-driven simulations for cardiac digital twins. △ Less

Submitted 7 March, 2024; originally announced March 2024.

POUR-Net: A Population-Prior-Aided Over-Under-Representation Network for Low-Count PET Attenuation Map Generation

Authors: Bo Zhou, Jun Hou, Tianqi Chen, Yinchi Zhou, Xiongchao Chen, Huidong Xie, Qiong Liu, Xueqi Guo, Yu-Jung Tsai, Vladimir Y. Panin, Takuya Toyonaga, James S. Duncan, Chi Liu

Abstract: Low-dose PET offers a valuable means of minimizing radiation exposure in PET imaging. However, the prevalent practice of employing additional CT scans for generating attenuation maps (u-map) for PET attenuation correction significantly elevates radiation doses. To address this concern and further mitigate radiation exposure in low-dose PET exams, we propose POUR-Net - an innovative population-prio… ▽ More Low-dose PET offers a valuable means of minimizing radiation exposure in PET imaging. However, the prevalent practice of employing additional CT scans for generating attenuation maps (u-map) for PET attenuation correction significantly elevates radiation doses. To address this concern and further mitigate radiation exposure in low-dose PET exams, we propose POUR-Net - an innovative population-prior-aided over-under-representation network that aims for high-quality attenuation map generation from low-dose PET. First, POUR-Net incorporates an over-under-representation network (OUR-Net) to facilitate efficient feature extraction, encompassing both low-resolution abstracted and fine-detail features, for assisting deep generation on the full-resolution level. Second, complementing OUR-Net, a population prior generation machine (PPGM) utilizing a comprehensive CT-derived u-map dataset, provides additional prior information to aid OUR-Net generation. The integration of OUR-Net and PPGM within a cascade framework enables iterative refinement of $μ$-map generation, resulting in the production of high-quality $μ$-maps. Experimental results underscore the effectiveness of POUR-Net, showing it as a promising solution for accurate CT-free low-count PET attenuation correction, which also surpasses the performance of previous baseline methods. △ Less

Submitted 25 January, 2024; originally announced January 2024.

Comments: 10 pages, 5 figures

Dual-Domain Coarse-to-Fine Progressive Estimation Network for Simultaneous Denoising, Limited-View Reconstruction, and Attenuation Correction of Cardiac SPECT

Authors: Xiongchao Chen, Bo Zhou, Xueqi Guo, Huidong Xie, Qiong Liu, James S. Duncan, Albert J. Sinusas, Chi Liu

Abstract: Single-Photon Emission Computed Tomography (SPECT) is widely applied for the diagnosis of coronary artery diseases. Low-dose (LD) SPECT aims to minimize radiation exposure but leads to increased image noise. Limited-view (LV) SPECT, such as the latest GE MyoSPECT ES system, enables accelerated scanning and reduces hardware expenses but degrades reconstruction accuracy. Additionally, Computed Tomog… ▽ More Single-Photon Emission Computed Tomography (SPECT) is widely applied for the diagnosis of coronary artery diseases. Low-dose (LD) SPECT aims to minimize radiation exposure but leads to increased image noise. Limited-view (LV) SPECT, such as the latest GE MyoSPECT ES system, enables accelerated scanning and reduces hardware expenses but degrades reconstruction accuracy. Additionally, Computed Tomography (CT) is commonly used to derive attenuation maps ($μ$-maps) for attenuation correction (AC) of cardiac SPECT, but it will introduce additional radiation exposure and SPECT-CT misalignments. Although various methods have been developed to solely focus on LD denoising, LV reconstruction, or CT-free AC in SPECT, the solution for simultaneously addressing these tasks remains challenging and under-explored. Furthermore, it is essential to explore the potential of fusing cross-domain and cross-modality information across these interrelated tasks to further enhance the accuracy of each task. Thus, we propose a Dual-Domain Coarse-to-Fine Progressive Network (DuDoCFNet), a multi-task learning method for simultaneous LD denoising, LV reconstruction, and CT-free $μ$-map generation of cardiac SPECT. Paired dual-domain networks in DuDoCFNet are cascaded using a multi-layer fusion mechanism for cross-domain and cross-modality feature fusion. Two-stage progressive learning strategies are applied in both projection and image domains to achieve coarse-to-fine estimations of SPECT projections and CT-derived $μ$-maps. Our experiments demonstrate DuDoCFNet's superior accuracy in estimating projections, generating $μ$-maps, and AC reconstructions compared to existing single- or multi-task learning methods, under various iterations and LD levels. The source code of this work is available at https://github.com/XiongchaoChen/DuDoCFNet-MultiTask. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: 11 Pages, 10 figures, 4 tables

An Adaptive Correspondence Scoring Framework for Unsupervised Image Registration of Medical Images

Authors: Xiaoran Zhang, John C. Stendahl, Lawrence Staib, Albert J. Sinusas, Alex Wong, James S. Duncan

Abstract: We propose an adaptive training scheme for unsupervised medical image registration. Existing methods rely on image reconstruction as the primary supervision signal. However, nuisance variables (e.g. noise and covisibility) often cause the loss of correspondence between medical images, violating the Lambertian assumption in physical waves (e.g. ultrasound) and consistent imaging acquisition. As the… ▽ More We propose an adaptive training scheme for unsupervised medical image registration. Existing methods rely on image reconstruction as the primary supervision signal. However, nuisance variables (e.g. noise and covisibility) often cause the loss of correspondence between medical images, violating the Lambertian assumption in physical waves (e.g. ultrasound) and consistent imaging acquisition. As the unsupervised learning scheme relies on intensity constancy to establish correspondence between images for reconstruction, this introduces spurious error residuals that are not modeled by the typical training objective. To mitigate this, we propose an adaptive framework that re-weights the error residuals with a correspondence scoring map during training, preventing the parametric displacement estimator from drifting away due to noisy gradients, which leads to performance degradations. To illustrate the versatility and effectiveness of our method, we tested our framework on three representative registration architectures across three medical image datasets along with other baselines. Our proposed adaptive framework consistently outperforms other methods both quantitatively and qualitatively. Paired t-tests show that our improvements are statistically significant. The code will be publicly available at \url{https://voldemort108x.github.io/AdaCS/}. △ Less

Submitted 30 November, 2023; originally announced December 2023.

Heteroscedastic Uncertainty Estimation for Probabilistic Unsupervised Registration of Noisy Medical Images

Authors: Xiaoran Zhang, Daniel H. Pak, Shawn S. Ahn, Xiaoxiao Li, Chenyu You, Lawrence Staib, Albert J. Sinusas, Alex Wong, James S. Duncan

Abstract: This paper proposes a heteroscedastic uncertainty estimation framework for unsupervised medical image registration. Existing methods rely on objectives (e.g. mean-squared error) that assume a uniform noise level across the image, disregarding the heteroscedastic and input-dependent characteristics of noise distribution in real-world medical images. This further introduces noisy gradients due to un… ▽ More This paper proposes a heteroscedastic uncertainty estimation framework for unsupervised medical image registration. Existing methods rely on objectives (e.g. mean-squared error) that assume a uniform noise level across the image, disregarding the heteroscedastic and input-dependent characteristics of noise distribution in real-world medical images. This further introduces noisy gradients due to undesired penalization on outliers, causing unnatural deformation and performance degradation. To mitigate this, we propose an adaptive weighting scheme with a relative $γ$-exponentiated signal-to-noise ratio (SNR) for the displacement estimator after modeling the heteroscedastic noise using a separate variance estimator to prevent the model from being driven away by spurious gradients from error residuals, leading to more accurate displacement estimation. To illustrate the versatility and effectiveness of the proposed method, we tested our framework on two representative registration architectures across three medical image datasets. Our proposed framework consistently outperforms other baselines both quantitatively and qualitatively while also providing accurate and sensible uncertainty measures. Paired t-tests show that our improvements in registration accuracy are statistically significant. The code will be publicly available at \url{https://voldemort108x.github.io/hetero_uncertainty/}. △ Less

Submitted 30 November, 2023; originally announced December 2023.

Incomplete resection of the icEEG seizure onset zone is not associated with post-surgical outcomes

Authors: Sarah J. Gascoigne, Nathan Evans, Gerard Hall, Csaba Kozma, Mariella Panagiotopoulou, Gabrielle M. Schroeder, Callum Simpson, Christopher Thornton, Frances Turner, Heather Woodhouse, Jess Blickwedel, Fahmida Chowdhury, Beate Diehl, John S. Duncan, Ryan Faulder, Rhys H. Thomas, Kevin Wilson, Peter N. Taylor, Yujiang Wang

Abstract: Delineation of seizure onset regions from EEG is important for effective surgical workup. However, it is unknown if their complete resection is required for seizure freedom, or in other words, if post-surgical seizure recurrence is due to incomplete removal of the seizure onset regions. Retrospective analysis of icEEG recordings from 63 subjects (735 seizures) identified seizure onset regions th… ▽ More Delineation of seizure onset regions from EEG is important for effective surgical workup. However, it is unknown if their complete resection is required for seizure freedom, or in other words, if post-surgical seizure recurrence is due to incomplete removal of the seizure onset regions. Retrospective analysis of icEEG recordings from 63 subjects (735 seizures) identified seizure onset regions through visual inspection and algorithmic delineation. We analysed resection of onset regions and correlated this with post-surgical seizure control. Most subjects had over half of onset regions resected (70.7% and 60.5% of subjects for visual and algorithmic methods, respectively). In investigating spatial extent of onset or resection, and presence of diffuse onsets, we found no substantial evidence of association with post-surgical seizure control (all AUC<0.7, p>0.05). Seizure onset regions tends to be at least partially resected, however a less complete resection is not associated with worse post-surgical outcome. We conclude that seizure recurrence after epilepsy surgery is not necessarily a result of failing to completely resect the seizure onset zone, as defined by icEEG. Other network mechanisms must be involved, which are not limited to seizure onset regions alone. △ Less

Submitted 24 November, 2023; originally announced November 2023.

Diminished circadian and ultradian rhythms in pathological brain tissue in human in vivo

Authors: Mariella Panagiotopoulou, Christopher Thornton, Fahmida A Chowdhury, Beate Diehl, John S Duncan, Sarah J Gascoigne, Andrew W McEvoy, Anna Miserocchi, Billy C Smith, Jane de Tisi, Peter N Taylor, Yujiang Wang

Abstract: Chronobiological rhythms, such as the circadian rhythm, have long been linked to neurological disorders, but it is currently unknown how pathological processes affect the expression of biological rhythms in the brain. Here, we use the unique opportunity of long-term, continuous intracranially recorded EEG from 38 patients (totalling 6338 hours) to delineate circadian and ultradian rhythms in diffe… ▽ More Chronobiological rhythms, such as the circadian rhythm, have long been linked to neurological disorders, but it is currently unknown how pathological processes affect the expression of biological rhythms in the brain. Here, we use the unique opportunity of long-term, continuous intracranially recorded EEG from 38 patients (totalling 6338 hours) to delineate circadian and ultradian rhythms in different brain regions. We show that functional circadian and ultradian rhythms are diminished in pathological tissue, independent of regional variations. We further demonstrate that these diminished rhythms are persistent in time, regardless of load or occurrence of pathological events. These findings provide the first evidence that brain pathology is functionally associated with persistently diminished chronobiological rhythms in vivo in humans, independent of regional variations or pathological events. Future work interacting with, and restoring, these modulatory chronobiological rhythms may allow for novel therapies. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Preserved Edge Convolutional Neural Network for Sensitivity Enhancement of Deuterium Metabolic Imaging (DMI)

Authors: Siyuan Dong, Henk M. De Feyter, Monique A. Thomas, Robin A. de Graaf, James S. Duncan

Abstract: Purpose: Common to most MRSI techniques, the spatial resolution and the minimal scan duration of Deuterium Metabolic Imaging (DMI) are limited by the achievable SNR. This work presents a deep learning method for sensitivity enhancement of DMI. Methods: A convolutional neural network (CNN) was designed to estimate the 2H-labeled metabolite concentrations from low SNR and distorted DMI FIDs. The C… ▽ More Purpose: Common to most MRSI techniques, the spatial resolution and the minimal scan duration of Deuterium Metabolic Imaging (DMI) are limited by the achievable SNR. This work presents a deep learning method for sensitivity enhancement of DMI. Methods: A convolutional neural network (CNN) was designed to estimate the 2H-labeled metabolite concentrations from low SNR and distorted DMI FIDs. The CNN was trained with synthetic data that represent a range of SNR levels typically encountered in vivo. The estimation precision was further improved by fine-tuning the CNN with MRI-based edge-preserving regularization for each DMI dataset. The proposed processing method, PReserved Edge ConvolutIonal neural network for Sensitivity Enhanced DMI (PRECISE-DMI), was applied to simulation studies and in vivo experiments to evaluate the anticipated improvements in SNR and investigate the potential for inaccuracies. Results: PRECISE-DMI visually improved the metabolic maps of low SNR datasets, and quantitatively provided higher precision than the standard Fourier reconstruction. Processing of DMI data acquired in rat brain tumor models resulted in more precise determination of 2H-labeled lactate and glutamate + glutamine levels, at increased spatial resolution (from >8 to 2 $μ$L) or shortened scan time (from 32 to 4 min) compared to standard acquisitions. However, rigorous SD-bias analyses showed that overuse of the edge-preserving regularization can compromise the accuracy of the results. Conclusion: PRECISE-DMI allows a flexible trade-off between enhancing the sensitivity of DMI and minimizing the inaccuracies. With typical settings, the DMI sensitivity can be improved by 3-fold while retaining the capability to detect local signal variations. △ Less

Submitted 13 September, 2023; v1 submitted 7 September, 2023; originally announced September 2023.

Learning Sequential Information in Task-based fMRI for Synthetic Data Augmentation

Authors: Jiyao Wang, Nicha C. Dvornek, Lawrence H. Staib, James S. Duncan

Abstract: Insufficiency of training data is a persistent issue in medical image analysis, especially for task-based functional magnetic resonance images (fMRI) with spatio-temporal imaging data acquired using specific cognitive tasks. In this paper, we propose an approach for generating synthetic fMRI sequences that can then be used to create augmented training datasets in downstream learning tasks. To synt… ▽ More Insufficiency of training data is a persistent issue in medical image analysis, especially for task-based functional magnetic resonance images (fMRI) with spatio-temporal imaging data acquired using specific cognitive tasks. In this paper, we propose an approach for generating synthetic fMRI sequences that can then be used to create augmented training datasets in downstream learning tasks. To synthesize high-resolution task-specific fMRI, we adapt the $α$-GAN structure, leveraging advantages of both GAN and variational autoencoder models, and propose different alternatives in aggregating temporal information. The synthetic images are evaluated from multiple perspectives including visualizations and an autism spectrum disorder (ASD) classification task. The results show that the synthetic task-based fMRI can provide effective data augmentation in learning the ASD classification task. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Comments: Accepted by Machine Learning in Clinical Neuroimaging 2023 (MICCAI workshop), preprint version

Copy Number Variation Informs fMRI-based Prediction of Autism Spectrum Disorder

Authors: Nicha C. Dvornek, Catherine Sullivan, James S. Duncan, Abha R. Gupta

Abstract: The multifactorial etiology of autism spectrum disorder (ASD) suggests that its study would benefit greatly from multimodal approaches that combine data from widely varying platforms, e.g., neuroimaging, genetics, and clinical characterization. Prior neuroimaging-genetic analyses often apply naive feature concatenation approaches in data-driven work or use the findings from one modality to guide p… ▽ More The multifactorial etiology of autism spectrum disorder (ASD) suggests that its study would benefit greatly from multimodal approaches that combine data from widely varying platforms, e.g., neuroimaging, genetics, and clinical characterization. Prior neuroimaging-genetic analyses often apply naive feature concatenation approaches in data-driven work or use the findings from one modality to guide posthoc analysis of another, missing the opportunity to analyze the paired multimodal data in a truly unified approach. In this paper, we develop a more integrative model for combining genetic, demographic, and neuroimaging data. Inspired by the influence of genotype on phenotype, we propose using an attention-based approach where the genetic data guides attention to neuroimaging features of importance for model prediction. The genetic data is derived from copy number variation parameters, while the neuroimaging data is from functional magnetic resonance imaging. We evaluate the proposed approach on ASD classification and severity prediction tasks, using a sex-balanced dataset of 228 ASD and typically developing subjects in a 10-fold cross-validation framework. We demonstrate that our attention-based model combining genetic information, demographic data, and functional magnetic resonance imaging results in superior prediction performance compared to other multimodal approaches. △ Less

Submitted 8 August, 2023; originally announced August 2023.

Comments: Accepted by Machine Learning in Clinical Neuroimaging 2023 (MICCAI workshop), preprint version

Rapid Brain Meninges Surface Reconstruction with Layer Topology Guarantee

Authors: Peiyu Duan, Yuan Xue, Shuo Han, Lianrui Zuo, Aaron Carass, Caitlyn Bernhard, Savannah Hays, Peter A. Calabresi, Susan M. Resnick, James S. Duncan, Jerry L. Prince

Abstract: The meninges, located between the skull and brain, are composed of three membrane layers: the pia, the arachnoid, and the dura. Reconstruction of these layers can aid in studying volume differences between patients with neurodegenerative diseases and normal aging subjects. In this work, we use convolutional neural networks (CNNs) to reconstruct surfaces representing meningeal layer boundaries from… ▽ More The meninges, located between the skull and brain, are composed of three membrane layers: the pia, the arachnoid, and the dura. Reconstruction of these layers can aid in studying volume differences between patients with neurodegenerative diseases and normal aging subjects. In this work, we use convolutional neural networks (CNNs) to reconstruct surfaces representing meningeal layer boundaries from magnetic resonance (MR) images. We first use the CNNs to predict the signed distance functions (SDFs) representing these surfaces while preserving their anatomical ordering. The marching cubes algorithm is then used to generate continuous surface representations; both the subarachnoid space (SAS) and the intracranial volume (ICV) are computed from these surfaces. The proposed method is compared to a state-of-the-art deformable model-based reconstruction method, and we show that our method can reconstruct smoother and more accurate surfaces using less computation time. Finally, we conduct experiments with volumetric analysis on both subjects with multiple sclerosis and healthy controls. For healthy and MS subjects, ICVs and SAS volumes are found to be significantly correlated to sex (p<0.01) and age (p<0.03) changes, respectively. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Comments: ISBI 2023 Oral

Interictal MEG abnormalities to guide intracranial electrode implantation and predict surgical outcome

Authors: Thomas W. Owen, Vytene Janiukstyte, Gerard R. Hall, Fahmida A. Chowdhury, Beate Diehl, Andrew McEvoy, Anna Miserocchi, Jane de Tisi, John S. Duncan, Fergus Rugg-Gunn, Yujiang Wang, Peter N. Taylor

Abstract: Intracranial EEG (iEEG) is the gold standard technique for epileptogenic zone (EZ) localisation, but requires a hypothesis of which tissue is epileptogenic, guided by qualitative analysis of seizure semiology and other imaging modalities such as magnetoencephalography (MEG). We hypothesised that if quantifiable MEG band power abnormalities were sampled by iEEG, then patients' post-resection seizur… ▽ More Intracranial EEG (iEEG) is the gold standard technique for epileptogenic zone (EZ) localisation, but requires a hypothesis of which tissue is epileptogenic, guided by qualitative analysis of seizure semiology and other imaging modalities such as magnetoencephalography (MEG). We hypothesised that if quantifiable MEG band power abnormalities were sampled by iEEG, then patients' post-resection seizure outcome were better. Thirty-two individuals with neocortical epilepsy underwent MEG and iEEG recordings as part of pre-surgical evaluation. Interictal MEG band power abnormalities were derived using 70 healthy controls as a normative baseline. MEG abnormality maps were compared to electrode implantation, with the spatial overlap of iEEG electrodes and MEG abnormalities recorded. Finally, we assessed if the implantation of electrodes in abnormal tissue, and resection of the strongest abnormalities determined by MEG and iEEG explained surgical outcome. Intracranial electrodes were implanted in brain tissue with the most abnormal MEG findings in individuals that were seizure-free post-resection (T=3.9, p=0.003). The overlap between MEG abnormalities and iEEG electrodes distinguished outcome groups moderately well (AUC=0.68). In isolation, the resection of the strongest MEG and iEEG abnormalities separated surgical outcome groups well (AUC=0.71, AUC=0.74 respectively). A model incorporating all three features separated outcome groups best (AUC=0.80). Intracranial EEG is a key tool to delineate the EZ and help render patients seizure-free after resection. We showed that data-driven abnormalities derived from interictal MEG recordings have clinical value and may help guide electrode placement in individuals with neocortical epilepsy. Finally, our predictive model of post-operative seizure-freedom, which leverages both MEG and iEEG recordings, may aid patient counselling of expected outcome. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Comments: 22 pages, 6 figures

Identifying epileptogenic abnormalities through spatial clustering of MEG interictal band power

Authors: Thomas W. Owen, Vytene Janiukstyte, Gerard R. Hall, Jonathan J. Horsley, Andrew McEvoy, Anna Miserocchi, Jane de Tisi, John S. Duncan, Fergus Rugg-Gunn, Yujiang Wang, Peter N. Taylor

Abstract: Successful epilepsy surgery depends on localising and resecting cerebral abnormalities and networks that generate seizures. Abnormalities, however, may be widely distributed across multiple discontiguous areas. We propose spatially constrained clusters as candidate areas for further investigation, and potential resection. We quantified the spatial overlap between the abnormality cluster and subseq… ▽ More Successful epilepsy surgery depends on localising and resecting cerebral abnormalities and networks that generate seizures. Abnormalities, however, may be widely distributed across multiple discontiguous areas. We propose spatially constrained clusters as candidate areas for further investigation, and potential resection. We quantified the spatial overlap between the abnormality cluster and subsequent resection, hypothesising a greater overlap in seizure-free patients. Thirty-four individuals with refractory focal epilepsy underwent pre-surgical resting-state interictal MEG recording. Fourteen individuals were totally seizure free (ILAE 1) after surgery and 20 continued to have some seizures post-operatively (ILAE 2+). Band power abnormality maps were derived using controls as a baseline. Patient abnormalities were spatially clustered using the k-means algorithm. The tissue within the cluster containing the most abnormal region was compared with the resection volume using the dice score. The proposed abnormality cluster overlapped with the resection in 71% of ILAE 1 patients. Conversely, an overlap only occurred in 15% of ILAE 2+ patients. This effect discriminated outcome groups well (AUC=0.82). Our novel approach identifies clusters of spatially similar tissue with high abnormality. This is clinically valuable, providing (i) a data-driven framework to validate current hypotheses of the epileptogenic zone localisation or (ii) to guide further investigation. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Comments: 16 pages, 3 figures

Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts

Authors: Chenyu You, Weicheng Dai, Yifei Min, Lawrence Staib, James S. Duncan

Abstract: Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the… ▽ More Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the high-frequency regions, i.e., boundary regions. In this work, we propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our method is motivated by the fact that implicit neural representation has been shown to be more effective in fitting complex signals and solving computer graphics problems than discrete grid-based representation. The core of our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Specifically, we continuously align the coarse segmentation prediction with the ambiguous coordinate-based point representations and aggregate these features to adaptively refine the boundary region. To parallelly optimize multi-scale pixel-level features, we leverage the idea from Mixture-of-Expert (MoE) to design and train our MORSE with a stochastic gating mechanism. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones, consistently achieving competitive performance improvements in both 2D and 3D supervised medical segmentation methods. We also theoretically analyze the superiority of MORSE. △ Less

Submitted 17 July, 2023; v1 submitted 6 April, 2023; originally announced April 2023.

Comments: Accepted at International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2023)

Normative brain mapping using scalp EEG and potential clinical application

Authors: Vytene Janiukstyte, Thomas W Owen, Umair J Chaudhary, Beate Diehl, Louis Lemieux, John S Duncan, Jane de Tisi, Yujiang Wang, Peter N Taylor

Abstract: A normative electrographic activity map could be a powerful resource to understand normal brain function and identify abnormal activity. Here, we present a normative brain map using scalp EEG in terms of relative band power. In this exploratory study we investigate its temporal stability, its similarity to other imaging modalities, and explore a potential clinical application. We constructed sca… ▽ More A normative electrographic activity map could be a powerful resource to understand normal brain function and identify abnormal activity. Here, we present a normative brain map using scalp EEG in terms of relative band power. In this exploratory study we investigate its temporal stability, its similarity to other imaging modalities, and explore a potential clinical application. We constructed scalp EEG normative maps of brain dynamics from 17 healthy controls using source-localised resting-state scalp recordings. We then correlated these maps with those acquired from MEG and intracranial EEG to investigate their similarity. Lastly, we use the normative maps to lateralise abnormal regions in epilepsy. Spatial patterns of band powers were broadly consistent with previous literature and stable across recordings. Scalp EEG normative maps were most similar to other modalities in the alpha band, and relatively similar across most bands. Towards a clinical application in epilepsy, we found abnormal temporal regions ipsilateral to the epileptogenic hemisphere. Scalp EEG relative band power normative maps are spatially stable across time, in keeping with MEG and intracranial EEG results. Normative mapping is feasible and may be potentially clinically useful in epilepsy. Future studies with larger sample sizes and high-density EEG are now required for validation. △ Less

Submitted 6 April, 2023; originally announced April 2023.

Comments: 4 figures

Complementary structural and functional abnormalities to localise epileptogenic tissue

Authors: Jonathan J Horsley, Rhys H Thomas, Fahmida A Chowdhury, Beate Diehl, Andrew W McEvoy, Anna Miserocchi, Jane de Tisi, Sjoerd B Vos, Matthew C Walker, Gavin P Winston, John S Duncan, Yujiang Wang, Peter N Taylor

Abstract: When investigating suitability for surgery, people with drug-refractory focal epilepsy may have intracranial EEG (iEEG) electrodes implanted to localise seizure onset. Diffusion-weighted magnetic resonance imaging (dMRI) may be acquired to identify key white matter tracts for surgical avoidance. Here, we investigate whether structural connectivity abnormalities, inferred from dMRI, may be used in… ▽ More When investigating suitability for surgery, people with drug-refractory focal epilepsy may have intracranial EEG (iEEG) electrodes implanted to localise seizure onset. Diffusion-weighted magnetic resonance imaging (dMRI) may be acquired to identify key white matter tracts for surgical avoidance. Here, we investigate whether structural connectivity abnormalities, inferred from dMRI, may be used in conjunction with functional iEEG abnormalities to aid localisation and resection of the epileptogenic zone (EZ), and improve surgical outcomes in epilepsy. We retrospectively investigated data from 43 patients with epilepsy who had surgery following iEEG. Twenty five patients (58%) were free from disabling seizures (ILAE 1 or 2) at one year. For all patients, T1-weighted and diffusion-weighted MRIs were acquired prior to iEEG implantation. Interictal iEEG functional, and dMRI structural connectivity abnormalities were quantified by comparison to a normative map and healthy controls respectively. First, we explored whether the resection of maximal (dMRI and iEEG) abnormalities related to improved surgical outcomes. Second, we investigated whether the modalities provided complementary information for improved prediction of surgical outcome. Third, we suggest how dMRI abnormalities may be useful to inform the placement of iEEG electrodes as part of the pre-surgical evaluation using a patient case study. Seizure freedom was 15 times more likely in those patients with resection of maximal dMRI and iEEG abnormalities (p=0.008). Both modalities were separately able to distinguish patient outcome groups and when combined, a decision tree correctly separated 36 out of 43 (84%) patients based on surgical outcome. Structural dMRI could be used in pre-surgical evaluations, particularly when localisation of the EZ is uncertain, to inform personalised iEEG implantation and resection. △ Less

Submitted 24 October, 2023; v1 submitted 6 April, 2023; originally announced April 2023.

Comments: 5 figures

ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast

Authors: Chenyu You, Weicheng Dai, Yifei Min, Lawrence Staib, Jasjeet S. Sekhon, James S. Duncan

Abstract: Medical data often exhibits long-tail distributions with heavy class imbalance, which naturally leads to difficulty in classifying the minority classes (i.e., boundary regions or rare objects). Recent work has significantly improved semi-supervised medical image segmentation in long-tailed scenarios by equipping them with unsupervised contrastive criteria. However, it remains unclear how well they… ▽ More Medical data often exhibits long-tail distributions with heavy class imbalance, which naturally leads to difficulty in classifying the minority classes (i.e., boundary regions or rare objects). Recent work has significantly improved semi-supervised medical image segmentation in long-tailed scenarios by equipping them with unsupervised contrastive criteria. However, it remains unclear how well they will perform in the labeled portion of data where class distribution is also highly imbalanced. In this work, we present ACTION++, an improved contrastive learning framework with adaptive anatomical contrast for semi-supervised medical segmentation. Specifically, we propose an adaptive supervised contrastive loss, where we first compute the optimal locations of class centers uniformly distributed on the embedding space (i.e., off-line), and then perform online contrastive matching training by encouraging different class features to adaptively match these distinct and uniformly distributed class centers. Moreover, we argue that blindly adopting a constant temperature $τ$ in the contrastive loss on long-tailed medical data is not optimal, and propose to use a dynamic $τ$ via a simple cosine schedule to yield better separation between majority and minority classes. Empirically, we evaluate ACTION++ on ACDC and LA benchmarks and show that it achieves state-of-the-art across two semi-supervised settings. Theoretically, we analyze the performance of adaptive anatomical contrast and confirm its superiority in label efficiency. △ Less

Submitted 17 July, 2023; v1 submitted 5 April, 2023; originally announced April 2023.

Comments: Accepted by International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2023)

FedFTN: Personalized Federated Learning with Deep Feature Transformation Network for Multi-institutional Low-count PET Denoising

Authors: Bo Zhou, Huidong Xie, Qiong Liu, Xiongchao Chen, Xueqi Guo, Zhicheng Feng, Jun Hou, S. Kevin Zhou, Biao Li, Axel Rominger, Kuangyu Shi, James S. Duncan, Chi Liu

Abstract: Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutio… ▽ More Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutions for training a robust model is difficult due to privacy and security concerns of patient data. Moreover, low-count PET data at different institutions may have different data distribution, thus requiring personalized models. While previous federated learning (FL) algorithms enable multi-institution collaborative training without the need of aggregating local data, addressing the large domain shift in the application of multi-institutional low-count PET denoising remains a challenge and is still highly under-explored. In this work, we propose FedFTN, a personalized federated learning strategy that addresses these challenges. FedFTN uses a local deep feature transformation network (FTN) to modulate the feature outputs of a globally shared denoising network, enabling personalized low-count PET denoising for each institution. During the federated learning process, only the denoising network's weights are communicated and aggregated, while the FTN remains at the local institutions for feature transformation. We evaluated our method using a large-scale dataset of multi-institutional low-count PET imaging data from three medical centers located across three continents, and showed that FedFTN provides high-quality low-count PET images, outperforming previous baseline FL reconstruction methods across all low-count levels at all three institutions. △ Less

Submitted 6 October, 2023; v1 submitted 2 April, 2023; originally announced April 2023.

Comments: 13 pages, 6 figures, Accepted at Medical Image Analysis Journal (MedIA)

Meta-information-aware Dual-path Transformer for Differential Diagnosis of Multi-type Pancreatic Lesions in Multi-phase CT

Authors: Bo Zhou, Yingda Xia, Jiawen Yao, Le Lu, Jingren Zhou, Chi Liu, James S. Duncan, Ling Zhang

Abstract: Pancreatic cancer is one of the leading causes of cancer-related death. Accurate detection, segmentation, and differential diagnosis of the full taxonomy of pancreatic lesions, i.e., normal, seven major types of lesions, and other lesions, is critical to aid the clinical decision-making of patient management and treatment. However, existing works focus on segmentation and classification for very s… ▽ More Pancreatic cancer is one of the leading causes of cancer-related death. Accurate detection, segmentation, and differential diagnosis of the full taxonomy of pancreatic lesions, i.e., normal, seven major types of lesions, and other lesions, is critical to aid the clinical decision-making of patient management and treatment. However, existing works focus on segmentation and classification for very specific lesion types (PDAC) or groups. Moreover, none of the previous work considers using lesion prevalence-related non-imaging patient information to assist the differential diagnosis. To this end, we develop a meta-information-aware dual-path transformer and exploit the feasibility of classification and segmentation of the full taxonomy of pancreatic lesions. Specifically, the proposed method consists of a CNN-based segmentation path (S-path) and a transformer-based classification path (C-path). The S-path focuses on initial feature extraction by semantic segmentation using a UNet-based network. The C-path utilizes both the extracted features and meta-information for patient-level classification based on stacks of dual-path transformer blocks that enhance the modeling of global contextual information. A large-scale multi-phase CT dataset of 3,096 patients with pathology-confirmed pancreatic lesion class labels, voxel-wise manual annotations of lesions from radiologists, and patient meta-information, was collected for training and evaluations. Our results show that our method can enable accurate classification and segmentation of the full taxonomy of pancreatic lesions, approaching the accuracy of the radiologist's report and significantly outperforming previous baselines. Results also show that adding the common meta-information, i.e., gender and age, can boost the model's performance, thus demonstrating the importance of meta-information for aiding pancreatic disease diagnosis. △ Less

Submitted 1 March, 2023; originally announced March 2023.

Comments: Accepted at Information Processing in Medical Imaging (IPMI 2023)

Dual-Domain Self-Supervised Learning for Accelerated Non-Cartesian MRI Reconstruction

Authors: Bo Zhou, Jo Schlemper, Neel Dey, Seyed Sadegh Mohseni Salehi, Kevin Sheth, Chi Liu, James S. Duncan, Michal Sofka

Abstract: While enabling accelerated acquisition and improved reconstruction accuracy, current deep MRI reconstruction networks are typically supervised, require fully sampled data, and are limited to Cartesian sampling patterns. These factors limit their practical adoption as fully-sampled MRI is prohibitively time-consuming to acquire clinically. Further, non-Cartesian sampling patterns are particularly d… ▽ More While enabling accelerated acquisition and improved reconstruction accuracy, current deep MRI reconstruction networks are typically supervised, require fully sampled data, and are limited to Cartesian sampling patterns. These factors limit their practical adoption as fully-sampled MRI is prohibitively time-consuming to acquire clinically. Further, non-Cartesian sampling patterns are particularly desirable as they are more amenable to acceleration and show improved motion robustness. To this end, we present a fully self-supervised approach for accelerated non-Cartesian MRI reconstruction which leverages self-supervision in both k-space and image domains. In training, the undersampled data are split into disjoint k-space domain partitions. For the k-space self-supervision, we train a network to reconstruct the input undersampled data from both the disjoint partitions and from itself. For the image-level self-supervision, we enforce appearance consistency obtained from the original undersampled data and the two partitions. Experimental results on our simulated multi-coil non-Cartesian MRI dataset demonstrate that DDSS can generate high-quality reconstruction that approaches the accuracy of the fully supervised reconstruction, outperforming previous baseline methods. Finally, DDSS is shown to scale to highly challenging real-world clinical MRI reconstruction acquired on a portable low-field (0.064 T) MRI scanner with no data available for supervised training while demonstrating improved image quality as compared to traditional reconstruction, as determined by a radiologist study. △ Less

Submitted 18 February, 2023; originally announced February 2023.

Comments: 14 pages, 10 figures, published at Medical Image Analysis (MedIA)

Fast-MC-PET: A Novel Deep Learning-aided Motion Correction and Reconstruction Framework for Accelerated PET

Authors: Bo Zhou, Yu-Jung Tsai, Jiazhen Zhang, Xueqi Guo, Huidong Xie, Xiongchao Chen, Tianshun Miao, Yihuan Lu, James S. Duncan, Chi Liu

Abstract: Patient motion during PET is inevitable. Its long acquisition time not only increases the motion and the associated artifacts but also the patient's discomfort, thus PET acceleration is desirable. However, accelerating PET acquisition will result in reconstructed images with low SNR, and the image quality will still be degraded by motion-induced artifacts. Most of the previous PET motion correctio… ▽ More Patient motion during PET is inevitable. Its long acquisition time not only increases the motion and the associated artifacts but also the patient's discomfort, thus PET acceleration is desirable. However, accelerating PET acquisition will result in reconstructed images with low SNR, and the image quality will still be degraded by motion-induced artifacts. Most of the previous PET motion correction methods are motion type specific that require motion modeling, thus may fail when multiple types of motion present together. Also, those methods are customized for standard long acquisition and could not be directly applied to accelerated PET. To this end, modeling-free universal motion correction reconstruction for accelerated PET is still highly under-explored. In this work, we propose a novel deep learning-aided motion correction and reconstruction framework for accelerated PET, called Fast-MC-PET. Our framework consists of a universal motion correction (UMC) and a short-to-long acquisition reconstruction (SL-Reon) module. The UMC enables modeling-free motion correction by estimating quasi-continuous motion from ultra-short frame reconstructions and using this information for motion-compensated reconstruction. Then, the SL-Recon converts the accelerated UMC image with low counts to a high-quality image with high counts for our final reconstruction output. Our experimental results on human studies show that our Fast-MC-PET can enable 7-fold acceleration and use only 2 minutes acquisition to generate high-quality reconstruction images that outperform/match previous motion correction reconstruction methods using standard 15 minutes long acquisition data. △ Less

Submitted 14 February, 2023; originally announced February 2023.

Comments: Accepted at Information Processing in Medical Imaging (IPMI 2023)

Temporal stability of intracranial EEG abnormality maps for localising epileptogenic tissue

Authors: Yujiang Wang, Gabrielle M Schroeder, Jonathan J Horsley, Mariella Panagiotopoulou, Fahmida A Chowdhury, Beate Diehl, John S Duncan, Andrew W McEvoy, Anna Miserocchi, Jane de Tisi, Peter N Taylor

Abstract: Objective: Identifying abnormalities in interictal intracranial EEG, by comparing patient data to a normative map, has shown promise for the localisation of epileptogenic tissue and prediction of outcome. The approach typically uses short interictal segments of around one minute. However, the temporal stability of findings has not been established. Methods: Here, we generated a normative map of… ▽ More Objective: Identifying abnormalities in interictal intracranial EEG, by comparing patient data to a normative map, has shown promise for the localisation of epileptogenic tissue and prediction of outcome. The approach typically uses short interictal segments of around one minute. However, the temporal stability of findings has not been established. Methods: Here, we generated a normative map of iEEG in non-pathological brain tissue from 249 patients. We computed regional band power abnormalities in a separate cohort of 39 patients for the duration of their monitoring period (0.92-8.62 days of iEEG data, mean 4.58 days per patient, over 4,800 hours recording). To assess the localising value of band power abnormality, we computed DRS - a measure of how different the surgically resected and spared tissue were in terms of band power abnormalities - over time. Results: In each patient, band power abnormality was relatively consistent over time. The median DRS of the entire recording period separated seizure free (ILAE = 1) and not seizure free (ILAE > 1) patients well (AUC = 0.69). This effect was similar interictally (AUC = 0.69) and peri-ictally (AUC = 0.71). Significance: Our results suggest that band power abnormality DRS, as a predictor of outcomes from epilepsy surgery, is a relatively robust metric over time. These findings add further support for abnormality mapping of neurophysiology data during presurgical evaluation. △ Less

Submitted 11 February, 2023; originally announced February 2023.

Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective

Authors: Chenyu You, Weicheng Dai, Yifei Min, Fenglin Liu, David A. Clifton, S Kevin Zhou, Lawrence Hamilton Staib, James S Duncan

Abstract: For medical image segmentation, contrastive learning is the dominant practice to improve the quality of visual representations by contrasting semantically similar and dissimilar pairs of samples. This is enabled by the observation that without accessing ground truth labels, negative examples with truly dissimilar anatomical features, if sampled, can significantly improve the performance. In realit… ▽ More For medical image segmentation, contrastive learning is the dominant practice to improve the quality of visual representations by contrasting semantically similar and dissimilar pairs of samples. This is enabled by the observation that without accessing ground truth labels, negative examples with truly dissimilar anatomical features, if sampled, can significantly improve the performance. In reality, however, these samples may come from similar anatomical regions and the models may struggle to distinguish the minority tail-class samples, making the tail classes more prone to misclassification, both of which typically lead to model collapse. In this paper, we propose ARCO, a semi-supervised contrastive learning (CL) framework with stratified group theory for medical image segmentation. In particular, we first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks with extremely limited labels. Furthermore, we theoretically prove these sampling techniques are universal in variance reduction. Finally, we experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings, and our methods consistently outperform state-of-the-art semi-supervised methods. Additionally, we augment the CL frameworks with these sampling techniques and demonstrate significant gains over previous methods. We believe our work is an important step towards semi-supervised medical image segmentation by quantifying the limitation of current self-supervision objectives for accomplishing such challenging safety-critical tasks. △ Less

Submitted 23 October, 2023; v1 submitted 3 February, 2023; originally announced February 2023.

Comments: Accepted by Advances in Neural Information Processing Systems (NeurIPS 2023)

Effects of anterior temporal lobe resection on cortical morphology

Authors: Karoline Leiberg, Jane de Tisi, John S Duncan, Bethany Little, Peter N Taylor, Sjoerd B Vos, Gavin P Winston, Bruno Mota, Yujiang Wang

Abstract: Anterior temporal lobe resection (ATLR) is a surgical procedure to treat drug-resistant temporal lobe epilepsy (TLE). Resection may involve large amounts of cortical tissue. Here, we examine the effects of this surgery on cortical morphology measured in independent variables both near the resection and remotely. We studied 101 individuals with TLE (55 left, 46 right onset) who underwent ATLR. Fo… ▽ More Anterior temporal lobe resection (ATLR) is a surgical procedure to treat drug-resistant temporal lobe epilepsy (TLE). Resection may involve large amounts of cortical tissue. Here, we examine the effects of this surgery on cortical morphology measured in independent variables both near the resection and remotely. We studied 101 individuals with TLE (55 left, 46 right onset) who underwent ATLR. For each individual we considered one pre-surgical MRI and one follow-up MRI 2 to 13 months after surgery. We used our newly developed surface-based method to locally compute traditional morphological variables (average cortical thickness, exposed surface area, and total surface area), and the independent measures $K$, $I$, and $S$, where $K$ measures white matter tension, $I$ captures isometric scaling, and $S$ contains the remaining information about cortical shape. Data from 924 healthy controls was included to account for healthy ageing effects occurring during scans. A SurfStat random field theory clustering approach assessed changes across the cortex caused by ATLR. Compared to preoperative data, surgery had marked effects on all morphological measures. Ipsilateral effects were located in the orbitofrontal and inferior frontal gyri, the pre- and postcentral gyri and supramarginal gyrus, and the lateral occipital gyrus and lingual cortex. Contralateral effects were in the lateral occipital gyrus, and inferior frontal gyrus and frontal pole. The restructuring following ATLR is reflected in widespread morphological changes, mainly in regions near the resection, but also remotely in regions that are structurally connected to the anterior temporal lobe. The causes could include mechanical effects, Wallerian degeneration, or compensatory plasticity. The study of independent measures revealed additional effects compared to traditional measures. △ Less

Submitted 13 December, 2022; originally announced December 2022.

Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels

Authors: Chenyu You, Weicheng Dai, Fenglin Liu, Yifei Min, Haoran Su, Xiaoran Zhang, Xiaoxiao Li, David A. Clifton, Lawrence Staib, James S. Duncan

Abstract: Recent studies on contrastive learning have achieved remarkable performance solely by leveraging few labels in the context of medical image segmentation. Existing methods mainly focus on instance discrimination and invariant mapping. However, they face three common pitfalls: (1) tailness: medical image data usually follows an implicit long-tail class distribution. Blindly leveraging all pixels in… ▽ More Recent studies on contrastive learning have achieved remarkable performance solely by leveraging few labels in the context of medical image segmentation. Existing methods mainly focus on instance discrimination and invariant mapping. However, they face three common pitfalls: (1) tailness: medical image data usually follows an implicit long-tail class distribution. Blindly leveraging all pixels in training hence can lead to the data imbalance issues, and cause deteriorated performance; (2) consistency: it remains unclear whether a segmentation model has learned meaningful and yet consistent anatomical features due to the intra-class variations between different anatomical features; and (3) diversity: the intra-slice correlations within the entire dataset have received significantly less attention. This motivates us to seek a principled approach for strategically making use of the dataset itself to discover similar yet distinct samples from different anatomical views. In this paper, we introduce a novel semi-supervised 2D medical image segmentation framework termed Mine yOur owN Anatomy (MONA), and make three contributions. First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features, mainly due to lacking the supervision signal. We show two simple solutions towards learning invariances - through the use of stronger data augmentations and nearest neighbors. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features in an unsupervised manner. Lastly, we both empirically and theoretically, demonstrate the efficacy of our MONA on three benchmark datasets with different labeled settings, achieving new state-of-the-art under different labeled semi-supervised settings △ Less

Submitted 16 March, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Comments: In this version: Add theoretical analysis and correct some typos

Flow-based Visual Quality Enhancer for Super-resolution Magnetic Resonance Spectroscopic Imaging

Authors: Siyuan Dong, Gilbert Hangel, Eric Z. Chen, Shanhui Sun, Wolfgang Bogner, Georg Widhalm, Chenyu You, John A. Onofrey, Robin de Graaf, James S. Duncan

Abstract: Magnetic Resonance Spectroscopic Imaging (MRSI) is an essential tool for quantifying metabolites in the body, but the low spatial resolution limits its clinical applications. Deep learning-based super-resolution methods provided promising results for improving the spatial resolution of MRSI, but the super-resolved images are often blurry compared to the experimentally-acquired high-resolution imag… ▽ More Magnetic Resonance Spectroscopic Imaging (MRSI) is an essential tool for quantifying metabolites in the body, but the low spatial resolution limits its clinical applications. Deep learning-based super-resolution methods provided promising results for improving the spatial resolution of MRSI, but the super-resolved images are often blurry compared to the experimentally-acquired high-resolution images. Attempts have been made with the generative adversarial networks to improve the image visual quality. In this work, we consider another type of generative model, the flow-based model, of which the training is more stable and interpretable compared to the adversarial networks. Specifically, we propose a flow-based enhancer network to improve the visual quality of super-resolution MRSI. Different from previous flow-based models, our enhancer network incorporates anatomical information from additional image modalities (MRI) and uses a learnable base distribution. In addition, we impose a guide loss and a data-consistency loss to encourage the network to generate images with high visual quality while maintaining high fidelity. Experiments on a 1H-MRSI dataset acquired from 25 high-grade glioma patients indicate that our enhancer network outperforms the adversarial networks and the baseline flow-based methods. Our method also allows visual quality adjustment and uncertainty estimation. △ Less

Submitted 20 July, 2022; originally announced July 2022.

Comments: Accepted by DGM4MICCAI 2022

MEG abnormalities and mechanisms of surgical failure in neocortical epilepsy

Authors: Thomas W. Owen, Gabrielle M. Schroeder, Vytene Janiukstyte, Gerard R. Hall, Andrew McEvoy, Anna Miserocchi, Jane de Tisi, John S. Duncan, Fergus Rugg-Gunn, Yujiang Wang, Peter N. Taylor

Abstract: Neocortical epilepsy surgery fails to achieve post-operative seizure freedom in 30-40% of cases. It is not fully understood why surgery in some patients is unsuccessful. Comparing interictal MEG bandpower from patients to normative maps, which describe healthy spatial and population variability, we identify patient specific abnormalities relating to surgical failure. We propose three mechanisms co… ▽ More Neocortical epilepsy surgery fails to achieve post-operative seizure freedom in 30-40% of cases. It is not fully understood why surgery in some patients is unsuccessful. Comparing interictal MEG bandpower from patients to normative maps, which describe healthy spatial and population variability, we identify patient specific abnormalities relating to surgical failure. We propose three mechanisms contributing to poor surgical outcome; 1) failure to resect abnormalities, 2) failing to remove all epileptogenic abnormalities, and 3) insufficiently impacting the overall cortical abnormality. We develop markers of these mechanisms, validating them against patient outcomes. Resting-state MEG data were acquired for 70 healthy controls and 32 patients with refractory neocortical epilepsy. Relative bandpower maps were computed using source localised recordings from healthy controls. Patient and region-specific bandpower abnormalities were estimated as the maximum absolute z-score, using healthy data as a baseline. Resected regions were identified from post-operative MRI. We hypothesised our mechanism markers would discriminate patient's post-surgery seizure outcomes. Mechanisms of surgical failure discriminate surgical outcome groups (Abnormalities not targeted: AUC=0.80, Partial resection of the epileptogenic zone: AUC=0.68, Insufficient cortical abnormality impact: AUC=0.64). Leveraging all markers together found that 95% of those who were not seizure free had markers of surgical failure in at least one of the three proposed mechanisms. In contrast, of those patients markers for any mechanism, 80% were seizure-free. Abnormality mapping across the brain is important for a wide range of neurological conditions. Here we demonstrated that interictal MEG bandpower mapping has merit for localising pathology and improving our mechanistic understanding of epilepsy. △ Less

Submitted 11 February, 2023; v1 submitted 13 July, 2022; originally announced July 2022.

A library of quantitative markers of seizure severity

Authors: Sarah J. Gascoigne, Leonard Waldmann, Mariella Panagiotopoulou, Fahmida Chowdhury, Alison Cronie, Beate Diehl, John S. Duncan, Jennifer Falconer, Yu Guan, Veronica Leach, Shona Livingstone, Christoforos Papasavvas, Ryan Faulder, Jess Blickwedel, Gabrielle M. Schroeder, Rhys H. Thomas, Kevin Wilson, Peter N. Taylor, Yujiang Wang

Abstract: Purpose: Understanding fluctuations of seizure severity within individuals is important for defining treatment outcomes and response to therapy, as well as developing novel treatments for epilepsy. Current methods for grading seizure severity rely on qualitative interpretations from patients and clinicians. Quantitative measures of seizure severity would complement existing approaches, for EEG mon… ▽ More Purpose: Understanding fluctuations of seizure severity within individuals is important for defining treatment outcomes and response to therapy, as well as developing novel treatments for epilepsy. Current methods for grading seizure severity rely on qualitative interpretations from patients and clinicians. Quantitative measures of seizure severity would complement existing approaches, for EEG monitoring, outcome monitoring, and seizure prediction. Therefore, we developed a library of quantitative electroencephalographic (EEG) markers that assess the spread and intensity of abnormal electrical activity during and after seizures. Methods: We analysed intracranial EEG (iEEG) recordings of 1056 seizures from 63 patients. For each seizure, we computed 16 markers of seizure severity that capture the signal magnitude, spread, duration, and post-ictal suppression of seizures. Results: Quantitative EEG markers of seizure severity distinguished focal vs. subclinical and focal vs. FTBTC seizures across patients. In individual patients, 71% had a moderate to large difference (ranksum r > 0.3) between focal and subclinical seizures in three or more markers. Circadian and longer-term changes in severity were found for 67% and 53% of patients, respectively. Conclusion: We demonstrate the feasibility of using quantitative iEEG markers to measure seizure severity. Our quantitative markers distinguish between seizure types and are therefore sensitive to established qualitative differences in seizure severity. Our results also suggest that seizure severity is modulated over different timescales. We envisage that our proposed seizure severity library will be expanded and updated in collaboration with the epilepsy research community to include more measures and modalities. △ Less

Submitted 27 January, 2023; v1 submitted 30 June, 2022; originally announced June 2022.

Comments: 30 pages main text, 18 pages supplementary. Six main figures, two supplementary figures

Bootstrapping Semi-supervised Medical Image Segmentation with Anatomical-aware Contrastive Distillation

Authors: Chenyu You, Weicheng Dai, Yifei Min, Lawrence Staib, James S. Duncan

Abstract: Contrastive learning has shown great promise over annotation scarcity problems in the context of medical image segmentation. Existing approaches typically assume a balanced class distribution for both labeled and unlabeled medical images. However, medical image data in reality is commonly imbalanced (i.e., multi-class label imbalance), which naturally yields blurry contours and usually incorrectly… ▽ More Contrastive learning has shown great promise over annotation scarcity problems in the context of medical image segmentation. Existing approaches typically assume a balanced class distribution for both labeled and unlabeled medical images. However, medical image data in reality is commonly imbalanced (i.e., multi-class label imbalance), which naturally yields blurry contours and usually incorrectly labels rare objects. Moreover, it remains unclear whether all negative samples are equally negative. In this work, we present ACTION, an Anatomical-aware ConTrastive dIstillatiON framework, for semi-supervised medical image segmentation. Specifically, we first develop an iterative contrastive distillation algorithm by softly labeling the negatives rather than binary supervision between positive and negative pairs. We also capture more semantically similar features from the randomly chosen negative set compared to the positives to enforce the diversity of the sampled data. Second, we raise a more important question: Can we really handle imbalanced samples to yield better performance? Hence, the key innovation in ACTION is to learn global semantic relationship across the entire dataset and local anatomical features among the neighbouring pixels with minimal additional memory footprint. During the training, we introduce anatomical contrast by actively sampling a sparse set of hard negative pixels, which can generate smoother segmentation boundaries and more accurate predictions. Extensive experiments across two benchmark datasets and different unlabeled settings show that ACTION significantly outperforms the current state-of-the-art semi-supervised methods. △ Less

Submitted 10 March, 2023; v1 submitted 5 June, 2022; originally announced June 2022.

Comments: Accepted at Information Processing in Medical Imaging (IPMI 2023)

Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation

Authors: Chenyu You, Jinlin Xiang, Kun Su, Xiaoran Zhang, Siyuan Dong, John Onofrey, Lawrence Staib, James S. Duncan

Abstract: Many medical datasets have recently been created for medical image segmentation tasks, and it is natural to question whether we can use them to sequentially train a single model that (1) performs better on all these datasets, and (2) generalizes well and transfers better to the unknown target site domain. Prior works have achieved this goal by jointly training one model on multi-site datasets, whi… ▽ More Many medical datasets have recently been created for medical image segmentation tasks, and it is natural to question whether we can use them to sequentially train a single model that (1) performs better on all these datasets, and (2) generalizes well and transfers better to the unknown target site domain. Prior works have achieved this goal by jointly training one model on multi-site datasets, which achieve competitive performance on average but such methods rely on the assumption about the availability of all training data, thus limiting its effectiveness in practical deployment. In this paper, we propose a novel multi-site segmentation framework called incremental-transfer learning (ITL), which learns a model from multi-site datasets in an end-to-end sequential fashion. Specifically, "incremental" refers to training sequentially constructed datasets, and "transfer" is achieved by leveraging useful information from the linear combination of embedding features on each dataset. In addition, we introduce our ITL framework, where we train the network including a site-agnostic encoder with pre-trained weights and at most two segmentation decoder heads. We also design a novel site-level incremental loss in order to generalize well on the target domain. Second, we show for the first time that leveraging our ITL training scheme is able to alleviate challenging catastrophic forgetting problems in incremental learning. We conduct experiments using five challenging benchmark datasets to validate the effectiveness of our incremental-transfer learning approach. Our approach makes minimal assumptions on computation resources and domain-specific expertise, and hence constitutes a strong starting point in multi-site medical image segmentation. △ Less

Submitted 30 July, 2022; v1 submitted 2 June, 2022; originally announced June 2022.

Intracranial EEG structure-function coupling predicts surgical outcomes in focal epilepsy

Authors: Nishant Sinha, John S. Duncan, Beate Diehl, Fahmida A. Chowdhury, Jane de Tisi, Anna Miserocchi, Andrew W. McEvoy, Kathryn A. Davis, Sjoerd B. Vos, Gavin P. Winston, Yujiang Wang, Peter N. Taylor

Abstract: Alterations to structural and functional brain networks have been reported across many neurological conditions. However, the relationship between structure and function -- their coupling -- is relatively unexplored, particularly in the context of an intervention. Epilepsy surgery alters the brain structure and networks to control the functional abnormality of seizures. Given that surgery is a stru… ▽ More Alterations to structural and functional brain networks have been reported across many neurological conditions. However, the relationship between structure and function -- their coupling -- is relatively unexplored, particularly in the context of an intervention. Epilepsy surgery alters the brain structure and networks to control the functional abnormality of seizures. Given that surgery is a structural modification aiming to alter the function, we hypothesized that stronger structure-function coupling preoperatively is associated with a greater chance of post-operative seizure control. We constructed structural and functional brain networks in 39 subjects with medication-resistant focal epilepsy using data from intracranial EEG (pre-surgery), structural MRI (pre-and post-surgery), and diffusion MRI (pre-surgery). We investigated pre-operative structure-function coupling at two spatial scales a) at the global iEEG network level and b) at the resolution of individual iEEG electrode contacts using virtual surgeries. At global network level, seizure-free individuals had stronger structure-function coupling pre-operatively than those that were not seizure-free regardless of the choice of interictal segment or frequency band. At the resolution of individual iEEG contacts, the virtual surgery approach provided complementary information to localize epileptogenic tissues. In predicting seizure outcomes, structure-function coupling measures were more important than clinical attributes, and together they predicted seizure outcomes with an accuracy of 85% and sensitivity of 87%. The underlying assumption that the structural changes induced by surgery translate to the functional level to control seizures is valid when the structure-functional coupling is strong. Mapping the regions that contribute to structure-functional coupling using virtual surgeries may help aid surgical planning. △ Less

Submitted 17 April, 2022; originally announced April 2022.

Volumetric and structural connectivity abnormalities co-localise in TLE

Authors: Jonathan J. Horsley, Gabrielle M. Schroeder, Rhys H. Thomas, Jane de Tisi, Sjoerd B. Vos, Gavin P. Winston, John S. Duncan, Yujiang Wang, Peter N. Taylor

Abstract: Patients with temporal lobe epilepsy (TLE) exhibit both volumetric and structural connectivity abnormalities relative to healthy controls. How these abnormalities inter-relate and their mechanisms are unclear. We computed grey matter volumetric changes and white matter structural connectivity abnormalities in 144 patients with unilateral TLE and 96 healthy controls. Regional volumes were calculate… ▽ More Patients with temporal lobe epilepsy (TLE) exhibit both volumetric and structural connectivity abnormalities relative to healthy controls. How these abnormalities inter-relate and their mechanisms are unclear. We computed grey matter volumetric changes and white matter structural connectivity abnormalities in 144 patients with unilateral TLE and 96 healthy controls. Regional volumes were calculated using T1-weighted MRI, while structural connectivity was derived using white matter fibre tractography from diffusion-weighted MRI. For each regional volume and each connection strength, we calculated the effect size between patient and control groups in a group-level analysis. We then applied hierarchical regression to investigate the relationship between volumetric and structural connectivity abnormalities in individuals. Additionally, we quantified whether abnormalities co-localised within individual patients by computing Dice similarity scores. In TLE, white matter connectivity abnormalities were greater when joining two grey matter regions with abnormal volumes. Similarly, grey matter volumetric abnormalities were greater when joined by abnormal white matter connections. The extent of volumetric and connectivity abnormalities related to epilepsy duration, but co-localisation did not. Co-localisation was primarily driven by neighbouring abnormalities in the ipsilateral hemisphere. Overall, volumetric and structural connectivity abnormalities were related in TLE. Our results suggest that shared mechanisms may underlie changes in both volume and connectivity alterations in patients with TLE. △ Less

Submitted 5 February, 2022; originally announced February 2022.

DSFormer: A Dual-domain Self-supervised Transformer for Accelerated Multi-contrast MRI Reconstruction

Authors: Bo Zhou, Neel Dey, Jo Schlemper, Seyed Sadegh Mohseni Salehi, Chi Liu, James S. Duncan, Michal Sofka

Abstract: Multi-contrast MRI (MC-MRI) captures multiple complementary imaging modalities to aid in radiological decision-making. Given the need for lowering the time cost of multiple acquisitions, current deep accelerated MRI reconstruction networks focus on exploiting the redundancy between multiple contrasts. However, existing works are largely supervised with paired data and/or prohibitively expensive fu… ▽ More Multi-contrast MRI (MC-MRI) captures multiple complementary imaging modalities to aid in radiological decision-making. Given the need for lowering the time cost of multiple acquisitions, current deep accelerated MRI reconstruction networks focus on exploiting the redundancy between multiple contrasts. However, existing works are largely supervised with paired data and/or prohibitively expensive fully-sampled MRI sequences. Further, reconstruction networks typically rely on convolutional architectures which are limited in their capacity to model long-range interactions and may lead to suboptimal recovery of fine anatomical detail. To these ends, we present a dual-domain self-supervised transformer (DSFormer) for accelerated MC-MRI reconstruction. DSFormer develops a deep conditional cascade transformer (DCCT) consisting of several cascaded Swin transformer reconstruction networks (SwinRN) trained under two deep conditioning strategies to enable MC-MRI information sharing. We further present a dual-domain (image and k-space) self-supervised learning strategy for DCCT to alleviate the costs of acquiring fully sampled training data. DSFormer generates high-fidelity reconstructions which experimentally outperform current fully-supervised baselines. Moreover, we find that DSFormer achieves nearly the same performance when trained either with full supervision or with our proposed dual-domain self-supervision. △ Less

Submitted 16 August, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

Comments: Accepted at WACV 2023

Class-Aware Adversarial Transformers for Medical Image Segmentation

Authors: Chenyu You, Ruihan Zhao, Fenglin Liu, Siyuan Dong, Sandeep Chinchali, Ufuk Topcu, Lawrence Staib, James S. Duncan

Abstract: Transformers have made remarkable progress towards modeling long-range dependencies within the medical image analysis domain. However, current transformer-based models suffer from several disadvantages: (1) existing methods fail to capture the important features of the images due to the naive tokenization scheme; (2) the models suffer from information loss because they only consider single-scale f… ▽ More Transformers have made remarkable progress towards modeling long-range dependencies within the medical image analysis domain. However, current transformer-based models suffer from several disadvantages: (1) existing methods fail to capture the important features of the images due to the naive tokenization scheme; (2) the models suffer from information loss because they only consider single-scale feature representations; and (3) the segmentation label maps generated by the models are not accurate enough without considering rich semantic contexts and anatomical textures. In this work, we present CASTformer, a novel type of adversarial transformers, for 2D medical image segmentation. First, we take advantage of the pyramid structure to construct multi-scale representations and handle multi-scale variations. We then design a novel class-aware transformer module to better learn the discriminative regions of objects with semantic structures. Lastly, we utilize an adversarial training strategy that boosts segmentation accuracy and correspondingly allows a transformer-based discriminator to capture high-level semantically correlated contents and low-level anatomical features. Our experiments demonstrate that CASTformer dramatically outperforms previous state-of-the-art transformer-based approaches on three benchmarks, obtaining 2.54%-5.88% absolute improvements in Dice over previous models. Further qualitative experiments provide a more detailed picture of the model's inner workings, shed light on the challenges in improved transparency, and demonstrate that transfer learning can greatly improve performance and reduce the size of medical image datasets in training, making CASTformer a strong starting point for downstream medical image analysis tasks. △ Less

Submitted 15 December, 2022; v1 submitted 25 January, 2022; originally announced January 2022.

Momentum Centering and Asynchronous Update for Adaptive Gradient Methods

Authors: Juntang Zhuang, Yifan Ding, Tommy Tang, Nicha Dvornek, Sekhar Tatikonda, James S. Duncan

Abstract: We propose ACProp (Asynchronous-centering-Prop), an adaptive optimizer which combines centering of second momentum and asynchronous update (e.g. for $t$-th update, denominator uses information up to step $t-1$, while numerator uses gradient at $t$-th step). ACProp has both strong theoretical properties and empirical performance. With the example by Reddi et al. (2018), we show that asynchronous op… ▽ More We propose ACProp (Asynchronous-centering-Prop), an adaptive optimizer which combines centering of second momentum and asynchronous update (e.g. for $t$-th update, denominator uses information up to step $t-1$, while numerator uses gradient at $t$-th step). ACProp has both strong theoretical properties and empirical performance. With the example by Reddi et al. (2018), we show that asynchronous optimizers (e.g. AdaShift, ACProp) have weaker convergence condition than synchronous optimizers (e.g. Adam, RMSProp, AdaBelief); within asynchronous optimizers, we show that centering of second momentum further weakens the convergence condition. We demonstrate that ACProp has a convergence rate of $O(\frac{1}{\sqrt{T}})$ for the stochastic non-convex case, which matches the oracle rate and outperforms the $O(\frac{logT}{\sqrt{T}})$ rate of RMSProp and Adam. We validate ACProp in extensive empirical studies: ACProp outperforms both SGD and other adaptive optimizers in image classification with CNN, and outperforms well-tuned adaptive optimizers in the training of various GAN models, reinforcement learning and transformers. To sum up, ACProp has good theoretical properties including weak convergence condition and optimal convergence rate, and strong empirical performance including good generalization like SGD and training stability like Adam. We provide the implementation at https://github.com/juntang-zhuang/ACProp-Optimizer. △ Less

Submitted 1 December, 2021; v1 submitted 11 October, 2021; originally announced October 2021.

Seizure pathways and seizure durations can vary independently within individual patients with focal epilepsy

Authors: Gabrielle M. Schroeder, Fahmida A. Chowdhury, Mark J. Cook, Beate Diehl, John S. Duncan, Philippa J. Karoly, Peter N. Taylor, Yujiang Wang

Abstract: A seizure's electrographic dynamics are characterised by its spatiotemporal evolution, also termed dynamical "pathway" and the time it takes to complete that pathway, which results in the seizure's duration. Both seizure pathways and durations can vary within the same patient, producing seizures with different dynamics, severity, and clinical implications. However, it is unclear whether seizures f… ▽ More A seizure's electrographic dynamics are characterised by its spatiotemporal evolution, also termed dynamical "pathway" and the time it takes to complete that pathway, which results in the seizure's duration. Both seizure pathways and durations can vary within the same patient, producing seizures with different dynamics, severity, and clinical implications. However, it is unclear whether seizures following the same pathway will have the same duration or if these features can vary independently. We compared within-subject variability in these seizure features using 1) epilepsy monitoring unit intracranial EEG (iEEG) recordings of 31 patients (mean 6.7 days, 16.5 seizures/subject), 2) NeuroVista chronic iEEG recordings of 10 patients (mean 521.2 days, 252.6 seizures/subject), and 3) chronic iEEG recordings of 3 dogs with focal-onset seizures (mean 324.4 days, 62.3 seizures/subject). While the strength of the relationship between seizure pathways and durations was highly subject-specific, in most subjects, changes in seizure pathways were only weakly to moderately associated with differences in seizure durations. The relationship between seizure pathways and durations was weakened by seizures that 1) had a common pathway, but different durations ("elastic pathways"), or 2) had similar durations, but followed different pathways ("duplicate durations"). Even in subjects with distinct populations of short and long seizures, seizure durations were not a reliable indicator of different seizure pathways. These findings suggest that seizure pathways and durations are modulated by different processes. Uncovering such modulators may reveal novel therapeutic targets for reducing seizure duration and severity. △ Less

Submitted 14 September, 2021; originally announced September 2021.

SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation

Authors: Chenyu You, Yuan Zhou, Ruihan Zhao, Lawrence Staib, James S. Duncan

Abstract: Automated segmentation in medical image analysis is a challenging task that requires a large amount of manually labeled data. However, most existing learning-based approaches usually suffer from limited manually annotated medical data, which poses a major practical problem for accurate and robust medical image segmentation. In addition, most existing semi-supervised approaches are usually not robu… ▽ More Automated segmentation in medical image analysis is a challenging task that requires a large amount of manually labeled data. However, most existing learning-based approaches usually suffer from limited manually annotated medical data, which poses a major practical problem for accurate and robust medical image segmentation. In addition, most existing semi-supervised approaches are usually not robust compared with the supervised counterparts, and also lack explicit modeling of geometric structure and semantic information, both of which limit the segmentation accuracy. In this work, we present SimCVD, a simple contrastive distillation framework that significantly advances state-of-the-art voxel-wise representation learning. We first describe an unsupervised training strategy, which takes two views of an input volume and predicts their signed distance maps of object boundaries in a contrastive objective, with only two independent dropout as mask. This simple approach works surprisingly well, performing on the same level as previous fully supervised methods with much less labeled data. We hypothesize that dropout can be viewed as a minimal form of data augmentation and makes the network robust to representation collapse. Then, we propose to perform structural distillation by distilling pair-wise similarities. We evaluate SimCVD on two popular datasets: the Left Atrial Segmentation Challenge (LA) and the NIH pancreas CT dataset. The results on the LA dataset demonstrate that, in two types of labeled ratios (i.e., 20% and 10%), SimCVD achieves an average Dice score of 90.85% and 89.03% respectively, a 0.91% and 2.22% improvement compared to previous best results. Our method can be trained in an end-to-end fashion, showing the promise of utilizing SimCVD as a general framework for downstream tasks, such as medical image synthesis, enhancement, and registration. △ Less

Submitted 21 March, 2022; v1 submitted 13 August, 2021; originally announced August 2021.

Comments: IEEE Transactions on Medical Imaging (IEEE-TMI) 2022

Synthesizing Multi-Tracer PET Images for Alzheimer's Disease Patients using a 3D Unified Anatomy-aware Cyclic Adversarial Network

Authors: Bo Zhou, Rui Wang, Ming-Kai Chen, Adam P. Mecca, Ryan S. O'Dell, Christopher H. Van Dyck, Richard E. Carson, James S. Duncan, Chi Liu

Abstract: Positron Emission Tomography (PET) is an important tool for studying Alzheimer's disease (AD). PET scans can be used as diagnostics tools, and to provide molecular characterization of patients with cognitive disorders. However, multiple tracers are needed to measure glucose metabolism (18F-FDG), synaptic vesicle protein (11C-UCB-J), and $β$-amyloid (11C-PiB). Administering multiple tracers to pati… ▽ More Positron Emission Tomography (PET) is an important tool for studying Alzheimer's disease (AD). PET scans can be used as diagnostics tools, and to provide molecular characterization of patients with cognitive disorders. However, multiple tracers are needed to measure glucose metabolism (18F-FDG), synaptic vesicle protein (11C-UCB-J), and $β$-amyloid (11C-PiB). Administering multiple tracers to patient will lead to high radiation dose and cost. In addition, access to PET scans using new or less-available tracers with sophisticated production methods and short half-life isotopes may be very limited. Thus, it is desirable to develop an efficient multi-tracer PET synthesis model that can generate multi-tracer PET from single-tracer PET. Previous works on medical image synthesis focus on one-to-one fixed domain translations, and cannot simultaneously learn the feature from multi-tracer domains. Given 3 or more tracers, relying on previous methods will also create a heavy burden on the number of models to be trained. To tackle these issues, we propose a 3D unified anatomy-aware cyclic adversarial network (UCAN) for translating multi-tracer PET volumes with one unified generative model, where MR with anatomical information is incorporated. Evaluations on a multi-tracer PET dataset demonstrate the feasibility that our UCAN can generate high-quality multi-tracer PET volumes, with NMSE less than 15% for all PET tracers. △ Less

Submitted 12 July, 2021; originally announced July 2021.

Comments: Accepted at MICCAI 2021