-
Beneath the Surface: Revealing Deep-Tissue Blood Flow in Human Subjects with Massively Parallelized Diffuse Correlation Spectroscopy
Authors:
Lucas Kreiss,
Melissa Wu,
Michael Wayne,
Shiqi Xu,
Paul McKee,
Derrick Dwamena,
Kanghyun Kim,
Kyung Chul Lee,
Wenhui Liu,
Aarin Ulku,
Mark Harfouche,
Xi Yang,
Clare Cook,
Amey Chaware,
Seung Ah Lee,
Erin Buckley,
Claudio Bruschini,
Edoardo Charbon,
Scott Huettel,
Roarke Horstmeyer
Abstract:
Diffuse Correlation Spectroscopy (DCS) allows the label-free investigation of microvascular dynamics deep within living tissue. However, common implementations of DCS are currently limited to measurement depths of $\sim 1-1.5cm$, which can limit the accuracy of cerebral hemodynamics measurement. Here we present massively parallelized DCS (pDCS) using novel single photon avalanche detector (SPAD) a…
▽ More
Diffuse Correlation Spectroscopy (DCS) allows the label-free investigation of microvascular dynamics deep within living tissue. However, common implementations of DCS are currently limited to measurement depths of $\sim 1-1.5cm$, which can limit the accuracy of cerebral hemodynamics measurement. Here we present massively parallelized DCS (pDCS) using novel single photon avalanche detector (SPAD) arrays with up to 500x500 individual channels. The new SPAD array technology can boost the signal-to-noise ratio by a factor of up to 500 compared to single-pixel DCS, or by more than 15-fold compared to the most recent state-of-the-art pDCS demonstrations. Our results demonstrate the first in vivo use of this massively parallelized DCS system to measure cerebral blood flow changes at $\sim 2cm$ depth in human adults. We compared different modes of operation and applied a dual detection strategy, where a secondary SPAD array is used to simultaneously assess the superficial blood flow as a built-in reference measurement. While the blood flow in the superficial scalp tissue showed no significant change during cognitive activation, the deep pDCS measurement showed a statistically significant increase in the derived blood flow index of 8-12% when compared to the control rest state.
△ Less
Submitted 25 June, 2024; v1 submitted 6 March, 2024;
originally announced March 2024.
-
Tensorial tomographic Fourier Ptychography with applications to muscle tissue imaging
Authors:
Shiqi Xu,
Xiang Dai,
Paul Ritter,
Kyung Chul Lee,
Xi Yang,
Lucas Kreiss,
Kevin C. Zhou,
Kanghyun Kim,
Amey Chaware,
Jadee Neff,
Carolyn Glass,
Seung Ah Lee,
Oliver Friedrich,
Roarke Horstmeyer
Abstract:
We report Tensorial tomographic Fourier Ptychography (ToFu), a new non-scanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier Ptychography, a quantitative phase imaging technique, ToFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equ…
▽ More
We report Tensorial tomographic Fourier Ptychography (ToFu), a new non-scanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier Ptychography, a quantitative phase imaging technique, ToFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equipped with an LED matrix, a polarization generator, and a polarization-sensitive camera. Permittivity tensors of anisotropic samples are computationally recovered from polarized intensity measurements across three dimensions. We demonstrate ToFu's efficiency through volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, as well as tissue samples from muscle fibers and diseased heart tissue. Our reconstructions of muscle fibers resolve their 3D fine-filament structure and yield consistent morphological measurements compared to gold-standard second harmonic generation scanning confocal microscope images found in the literature. Additionally, we demonstrate reconstructions of a heart tissue sample that carries important polarization information for detecting cardiac amyloidosis.
△ Less
Submitted 13 May, 2023; v1 submitted 8 May, 2023;
originally announced May 2023.
-
Digital staining in optical microscopy using deep learning -- a review
Authors:
Lucas Kreiss,
Shaowei Jiang,
Xiang Li,
Shiqi Xu,
Kevin C. Zhou,
Alexander Mühlberg,
Kyung Chul Lee,
Kanghyun Kim,
Amey Chaware,
Michael Ando,
Laura Barisoni,
Seung Ah Lee,
Guoan Zheng,
Kyle Lafata,
Oliver Friedrich,
Roarke Horstmeyer
Abstract:
Until recently, conventional biochemical staining had the undisputed status as well-established benchmark for most biomedical problems related to clinical diagnostics, fundamental research and biotechnology. Despite this role as gold-standard, staining protocols face several challenges, such as a need for extensive, manual processing of samples, substantial time delays, altered tissue homeostasis,…
▽ More
Until recently, conventional biochemical staining had the undisputed status as well-established benchmark for most biomedical problems related to clinical diagnostics, fundamental research and biotechnology. Despite this role as gold-standard, staining protocols face several challenges, such as a need for extensive, manual processing of samples, substantial time delays, altered tissue homeostasis, limited choice of contrast agents for a given sample, 2D imaging instead of 3D tomography and many more. Label-free optical technologies, on the other hand, do not rely on exogenous and artificial markers, by exploiting intrinsic optical contrast mechanisms, where the specificity is typically less obvious to the human observer. Over the past few years, digital staining has emerged as a promising concept to use modern deep learning for the translation from optical contrast to established biochemical contrast of actual stainings. In this review article, we provide an in-depth analysis of the current state-of-the-art in this field, suggest methods of good practice, identify pitfalls and challenges and postulate promising advances towards potential future implementations and applications.
△ Less
Submitted 14 March, 2023;
originally announced March 2023.
-
Imaging across multiple spatial scales with the multi-camera array microscope
Authors:
Mark Harfouche,
Kanghyun Kim,
Kevin C. Zhou,
Pavan Chandra Konda,
Sunanda Sharma,
Eric E. Thomson,
Colin Cooke,
Shiqi Xu,
Lucas Kreiss,
Amey Chaware,
Xi Yang,
Xing Yao,
Vinayak Pathak,
Martin Bohlen,
Ron Appel,
Aurélien Bègue,
Clare Cook,
Jed Doman,
John Efromson,
Gregor Horstmeyer,
Jaehee Park,
Paul Reamey,
Veton Saliu,
Eva Naumann,
Roarke Horstmeyer
Abstract:
This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digit…
▽ More
This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digitally combined into composite frames, whose total pixel counts significantly exceed the pixel counts of standard microscope systems. We present results from three unique MCAM configurations for different use cases. First, we demonstrate a configuration that simultaneously images and estimates the 3D object depth across a 100 x 135 mm^2 field-of-view (FOV) at approximately 20 um resolution, which results in 0.15 gigapixels (GP) per snapshot. Second, we demonstrate an MCAM configuration that records video across a continuous 83 x 123 mm^2 FOV with two-fold increased resolution (0.48 GP per frame). Finally, we report a third high-resolution configuration (2 um resolution) that can rapidly produce 9.8 GP composites of large histopathology specimens.
△ Less
Submitted 28 February, 2023; v1 submitted 30 November, 2022;
originally announced December 2022.
-
Increasing a microscope's effective field of view via overlapped imaging and machine learning
Authors:
Xing Yao,
Vinayak Pathak,
Haoran Xi,
Amey Chaware,
Colin Cooke,
Kanghyun Kim,
Shiqi Xu,
Yuting Li,
Timothy Dunn,
Pavan Chandra Konda,
Kevin C. Zhou,
Roarke Horstmeyer
Abstract:
This work demonstrates a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for high-efficiency automated specimen analysis. Automatic detection, classification and counting of various morphological features of interest is now a crucial component of both biomedical research and disease diagnosis. While convolutional neural networks (CNNs) hav…
▽ More
This work demonstrates a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for high-efficiency automated specimen analysis. Automatic detection, classification and counting of various morphological features of interest is now a crucial component of both biomedical research and disease diagnosis. While convolutional neural networks (CNNs) have dramatically improved the accuracy of counting cells and sub-cellular features from acquired digital image data, the overall throughput is still typically hindered by the limited space-bandwidth product (SBP) of conventional microscopes. Here, we show both in simulation and experiment that overlapped imaging and co-designed analysis software can achieve accurate detection of diagnostically-relevant features for several applications, including counting of white blood cells and the malaria parasite, leading to multi-fold increase in detection and processing throughput with minimal reduction in accuracy.
△ Less
Submitted 10 October, 2021;
originally announced October 2021.
-
Physics-enhanced machine learning for virtual fluorescence microscopy
Authors:
Colin L. Cooke,
Fanjie Kong,
Amey Chaware,
Kevin C. Zhou,
Kanghyun Kim,
Rong Xu,
D. Michael Ando,
Samuel J. Yang,
Pavan Chandra Konda,
Roarke Horstmeyer
Abstract:
This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy ima…
▽ More
This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy images. We validated our method on two different experimental setups, with different magnifications and different sample types, to show a consistent improvement in performance as compared to conventional illumination methods. Additionally, to understand the importance of learned illumination on inference task, we varied the dynamic range of the fluorescent image targets (from one to seven bits), and showed that the margin of improvement for learned patterns increased with the information content of the target. This work demonstrates the power of programmable optical elements at enabling better machine learning algorithm performance and at providing physical insight into next generation of machine-controlled imaging systems.
△ Less
Submitted 21 April, 2020; v1 submitted 8 April, 2020;
originally announced April 2020.
-
Towards an Intelligent Microscope: adaptively learned illumination for optimal sample classification
Authors:
Amey Chaware,
Colin L. Cooke,
Kanghyun Kim,
Roarke Horstmeyer
Abstract:
Recent machine learning techniques have dramatically changed how we process digital images. However, the way in which we capture images is still largely driven by human intuition and experience. This restriction is in part due to the many available degrees of freedom that alter the image acquisition process (lens focus, exposure, filtering, etc). Here we focus on one such degree of freedom - illum…
▽ More
Recent machine learning techniques have dramatically changed how we process digital images. However, the way in which we capture images is still largely driven by human intuition and experience. This restriction is in part due to the many available degrees of freedom that alter the image acquisition process (lens focus, exposure, filtering, etc). Here we focus on one such degree of freedom - illumination within a microscope - which can drastically alter information captured by the image sensor. We present a reinforcement learning system that adaptively explores optimal patterns to illuminate specimens for immediate classification. The agent uses a recurrent latent space to encode a large set of variably-illuminated samples and illumination patterns. We train our agent using a reward that balances classification confidence with image acquisition cost. By synthesizing knowledge over multiple snapshots, the agent can classify on the basis of all previous images with higher accuracy than from naively illuminated images, thus demonstrating a smarter way to physically capture task-specific information.
△ Less
Submitted 13 February, 2020; v1 submitted 22 October, 2019;
originally announced October 2019.