Showing 1–38 of 38 results for author: Horstmeyer, R

A comprehensive overview of diffuse correlation spectroscopy: theoretical framework, recent advances in hardware, analysis, and applications

Authors: Quan Wang, Mingliang Pan, Lucas Kreiss, Saeed Samaei, Stefan A. Carp, Johannes D. Johansson, Yuanzhe Zhang, Melissa Wu, Roarke Horstmeyer, Mamadou Diop, David Day-Uei Li

Abstract: Diffuse correlation spectroscopy (DCS) is a powerful tool for assessing microvascular hemodynamic in deep tissues. Recent advances in sensors, lasers, and deep learning have further boosted the development of new DCS methods. However, newcomers might feel overwhelmed, not only by the already complex DCS theoretical framework but also by the broad range of component options and system architectures… ▽ More Diffuse correlation spectroscopy (DCS) is a powerful tool for assessing microvascular hemodynamic in deep tissues. Recent advances in sensors, lasers, and deep learning have further boosted the development of new DCS methods. However, newcomers might feel overwhelmed, not only by the already complex DCS theoretical framework but also by the broad range of component options and system architectures. To facilitate new entry into this exciting field, we present a comprehensive review of DCS hardware architectures (continuous-wave, frequency-domain, and time-domain) and summarize corresponding theoretical models. Further, we discuss new applications of highly integrated silicon single-photon avalanche diode (SPAD) sensors in DCS, compare SPADs with existing sensors, and review other components (lasers, fibers, and correlators), as well as new data analysis tools, including deep learning. Potential applications in medical diagnosis are discussed, and an outlook for the future directions is provided, to offer effective guidance to embark on DCS research. △ Less

Submitted 18 May, 2024; originally announced June 2024.

Beneath the Surface: Revealing Deep-Tissue Blood Flow in Human Subjects with Massively Parallelized Diffuse Correlation Spectroscopy

Authors: Lucas Kreiss, Melissa Wu, Michael Wayne, Shiqi Xu, Paul McKee, Derrick Dwamena, Kanghyun Kim, Kyung Chul Lee, Wenhui Liu, Aarin Ulku, Mark Harfouche, Xi Yang, Clare Cook, Amey Chaware, Seung Ah Lee, Erin Buckley, Claudio Bruschini, Edoardo Charbon, Scott Huettel, Roarke Horstmeyer

Abstract: Diffuse Correlation Spectroscopy (DCS) allows the label-free investigation of microvascular dynamics deep within living tissue. However, common implementations of DCS are currently limited to measurement depths of $\sim 1-1.5cm$, which can limit the accuracy of cerebral hemodynamics measurement. Here we present massively parallelized DCS (pDCS) using novel single photon avalanche detector (SPAD) a… ▽ More Diffuse Correlation Spectroscopy (DCS) allows the label-free investigation of microvascular dynamics deep within living tissue. However, common implementations of DCS are currently limited to measurement depths of $\sim 1-1.5cm$, which can limit the accuracy of cerebral hemodynamics measurement. Here we present massively parallelized DCS (pDCS) using novel single photon avalanche detector (SPAD) arrays with up to 500x500 individual channels. The new SPAD array technology can boost the signal-to-noise ratio by a factor of up to 500 compared to single-pixel DCS, or by more than 15-fold compared to the most recent state-of-the-art pDCS demonstrations. Our results demonstrate the first in vivo use of this massively parallelized DCS system to measure cerebral blood flow changes at $\sim 2cm$ depth in human adults. We compared different modes of operation and applied a dual detection strategy, where a secondary SPAD array is used to simultaneously assess the superficial blood flow as a built-in reference measurement. While the blood flow in the superficial scalp tissue showed no significant change during cognitive activation, the deep pDCS measurement showed a statistically significant increase in the derived blood flow index of 8-12% when compared to the control rest state. △ Less

Submitted 25 June, 2024; v1 submitted 6 March, 2024; originally announced March 2024.

Computational 3D topographic microscopy from terabytes of data per sample

Authors: Kevin C. Zhou, Mark Harfouche, Maxwell Zheng, Joakim Jönsson, Kyung Chul Lee, Ron Appel, Paul Reamey, Thomas Doman, Veton Saliu, Gregor Horstmeyer, Roarke Horstmeyer

Abstract: We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis… ▽ More We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis translation to capture, for each sample of interest, a multi-dimensional, 2.1-terabyte (TB) dataset, consisting of a total of 224,640 9.4-megapixel images. We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map across the entire field of view, using multi-view stereo information and image sharpness as a focal metric. The memory-efficient, compressed differentiable representation offered by the neural network effectively enables joint participation of the entire multi-TB dataset during the reconstruction process. To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects, with applications ranging from cultural heritage to industrial inspection. △ Less

Submitted 5 June, 2023; originally announced June 2023.

Tensorial tomographic Fourier Ptychography with applications to muscle tissue imaging

Authors: Shiqi Xu, Xiang Dai, Paul Ritter, Kyung Chul Lee, Xi Yang, Lucas Kreiss, Kevin C. Zhou, Kanghyun Kim, Amey Chaware, Jadee Neff, Carolyn Glass, Seung Ah Lee, Oliver Friedrich, Roarke Horstmeyer

Abstract: We report Tensorial tomographic Fourier Ptychography (ToFu), a new non-scanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier Ptychography, a quantitative phase imaging technique, ToFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equ… ▽ More We report Tensorial tomographic Fourier Ptychography (ToFu), a new non-scanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier Ptychography, a quantitative phase imaging technique, ToFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equipped with an LED matrix, a polarization generator, and a polarization-sensitive camera. Permittivity tensors of anisotropic samples are computationally recovered from polarized intensity measurements across three dimensions. We demonstrate ToFu's efficiency through volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, as well as tissue samples from muscle fibers and diseased heart tissue. Our reconstructions of muscle fibers resolve their 3D fine-filament structure and yield consistent morphological measurements compared to gold-standard second harmonic generation scanning confocal microscope images found in the literature. Additionally, we demonstrate reconstructions of a heart tissue sample that carries important polarization information for detecting cardiac amyloidosis. △ Less

Submitted 13 May, 2023; v1 submitted 8 May, 2023; originally announced May 2023.

Journal ref: Tensorial tomographic Fourier Ptychography with applications to muscle tissue imaging, Adv. Photon. 6(2), 026004 (2024)

Digital staining in optical microscopy using deep learning -- a review

Authors: Lucas Kreiss, Shaowei Jiang, Xiang Li, Shiqi Xu, Kevin C. Zhou, Alexander Mühlberg, Kyung Chul Lee, Kanghyun Kim, Amey Chaware, Michael Ando, Laura Barisoni, Seung Ah Lee, Guoan Zheng, Kyle Lafata, Oliver Friedrich, Roarke Horstmeyer

Abstract: Until recently, conventional biochemical staining had the undisputed status as well-established benchmark for most biomedical problems related to clinical diagnostics, fundamental research and biotechnology. Despite this role as gold-standard, staining protocols face several challenges, such as a need for extensive, manual processing of samples, substantial time delays, altered tissue homeostasis,… ▽ More Until recently, conventional biochemical staining had the undisputed status as well-established benchmark for most biomedical problems related to clinical diagnostics, fundamental research and biotechnology. Despite this role as gold-standard, staining protocols face several challenges, such as a need for extensive, manual processing of samples, substantial time delays, altered tissue homeostasis, limited choice of contrast agents for a given sample, 2D imaging instead of 3D tomography and many more. Label-free optical technologies, on the other hand, do not rely on exogenous and artificial markers, by exploiting intrinsic optical contrast mechanisms, where the specificity is typically less obvious to the human observer. Over the past few years, digital staining has emerged as a promising concept to use modern deep learning for the translation from optical contrast to established biochemical contrast of actual stainings. In this review article, we provide an in-depth analysis of the current state-of-the-art in this field, suggest methods of good practice, identify pitfalls and challenges and postulate promising advances towards potential future implementations and applications. △ Less

Submitted 14 March, 2023; originally announced March 2023.

Comments: Review article, 4 main Figures, 3 Tables, 2 supplementary figures

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second

Authors: Kevin C. Zhou, Mark Harfouche, Colin L. Cooke, Jaehee Park, Pavan C. Konda, Lucas Kreiss, Kanghyun Kim, Joakim Jönsson, Jed Doman, Paul Reamey, Veton Saliu, Clare B. Cook, Maxwell Zheng, Jack P. Bechtel, Aurélien Bègue, Matthew McCarroll, Jennifer Bagwell, Gregor Horstmeyer, Michel Bagnat, Roarke Horstmeyer

Abstract: To study the behavior of freely moving model organisms such as zebrafish (Danio rerio) and fruit flies (Drosophila) across multiple spatial scales, it would be ideal to use a light microscope that can resolve 3D information over a wide field of view (FOV) at high speed and high spatial resolution. However, it is challenging to design an optical instrument to achieve all of these properties simulta… ▽ More To study the behavior of freely moving model organisms such as zebrafish (Danio rerio) and fruit flies (Drosophila) across multiple spatial scales, it would be ideal to use a light microscope that can resolve 3D information over a wide field of view (FOV) at high speed and high spatial resolution. However, it is challenging to design an optical instrument to achieve all of these properties simultaneously. Existing techniques for large-FOV microscopic imaging and for 3D image measurement typically require many sequential image snapshots, thus compromising speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over a 135-cm^2 area, achieving up to 230 frames per second at throughputs exceeding 5 gigapixels (GPs) per second. 3D-RAPID features a 3D reconstruction algorithm that, for each synchronized temporal snapshot, simultaneously fuses all 54 images seamlessly into a globally-consistent composite that includes a coregistered 3D height map. The self-supervised 3D reconstruction algorithm itself trains a spatiotemporally-compressed convolutional neural network (CNN) that maps raw photometric images to 3D topography, using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. As a result, our end-to-end 3D reconstruction algorithm is robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. The scalable hardware and software design of 3D-RAPID addresses a longstanding problem in the field of behavioral imaging, enabling parallelized 3D observation of large collections of freely moving organisms at high spatiotemporal throughputs, which we demonstrate in ants (Pogonomyrmex barbatus), fruit flies, and zebrafish larvae. △ Less

Submitted 19 January, 2023; originally announced January 2023.

Large-scale single-photon imaging

Authors: Liheng Bian, Haoze Song, Lintao Peng, Xuyang Chang, Xi Yang, Roarke Horstmeyer, Lin Ye, Tong Qin, Dezhi Zheng, Jun Zhang

Abstract: Benefiting from its single-photon sensitivity, single-photon avalanche diode (SPAD) array has been widely applied in various fields such as fluorescence lifetime imaging and quantum computing. However, large-scale high-fidelity single-photon imaging remains a big challenge, due to the complex hardware manufacture craft and heavy noise disturbance of SPAD arrays. In this work, we introduce deep lea… ▽ More Benefiting from its single-photon sensitivity, single-photon avalanche diode (SPAD) array has been widely applied in various fields such as fluorescence lifetime imaging and quantum computing. However, large-scale high-fidelity single-photon imaging remains a big challenge, due to the complex hardware manufacture craft and heavy noise disturbance of SPAD arrays. In this work, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging over an order of magnitude, with significant enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 $\times$ 32 pixels, 90 scenes, 10 different bit depth, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this real-world physical noise model, we for the first time synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depth, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique on a series of experiments including macroscopic and microscopic imaging, microfluidic inspection, and Fourier ptychography. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance, with more than 5 dB superiority on PSNR compared to the existing methods. △ Less

Submitted 27 December, 2022; originally announced December 2022.

Imaging across multiple spatial scales with the multi-camera array microscope

Authors: Mark Harfouche, Kanghyun Kim, Kevin C. Zhou, Pavan Chandra Konda, Sunanda Sharma, Eric E. Thomson, Colin Cooke, Shiqi Xu, Lucas Kreiss, Amey Chaware, Xi Yang, Xing Yao, Vinayak Pathak, Martin Bohlen, Ron Appel, Aurélien Bègue, Clare Cook, Jed Doman, John Efromson, Gregor Horstmeyer, Jaehee Park, Paul Reamey, Veton Saliu, Eva Naumann, Roarke Horstmeyer

Abstract: This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digit… ▽ More This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digitally combined into composite frames, whose total pixel counts significantly exceed the pixel counts of standard microscope systems. We present results from three unique MCAM configurations for different use cases. First, we demonstrate a configuration that simultaneously images and estimates the 3D object depth across a 100 x 135 mm^2 field-of-view (FOV) at approximately 20 um resolution, which results in 0.15 gigapixels (GP) per snapshot. Second, we demonstrate an MCAM configuration that records video across a continuous 83 x 123 mm^2 FOV with two-fold increased resolution (0.48 GP per frame). Finally, we report a third high-resolution configuration (2 um resolution) that can rapidly produce 9.8 GP composites of large histopathology specimens. △ Less

Submitted 28 February, 2023; v1 submitted 30 November, 2022; originally announced December 2022.

Multi-modal imaging using a cascaded microscope design

Authors: Xi Yang, Mark Harfouche, Kevin C. Zhou, Lucas Kreiss, Shiqi Xu, Kanghyun Kim, Roarke Horstmeyer

Abstract: We present a new Multimodal Fiber Array Snapshot Technique (M-FAST), based on an array of 96 compact cameras placed behind a primary objective lens and a fiber bundle array. which is capable of large-area, high-resolution, multi-channel video acquisition. The proposed design provides two key improvements to prior cascaded imaging system approaches: a novel optical arrangement that accommodates the… ▽ More We present a new Multimodal Fiber Array Snapshot Technique (M-FAST), based on an array of 96 compact cameras placed behind a primary objective lens and a fiber bundle array. which is capable of large-area, high-resolution, multi-channel video acquisition. The proposed design provides two key improvements to prior cascaded imaging system approaches: a novel optical arrangement that accommodates the use of planar camera arrays, and the new ability to acquire multi-modal image data acquisition. M-FAST is a multi-modal, scalable imaging system that can acquire snapshot dual-channel fluorescence images as well as d phase contrast measurements over a large 8x10mm^2 FOV at 2.2um full-pitch resolution. △ Less

Submitted 18 August, 2022; originally announced August 2022.

Tensorial tomographic differential phase-contrast microscopy

Authors: Shiqi Xu, Xiang Dai, Xi Yang, Kevin C. Zhou, Kanghyun Kim, Vinayak Pathak, Carolyn Glass, Roarke Horstmeyer

Abstract: We report Tensorial Tomographic Differential Phase-Contrast microscopy (T2DPC), a quantitative label-free tomographic imaging method for simultaneous measurement of phase and anisotropy. T2DPC extends differential phase-contrast microscopy, a quantitative phase imaging technique, to highlight the vectorial nature of light. The method solves for permittivity tensor of anisotropic samples from inten… ▽ More We report Tensorial Tomographic Differential Phase-Contrast microscopy (T2DPC), a quantitative label-free tomographic imaging method for simultaneous measurement of phase and anisotropy. T2DPC extends differential phase-contrast microscopy, a quantitative phase imaging technique, to highlight the vectorial nature of light. The method solves for permittivity tensor of anisotropic samples from intensity measurements acquired with a standard microscope equipped with an LED matrix, a circular polarizer, and a polarization-sensitive camera. We demonstrate accurate volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, and show that the reconstructed polarization structures of a biological specimen are predictive of pathology. △ Less

Submitted 6 September, 2022; v1 submitted 24 April, 2022; originally announced April 2022.

Transient motion classification through turbid volumes via parallelized single-photon detection and deep contrastive embedding

Authors: Shiqi Xu, Wenhui Liu, Xi Yang, Joakim Jönsson, Ruobing Qian, Paul McKee, Kanghyun Kim, Pavan Chandra Konda, Kevin C. Zhou, Lucas Kreiß, Haoqian Wang, Edouard Berrocal, Scott Huettel, Roarke Horstmeyer

Abstract: Fast noninvasive probing of spatially varying decorrelating events, such as cerebral blood flow beneath the human skull, is an essential task in various scientific and clinical settings. One of the primary optical techniques used is diffuse correlation spectroscopy (DCS), whose classical implementation uses a single or few single-photon detectors, resulting in poor spatial localization accuracy an… ▽ More Fast noninvasive probing of spatially varying decorrelating events, such as cerebral blood flow beneath the human skull, is an essential task in various scientific and clinical settings. One of the primary optical techniques used is diffuse correlation spectroscopy (DCS), whose classical implementation uses a single or few single-photon detectors, resulting in poor spatial localization accuracy and relatively low temporal resolution. Here, we propose a technique termed Classifying Rapid decorrelation Events via Parallelized single photon dEtection (CREPE)}, a new form of DCS that can probe and classify different decorrelating movements hidden underneath turbid volume with high sensitivity using parallelized speckle detection from a $32\times32$ pixel SPAD array. We evaluate our setup by classifying different spatiotemporal-decorrelating patterns hidden beneath a 5mm tissue-like phantom made with rapidly decorrelating dynamic scattering media. Twelve multi-mode fibers are used to collect scattered light from different positions on the surface of the tissue phantom. To validate our setup, we generate perturbed decorrelation patterns by both a digital micromirror device (DMD) modulated at multi-kilo-hertz rates, as well as a vessel phantom containing flowing fluid. Along with a deep contrastive learning algorithm that outperforms classic unsupervised learning methods, we demonstrate our approach can accurately detect and classify different transient decorrelation events (happening in 0.1-0.4s) underneath turbid scattering media, without any data labeling. This has the potential to be applied to noninvasively monitor deep tissue motion patterns, for example identifying normal or abnormal cerebral blood flow events, at multi-Hertz rates within a compact and static detection probe. △ Less

Submitted 12 June, 2022; v1 submitted 4 April, 2022; originally announced April 2022.

Comments: Journal submission

Roadmap on Wavefront Shaping and deep imaging in complex media

Authors: Sylvain Gigan, Ori Katz, Hilton B. de Aguiar, Esben Ravn Andresen, Alexandre Aubry, Jacopo Bertolotti, Emmanuel Bossy, Dorian Bouchet, Joshua Brake, Sophie Brasselet, Yaron Bromberg, Hui Cao, Thomas Chaigne, Zhongtao Cheng, Wonshik Choi, Tomáš Čižmár, Meng Cui, Vincent R Curtis, Hugo Defienne, Matthias Hofer, Ryoichi Horisaki, Roarke Horstmeyer, Na Ji, Aaron K. LaViolette, Jerome Mertz , et al. (20 additional authors not shown)

Abstract: The last decade has seen the development of a wide set of tools, such as wavefront shaping, computational or fundamental methods, that allow to understand and control light propagation in a complex medium, such as biological tissues or multimode fibers. A vibrant and diverse community is now working on this field, that has revolutionized the prospect of diffraction-limited imaging at depth in tiss… ▽ More The last decade has seen the development of a wide set of tools, such as wavefront shaping, computational or fundamental methods, that allow to understand and control light propagation in a complex medium, such as biological tissues or multimode fibers. A vibrant and diverse community is now working on this field, that has revolutionized the prospect of diffraction-limited imaging at depth in tissues. This roadmap highlights several key aspects of this fast developing field, and some of the challenges and opportunities ahead. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: submitted to J.Phys Photonics (IOP), 116 pages, 23 sections

Increasing a microscope's effective field of view via overlapped imaging and machine learning

Authors: Xing Yao, Vinayak Pathak, Haoran Xi, Amey Chaware, Colin Cooke, Kanghyun Kim, Shiqi Xu, Yuting Li, Timothy Dunn, Pavan Chandra Konda, Kevin C. Zhou, Roarke Horstmeyer

Abstract: This work demonstrates a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for high-efficiency automated specimen analysis. Automatic detection, classification and counting of various morphological features of interest is now a crucial component of both biomedical research and disease diagnosis. While convolutional neural networks (CNNs) hav… ▽ More This work demonstrates a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for high-efficiency automated specimen analysis. Automatic detection, classification and counting of various morphological features of interest is now a crucial component of both biomedical research and disease diagnosis. While convolutional neural networks (CNNs) have dramatically improved the accuracy of counting cells and sub-cellular features from acquired digital image data, the overall throughput is still typically hindered by the limited space-bandwidth product (SBP) of conventional microscopes. Here, we show both in simulation and experiment that overlapped imaging and co-designed analysis software can achieve accurate detection of diagnostically-relevant features for several applications, including counting of white blood cells and the malaria parasite, leading to multi-fold increase in detection and processing throughput with minimal reduction in accuracy. △ Less

Submitted 10 October, 2021; originally announced October 2021.

Quantitative Jones matrix imaging using vectorial Fourier ptychography

Authors: Xiang Dai, Shiqi Xu, Xi Yang, Kevin C. Zhou, Carolyn Glass, Pavan Chandra Konda, Roarke Horstmeyer

Abstract: This paper presents a microscopic imaging technique that uses variable-angle illumination to recover the complex polarimetric properties of a specimen at high resolution and over a large field-of-view. The approach extends Fourier ptychography, which is a synthetic aperture-based imaging approach to improve resolution with phaseless measurements, to additionally account for the vectorial nature of… ▽ More This paper presents a microscopic imaging technique that uses variable-angle illumination to recover the complex polarimetric properties of a specimen at high resolution and over a large field-of-view. The approach extends Fourier ptychography, which is a synthetic aperture-based imaging approach to improve resolution with phaseless measurements, to additionally account for the vectorial nature of light. After images are acquired using a standard microscope outfitted with an LED illumination array and two polarizers, our vectorial Fourier Ptychography (vFP) algorithm solves for the complex 2x2 Jones matrix of the anisotropic specimen of interest at each resolved spatial location. We introduce a new sequential Gauss-Newton-based solver that additionally jointly estimates and removes polarization-dependent imaging system aberrations. We demonstrate effective vFP performance by generating large-area (29 mm$^2$), high-resolution (1.24 $μ$m full-pitch) reconstructions of sample absorption, phase, orientation, diattenuation, and retardance for a variety of calibration samples and biological specimens. △ Less

Submitted 15 October, 2021; v1 submitted 30 September, 2021; originally announced October 2021.

Imaging dynamics beneath turbid media via parallelized single-photon detection

Authors: Shiqi Xu, Xi Yang, Wenhui Liu, Joakim Jonsson, Ruobing Qian, Pavan Chandra Konda, Kevin C. Zhou, Lucas Kreiss, Qionghai Dai, Haoqian Wang, Edouard Berrocal, Roarke Horstmeyer

Abstract: Noninvasive optical imaging through dynamic scattering media has numerous important biomedical applications but still remains a challenging task. While standard diffuse imaging methods measure optical absorption or fluorescent emission, it is also well-established that the temporal correlation of scattered coherent light diffuses through tissue much like optical intensity. Few works to date, howev… ▽ More Noninvasive optical imaging through dynamic scattering media has numerous important biomedical applications but still remains a challenging task. While standard diffuse imaging methods measure optical absorption or fluorescent emission, it is also well-established that the temporal correlation of scattered coherent light diffuses through tissue much like optical intensity. Few works to date, however, have aimed to experimentally measure and process such temporal correlation data to demonstrate deep-tissue video reconstruction of decorrelation dynamics. In this work, we utilize a single-photon avalanche diode (SPAD) array camera to simultaneously monitor the temporal dynamics of speckle fluctuations at the single-photon level from 12 different phantom tissue surface locations delivered via a customized fiber bundle array. We then apply a deep neural network to convert the acquired single-photon measurements into video of scattering dynamics beneath rapidly decorrelating tissue phantoms. We demonstrate the ability to reconstruct images of transient (0.1-0.4s) dynamic events occurring up to 8 mm beneath a decorrelating tissue phantom with millimeter-scale resolution, and highlight how our model can flexibly extend to monitor flow speed within buried phantom vessels. △ Less

Submitted 12 June, 2022; v1 submitted 3 July, 2021; originally announced July 2021.

Mesoscopic photogrammetry with an unstabilized phone camera

Authors: Kevin C. Zhou, Colin Cooke, Jaehee Park, Ruobing Qian, Roarke Horstmeyer, Joseph A. Izatt, Sina Farsiu

Abstract: We present a feature-free photogrammetric technique that enables quantitative 3D mesoscopic (mm-scale height variation) imaging with tens-of-micron accuracy from sequences of images acquired by a smartphone at close range (several cm) under freehand motion without additional hardware. Our end-to-end, pixel-intensity-based approach jointly registers and stitches all the images by estimating a coali… ▽ More We present a feature-free photogrammetric technique that enables quantitative 3D mesoscopic (mm-scale height variation) imaging with tens-of-micron accuracy from sequences of images acquired by a smartphone at close range (several cm) under freehand motion without additional hardware. Our end-to-end, pixel-intensity-based approach jointly registers and stitches all the images by estimating a coaligned height map, which acts as a pixel-wise radial deformation field that orthorectifies each camera image to allow homographic registration. The height maps themselves are reparameterized as the output of an untrained encoder-decoder convolutional neural network (CNN) with the raw camera images as the input, which effectively removes many reconstruction artifacts. Our method also jointly estimates both the camera's dynamic 6D pose and its distortion using a nonparametric model, the latter of which is especially important in mesoscopic applications when using cameras not designed for imaging at short working distances, such as smartphone cameras. We also propose strategies for reducing computation time and memory, applicable to other multi-frame registration problems. Finally, we demonstrate our method using sequences of multi-megapixel images captured by an unstabilized smartphone on a variety of samples (e.g., painting brushstrokes, circuit board, seeds). △ Less

Submitted 10 December, 2020; originally announced December 2020.

Journal ref: CVPR 2021

Deep image prior for undersampling high-speed photoacoustic microscopy

Authors: Tri Vu, Anthony DiSpirito III, Daiwei Li, Zixuan Zhang, Xiaoyi Zhu, Maomao Chen, Laiming Jiang, Dong Zhang, Jianwen Luo, Yu Shrike Zhang, Qifa Zhou, Roarke Horstmeyer, Junjie Yao

Abstract: Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser's repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however,… ▽ More Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser's repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however, these methods often require time-consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image prior (DIP) to improve the image quality of undersampled PAM images. Unlike other DL approaches, DIP requires neither pre-training nor fully-sampled ground truth, enabling its flexible and fast implementation on various imaging targets. Our results have demonstrated substantial improvement in PAM images with as few as 1.4$\%$ of the fully sampled pixels on high-speed PAM. Our approach outperforms interpolation, is competitive with pre-trained supervised DL method, and is readily translated to other high-speed, undersampling imaging modalities. △ Less

Submitted 7 April, 2021; v1 submitted 15 October, 2020; originally announced October 2020.

Multi-element microscope optimization by a learned sensing network with composite physical layers

Authors: Kanghyun Kim, Pavan Chandra Konda, Colin L. Cooke, Ron Appel, Roarke Horstmeyer

Abstract: Standard microscopes offer a variety of settings to help improve the visibility of different specimens to the end microscope user. Increasingly, however, digital microscopes are used to capture images for automated interpretation by computer algorithms (e.g., for feature classification, detection or segmentation), often without any human involvement. In this work, we investigate an approach to joi… ▽ More Standard microscopes offer a variety of settings to help improve the visibility of different specimens to the end microscope user. Increasingly, however, digital microscopes are used to capture images for automated interpretation by computer algorithms (e.g., for feature classification, detection or segmentation), often without any human involvement. In this work, we investigate an approach to jointly optimize multiple microscope settings, together with a classification network, for improved performance with such automated tasks. We explore the interplay between optimization of programmable illumination and pupil transmission, using experimentally imaged blood smears for automated malaria parasite detection, to show that multi-element "learned sensing" outperforms its single-element counterpart. While not necessarily ideal for human interpretation, the network's resulting low-resolution microscope images (20X-comparable) offer a machine learning network sufficient contrast to match the classification performance of corresponding high-resolution imagery (100X-comparable), pointing a path towards accurate automation over large fields-of-view. △ Less

Submitted 27 June, 2020; originally announced June 2020.

Comments: 6 pages, 3 figures

Journal ref: Optics Letters Vol. 45, Issue 20, pp. 5684-5687 (2020)

Reconstructing undersampled photoacoustic microscopy images using deep learning

Authors: Anthony DiSpirito III, Daiwei Li, Tri Vu, Maomao Chen, Dong Zhang, Jianwen Luo, Roarke Horstmeyer, Junjie Yao

Abstract: One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and… ▽ More One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and selected a fully dense U-net (FD U-net) model that produced the best results. To mimic various undersampling conditions in practice, we artificially downsampled fully-sampled PAM images of mouse brain vasculature at different ratios. This allowed us to not only definitively establish the ground truth, but also train and test our deep learning model at various imaging conditions. Our results and numerical analysis have collectively demonstrated the robust performance of our model to reconstruct PAM images with as few as 2% of the original pixels, which may effectively shorten the imaging time without substantially sacrificing the image quality. △ Less

Submitted 30 May, 2020; originally announced June 2020.

Comments: 12 pages, 7 main figures, 3 supplemental figures (see last 2 pages)

Physics-enhanced machine learning for virtual fluorescence microscopy

Authors: Colin L. Cooke, Fanjie Kong, Amey Chaware, Kevin C. Zhou, Kanghyun Kim, Rong Xu, D. Michael Ando, Samuel J. Yang, Pavan Chandra Konda, Roarke Horstmeyer

Abstract: This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy ima… ▽ More This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy images. We validated our method on two different experimental setups, with different magnifications and different sample types, to show a consistent improvement in performance as compared to conventional illumination methods. Additionally, to understand the importance of learned illumination on inference task, we varied the dynamic range of the fluorescent image targets (from one to seven bits), and showed that the margin of improvement for learned patterns increased with the information content of the target. This work demonstrates the power of programmable optical elements at enabling better machine learning algorithm performance and at providing physical insight into next generation of machine-controlled imaging systems. △ Less

Submitted 21 April, 2020; v1 submitted 8 April, 2020; originally announced April 2020.

Comments: 12 pages, 13 figures

Diffraction tomography with a deep image prior

Authors: Kevin C. Zhou, Roarke Horstmeyer

Abstract: We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which rep… ▽ More We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy. △ Less

Submitted 10 December, 2019; originally announced December 2019.

Journal ref: Optics Express 28(9), 12872-12896 (2020)

Towards an Intelligent Microscope: adaptively learned illumination for optimal sample classification

Authors: Amey Chaware, Colin L. Cooke, Kanghyun Kim, Roarke Horstmeyer

Abstract: Recent machine learning techniques have dramatically changed how we process digital images. However, the way in which we capture images is still largely driven by human intuition and experience. This restriction is in part due to the many available degrees of freedom that alter the image acquisition process (lens focus, exposure, filtering, etc). Here we focus on one such degree of freedom - illum… ▽ More Recent machine learning techniques have dramatically changed how we process digital images. However, the way in which we capture images is still largely driven by human intuition and experience. This restriction is in part due to the many available degrees of freedom that alter the image acquisition process (lens focus, exposure, filtering, etc). Here we focus on one such degree of freedom - illumination within a microscope - which can drastically alter information captured by the image sensor. We present a reinforcement learning system that adaptively explores optimal patterns to illuminate specimens for immediate classification. The agent uses a recurrent latent space to encode a large set of variably-illuminated samples and illumination patterns. We train our agent using a reward that balances classification confidence with image acquisition cost. By synthesizing knowledge over multiple snapshots, the agent can classify on the basis of all previous images with higher accuracy than from naively illuminated images, thus demonstrating a smarter way to physically capture task-specific information. △ Less

Submitted 13 February, 2020; v1 submitted 22 October, 2019; originally announced October 2019.

Comments: 5 pages, 5 figures

Artificial Neural Network with Physical Dynamic Metasurface Layer for Optimal Sensing

Authors: Philipp del Hougne, Mohammadreza F. Imani, Aaron V. Diebold, Roarke Horstmeyer, David R. Smith

Abstract: We address the fundamental question of how to optimally probe a scene with electromagnetic (EM) radiation to yield a maximum amount of information relevant to a particular task. Machine learning (ML) techniques have emerged as powerful tools to extract task-relevant information from a wide variety of EM measurements, ranging from optics to the microwave domain. However, given the ability to active… ▽ More We address the fundamental question of how to optimally probe a scene with electromagnetic (EM) radiation to yield a maximum amount of information relevant to a particular task. Machine learning (ML) techniques have emerged as powerful tools to extract task-relevant information from a wide variety of EM measurements, ranging from optics to the microwave domain. However, given the ability to actively illuminate a particular scene with a programmable EM wavefront, it is often not clear what wavefronts optimally encode information for the task at hand (e.g., object detection, classification). Here, we show that by integrating a physical model of scene illumination and detection into a ML pipeline, we can jointly learn optimal sampling and measurement processing strategies for a given task. We consider in simulation the example of classifying objects using microwave radiation produced by dynamic metasurfaces. By integrating an analytical forward model describing the metamaterial elements as coupled dipoles into the ML pipeline, we jointly train analog model weights with digital neural network weights. The learned non-intuitive illumination settings yield a higher classification accuracy using fewer measurements. On the practical level, these results are highly relevant to emerging context-aware systems such as autonomous vehicles, touchless human-interactive devices or within smart health care, where strict time constraints place severe limits on measurement strategies. On the conceptual level, our work serves as a bridge between wavefront shaping and tunable metasurface design on the physical layer and ML techniques on the processing layer. △ Less

Submitted 24 June, 2019; originally announced June 2019.

Comments: 14 pages including 5 figures + 6 pages Supplemental Material

Journal ref: Adv. Sci. 1901913 (2019)

Convolutional neural networks that teach microscopes how to image

Authors: Roarke Horstmeyer, Richard Y. Chen, Barbara Kappes, Benjamin Judkewitz

Abstract: Deep learning algorithms offer a powerful means to automatically analyze the content of medical images. However, many biological samples of interest are primarily transparent to visible light and contain features that are difficult to resolve with a standard optical microscope. Here, we use a convolutional neural network (CNN) not only to classify images, but also to optimize the physical layout o… ▽ More Deep learning algorithms offer a powerful means to automatically analyze the content of medical images. However, many biological samples of interest are primarily transparent to visible light and contain features that are difficult to resolve with a standard optical microscope. Here, we use a convolutional neural network (CNN) not only to classify images, but also to optimize the physical layout of the imaging device itself. We increase the classification accuracy of a microscope's recorded images by merging an optical model of image formation into the pipeline of a CNN. The resulting network simultaneously determines an ideal illumination arrangement to highlight important sample features during image acquisition, along with a set of convolutional weights to classify the detected images post-capture. We demonstrate our joint optimization technique with an experimental microscope configuration that automatically identifies malaria-infected cells with 5-10% higher accuracy than standard and alternative microscope lighting designs. △ Less

Submitted 21 September, 2017; originally announced September 2017.

Comments: 13 pages, 6 figures

Scattering correlations of time-gated light

Authors: Mykola Kadobianskyi, Ioannis N. Papadopoulos, Thomas Chaigne, Roarke Horstmeyer, Benjamin Judkewitz

Abstract: Manipulating the propagation of light through scattering media remains a major challenge for many applications, including astronomy, biomedical imaging and colloidal optics. Wavefront shaping is one of the most promising ways to mitigate scattering and focus through inhomogeneous samples. However, wavefront correction remains accurate over only a limited spatial extent within the scattering medium… ▽ More Manipulating the propagation of light through scattering media remains a major challenge for many applications, including astronomy, biomedical imaging and colloidal optics. Wavefront shaping is one of the most promising ways to mitigate scattering and focus through inhomogeneous samples. However, wavefront correction remains accurate over only a limited spatial extent within the scattering medium - a correlation range referred to as the optical memory effect. Here, by selecting only the weakly scattered light for wavefront shaping, we show that the addition of temporal degrees of freedom enhances this correlation range. We investigate spatial scattering correlations by digitally time-gating the early arriving light in the spectral domain. We demonstrate that the range of the translational memory effect for the early arriving light is increased almost fourfold, paving the way for a range of scattering media imaging applications. △ Less

Submitted 21 July, 2017; originally announced July 2017.

The generalized optical memory effect

Authors: Gerwin Osnabrugge, Roarke Horstmeyer, Ioannis N. Papadopoulos, Benjamin Judkewitz, Ivo M. Vellekoop

Abstract: The optical memory effect is a well-known type of wave correlation that is observed in coherent fields that scatter through thin and diffusive materials, like biological tissue. It is a fundamental physical property of scattering media that can be harnessed for deep-tissue microscopy or 'through-the-wall' imaging applications. Here we show that the optical memory effect is a special case of a far… ▽ More The optical memory effect is a well-known type of wave correlation that is observed in coherent fields that scatter through thin and diffusive materials, like biological tissue. It is a fundamental physical property of scattering media that can be harnessed for deep-tissue microscopy or 'through-the-wall' imaging applications. Here we show that the optical memory effect is a special case of a far more general class of wave correlation. Our new theoretical framework explains how waves remain correlated over both space and angle when they are jointly shifted and tilted inside scattering media of arbitrary geometry. We experimentally demonstrate the existence of such coupled correlations and describe how they can be used to optimize the scanning range in adaptive optics microscopes. △ Less

Submitted 3 May, 2017; originally announced May 2017.

Comments: 11 pages, 4 figures, supplementary materials included

Subsampled Phase Retrieval for On-chip Lensless Holographic Video

Authors: Donghun Ryu, Zihao Wang, Kuan He, Roarke Horstmeyer, Oliver Cossairt

Abstract: On-chip holographic video is a convenient way to monitor biological samples simultaneously at high spatial resolution and over a wide field-of-view. However, due to the limited readout rate of digital detector arrays, one often faces a tradeoff between the per-frame pixel count and frame rate of the captured video. In this report, we propose a subsampled phase retrieval (SPR) algorithm to overcome… ▽ More On-chip holographic video is a convenient way to monitor biological samples simultaneously at high spatial resolution and over a wide field-of-view. However, due to the limited readout rate of digital detector arrays, one often faces a tradeoff between the per-frame pixel count and frame rate of the captured video. In this report, we propose a subsampled phase retrieval (SPR) algorithm to overcome the spatial-temporal trade-off in holographic video. Compared to traditional phase retrieval approaches, our SPR algorithm uses over an order of magnitude less pixel measurements while maintaining suitable reconstruction quality. We use an on-chip holographic video setup with pixel sub-sampling to experimentally demonstrate a factor of 5.5 increase in sensor frame rate while monitoring the in vivo movement of Peranema microorganisms. △ Less

Submitted 7 December, 2016; originally announced December 2016.

Diffraction tomography with Fourier ptychography

Authors: Roarke Horstmeyer, Changhuei Yang

Abstract: This article presents a method to perform diffraction tomography in a standard microscope that includes an LED array for illumination. After acquiring a sequence of intensity-only images of a thick sample, a ptychography-based reconstruction algorithm solves for its unknown complex index of refraction across three dimensions. The experimental microscope demonstrates a spatial resolution of 0.39… ▽ More This article presents a method to perform diffraction tomography in a standard microscope that includes an LED array for illumination. After acquiring a sequence of intensity-only images of a thick sample, a ptychography-based reconstruction algorithm solves for its unknown complex index of refraction across three dimensions. The experimental microscope demonstrates a spatial resolution of 0.39 $μ$m and an axial resolution of 3.7 $μ$m at the Nyquist-Shannon sampling limit (0.54 $μ$m and 5.0 $μ$m at the Sparrow limit, respectively), across a total imaging volume of 2.2 mm $\times$ 2.2 mm $\times$ 110 $μ$m. Unlike competing methods, the 3D tomograms presented in this article are continuous, quantitative, and formed without the need for interferometry or any moving parts. Wide field-of-view reconstructions of thick biological specimens demonstrate potential applications in pathology and developmental biology. △ Less

Submitted 29 October, 2015; originally announced October 2015.

Comments: 22 pages, 7 figures

Toward Long Distance, Sub-diffraction Imaging Using Coherent Camera Arrays

Authors: Jason Holloway, M. Salman Asif, Manoj Kumar Sharma, Nathan Matsuda, Roarke Horstmeyer, Oliver Cossairt, Ashok Veeraraghavan

Abstract: In this work, we propose using camera arrays coupled with coherent illumination as an effective method of improving spatial resolution in long distance images by a factor of ten and beyond. Recent advances in ptychography have demonstrated that one can image beyond the diffraction limit of the objective lens in a microscope. We demonstrate a similar imaging system to image beyond the diffraction l… ▽ More In this work, we propose using camera arrays coupled with coherent illumination as an effective method of improving spatial resolution in long distance images by a factor of ten and beyond. Recent advances in ptychography have demonstrated that one can image beyond the diffraction limit of the objective lens in a microscope. We demonstrate a similar imaging system to image beyond the diffraction limit in long range imaging. We emulate a camera array with a single camera attached to an X-Y translation stage. We show that an appropriate phase retrieval based reconstruction algorithm can be used to effectively recover the lost high resolution details from the multiple low resolution acquired images. We analyze the effects of noise, required degree of image overlap, and the effect of increasing synthetic aperture size on the reconstructed image quality. We show that coherent camera arrays have the potential to greatly improve imaging performance. Our simulations show resolution gains of 10x and more are achievable. Furthermore, experimental results from our proof-of-concept systems show resolution gains of 4x-7x for real scenes. Finally, we introduce and analyze in simulation a new strategy to capture macroscopic Fourier Ptychography images in a single snapshot, albeit using a camera array. △ Less

Submitted 28 October, 2015; originally announced October 2015.

Comments: 13 pages, 16 figures, submitted to IEEE Transactions on Computational Imaging

Solving ptychography with a convex relaxation

Authors: Roarke Horstmeyer, Richard Y. Chen, Xiaoze Ou, Brendan Ames, Joel A. Tropp, Changhuei Yang

Abstract: Ptychography is a powerful computational imaging technique that transforms a collection of low-resolution images into a high-resolution sample reconstruction. Unfortunately, algorithms that are currently used to solve this reconstruction problem lack stability, robustness, and theoretical guarantees. Recently, convex optimization algorithms have improved the accuracy and reliability of several rel… ▽ More Ptychography is a powerful computational imaging technique that transforms a collection of low-resolution images into a high-resolution sample reconstruction. Unfortunately, algorithms that are currently used to solve this reconstruction problem lack stability, robustness, and theoretical guarantees. Recently, convex optimization algorithms have improved the accuracy and reliability of several related reconstruction efforts. This paper proposes a convex formulation of the ptychography problem. This formulation has no local minima, it can be solved using a wide range of algorithms, it can incorporate appropriate noise models, and it can include multiple a priori constraints. The paper considers a specific algorithm, based on low-rank factorization, whose runtime and memory usage are near-linear in the size of the output image. Experiments demonstrate that this approach offers a 25% lower background variance on average than alternating projections, the current standard algorithm for ptychographic reconstruction. △ Less

Submitted 3 December, 2014; originally announced December 2014.

Comments: 8 pages, 8 figures

Translation correlations in anisotropically scattering media

Authors: Benjamin Judkewitz, Roarke Horstmeyer, Ivo M. Vellekoop, Ioannis N. Papadopoulos, Changhuei Yang

Abstract: Controlling light propagation across scattering media by wavefront shaping holds great promise for a wide range of communications and imaging applications. However, finding the right wavefront to shape is a challenge when the mapping between input and output scattered wavefronts (i.e. the transmission matrix) is not known. Correlations in transmission matrices, especially the so-called memory-effe… ▽ More Controlling light propagation across scattering media by wavefront shaping holds great promise for a wide range of communications and imaging applications. However, finding the right wavefront to shape is a challenge when the mapping between input and output scattered wavefronts (i.e. the transmission matrix) is not known. Correlations in transmission matrices, especially the so-called memory-effect, have been exploited to address this limitation. However, the traditional memory-effect applies to thin scattering layers at a distance from the target, which precludes its use within thick scattering media, such as fog and biological tissue. Here, we theoretically predict and experimentally verify new transmission matrix correlations within thick anisotropically scattering media, with important implications for biomedical imaging and adaptive optics. △ Less

Submitted 9 June, 2015; v1 submitted 26 November, 2014; originally announced November 2014.

Comments: main article (18 pages) and appendices (6 pages)

Wide-field, high-resolution Fourier ptychographic microscopy

Authors: Guoan Zheng, Roarke Horstmeyer, Changhuei Yang

Abstract: In this article, we report an imaging method, termed Fourier ptychographic microscopy (FPM), which iteratively stitches together a number of variably illuminated, low-resolution intensity images in Fourier space to produce a wide-field, high-resolution complex sample image. By adopting a wavefront correction strategy, the FPM method can also correct for aberrations and digitally extend a microscop… ▽ More In this article, we report an imaging method, termed Fourier ptychographic microscopy (FPM), which iteratively stitches together a number of variably illuminated, low-resolution intensity images in Fourier space to produce a wide-field, high-resolution complex sample image. By adopting a wavefront correction strategy, the FPM method can also correct for aberrations and digitally extend a microscope's depth-of-focus beyond the physical limitations of its optics. As a demonstration, we built a microscope prototype with a resolution of 0.78 um, a field-of-view of ~120 mm2, and a resolution-invariant depth-of-focus of 0.3 mm (characterized at 632 nm). Gigapixel colour images of histology slides verify FPM's successful operation. The reported imaging procedure transforms the general challenge of high-throughput, high-resolution microscopy from one that is coupled to the physical limitations of the system's optics to one that is solvable through computation. △ Less

Submitted 1 May, 2014; originally announced May 2014.

Comments: 11 pages, 4 figures. Nature Photonics,2013

Reconfigurable random bit storage using polymer-dispersed liquid crystal

Authors: Roarke Horstmeyer, Sid Assawaworrarit, Changhuei Yang

Abstract: We present an optical method of storing random cryptographic keys, at high densities, within an electronically reconfigurable volume of polymer-dispersed liquid crystal (PDLC) film. We demonstrate how temporary application of a voltage above PDLC's saturation threshold can completely randomize (i.e., decorrelate) its optical scattering potential in less than a second. A unique optical setup is bui… ▽ More We present an optical method of storing random cryptographic keys, at high densities, within an electronically reconfigurable volume of polymer-dispersed liquid crystal (PDLC) film. We demonstrate how temporary application of a voltage above PDLC's saturation threshold can completely randomize (i.e., decorrelate) its optical scattering potential in less than a second. A unique optical setup is built around this resettable PDLC film to non-electronically save many random cryptographic bits, with minimal error, over a period of one day. These random bits, stored at an unprecedented density (10 Gb per cubic millimeter), can then be erased and transformed into a new random key space in less than one second. Cryptographic applications of such a volumetric memory device include use as a crypto-currency wallet and as a source of resettable "fingerprints" for time-sensitive authentication. △ Less

Submitted 10 March, 2014; originally announced March 2014.

Comments: 5 pages, 6 figures

Quantitative phase imaging via Fourier ptychographic microscopy

Authors: Xiaoze Ou, Roarke Horstmeyer, Changhuei Yang, Guoan Zheng

Abstract: Fourier ptychographic microscopy (FPM) is a recently developed imaging modality that uses angularly varying illumination to extend a system performance beyond the limit defined by its optical elements. The FPM technique applies a novel phase retrieval procedure to achieve both resolution enhancement and complex image recovery. In this letter, we compare FPM data to both theoretical prediction and… ▽ More Fourier ptychographic microscopy (FPM) is a recently developed imaging modality that uses angularly varying illumination to extend a system performance beyond the limit defined by its optical elements. The FPM technique applies a novel phase retrieval procedure to achieve both resolution enhancement and complex image recovery. In this letter, we compare FPM data to both theoretical prediction and phase-shifting digital holography measurement to show that its acquired phase maps are quantitative and artifact-free. We additionally explore the relationship between the achievable spatial and optical thickness resolution offered by a reconstructed FPM phase image. We conclude by demonstrating both enhanced visualization and the collection of otherwise unobservable sample information using FPM quantitative phase. △ Less

Submitted 28 September, 2013; originally announced September 2013.

Comments: 4 pages, 5 figures

Physical key-protected one-time pad

Authors: Roarke Horstmeyer, Benjamin Judkewitz, Ivo Vellekoop, Sid Assawaworrarit, Changhuei Yang

Abstract: We describe an encrypted communication principle that can form a perfectly secure link between two parties without electronically saving either of their keys. Instead, cryptographic key bits are kept safe within the unique mesoscopic randomness of two volumetric scattering materials. We demonstrate how a shared set of patterned optical probes can generate 10 gigabits of statistically verified rand… ▽ More We describe an encrypted communication principle that can form a perfectly secure link between two parties without electronically saving either of their keys. Instead, cryptographic key bits are kept safe within the unique mesoscopic randomness of two volumetric scattering materials. We demonstrate how a shared set of patterned optical probes can generate 10 gigabits of statistically verified randomness between a pair of unique 2 cubic millimeter scattering objects. This shared randomness is used to facilitate information-theoretically secure communication following a modified one-time pad protocol. Benefits of volumetric physical storage over electronic memory include the inability to probe, duplicate or selectively reset any random bits without fundamentally altering the entire key space. Beyond the demonstrated communication scheme, our ability to securely couple the randomness contained within two unique physical objects may help strengthen the hardware for a large class of cryptographic protocols, which is currently a critically weak link in the security pipeline of our increasingly mobile communication culture. △ Less

Submitted 26 November, 2013; v1 submitted 16 May, 2013; originally announced May 2013.

Comments: 6 pages, 4 figures

Characterization of shift-variant pupil aberrations for wide field-of-view microscopy

Authors: Guoan Zheng, Xiaoze Ou, Roarke Horstmeyer, Changhuei Yang

Abstract: We describe a simple and robust approach for characterizing the shift-variant pupil aberrations of wide field-of-view microscopy systems. We derive the location-dependent pupil transfer functions by first capturing multiple intensity images at different defocus settings; a generalized pattern search (GPS) algorithm is then applied to recover the complex pupil functions at ~350 different spatial lo… ▽ More We describe a simple and robust approach for characterizing the shift-variant pupil aberrations of wide field-of-view microscopy systems. We derive the location-dependent pupil transfer functions by first capturing multiple intensity images at different defocus settings; a generalized pattern search (GPS) algorithm is then applied to recover the complex pupil functions at ~350 different spatial locations over the entire field-of-view. Parameter fitting transforms these pupil functions into accurate 2D aberration maps. We demonstrate shift-variant aberration compensation by using an information-preserving image deconvolution scheme over the entire field-of-view. Such automated shift-variant pupil characterization will facilitate new approaches of aberration correction for future gigapixel imaging platforms. △ Less

Submitted 20 June, 2013; v1 submitted 27 April, 2013; originally announced April 2013.

View-Dependent Displays and the Space of Light Fields

Authors: Roarke Horstmeyer, Se Baek Oh, Ramesh Raskar

Abstract: In this paper we explore how light propagates from thin elements into a volume for viewing. In particular, devices that are typically connected with geometric optics, like parallax barriers, differ in treatment with those that obey physical optics, like holograms. However, the two concepts are often used to achieve the same effect of capturing or displaying a combination of spatial and angular inf… ▽ More In this paper we explore how light propagates from thin elements into a volume for viewing. In particular, devices that are typically connected with geometric optics, like parallax barriers, differ in treatment with those that obey physical optics, like holograms. However, the two concepts are often used to achieve the same effect of capturing or displaying a combination of spatial and angular information. This paper attempts to connect the two approaches under a general framework based in ray space, from which insights into applications and limitations of both parallax-based and holography-based systems can be observed. We show that each display form can generate a light distribution that can always be expressed as a rank-1 matrix. Knowledge of this limitation is then discussed in the context of extending the capabilities of current display forms by considering the use of partially coherent light. △ Less

Submitted 30 July, 2010; originally announced August 2010.

Comments: 9 pages, 13 figures

$^{12}$C/$^{13}$C ratio in planetary nebulae from the IUE archives

Authors: R. H. Rubin, G. J. Ferland, E. E. Chollet, R. Horstmeyer

Abstract: We investigated the abundance ratio of $^{12}$C/$^{13}$C in planetary nebulae by examining emission lines arising from \ion{C}{3} $2s2p ^3P_{2,1,0} \to 2s^2 ^1S_0$. Spectra were retrieved from the International Ultraviolet Explorer archives, and multiple spectra of the same object were coadded to achieve improved signal-to-noise. The $^{13}$C hyperfine structure line at 1909.6 Åwas detected in N… ▽ More We investigated the abundance ratio of $^{12}$C/$^{13}$C in planetary nebulae by examining emission lines arising from \ion{C}{3} $2s2p ^3P_{2,1,0} \to 2s^2 ^1S_0$. Spectra were retrieved from the International Ultraviolet Explorer archives, and multiple spectra of the same object were coadded to achieve improved signal-to-noise. The $^{13}$C hyperfine structure line at 1909.6 Åwas detected in NGC 2440. The $^{12}$C/$^{13}$C ratio was found to be $\sim4.4\pm$1.2. In all other objects, we provide an upper limit for the flux of the 1910 Åline. For 23 of these sources, a lower limit for the $^{12}$C/$^{13}$C ratio was established. The impact on our current understanding of stellar evolution is discussed. The resulting high signal-to-noise \ion{C}{3} spectrum helps constrain the atomic physics of the line formation process. Some objects have the measured 1907/1909 flux ratio outside the low-electron density theoretical limit for $^{12}$C. A mixture of $^{13}$C with $^{12}$C helps to close the gap somewhat. Nevertheless, some observed 1907/1909 flux ratios still appear too high to conform to the presently predicted limits. It is shown that this limit, as well as the 1910/1909 flux ratio, are predominantly influenced by using the standard partitioning among the collision strengths for the multiplet $^1S_0$--$^3P_J$ according to the statistical weights. A detailed calculation for the fine structure collision strengths between these individual levels would be valuable. △ Less

Submitted 6 January, 2004; originally announced January 2004.

Comments: ApJ accepted: 19 pages, 3 Figures, 2 Tables

Journal ref: Astrophys.J. 605 (2004) 784-792