Cellular analysis using label-free parallel array microscopy with Fourier ptychography

Quantitative phase imaging (QPI) is an ideal method to non-invasively monitor cell populations and provide label-free imaging and analysis. QPI offers enhanced sample characterization and cell counting compared to conventional label-free techniques. We demonstrate this in the current study through a comparison of cell counting data from digital phase contrast (DPC) imaging and from QPI using a system based on Fourier ptychographic microscopy (FPM). Our FPM system offers multi-well, parallel imaging and a QPI-specific cell segmentation method to establish automated and reliable cell counting. Three cell types were studied and FPM showed improvement in the ability to resolve fine details and thin cells, despite limitations of the FPM system incurred by imaging artifacts. Relative to manually counted fluorescence ground-truth, cell counting results after automated segmentation showed improved accuracy with QPI over DPC.     

Robust Fourier ptychographic microscopy via a physics-based defocusing strategy for calibrating angle-varied LED illumination

Fourier ptychographic microscopy (FPM) is a recently developed computational imaging technique for wide-field, high-resolution microscopy with a high space-bandwidth product. It integrates the concepts of synthetic aperture and phase retrieval to surpass the resolution limit imposed by the employed objective lens. In the FPM framework, the position of each sub-spectrum needs to be accurately known to ensure the success of the phase retrieval process. Different from the conventional methods with mechanical adjustment or data-driven optimization strategies, here we report a physics-based defocusing strategy for correcting large-scale positional deviation of the LED illumination in FPM. Based on a subpixel image registration process with a defocused object, we can directly infer the illumination parameters including the lateral offsets of the light source, the in-plane rotation angle of the LED array, and the distance between the sample and the LED board. The feasibility and effectiveness of our method are validated with both simulations and experiments. We show that the reported strategy can obtain high-quality reconstructions of both the complex object and pupil function even the LED array is randomly placed under the sample with both unknown lateral offsets and rotations. As such, it enables the development of robust FPM systems by reducing the requirements on fine mechanical adjustment and data-driven correction in the construction process.     

Fourier phase microscopy with white light

Laser-based Fourier phase microscopy (FPM) works on the principle of decomposition of an image field in two spatial components that can be controllably shifted in phase with respect to each other. However, due to the coherent illumination, the contrast in phase images is degraded by speckles. In this paper we present FPM with spatially coherent white light (wFPM), which offers high spatial phase sensitivity due to the low temporal coherence and high temporal phase stability due to common path geometry. Further, by using a fast spatial light modulator (SLM) and a fast scientific-grade complementary metal oxide semiconductor (sCMOS) camera, we report imaging at a maximum rate of 12.5 quantitative phase frames per second with 5.5 mega pixels image size. We illustrate the utility of wFPM as a contrast enhancement as well as dynamic phase measurement method by imaging section of benign colonic glands and red blood cell membrane fluctuation.OCIS codes: (170.0180) Microscopy, (070.0070) Fourier optics and signal processing, (070.6120) Spatial light modulators, (120.5050) Phase measurement     

Deep-learning super-resolution light-sheet add-on microscopy (Deep-SLAM) for easy isotropic volumetric imaging of large biological specimens

Isotropic 3D histological imaging of large biological specimens is highly desired but remains highly challenging to current fluorescence microscopy technique. Here we present a new method, termed deep-learning super-resolution light-sheet add-on microscopy (Deep-SLAM), to enable fast, isotropic light-sheet fluorescence imaging on a conventional wide-field microscope. After integrating a minimized add-on device that transforms an inverted microscope into a 3D light-sheet microscope, we further integrate a deep neural network (DNN) procedure to quickly restore the ambiguous z-reconstructed planes that suffer from still insufficient axial resolution of light-sheet illumination, thereby achieving isotropic 3D imaging of thick biological specimens at single-cell resolution. We apply this easy and cost-effective Deep-SLAM approach to the anatomical imaging of single neurons in a meso-scale mouse brain, demonstrating its potential for readily converting commonly-used commercialized 2D microscopes to high-throughput 3D imaging, which is previously exclusive for high-end microscopy implementations.     

Enhancement of image quality and imaging depth with Airy light-sheet microscopy in cleared and non-cleared neural tissue

We have investigated the effect of Airy illumination on the image quality and depth penetration of digitally scanned light-sheet microscopy in turbid neural tissue. We used Fourier analysis of images acquired using Gaussian and Airy light-sheets to assess their respective image quality versus penetration into the tissue. We observed a three-fold average improvement in image quality at 50 μm depth with the Airy light-sheet. We also used optical clearing to tune the scattering properties of the tissue and found that the improvement when using an Airy light-sheet is greater in the presence of stronger sample-induced aberrations. Finally, we used homogeneous resolution probes in these tissues to quantify absolute depth penetration in cleared samples with each beam type. The Airy light-sheet method extended depth penetration by 30% compared to a Gaussian light-sheet.OCIS codes: (070.0070) Fourier optics and signal processing, (100.2960) Image analysis, (100.2980) Image enhancement, (170.3880) Medical and biological imaging, (180.0180) Microscopy, (180.2520) Fluorescence microscopy     

Achromatic miniature lens system for coherent Raman scattering microscopy

We discuss the design and performance of a miniature objective lens optimized for coherent Raman scattering microscopy. The packaged lens assembly has a numerical aperture of 0.51 in water and an outer diameter of 8 mm. The lens system exhibits minimum chromatic aberrations, and produces coherent Raman scattering images with sub-micrometer lateral resolution (0.648 μm) using near-infrared excitation pulses. We demonstrate that despite the small dimensions of the miniature objective, the performance of this lens system is comparable to standard microscope objective lenses, offering opportunities for miniaturizing coherent Raman scattering imaging probes without sacrificing the image quality.OCIS codes: (180.4315) Nonlinear microscopy, (350.3950) Micro-optics, (220.3630) Lenses     

Field-portable reflection and transmission microscopy based on lensless holography

We demonstrate a lensfree dual-mode holographic microscope that can image specimens in both transmission and reflection geometries using in-line transmission and off-axis reflection holography, respectively. This field-portable dual-mode holographic microscope has a weight of ~200 g with dimensions of 15 x 5.5 x 5cm, where a laser source is powered by two batteries. Based on digital in-line holography, our transmission microscope achieves a sub-pixel lateral resolution of ≤2 μm over a wide field-of-view (FOV) of ~24 mm(2) due to its unit fringe magnification geometry. Despite its simplicity and ease of operation, in-line transmission geometry is not suitable to image dense or connected objects such as tissue slides since the reference beam gets distorted causing severe aberrations in reconstruction of such objects. To mitigate this challenge, on the same cost-effective and field-portable assembly we built a lensless reflection mode microscope based on digital off-axis holography where a beam-splitter is used to interfere a tilted reference wave with the reflected light from the object surface, creating an off-axis hologram of the specimens on a CMOS sensor-chip. As a result of the reduced space-bandwidth product of the off-axis geometry compared to its in-line counterpart, the imaging FOV of our reflection mode is reduced to ~9 mm(2), while still achieving a similar sub-pixel resolution of ≤2 μm. We tested the performance of this compact dual-mode microscopy unit by imaging a US-air force resolution test target, various micro-particles as well as a histopathology slide corresponding to skin tissue. Due to its compact, cost-effective, and lightweight design, this dual-mode lensless holographic microscope might especially be useful for field-use or for conducting microscopic analysis in resource-poor settings.     

Single full-FOV reconstruction Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a recently developed computational imaging technique that has high-resolution and wide field-of-view (FOV). FPM bypasses the NA limit of the system by stitching a number of variable-illuminated measured images in Fourier space. On the basis of the wide FOV of the low NA objective, the high-resolution image with a wide FOV can be reconstructed through the phase recovery algorithm. However, the high-resolution reconstruction images are affected by the LED array point light source. The results are: (1) the intensities collected by the sample are severely declined when edge LEDs illuminate the sample; (2) the multiple reconstructions are caused by wavevectors inconsistency for the full FOV images. Here, we propose a new lighting scheme termed full FOV Fourier ptychographic microscopy (F³PM). By combining the LED array and telecentric lens, the method can provide plane waves with different angles while maintaining uniform intensity. Benefiting from the telecentric performance and f‒θ property of the telecentric lens, the system stability is improved and the relationship between the position of LED and its illumination angle is simplified. The excellent plane wave provided by the telecentric lens guarantees the same wavevector in the full FOV, and we use this wavevector to reconstruct the full FOV during one time. The area and diameter of the single reconstruction FOV reached 14.6mm² and 5.4 mm, respectively, and the diameter is very close to the field number (5.5 mm) of the 4× objective. Compared with the traditional FPM, we have increased the diameter of FOV in a single reconstruction by ∼ 10 times, eliminating the complicated steps of computational redundancy and image stitching.     

Fast digital refocusing and depth of field extended Fourier ptychography microscopy

Fourier ptychography microscopy (FPM) shares its roots with the synthetic aperture technique and phase retrieval method, and is a recently developed computational microscopic super-resolution technique. By turning on the light-emitting diode (LED) elements sequentially and acquiring the corresponding images that contain different spatial frequencies, FPM can achieve a wide field-of-view (FOV), high-spatial-resolution imaging and phase recovery simultaneously. Conventional FPM assumes that the sample is sufficiently thin and strictly in focus. Nevertheless, even for a relatively thin sample, the non-planar distribution characteristics and the non-ideal position/posture of the sample will cause all or part of FOV to be defocused. In this paper, we proposed a fast digital refocusing and depth-of-field (DOF) extended FPM strategy by taking the advantages of image lateral shift caused by sample defocusing and varied-angle illuminations. The lateral shift amount is proportional to the defocus distance and the tangent of the illumination angle. Instead of searching the optimal defocus distance with the optimization search strategy, which is time consuming, the defocus distance of each subregion of the sample can be precisely and quickly obtained by calculating the relative lateral shift amounts corresponding to different oblique illuminations. And then, digital refocusing strategy rooting in the angular spectrum (AS) method is integrated into FPM framework to achieve the high-resolution and phase information reconstruction for each part of the sample, which means the DOF of the FPM can be effectively extended. The feasibility of the proposed method in fast digital refocusing and DOF extending is verified in the actual experiments with the USAF chart and biological samples.     

Microscopy illumination engineering using a low-cost liquid crystal display

Illumination engineering is critical for obtaining high-resolution, high-quality images in microscope settings. In a typical microscope, the condenser lens provides sample illumination that is uniform and free from glare. The associated condenser diaphragm can be manually adjusted to obtain the optimal illumination numerical aperture. In this paper, we report a programmable condenser lens for active illumination control. In our prototype setup, we used a $15 liquid crystal display as a transparent spatial light modulator and placed it at the back focal plane of the condenser lens. By setting different binary patterns on the display, we can actively control the illumination and the spatial coherence of the microscope platform. We demonstrated the use of such a simple scheme for multimodal imaging, including bright-field microscopy, darkfield microscopy, phase-contrast microscopy, polarization microscopy, 3D tomographic imaging, and super-resolution Fourier ptychographic imaging. The reported illumination engineering scheme is cost-effective and compatible with most existing platforms. It enables a turnkey solution with high flexibility for researchers in various communities. From the engineering point-of-view, the reported illumination scheme may also provide new insights for the development of multimodal microscopy and Fourier ptychographic imaging.OCIS codes: (170.2945) Illumination design, (170.0180) Microscopy, (170.3010) Image reconstruction techniques, (100.3190) Inverse problems     

Quantitative analysis of illumination and detection corrections in adaptive light sheet fluorescence microscopy

Light-sheet fluorescence microscopy (LSFM) is a high-speed, high-resolution and minimally phototoxic technique for 3D imaging of in vivo and in vitro specimens. LSFM exhibits optical sectioning and when combined with tissue clearing techniques, it facilitates imaging of centimeter scale specimens with micrometer resolution. Although LSFM is ubiquitous, it still faces two main challenges that effect image quality especially when imaging large volumes with high-resolution. First, the light-sheet illumination plane and detection lens focal plane need to be coplanar, however sample-induced aberrations can violate this requirement and degrade image quality. Second, introduction of sample-induced optical aberrations in the detection path. These challenges intensify when imaging whole organisms or structurally complex specimens like cochleae and bones that exhibit many transitions from soft to hard tissue or when imaging deep (> 2 mm). To resolve these challenges, various illumination and aberration correction methods have been developed, yet no adaptive correction in both the illumination and the detection path have been applied to improve LSFM imaging. Here, we bridge this gap, by implementing the two correction techniques on a custom built adaptive LSFM. The illumination beam angular properties are controlled by two galvanometer scanners, while a deformable mirror is positioned in the detection path to correct for aberrations. By imaging whole porcine cochlea, we compare and contrast these correction methods and their influence on the image quality. This knowledge will greatly contribute to the field of adaptive LSFM, and imaging of large volumes of tissue cleared specimens.     

Compact plane illumination plugin device to enable light sheet fluorescence imaging of multi-cellular organisms on an inverted wide-field microscope

We developed a compact plane illumination plugin (PIP) device which enabled plane illumination and light sheet fluorescence imaging on a conventional inverted microscope. The PIP device allowed the integration of microscope with tunable laser sheet profile, fast image acquisition, and 3-D scanning. The device is both compact, measuring approximately 15 by 5 by 5 cm, and cost-effective, since we employed consumer electronics and an inexpensive device molding method. We demonstrated that PIP provided significant contrast and resolution enhancement to conventional microscopy through imaging different multi-cellular fluorescent structures, including 3-D branched cells in vitro and live zebrafish embryos. Imaging with the integration of PIP greatly reduced out-of-focus contamination and generated sharper contrast in acquired 2-D plane images when compared with the stand-alone inverted microscope. As a result, the dynamic fluid domain of the beating zebrafish heart was clearly segmented and the functional monitoring of the heart was achieved. Furthermore, the enhanced axial resolution established by thin plane illumination of PIP enabled the 3-D reconstruction of the branched cellular structures, which leads to the improvement on the functionality of the wide field microscopy.OCIS codes: (170.3880) Medical and biological imaging, (170.2520) Fluorescence microscopy, (110.3010) Image reconstruction techniques, (110.2945) Illumination design, (170.6900) Three-dimensional microscopy     

Fourier ptychography multi-parameunter neural network with composite physical priori optimization

Fourier ptychography microscopy(FPM) is a recently developed computational imaging approach for microscopic super-resolution imaging. By turning on each light-emitting-diode (LED) located on different position on the LED array sequentially and acquiring the corresponding images that contain different spatial frequency components, high spatial resolution and quantitative phase imaging can be achieved in the case of large field-of-view. Nevertheless, FPM has high requirements for the system construction and data acquisition processes, such as precise LEDs position, accurate focusing and appropriate exposure time, which brings many limitations to its practical applications. In this paper, inspired by artificial neural network, we propose a Fourier ptychography multi-parameter neural network (FPMN) with composite physical prior optimization. A hybrid parameter determination strategy combining physical imaging model and data-driven network training is proposed to recover the multi layers of the network corresponding to different physical parameters, including sample complex function, system pupil function, defocus distance, LED array position deviation and illumination intensity fluctuation, etc. Among these parameters, LED array position deviation is recovered based on the features of brightfield to darkfield transition low-resolution images while the others are recovered in the process of training of the neural network. The feasibility and effectiveness of FPMN are verified through simulations and actual experiments. Therefore FPMN can evidently reduce the requirement for practical applications of FPM.     

Quantitative Jones matrix imaging using vectorial Fourier ptychography

This paper presents a microscopic imaging technique that uses variable-angle illumination to recover the complex polarimetric properties of a specimen at high resolution and over a large field-of-view. The approach extends Fourier ptychography, which is a synthetic aperture-based imaging approach to improve resolution with phaseless measurements, to additionally account for the vectorial nature of light. After images are acquired using a standard microscope outfitted with an LED illumination array and two polarizers, our vectorial Fourier ptychography (vFP) algorithm solves for the complex 2x2 Jones matrix of the anisotropic specimen of interest at each resolved spatial location. We introduce a new sequential Gauss-Newton-based solver that additionally jointly estimates and removes polarization-dependent imaging system aberrations. We demonstrate effective vFP performance by generating large-area (29 mm²), high-resolution (1.24 μm full-pitch) reconstructions of sample absorption, phase, orientation, diattenuation, and retardance for a variety of calibration samples and biological specimens.     

FPscope: a field-portable high-resolution microscope using a cellphone lens

The large consumer market has made cellphone lens modules available at low-cost and in high-quality. In a conventional cellphone camera, the lens module is used to demagnify the scene onto the image plane of the camera, where image sensor is located. In this work, we report a 3D-printed high-resolution Fourier ptychographic microscope, termed FPscope, which uses a cellphone lens in a reverse manner. In our platform, we replace the image sensor with sample specimens, and use the cellphone lens to project the magnified image to the detector. To supersede the diffraction limit of the lens module, we use an LED array to illuminate the sample from different incident angles and synthesize the acquired images using the Fourier ptychographic algorithm. As a demonstration, we use the reported platform to acquire high-resolution images of resolution target and biological specimens, with a maximum synthetic numerical aperture (NA) of 0.5. We also show that, the depth-of-focus of the reported platform is about 0.1 mm, orders of magnitude longer than that of a conventional microscope objective with a similar NA. The reported platform may enable healthcare accesses in low-resource settings. It can also be used to demonstrate the concept of computational optics for educational purposes.OCIS codes: (110.0180) Microscopy, (170.3010) Image reconstruction techniques, (170.3880) Medical and biological imaging     

Minimally invasive microendoscopy system for in vivo functional imaging of deep nuclei in the mouse brain

The ability to image neurons anywhere in the mammalian brain is a major goal of optical microscopy. Here we describe a minimally invasive microendoscopy system for studying the morphology and function of neurons at depth. Utilizing a guide cannula with an ultrathin wall, we demonstrated in vivo two-photon fluorescence imaging of deeply buried nuclei such as the striatum (2.5 mm depth), substantia nigra (4.4 mm depth) and lateral hypothalamus (5.0 mm depth) in mouse brain. We reported, for the first time, the observation of neuronal activity with subcellular resolution in the lateral hypothalamus and substantia nigra of head-fixed awake mice.OCIS codes: (110.2760) Gradient-index lenses, (170.2150) Endoscopic imaging, (170.2655) Functional monitoring and imaging, (180.0180) Microscopy, (180.2520) Fluorescence microscopy, (180.4315) Nonlinear microscopy     

0.5 gigapixel microscopy using a flatbed scanner: erratum

When one uses USAF target to calibrate the resolution of an imaging system, the periodicity of the smallest resolvable line should be used to define the limit. However, in the original paper, the line width of the resolution target was used to characterize the resolution of our microscope system, resulting in an overestimation of the performance of the imaging system. In this erratum, we correct the parts that state incorrect resolution and also re-evaluate the performance of our micoscope.OCIS codes: (120.4820) Optical systems, (170.0180) Microscopy, (170.4730) Optical pathologyIn section 4 of the original paper [1], we characterized the resolution of our imaging system using a USAF target. In the experiment, group 9 element 3 (0.78 μm line width) of the resolution target was resolved, and we stated that ‘This establishes the resolution of our prototype system under the quasi-monochromatic 530 nm illumination, as 0.78 μm over the entire FOV.’ Here, we correct the statement as follows: This establishes the resolution of our prototype system under the quasi-monochromatic 530 nm illumination as 1.56 μm over the entire FOV. We also stated that ‘the effective pixel size at the object plane should be less than 0.39 μm (0.78 μm divided by 2).’ We correct the statement as: the effective pixel size at the object plane should be less than 0.78 μm (1.56 μm divided by 2).Because of the change of the resolution, the space-bandwidth product (SBP) of the imaging system needs to be recalculated. The imaging system has a field-of-view (FOV) of 10 mm × 7.5 mm, with effective pixel size of 0.78 μm x 0.78 μm, resulting in an SBP of 0.12 gigapixel. We hereby state that the previous estimation of a 0.5 gigapixel microscopy is incorrect. Instead, we got a microscope system with 0.12 gigapixel. In the following paragraphs, we listed all the incorrect statements and correct them accordingly.In the abstract, we stated that: ‘We show that such an imaging system is capable of capturing a 10 mm × 7.5 mm FOV image with 0.78 μm resolution, resulting in more than 0.5 billion pixels across the entire image... To demonstrate its application, 0.5 gigapixel images of histology slides were acquired using this system.’ We correct the statement as: We show that such an imaging system is capable of capturing a 10 mm × 7.5 mm FOV image with 1.56 μm resolution, resulting in more than 0.12 billion pixels across the entire image... To demonstrate its application, 0.12 gigapixel images of histology slides were acquired using this system.In the second last paragraph of section 1, we stated that: ‘We show that such a system is capable of capturing a 0.5-gigapixel pixel image with a FOV of 75 mm² and a resolution of 0.78 μm. Remarkably, the CCTV lens has a SBP of at least 0.5 gigapixel (10⁹pixels), two orders of magnitude larger than conventional microscope objectives.’ We correct the statement as: We show that such a system is capable of capturing a 0.12-gigapixel pixel image with an FOV of 75 mm² and a resolution of 1.56 μm. Remarkably, the CCTV lens has an SBP of at least 0.12 gigapixel (10⁹pixels), one order of magnitude larger than conventional microscope objectives.The title of section 2 was: ‘The prototype setup of the 0.5 gigapixel microscopy imaging system’, but it should be ‘The prototype setup of the 0.12 gigapixel microscopy imaging system’.In section 6, we stated that ‘In summary, we report a wide-FOV (10 mm × 7.5 mm) microscopy system which can generate a 0.5 gigapixel image with 0.78 μm resolution across the entire FOV.’ We correct the statement as: In summary, we report a wide-FOV (10 mm × 7.5 mm) microscopy system which can generate a 0.12 gigapixel image with 1.56 μm resolution across the entire FOV. We stated that: ‘Compared to typical 10 × and 4 × objectives, our system has both superior SBP and resolution.’ We correct the statement as: Compared to typical 4 × objectives, our system has both superior SBP and resolution.We also need to relabel the position of our CCTV lens in the space-bandwidth product plot shown in Fig. 7. The relabeled coordinate is shown as follows:Open in a separate windowFig. 7The SBP-resolution summary for microscope objectives and our current CCTV lensbased system.     

Correction of depth-induced spherical aberration for deep observation using two-photon excitation fluorescence microscopy with spatial light modulator

We demonstrate fluorescence imaging with high fluorescence intensity and depth resolution in which depth-induced spherical aberration (SA) caused by refractive-index mismatch between the medium and biological sample is corrected. To reduce the impact of SA, we incorporate a spatial light modulator into a two-photon excitation fluorescence microscope. Consequently, when fluorescent beads in epoxy resin were observed with this method of SA correction, the fluorescence signal of the observed images was ∼27 times higher and extension in the direction of the optical axes was ∼6.5 times shorter at a depth of ∼890 μm. Thus, the proposed method increases the depth observable at high resolution. Further, our results show that the method improved the fluorescence intensity of images of the fluorescent beads and the structure of a biological sample.OCIS codes: (170.3880) Medical and biological imaging, (170.2520) Fluorescence microscopy, (220.1000) Aberration compensation, (230.6120) Spatial light modulators     

Multiplexed coded illumination for Fourier Ptychography with an LED array microscope

Fourier Ptychography is a new computational microscopy technique that achieves gigapixel images with both wide field of view and high resolution in both phase and amplitude. The hardware setup involves a simple replacement of the microscope’s illumination unit with a programmable LED array, allowing one to flexibly pattern illumination angles without any moving parts. In previous work, a series of low-resolution images was taken by sequentially turning on each single LED in the array, and the data were then combined to recover a bandwidth much higher than the one allowed by the original imaging system. Here, we demonstrate a multiplexed illumination strategy in which multiple randomly selected LEDs are turned on for each image. Since each LED corresponds to a different area of Fourier space, the total number of images can be significantly reduced, without sacrificing image quality. We demonstrate this method experimentally in a modified commercial microscope. Compared to sequential scanning, our multiplexed strategy achieves similar results with approximately an order of magnitude reduction in both acquisition time and data capture requirements.OCIS codes: (170.1630) Coded aperture imaging, (170.0180) Microscopy, (110.1758) Computational imaging, (100.5070) Phase retrieval, (110.3010) Image reconstruction techniques