Improved 3D image reconstruction via deep-learning-based fusion of light-field microscopy and Fourier light-field microscopy images.

Significance: Light-field microscopy (LFM) is a scanning-free 3D imaging technique that is useful for observing dynamic biological systems due to its unique capability to capture both spatial and angular information from samples in a single exposure. However, LFM suffers from the spatial-angular information trade-off associated with microlens arrays, and its spatial resolution is usually unsatisfactory for fine-structure imaging. Aim: To overcome this bottleneck, we introduce a deep-learning-based image fusion technique that combines LFM images with Fourier LFM (FLFM) images. The high spatial resolution of FLFM is combined with the dense angular acquisition capability of LFM to improve 3D image reconstruction quality. Approach: The deep learning network was trained with LFM, FLFM, and epipolar plane image data. The proposed neural network employs specialized feature extraction modules for each modality, with a U-Net backbone for 3D reconstruction, and integrates a hierarchical cascade-based result-level fusion strategy to jointly optimize multimodal features. This approach significantly enhances detail preservation and depth recovery in the final output. Results: Results obtained using a publicly available dataset of synthetic tubulins demonstrate that the proposed method outperforms state-of-the-art techniques. Quantitatively, it achieved a peak signal-to-noise ratio (PSNR) of 38.4729 and a structural similarity index measure (SSIM) of 0.9876, significantly outperforming both traditional algorithms and single-modality deep learning approaches. Furthermore, validation on a mouse brain blood vessels dataset confirms the effectiveness of the method in reconstructing biological structures, achieving a PSNR of 35.0548 and an SSIM of 0.8424. Conclusions: We introduce an approach that combines LFM with FLFM, providing an efficient and reliable solution for practical LFM applications. The deep-learning-based framework demonstrates significant potential to simultaneously accelerate imaging acquisition and enhance 3D reconstruction quality, offering further possibilities for computational microscopy.