Efficient 2D Image Reconstruction with Modified CM2Net Enhancing Fluorescence Microscopy for Large-Scale Biological Dynamics

2 CM Net: Efficient 2D Image Reconstruction with Modified Enhancing Fluorescence Microscopy for Large-Scale Biological Dynamics Introduction Methods CM2 V2[2] v Fluorescence microscopy is indispensable for studying biological structures v Conventional optics suffers from the trade-off between field of view (FOV), resolution, and miniaturization v Solution: combine miniature optics and advanced computational algorithms with Computational Miniature Mesoscope (CM2) and CM2Net for high-quality 3D reconstruction v Linear shift-variant (LSV) model1 characterizes realistic shift-variant point spread functions (PSFs) and generates large-scale fluorescent bead data v Present Modified CM2Net for accurate and efficient 2D image reconstruction Spatially-varying PSFs v 3x3 Microlens array (MLA) to acquire multiple views v Results in strong multiplexing between neighboring views v Enables large FOV CM2Net (2D) 10 mm v Data Augmentation (training) v 20 Res-Blocks per net: use normalization for variance v Random patch sampling (256x256): reduce stabilization to speed up training convergence3 computational complexity v Training was conducted for for a max of 150 epochs or 48 hours v Mixed Poisson-Gaussian noise v Hyperparameter tuning Demixing-net Ablation Study With demixing-net demix_with_boxes.pdf 2400 x 2400 Without demixing-net no_demix_with_boxes.pdf 2400 x 2400 cropped_demix_3.pdf 400 x 400 Speed: 2.675 s F1: 0.9914 Results cropped_no_demix_3.pdf 400 x 400 Speed: 2.701 s F1: 0.1131 cropped_no_demix_1.pdf 400 x 400 cropped_demix_1.pdf 400 x 400 cropped_no_demix_2.pdf 400 x 400 cropped_demix_2.pdf 400 x 400 SSIM: 0.9592 PSNR: 27.77 Test Averages (with, w/o): Speed (2.791, 2.796 s), Recall Red = false positive (in recon* but not in gt*) *recon = reconstruction (0.9702, 0.0342), Precision (0.9887, 0.2382), F1 (0.9792, Green = missed (in gt but not in recon) *gt = ground truth Yellow = true positive (in gt and recon) Note: used best model from testing 0.0584), SSIM (0.9930, 0.9393), PSNR (40.73, 27.48) Two-Sample T-test for Difference in Mean PSNR Let µ be the population mean PSNR. ⍺ = 0.05 H0: μdemix = μno_demix Assumptions: Ha: μdemix > μno_demix Mouse Brain Slice meas_brainslice_test.pdf 4000 x 3000 Conclusion: v Since Welch’s t-test p < 0.0001 < 0.05, reject H0. Sufficient statistical evidence that μdemix > μno_demix i.e. demixing-net significantly improves model reconstruction. Discussion CM2Net v 2D achieves robust, high-resolution, wide FOV 2D reconstruction v LSV model generates large-scale realistic simulated fluorescent emitter data for model training v Ablation study shows that the demixing-net module significantly improves the true positive rate, F1 score, SSIM, and PSNR by removing crosstalk artifacts between adjacent views v Trained network performs exceptionally well on various unseen samples v 2D CM2Net performs reasonably well on mouse brain slices, which has different statistics than beads Future directions: v Explore different network structures: dropout, Deformable ConvNet4 v Sparsity restrictions of simulated data must be accounted for by transfer learning on various biological samples v Enhanced 2D CM2Net can be potentially used for recording large-scale biological dynamics Ground truth Reconstruction Experimental Beads (10 µm) ü Independence of the observations ü No extreme outliers ü Normality, checking Shapiro-Wilk test and QQ plot (normal probability plot) Ø demix: p = 0.0392 < 0.05 -> check QQ plot approx. linear Ø no_demix: p = 0.0633 > 0.05 -> assume normality ü Homogeneity of variances not satisfied -> Welch's ttest Ø Levene's test: p = 0.0041 < 0.05 -> variances significantly different Measurement Simulated Beads (5-24 μm) SSIM: 0.9953 PSNR: 45.63 Dropout in Res-Blocks Composite_RGB_dropout.pdf 2400 x 2400 Speed: 2.670 s (avg 2.776 s) F1: 0.9649 (0.9360) SSIM: 0.9932 (0.9854) PSNR: 40.32 (36.28) gt_brainslice.pdf 2328 x 2204 recon_brainslice.pdf 1779 x 1119