- Research Article
- Open access
- Published:

# Structured illumination microscopy based on principal component analysis

*eLight*
**volume 3**, Article number: 4 (2023)

## Abstract

Structured illumination microscopy (SIM) is one of the powerful super-resolution modalities in bioscience with the advantages of full-field imaging and high photon efficiency. However, artifact-free super-resolution image reconstruction requires precise knowledge about the illumination parameters. The sample- and environment-dependent on-the-fly experimental parameters need to be retrieved a posteriori from the acquired data, posing a major challenge for real-time, long-term live-cell imaging, where low photobleaching, phototoxicity, and light dose are a must. In this work, we present an efficient and robust SIM algorithm based on principal component analysis (PCA-SIM). PCA-SIM is based on the observation that the ideal phasor matrix of a SIM pattern is of rank one, leading to the low complexity, precise identification of noninteger pixel wave vector and pattern phase while rejecting components that are unrelated to the parameter estimation. We demonstrate that PCA-SIM achieves non-iteratively fast, accurate (below 0.01-pixel wave vector and 0.1\(\%\) of 2\(\pi\) relative phase under typical noise level), and robust parameter estimation at low SNRs, which allows real-time super-resolution imaging of live cells in complicated experimental scenarios where other state-of-the-art methods inevitably fail. In particular, we provide the open-source MATLAB toolbox of our PCA-SIM algorithm and associated datasets. The combination of iteration-free reconstruction, robustness to noise, and limited computational complexity makes PCA-SIM a promising method for high-speed, long-term, artifact-free super-resolution imaging of live cells.

## 1 Introduction

Fluorescence microscope is an essential tool in biological sciences thanks to its ability to visualize biomolecules of interest with great specificity and high contrast through fluorescent labeling [1,2,3]. In its classical form, however, the spatial resolution of fluorescence microscope is fundamentally limited to about 200 nm by the Abbe diffraction limit [4]. The past decades have witnessed the development of super-resolution techniques, which manage to circumvent the diffraction barrier and thus enable humans to observe the nano-scale subcellular features [5,6,7,8,9,10]. Among various super-resolution techniques, structured illumination microscopy (SIM) doubles the lateral resolution of fluorescence microscope by using spatially structured illumination to modulate high-frequency information into the passband of the microscope [8]. Notwithstanding the compromised lateral resolution with respect to other fluorescence super-resolution techniques, SIM is best suited for long-term super-resolution imaging of live cells due to its unique advantages of full-field imaging, fast imaging speed, low excitation intensity, high photon efficiency, and conventional fluorescent labels compatibility [11,12,13].

Despite all the advantages, SIM still suffers from several technical challenges that hinder its practical application and widespread adoption in the biomedical community for real-time long-term live-cell imaging. Generally, high-quality and robust SIM reconstructions rely heavily on post-processing algorithms, particularly the accurate estimation of illumination pattern parameters, such as wave vector, initial phase, and modulation depth [14, 15]. Even minor parameter errors may have a strong effect on the sensitive reconstruction results, resulting in substantial reconstruction artifacts [16,17,18,19]. Moreover, these parameters are sample- and environment-dependent, so they cannot be compensated by one-time calibration, necessitating posterior restoration from the acquired experimental data unless the experimental environment is harshly maintained stable [20, 21]. Many algorithms have been proposed to estimate illumination parameters, such as phase-of-peak (POP) [20], auto-correlation [22], non-iterative auto-correlation reconstruction (ACR) [21], and image recombination transform (IRT) [23]. These methods allow the retrieval of pattern parameters from the acquired data, thereby facilitating the artifact-free reconstruction of super-resolved images. However, their accuracy is generally limited to integer pixels, and the robustness can only be secured at a relatively low noise level. Among others, the cross-correlation approach (here we describe as COR) is probably the most popular and effective technique in regard to its robust determination of wave vectors with sub-pixel accuracy [2]. With the correlation-based optimization scheme, the wave vectors and the pattern phases can be determined with very high precision. However, its performance may still be compromised under noisy, low signal-to-noise ratio (SNR) conditions, posing a major challenge for its use for long-term live-cell imaging, where low photobleaching, phototoxicity, and light dose are a must. Moreover, the sub-pixel iterative interpolation operation is computationally lengthy and time-consuming, precluding its applications for real-time imaging of dynamic samples. Although video-rate high-speed SIM imaging has been recently demonstrated, the time-consuming parameter estimation step is not involved in the real-time workflow [24, 25]. A common practice is to calibrate the illumination parameters in advance, e.g., by using the COR algorithm, and then to use the estimated parameters for the subsequent time-lapse super-resolution reconstruction. Nevertheless, calibration at start-up requires that the imaging conditions remain invariant while continuous acquisitions are performed, which cannot follow the drift in the illumination parameters over time. Consequently, such a pre-calibration strategy may be compromised or even fails in complicated or harsh operating environments where the object or optical system cannot be maintained in a steady state. As a result, accurate, robust, efficient estimation of illumination pattern parameters in SIM is still an open quest for real-time, long-term live-cell imaging.

To this end, we present an efficient and robust SIM algorithm based on principal component analysis (PCA-SIM). Based on the observation that the ideal phasor matrix of a SIM pattern is of rank one, PCA is introduced as a “dimensionality reduction” tool for precise identification of noninteger pixel wave vector and pattern phase while rejecting components that are uncorrelated to the desired, parameter-dominating “principal component”. Experiments demonstrate that PCA-SIM achieves more accurate (0.01 pixel wave vector and 0.1\(\%\) of 2\(\pi\) relative phase) parameter estimation and superior noise immunity with more efficient non-iterative efficiency than conventional cross-correlation-based methods. The advantages of high efficiency and robustness enable flexible real-time SIM imaging of live cells based on PCA-SIM under complex experimental conditions. We also developed a MATLAB toolbox with associated datasets of our PCA-SIM algorithm, offering a ready-to-use, easy-to-operate, open-source solution for potential fast, long-term, artifact-free super-resolution live cell imaging applications.

## 2 Results

### 2.1 PCA-SIM principle

To break the optical diffraction limit, SIM modulates the high-frequency information of the sample, which is originally beyond the cutoff frequency of the optical transfer function (OTF), into the microscope passband by structured illumination, usually in the form of three-step phase-shifting fringe patterns. As illustrated in Fig.1a, SIM reconstruction is essentially the reorganization of diffraction-limited spectral components in the Fourier domain, consisting of 0-order spectrum [\(C_0=O({{\textbf {k}}}){{\tilde{S}}_0}({{\textbf {k}}})\)] representing the wide-field spectral information and \({{\pm }}\)1-order spectra [\(C_{{{\pm }}1}={m}O({{\textbf {k}}}){{\tilde{S}}_{ {{\pm }} 1}}({{\textbf {k}}} {{\mp }} {{{\textbf {k}}}_{ex}}){e^{ - j{\varphi _0}}}\)] for the super-resolved spectral information modulated into OTF’s support, where \({\textbf {k}}\) represents the spatial frequency coordinates, *S* is the desired sample function, \(\sim\) denotes the Fourier transform of the corresponding variable, *O* represents the system OTF, and \({{{\textbf {k}}}_{ex}}\), \({\varphi _0}\) and *m* are the wave vector, initial phase, and modulation depth of the sinusoidal pattern, respectively (detailed in Additional file 1: Note S1). In order to demodulate the high-frequency spectral information precisely, it is essential to reliably retrieve the illumination parameters \({{{\textbf {k}}}_{ex}}\), \({\varphi _0}\), and *m*, as even slight wave vector errors (within 1 pixel) and phase errors (of the order of 6\(\%\) of 2\(\pi\)) may lead to serious reconstruction artifacts [18]. We decompose \({{{\textbf {k}}}_{ex}}\) into an integer-pixel part \({{{\textbf {k}}}_{int}}\) and a sub-pixel part \({{{\textbf {k}}}_{sub}}\), where \({{{\textbf {k}}}_{int}}\) can be conveniently determined by locating the peak of the 1-order spectrum map. After a spectral shift of \({{{\textbf {k}}}_{int}}\), \(C_{{{\pm }}1}\) can be rewritten as \({m}O({{\textbf {k}}}\mathrm{{ + }}{{{\textbf {k}}}_{{\mathop \mathrm{int}} }}){\tilde{S}_1}({{\textbf {k}}} - {{{\textbf {k}}}_{sub}}){e^{j{\varphi _0}}}\) (see Fig. 1b). After inverse Fourier transform performed on \(C_{{{\pm }}1}\), the high-frequency sample information can be expressed as:

where \({\textbf {r}}\) represents the spatial coordinates. It can be seen that the sub-pixel wave vector \({{{\textbf {k}}}_{sub}}\) is exactly reflected in the 2D phase slope of the resultant phasor. Nevertheless, noise, optical aberrations, OTF-induced signal attenuation, dysregulated modulation depth, and other experimental imperfections all result in a noisy phase signal (see Fig. 1c), which prohibits reliable determination of the sub-pixel wave vector by using traditional methods, including COR (detailed in Additional file 1: Note S2). Therefore, eliminating these irrelevant disturbances is the key to the success of robust parameter estimation for SIM under low SNR conditions.

To devise a solution to such a challenging problem, we first consider an ideal case that is free from all above-mentioned interferences, and the phasor of an ideal sinusoidal pattern (free from sample modulation) can be denoted as \({e^{j({{{\textbf {k}}}_{sub}}{{\textbf {r}}} + {\varphi _0})}}\). A close inspection of the ideal pattern phasor function reveals that it is essentially a rank-one matrix, which can be represented as the product of two vectors:

where \({{{\textbf {s}}}_x} = exp[j({{{\textbf {k}}}_{x,sub}}{{{\textbf {r}}}_x} + {\varphi _{x,0}})]\), \({{{\textbf {s}}}_y} = exp[-j({{{\textbf {k}}}_{y,sub}}\) \({{{\textbf {r}}}_y} + {\varphi _{y,0}})]\), the subscripts *x* and *y* indicate the components along the horizontal and vertical directions, and \({\{ \cdot \} ^H}\) represents the complex-conjugate transpose. Such a decomposition inspires us that the ideal pattern phasor matrix \(exp[j({{{\textbf {k}}}_{sub}}{{\textbf {r}}} + {\varphi _0})]\) should have only one *principal component*, which describes the single best subspace of the data in the least-squares sense. However, under real conditions, experimental imperfections and other disturbances will inevitably produce noisy measurements, resulting in high dimensionality of the pattern phasor matrix. Then the extraction of \({{{\textbf {k}}}_{sub}}\) and \({\varphi _{x,0}}\) from the noisy measurements is recast as finding the first principal component of the high-dimensional pattern phasor matrix, which is precisely the task accomplished by *principal component analysis* (PCA) [26]. PCA is the process of reducing data dimensionality by geometrically projecting them onto lower subspace called principal components, with the goal of finding the best summary of the data using a limited number of principal components (here only the first principal component is kept and the rest is ignored). In PCA, the principal components are eigenvectors of the data’s covariance matrix. Thus, the principal components are often computed by singular value decomposition (SVD) of the data matrix. The 2D slope, *i.e.*, the sub-pixel wave vector (\({{{\textbf {k}}}_{x,sub}}\), \({{{\textbf {k}}}_{y,sub}}\)) in \({{{\textbf {s}}}_x}\) and \({{{\textbf {s}}}_y}\), can then be independently determined from 1D unwrapped phase distribution of the left and right dominant singular vectors based on linear regression. Without reference to prior knowledge, the unwanted noise and other disturbances in the raw pattern phasor matrix are effectively “cleaned up” by PCA after dimensionality reduction, as illustrated in Fig. 1d. More details about the principle and implementation of the PCA-SIM algorithm are summarized as *Algorithm 1* in Additional file 1: Note S3.

Though PCA-SIM improves the accuracy and robustness of parameter estimation without using interpolation, PCA itself can be a time-consuming operation for relatively large matrix dimensions. In addition, real-time, long-term live-cell imaging applications require low photobleaching, phototoxicity, and therefore very limited light dose. Under such low SNR conditions, it remains a major challenge to accurately extract the eigenvectors and eigenvalue of the first principal component from highly noised data. To further accelerate PCA-SIM and enhance its noise robustness, we introduce an additional frequency-domain masking operator on the noisy 1-order spectrum before applying PCA. The masking operator can be further divided into the inner signal window (for retaining the main signal) and the external padding region (for denoising). In Additional file 1: Note S4, we derive that the Fourier spectrum of an ideal pattern phasor matrix \(exp[j({{{\textbf {k}}}_{sub}}{{\textbf {r}}} + {\varphi _0})]\) with a sub-pixel wave vector is a downsampled 2D Dirichlet function with most of its energy is concentrated in limited support around the integer part of the wave vector (see Additional file 1: Fig. S1). Therefore, a frequency domain mask is applied to the noisy 1-order spectrum to retain the high-energy spectral peak region while zeroing out the low SNR data region outside this window, as illustrated in Additional file 1: Fig. S2. Because the limited mask almost encompasses the dominant energy of the downsampled 2D Dirichlet function (concentrated within small support) while the noise energy (that spread evenly over the entire 1-order spectrum) is significantly suppressed, significant improvement in accuracy and robustness of parameter estimation can be expected, especially under low SNR conditions (see Additional file 1: Fig. S3–S5 for more details). In addition, the masking operator significantly reduces the amount of data involved in the PCA computation, and thus the computational efficiency of the whole algorithm can be greatly improved. More details about the implementation of the PCA-SIM algorithm with the application of frequency domain masking operator are summarized as *Algorithm 2* in Additional file 1: Note S5.

After \({{{\textbf {k}}}_{sub}}\) is determined, the initial phase \({\varphi _0}\) can be obtained from the phase angle of \(exp[j({{{\textbf {k}}}_{sub}}{{\textbf {r}}} + {\varphi _0})]\) at \({{\textbf {r}}} = 0\). The modulation depth *m* is then calculated by complex linear regression of \(C_0\) against the sub-pixel shifted version of \(C_{{{\pm }}1}\). Then, the 0- and \({{\pm }}\)1-order spectrums can be accurately separated and merged in the frequency domain, as illustrated in Fig. 1e. To achieve isotropic super-resolution, a similar operation is performed on two other sets of SIM images with fringe orientations rotating by \({{\pm }}\) 120 degrees (see Fig. 1e). Finally, an image deconvolution algorithm (e.g., Wiener reconstruction [8], total variance (TV) [27], or High-Fidelity (HiFi) SIM [15]) is further applied on the synthetic spectrum to compensate the effects of the OTF and spectral overlapping regions, resulting in a super-resolved image of the sample. The complete flow diagram of the PCA-SIM algorithm is summarized in Fig. 1.

### 2.2 Simulations to demonstrate the superior comprehensive performance of PCA-SIM in terms of accuracy, efficiency, and noise immunity

A set of comparative simulations were carried out to verify the effectiveness of PCA-SIM. We used a high-resolution (\(1024 \times 1024\) pixel) image of bovine pulmonary artery endothelial (BPAE) cells with rich details, sharp edges, and smooth background as the ground truth. The high-resolution image was multiplied by ideal structured illumination patterns with frequencies matching the incoherent diffraction limit of a 40\(\times\) 0.6 NA objective lens. Then, its Fourier spectrum was attenuated with a corresponding OTF to simulate the ideal SIM images acquired by a 40\(\times\) 0.6 NA microscope (note that the main results obtained via these simulations do not significantly depend on the choice of imaging parameters). Finally, Gaussian noises with different powers were artificially added to generate the raw SIM images under different SNR conditions. Figure 2a shows the performances of PCA-SIM and COR for estimating wave vectors at different noise levels. It can be seen that PCA-SIM can obtain a more precise wave vector with an accuracy better than 0.01 pixel at low noise levels. When the noise level increases to 40 dBW, COR yields serious errors larger than 1 pixel, while the errors of PCA-SIM can still be controlled within 0.2 pixels. The initial phase errors of different methods at different noise levels are illustrated in Fig. 2b, from which we can see that the accuracy of PCA-SIM can stably achieve 0.1\(\%\) of 2\(\pi\) under typical noise conditions, thanks to its reliable wave vector estimation. In contrast, the POP, ACR, and IRT methods cannot provide high-accuracy phase estimation, because their wave vector estimation is limited to integer pixels only. Figure 2c shows the wide-field images of the sample under different noise conditions and the super-resolution reconstruction obtained by PCA-SIM. Figures 2d, f illustrate the super-resolution reconstruction comparisons of different methods at three low-to-high noise levels (25 dBW, 35 dBW, and 45 dBW). When the noise is less than 40 dBW, both PCA-SIM and COR yield good results, while the results of other methods suffer certain degrees of distortions. When the noise is greater than 40 dBW, the results obtained by COR deteriorate while PCA-SIM remains visually close to the ground truth. Note that the SIM reconstruction results shown in Figs. 2d-2f are obtained by using parameters estimated from different noise data but noise-free raw SIM images for better visual comparison. It should be also noted that the time cost of PCA-SIM for parameter estimation is only 0.8119 \({{\pm }}\) 0.0011 s, processed with MATLAB R2016a using the open source code in a Dell XPS 8930 computer (Intel(R) Core(TM) i7-9700 CPU, NVIDIA GeForce GTX 1660 Ti), while that of COR is 16.2810 \({{\pm }}\) 0.0355 s. In addition, we also provide the simulation comparisons of the standard simple structure, which can more intuitively demonstrate the advancement of PCA-SIM (Additional file 1: Fig. S6). These results demonstrate that parameter estimation has a considerable impact on the fidelity of SIM reconstruction, and PCA-SIM has the best performance in terms of accuracy, efficiency, and noise immunity, which in turn provides the highest quality image reconstructions than the state-of-the-art technologies.

### 2.3 Comparative experiments to demonstrate the superior super-resolution reconstruction capability of PCA-SIM

We constructed a tri-color laser-interference SIM system (Additional file 1: Fig. S7) based on an off-the-shelf inverted fluorescence microscope (IX73, Olympus, Japan) to compare the performance of PCA-SIM with COR, POP, ACR, and IRT through practical experiments. A CV-1 in Origin Simian-7 (COS-7) cell sample (with DAPI-labeled nucleus, Alexa Fluor\(^{TM}\) 568-labeled actin and MitoTracker\(^{TM}\) Green FM-labeled mitochondria) was fixed, and the raw SIM images were acquired by our SIM system. As shown in Fig. 3, though the image resolution is greatly improved in the results of POP, ACR, and IRT, the fine details of the mitochondria and actin filament are still blurred. In contrast, these fine structures are resolved with higher contrast and fidelity by COR and PCA-SIM, benefiting from more accurate estimates of illumination parameters. Note that when the difference of wave vector estimate is within 0.1 pixel, the difference in the resultant reconstruction can hardly be distinguished visually. Further analysis of the fluorescence intensity profile distributions in the super-resolution results reconstructed by different methods reveals a slight resolution improvement of PCA-SIM over COR (Fig. 3d). The other comparative results of COS-7 cells using different objective configurations also demonstrate that PCA-SIM yields super-resolution images with better quality (Additional file 1: Fig. S8). In addition, we applied PCA-SIM to different samples collected by different SIM microscopes, e.g., BPAE cells collected by our experimental setup (Additional file 1: Fig. S9), the autofluorescent ascaris sample collected by a commercial SIM system (N-SIM, Nikon, Japan) (Additional file 1: Fig. S10), microtubules in COS-7 cells collected by another commercial SIM system (GE DeltaVision OMX, GE, USA, data available in the literature [15] (Additional file 1: Fig. S11), and f-actin in COS-7 cells acquired by a 1.7 NA total internal reflection fluorescence (TIRF) objective (data available in the literature [11], Additional file 1: Fig. S12). In all these experimental datasets, PCA-SIM outperforms other approaches and achieves high-quality super-resolution reconstruction with higher efficiency.

### 2.4 Experiments to demonstrate the noise robustness of PCA-SIM

The potential of SIM for live-cell imaging is often difficult to be fulfilled owing to photobleaching and phototoxicity that can substantially perturb the desired physiology. Reducing the excitation power can effectively mitigate the photodamage, but the resulting low SNR poses a significant challenge to the post-processing algorithm. Here, we further verify that PCA-SIM is robust to low SNRs through experiments on fixed HeLa cells with DAPI-labeled nucleus and FITC-labeled \(\alpha\)-SMA (smooth muscle actins). Considering that photobleaching would make it difficult to image the sample repeatedly hundreds of times under dense SNR conditions, we artificially added simulated Gaussian noises to the practically captured raw SIM images to produce different SNR situations. As SNR decreases, the structural distortion and contrast imbalance in the super-resolution images reconstructed with COR increase significantly (Fig. 4a–e). In contrast, these artifacts are effectively suppressed by PCA-SIM, yielding high-quality results almost visually identical to those at high SNR. Note that here we also used the noise-added low-SNR images for parameter estimation and the raw captured images for image reconstruction to better visualize the impact of the parameter estimation. To further quantify the quality of super-resolution images obtained by COR and PCA-SIM, we calculated the SSIM value of the reconstruction results at different SNRs with respect to those at high SNR (without artificial noise). In constructing SIM images with added noise from 60 dBW to 80 dBW, the super-resolution images reconstructed with COR suffer a 92\(\%\) reduction in SSIM value (medians decreased from 0.99 to 0.07), while PCA-SIM only has a mild reduction (0.97 to 0.6) (Fig. 4f). In addition to Gaussian noise, PCA-SIM also demonstrates strong noise resistance under Poisson noise conditions, and the general trend of the SSIM distributions does not differ significantly from the cases of Gaussian noise (Additional file 1: Fig. S13). In Additional file 1: Fig. S14, we generate the noisy dataset on a BPAE sample under different SNR conditions by alternatively reducing the excitation powers. The corresponding experimental results coincide with those of simulations with artificially added noise, indicating that PCA-SIM provides more robust reconstruction quality at low SNR. All these experimental results demonstrate the potential of PCA-SIM for application to long-term live-cell imaging in low-excitation conditions.

### 2.5 Real-time live-cell SIM super-resolution imaging with illumination parameters correction

The high accuracy, robustness, and efficiency of PCA-SIM open the possibility of on-the-fly super-resolution reconstruction with illumination parameters compensation. Such an instant parameter updating strategy has the potential to significantly improve the live-cell imaging performance of SIM under imperfect imaging conditions with external disturbances. To validate this, we achieved real-time PCA-SIM acquisition, reconstruction, and display based on graphics processing unit (GPU) acceleration, and used it to measure the dynamics of mitochondria of live COS-7 cells labeled by MitoTracker\(^{TM}\) Green FM. Recent studies have indicated that mitochondrial dynamic tubulation (MDT) drives the formation of mitochondrial networks, which are essential for maintaining mitochondrial function (mitochondrial DNA integrity, apoptosis, *etc*.) [28, 29]. However, MDT tubules are approximately 100 nm in diameter and are highly dynamic, posing a challenge to the spatial and temporal resolution of the imaging. In our results, the dynamic events of mitochondrial tubules are well reconstructed by PCA-SIM (Fig. 5). We observed that the MDT tubule, different from the regular thicker and more static mitochondria, extended rapidly from the mitochondria (Fig. 5a, b). When fusing with another mitochondrion and forming a membrane bridge between them, the MDT tubule thickened and became the mitochondrial network, during which the transfer of material occurred (white dashed circle in Fig. 5b). More complex dynamic tubule events are recorded in Fig. 5c, where an MDT tubule grew a new one after fusing with the left mitochondrion (4.32 s), and the new tubule then fused with another mitochondrion (9.09 s). Figures 5d–f also illustrate interesting mitochondrial dynamics, where one mitochondrion was significantly deformed due to being dragged by an MDT tubule (Fig. 5e), and an extended tubule turned nearly 90 degrees in search of a mitochondrion with which to fuse (Fig. 5f). Additional file 1: Movie S1 provides the complete dynamic SIM super-resolution reconstruction results as well as the raw SIM images for comparison. We realized a reconstruction frame rate of about 11.1 frames per second with 30 ms exposure time based on the constructed SIM system. Despite frequent switching of the region of interest and adjustment of the focus, with some consequent environmental perturbations, PCA-SIM always maintained high imaging quality thanks to its ability to extract accurate experimental parameters from the raw SIM images with low SNRs per reconstruction. In addition, dynamic imaging scenes and results of a fixed BPAE cell sample with ambient light interference and artificial perturbations also demonstrate that PCA-SIM can provide immediate, high-quality super-resolution reconstruction in a complicated environment (Additional file 1: Fig. S15, and Additional file 3: Movie S2). Compared to the COR-based parameter calibration in advance, which caused subsequent reconstruction failures when the experimental environment varies, such as operators interacting with the experiment based on the results, PCA-SIM achieved adaptive parameter estimation, thus always obtaining stable and high-quality super-resolution images (Additional file 1: Fig. S16).

## 3 Discussion and conclusion

In summary, we have developed an efficient and robust SIM parameter estimation and image reconstruction approach based on PCA (PCA-SIM). By extracting the first principal component of the pattern phasor matrix, the noise and other interference components can be effectively removed, leading to precise retrieval of noninteger pixel wave vector and pattern phase in a simple and efficient manner. Compared with state-of-the-art SIM algorithms, including the established COR, PCA-SIM has demonstrated superior performance in terms of estimation accuracy, image quality, efficiency and noise robustness. Based on PCA-SIM, we implemented real-time SIM reconstruction in complex experimental scenarios, which allowed the operator to give feedback on the experiment instantly based on the displayed image, facilitating the utilization of the SIM microscope. In addition, we provide an associated open-source and easy-to-use MATLAB toolbox to help users process, analyze, and visualize SIM data based on PCA-SIM (Additional file 4). Simulations and experimental results suggest that PCA-SIM is a promising method for real-time, long-term, high-quality live-cell imaging under low-SNR, weak-excitation conditions.

It should be emphasized that the current work is mainly focused on parameter estimation algorithm in SIM. By further increasing the hardware speed, it is expected to achieve more rapid SIM imaging. Under extremely low SNR conditions, PCA-SIM can be combined with regularization techniques, e.g., Hessian-SIM [18], to gain additional capabilities of image denoising by multi-frame averaging under a priori of spatio-temporal continuity. Finally, while only 2D SIM parameter estimation is addressed here, PCA-SIM can be easily generalized to 3D SIM by extending PCA and associated operations from 2D to 3D. Since the data volume of 3D SIM (intensity stack) is much larger than that of 2D SIM, we believe that the improvement in accuracy and speed of parameter estimation over the traditional COR method [2] brought by the dimensionality reduction feature of PCA should be more significant. We leave this generalization of 2D PCA-SIM to 3D as future work.

## 4 Methods

### 4.1 SIM setup

Our tri-color SIM setup is constructed based on an off-the-shelf inverted fluorescence microscope (IX73, Olympus, Japan), as shown in Additional file 1: Fig. S7. Three laser beams (Laser 1: OBIS LX405, Coherent, USA; Laser 2: OBIS LX561, Coherent, USA; and Laser 3: Sapphire 488LP-200, Coherent, USA) are coupled by a plane mirror (M1) and two dichromatic mirrors (DM1: ZT561dcrb, Chroma, USA; and DM2: ZT488dcrb, Chroma, USA), and then filtered, expanded and collimated by a spatial filter and a collimating lens (L1: LSB08-A 150 mm, Thorlabs, USA). After being modulated by a half-wave plate (H1: GCL-0604, Daheng Optics, China) into p(parallel)-polarized light in order to reduce the subsequent light intensity loss, the laser beam passes through a polarization beam splitter (PBS: PBS251, Thorlabs, USA) and is projected onto a ferroelectric liquid crystal spatial light modulator (SLM: QXGA-3DM, Fourth Dimension Displays, UK) displaying dense gratings. The spatial light modulator is used to generate diffraction of light, which is then modulated to s(senkrecht)-polarized light by another half-wave plate (H2: GCL-0604, Daheng Optics, China) and reflected again through the polarization beam splitter. An achromatic lens (L2: LSB08-A 250 mm, Thorlabs, USA) converges diffracted beams to its focal plane, or spectral plane, with a mask designed in advance for blocking the 0-order beam and permitting the passage of the \({{{\pm }}1}\)-order diffracted light only. A polarization rotator (PR) is usually introduced to maximize the modulation contrast of the illumination pattern. The \({{{\pm }}1}\)-order diffracted light then passes through a lens pair (L3: LSB08-A 200 mm, Thorlabs, USA; L4: LSB08-A 175 mm, Thorlabs, USA), focuses on the back focal plane of the objective lens (OBS: UPlanSApo 100\(\times\)/1.40 Oil, Olympus, Japan; UPlanXApo 60\(\times\)/1.42 Oil, Olympus, Japan), and interferes in the sample plane. The emission fluorescence is collected by the same objective, propagated through the fluorescent interference filter block (FM: U-FVN, Olympus, Japan; U-FBW, Olympus, Japan; U-FYW, Olympus, Japan) and a tube lens (TL), and finally captured by an sCMOS camera (PCO Edge 5.5, PCO, Germany) with 60\(\%\) quantum efficiency to obtain fluorescent images, whose exposure is triggered by the spatial light modulator.

### 4.2 Image processing

The raw SIM images were first slightly edge-attenuated as a precaution against edge-related artifacts [2]. Before parameter estimation, the Richardson-Lucy deconvolution was applied to remove partial Poisson noise and out-of-focus background [30, 31]. For parameter estimation, PCA-SIM removed the interference signals unrelated to the illumination parameters from the phasor matrix (\(exp[j({{{\textbf {k}}}_{sub}}{{\textbf {r}}} + {\varphi _0})]\)) by SVD and using a masking operator, thus achieving efficient and accurate experimental parameter extraction. More details about the principle of PCA-SIM are summarized in Additional file 1: Notes S3–S5. For other comparison methods, COR followed the procedure in fairSIM [19], and POP, ACR and IRT were performed with reference to the literature [20, 21, 23]. For image reconstruction, we used the reconstruction algorithm of the literature [15], which is an improved Wiener reconstruction algorithm by point-spread-function engineering. All image processing of static results was implemented using customized codes in MATLAB R2016a (except for the image labeled ‘N-SIM’ in Additional file 1: Fig. S10, which was reconstructed with NIS-Elements). All image processing of dynamic results was implemented using developed SIM software in Visual Studio 2017.

### 4.3 GPU-accelerated SIM reconstruction

Since the SIM reconstruction algorithm contains numerous pixel-wise matrix operations and (inverse) Fourier transforms, it is well-suited and easily accelerated by GPU. We take advantage of GPU to implement the reconstruction through customized kernel functions based on CUDA, which is a parallel computing architecture developed by NVIDIA. For the convenience of the hardware control, we redevelop them in our own software, where exposure adjustment and image acquisition are realized through the Software Development Kit (SDK) of the sCMOS camera, and the communication between the lasers, SLM and the computer is established through the serial interface and the corresponding commands. All software programs are integrated into Qt framework (a cross-platform software for creating graphical user interfaces) based on C++ language. The software contains one main thread and several sub-threads that run in parallel with each other. The main thread is used for the creation and visualization of the graphical user interface (GUI), the parameter setting of the hardware and the communication between the threads. In the camera thread, the raw SIM images captured by the sCMOS camera are grabbed into a ring buffer, and simultaneously the corresponding image sequence numbers are added. In the reconstruction thread, each 9 raw SIM images in the ring buffer required for super-resolution reconstruction, which can be determined by the image sequence, are copied from the host memory (CPU) to the device memory (GPU), and then processed by a series of custom kernel functions to obtain the super-resolution image. In order to improve the temporal smoothness of the display, the sliding window strategy is adopted. Except for the first reconstruction which requires 9 images in the image buffer, subsequent reconstructions can be performed with only three images (raw images in a certain direction) updated. In the display thread, the reconstructed super-resolution image is copied back to CPU and transferred to the main thread along with the first of 9 raw images for display on GUI.

### 4.4 Sample preparation

The fixed COS-7 cell sample was obtained from Smart Computational Imaging Research Institute (SCIRI) of Nanjing University of Science and Technology, and cultured in Dulbecco’s modified Eagle’s medium (DMEM) on 0.17 mm coverslips, mixed with 10\(\%\) fetal bovine serum (FBS) and 1\(\%\) 100 mM sodium pyruvate solution at 37\(^{\circ }\)C and 5\(\%\) CO\(_2\). MitoTracker\(^{TM}\) Green FM (Thermo Fisher) was used to label mitochondria in DMEM (Gibco) at 37\(^{\circ }\)C for 30 min. After fixation, permeabilization and washing, F-actin is labeled by Alexa FluorTM 568 (Thermo Fisher) at 37\(^{\circ }\)C for 45 min and nucleus was labeled by DAPI (Thermo Fisher) at 37\(^{\circ }\)C for 30 min. Then the sample is sealed with glycerol after washed by Phosphate Buffered Saline (PBS) and pure water.

The fixed BPAE cell sample was obtained from Cell Applications, *Inc.* and cultured in Bovine Endothelial Cell Growth Medium in a 37\(^{\circ }\)C, 5\(\%\) CO2 humidified incubator. MitoTracker\(^{TM}\) Red CMXRos (Thermo Fisher) was used to stain mitochondria at 37\(^{\circ }\)C for 30 min. After fixation, permeabilization and washing, F-actin was stained with Alexa Fluor\(^{TM}\) 488 (Thermo Fisher) phalloidin at 37\(^{\circ }\)C for 45 min, and the nucleus was counterstained with blue fluorescent DNA staining DAPI (Thermo Fisher) at 37\(^{\circ }\)C for 30 min.

The HeLa cells were obtained from Smart Computational Imaging Research Institute (SCIRI) of Nanjing University of Science and Technology, and cultured in Minimum Essential Medium (MEM), mixed with 10\(\%\) FBS and 1\(\%\) Penicillin/Streptomycin (P/S) at 37\(^{\circ }\)C and 5\(\%\) CO2. The slice was dropped into anti-alpha smooth muscle Actin antibody (primary antibody, Abcam) added with PBS and incubated overnight in a wet box at 4\(^{\circ }\)C, then dropped into the Goat Anti-Mouse IgG H &L (secondary antibody, Abcam) and incubated in a dark greenhouse for 50 minutes. After being placed in PBS (pH 7.4), shaking and washing on the decolorization shaking table for 3 times, the nucleus was labeled by DAPI in a dark greenhouse for 10 min.

The live COS-7 cells were incubated in H-DMEM containing 10\(\%\) FBS and 1\(\%\) penicillin-streptomycin under a humidified environment of 5\(\%\) CO2 at 37\(^{\circ }\)C. COS-7 cells were seeded at a density of 5 \(\times\) \(10^8\) cells in the confocal dishes with three parallel samples, which were cultured overnight. MitoTracker\(^{TM}\) Green FM (Beyotime, China) were diluted to 200 nM in H-DMEM, which should be pre-warmed at 37\(^{\circ }\)C. They were washed by 0.5 mL PBS for 3 min. The prepared solution was incubated for 10 min at a dose of 0.5 mL/well under dark conditions. To remove interference from free dyes, all samples were washed by PBS for 3 times.

### 4.5 Statistical analysis

Except for Fig. 4, Additional file 1: Figs. S14–S16, all the figures showed the representative data from 10-30 representative experiments. The experimental parameter errors and the structural similarity index of the reconstruction results in Figs. 2, 4, Additional file 1: Figs. S3, S4, S6 and S13 were presented as box plots (center line, average; limits, 75\(\%\) and 25\(\%\); whiskers, maximum and minimum) in graphs. The intensity profiles in Figs. 2, 3, and Additional file 1: Fig. S6 are interpolated by linear interpolation in MATLAB. The running time of parameter estimation of COR and PCA-SIM was calculated by the stopwatch timer function in MATLAB, and was expressed as mean \({{\pm }}\) SEM (standard error of the mean). Note that these times were evaluated in the case of processing data through MATLAB R2016a in a Dell XPS 8930 computer (Intel(R) Core(TM) i7-9700 CPU, NVIDIA GeForce GTX 1660 Ti) using open source code.

## Availability of data and materials

Source codes and data for PCA-SIM (Additional file 4) are provided as supplementary material to this paper online. Other source data are available from the corresponding authors on reasonable request.

## Change history

### 27 February 2023

The 'Author Comments' file is updated

## References

J.W. Lichtman, J.-A. Conchello, Fluorescence microscopy. Nat. Methods

**2**(12), 910–919 (2005)M.G. Gustafsson, L. Shao, P.M. Carlton, C.R. Wang, I.N. Golubovskaya, W.Z. Cande, D.A. Agard, J.W. Sedat, Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J.

**94**(12), 4957–4970 (2008)P. Kner, B.B. Chhun, E.R. Griffis, L. Winoto, M.G. Gustafsson, Super-resolution video microscopy of live cells by structured illumination. Nat. Methods

**6**(5), 339–342 (2009)E. Abbe, Beiträge zur theorie des mikroskops und der mikroskopischen wahrnehmung. Arch. Mikrosk. Anat.

**9**(1), 413–468 (1873)E. Betzig, G.H. Patterson, R. Sougrat, O.W. Lindwasser, S. Olenych, J.S. Bonifacino, M.W. Davidson, J. Lippincott-Schwartz, H.F. Hess, Imaging intracellular fluorescent proteins at nanometer resolution. Science

**313**(5793), 1642–1645 (2006)M.J. Rust, M. Bates, X. Zhuang, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (storm). Nat. Methods

**3**(10), 793–796 (2006)V. Westphal, S.O. Rizzoli, M.A. Lauterbach, D. Kamin, R. Jahn, S.W. Hell, Video-rate far-field optical nanoscopy dissects synaptic vesicle movement. Science

**320**(5873), 246–249 (2008)M.G. Gustafsson, Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J Microsc

**198**(2), 82–87 (2000)C. Zuo, J. Sun, J. Li, J. Zhang, A. Asundi, Q. Chen, High-resolution transport-of-intensity quantitative phase microscopy with annular illumination. Sci Rep

**7**(1), 1–22 (2017)Y. Fan, J. Li, L. Lu, J. Sun, Y. Hu, J. Zhang, Z. Li, Q. Shen, B. Wang, R. Zhang et al., Smart computational light microscopes (sclms) of smart computational imaging laboratory (scilab). PhotoniX

**2**(1), 1–64 (2021)D. Li, L. Shao, B.C. Chen, X. Zhang, M. Zhang, B. Moses, D.E. Milkie, J.R. Beach, J.A. Hammer, M. Pasham et al., Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science (2015). https://doi.org/10.1126/science.aab3500

Y. Guo, D. Li, S. Zhang, Y. Yang, J.-J. Liu, X. Wang, C. Liu, D.E. Milkie, R.P. Moore, U.S. Tulu et al., Visualizing intracellular organelle and cytoskeletal interactions at nanoscale resolution on millisecond timescales. Cell

**175**(5), 1430–1442 (2018)W. Zhao, S. Zhao, L. Li, X. Huang, S. Xing, Y. Zhang, G. Qiu, Z. Han, Y. Shang, D.-E. Sun et al., Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy. Nat. Biotechnol.

**40**(4), 606–617 (2022)A. Lal, C. Shan, P. Xi, Structured illumination microscopy image reconstruction algorithm. IEEE J. Sel. Top. Quantum. Electron.

**22**(4), 50–63 (2016)G. Wen, S. Li, L. Wang, X. Chen, Z. Sun, Y. Liang, X. Jin, Y. Xing, Y. Jiu, Y. Tang et al., High-fidelity structured illumination microscopy by point-spread-function engineering. Light Sci Appl

**10**(1), 1–12 (2021)D.P. Hoffman, E. Betzig, Tiled reconstruction improves structured illumination microscopy. BioRxiv (2020). https://doi.org/10.1101/2020.01.06.895318

J. Demmerle, C. Innocent, A.J. North, G. Ball, M. Müller, E. Miron, A. Matsuda, I.M. Dobbie, Y. Markaki, L. Schermelleh, Strategic and practical guidelines for successful structured illumination microscopy. Nat. Protoc.

**12**(5), 988–1010 (2017)X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, P. Xi et al., Fast, long-term, super-resolution imaging with hessian structured illumination microscopy. Nat. Biotechnol.

**36**(5), 451–459 (2018)M. Müller, V. Mönkemöller, S. Hennig, W. Hübner, T. Huser, Open-source image reconstruction of super-resolution structured illumination microscopy data in imagej. Nat. Commun.

**7**(1), 1–6 (2016)S.A. Shroff, J.R. Fienup, D.R. Williams, Phase-shift estimation in sinusoidally illuminated images for lateral superresolution. J. Opt. Soc. Am. A

**26**(2), 413–424 (2009)K. Wicker, Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in fourier space. Opt. Express

**21**(21), 24692–24701 (2013)K. Wicker, O. Mandula, G. Best, R. Fiolka, R. Heintzmann, Phase optimisation for structured illumination microscopy. Opt. Express

**21**(2), 2032–2049 (2013)X. Zhou, M. Lei, D. Dan, B. Yao, Y. Yang, J. Qian, G. Chen, P.R. Bianco, Image recombination transform algorithm for superresolution structured illumination microscopy. J. Biomed. Opt.

**21**(9), 096009 (2016)A. Markwirth, M. Lachetta, V. Mönkemöller, R. Heintzmann, W. Hübner, T. Huser, M. Müller, Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction. Nat. Commun.

**10**(1), 1–11 (2019)Z. Wang, T. Zhao, H. Hao, Y. Cai, K. Feng, X. Yun, Y. Liang, S. Wang, Y. Sun, P.R. Bianco et al., High-speed image reconstruction for optically sectioned, super-resolution structured illumination microscopy. Adv. Photonics

**4**(2), 026003 (2022)C. Zuo, Q. Chen, W. Qu, A. Asundi, Phase aberration compensation in digital holographic microscopy based on principal component analysis. Opt. Lett.

**38**(10), 1724–1726 (2013)K. Chu, P.J. McMillan, Z.J. Smith, J. Yin, J. Atkins, P. Goodwin, S. Wachsmann-Hogiu, S. Lane, Image reconstruction for structured-illumination microscopy with low signal level. Opt. Express

**22**(7), 8687–8702 (2014)C. Wang, W. Du, Q.P. Su, M. Zhu, P. Feng, Y. Li, Y. Zhou, N. Mi, Y. Zhu, D. Jiang et al., Dynamic tubulation of mitochondria drives mitochondrial network formation. Cell Res.

**25**(10), 1108–1120 (2015)J. Qin, Y. Guo, B. Xue, P. Shi, Y. Chen, Q.P. Su, H. Hao, S. Zhao, C. Wu, L. Yu et al., Er-mitochondria contacts promote mtdna nucleoids active transportation via mitochondrial dynamic tubulation. Nat. Commun.

**11**(1), 1–12 (2020)V. Perez, B.-J. Chang, E.H.K. Stelzer, Optimal 2d-sim reconstruction by two filtering steps with richardson-lucy deconvolution. Sci. Rep.

**6**(1), 1–11 (2016)D. Sage, L. Donati, F. Soulez, D. Fortun, G. Schmit, A. Seitz, R. Guiet, C. Vonesch, M. Unser, Deconvolutionlab2: an open-source software for deconvolution microscopy. Methods

**115**, 28–41 (2017)

## Acknowledgements

Not applicable

## Funding

This work was supported by the National Natural Science Foundation of China (61905115, 62105151, 62175109, U21B2033), Leading Technology of Jiangsu Basic Research Plan (BK20192003), Youth Foundation of Jiangsu Province (BK20190445, BK20210338), Fundamental Research Funds for the Central Universities (30920032101), and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense (JSGP202105).

## Author information

### Authors and Affiliations

### Contributions

CZ and JQ conceived the project. CZ, YL, and QC supervised the research. JQ and YC programmed the reconstruction algorithm and analyzed the data. JQ, YC, and YB built the SIM system. JQ, HW, YC, and YB performed the experiments. CZ supervised the experiments. All authors discussed the results and contributed to writing the paper. All authors read and approved the final manuscript.

### Corresponding authors

## Ethics declarations

### Ethics approval and consent to participate

Not applicable.

### Consent for publication

Not applicable.

### Competing interests

The authors declare no competing interests.

## Supplementary Information

**Additional file 1: **

**Note S1. **Basic principle of structured illumination microscopy. **Note S2.** Parameter estimation based on cross-correlation. **Note S3.** Parameter estimation based on principal component analysis. **Note S4.** Derivation of analytic expressions for the Fourier spectrum of an ideal pattern phasor. **Note S5.** Frequency domain masking operator for robust, efficient PCA-SIM. **Note S6.** Supplementary experimental results. **Figure S1.** Spectral analysis and the energy concentration property of ideal pattern phasor. **Figure S3.** The proposed masking operator, which comprises a signal window and a padding window. **Figure S2.** Wave vector estimation errors under different noise environments for fixed padding window and varying signal windows. **Figure S4.** Wave vector estimation errors under different noise environments for fixed signal window and varying padding window. **Figure S5.** The unwrapped phase angle distribution of the pattern phasor matrix after different operations in the noise environment. **Figure S6.** Comparative simulations of different methods for parameter estimation and super-resolution reconstruction. **Figure S7.** The home-built tri-color SIM schematic diagram. **Figure S8.** Comparative experiments on super-resolution results of a COS-7 cell sample with DAPI-labeled nucleus, Alexa FluorTM 568-labeled actin and MitoTrackerTM Green FM-labeled mitochondria. **Figure S9.** Comparative experiments on super-resolution results of a BPAE cell sample with DAPI-labeled nucleus, Alexa FluorTM 488-labeled actin and MitoTracker RedTM CMXRos-labeled mitochondria. **Figure S10.** Comparative experiments on super-resolution results of an autofluorescent ascaris sample. **Figure S11.** Comparative experiments on super-resolution results of Qian et al. Page 12 of 13 the microtubules in a fixed COS-7 cell. **Figure S12.** Comparative experiments on super-resolution results of the f-actin in a fixed COS-7 cell. **Figure S13.** Comparative experiments on super-resolution results of a Hela cell sample with DAPI-labeled nucleus and FITC-labeled α-SMA under different SNRs. **Figure S14.** Comparative result of different methods on mitochondria of a BPAE cell sample at different laser operating powers. **Figure S15.** On-the-fly super-resolution reconstructions of mitochondria of a fixed BPAE cell sample obtained by PCA-SIM at different time points. **Figure S16.** Super-resolution results obtained by PCA-SIM and COR of two frames in supplementary movies. **Note S7.** Description of algorithm time consumption. **Table S1.** Reconstruction times for all super-resolution results run through open source code inthe main text and supplementary material.

**Additional file 2: Movie S1.** Real-time super-resolution results of mitochondria in live COS-7 cells.

**Additional file 3: Movie S2.** Real-time super-resolution imaging scenesand results of mitochondria of a fixed BPAE cell sample.

**Additional file 4.**

Additional code and data. User guide for Additional code and data.

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Qian, J., Cao, Y., Bi, Y. *et al.* Structured illumination microscopy based on principal component analysis.
*eLight* **3**, 4 (2023). https://doi.org/10.1186/s43593-022-00035-x

Received:

Revised:

Accepted:

Published:

DOI: https://doi.org/10.1186/s43593-022-00035-x