Abstract
Background
Superresolution optical fluctuation imaging (SOFI) achieves 3D superresolution by computing temporal cumulants or spatiotemporal crosscumulants of stochastically blinking fluorophores. In contrast to localization microscopy, SOFI is compatible with weakly emitting fluorophores and a wide range of blinking conditions. The main drawback of SOFI is the nonlinear response to brightness and blinking heterogeneities in the sample, which limits the use of higher cumulant orders for improving the resolution.
Balanced superresolution optical fluctuation imaging (bSOFI) analyses several cumulant orders for extracting molecular parameter maps, such as molecular state lifetimes, concentration and brightness distributions of fluorophores within biological samples. Moreover, the estimated blinking statistics are used to balance the image contrast, i.e. linearize the brightness and blinking response and to obtain a resolution improving linearly with the cumulant order.
Results
Using a widefield totalinternalreflection (TIR) fluorescence microscope, we acquired image sequences of fluorescently labelled microtubules in fixed HeLa cells. We demonstrate an up to fivefold resolution improvement as compared to the diffractionlimited image, despite low singleframe signaltonoise ratios. Due to the TIR illumination, the intensity profile in the sample decreases exponentially along the optical axis, which is reported by the estimated spatial distributions of the molecular brightness as well as the blinking onratio. Therefore, TIRbSOFI also encodes depth information through these parameter maps.
Conclusions
bSOFI is an extended version of SOFI that cancels the nonlinear response to brightness and blinking heterogeneities. The obtained balanced image contrast significantly enhances the visual perception of superresolution based on higherorder cumulants and thereby facilitates the access to higher resolutions. Furthermore, bSOFI provides microenvironmentrelated molecular parameter maps and paves the way for functional superresolution microscopy based on stochastic switching.
Keywords:
Fluorescence microscopy; Superresolution; Stochastic switching; Sofi; Cumulants; Balanced contrast; molecular statistics; Functional imagingBackground
The spatial resolution in classical optical microscopes is limited by diffraction to about half the wavelength of light. During the last two decades, several superresolution concepts have been developed. Based on the onoffswitching of fluorescent probes, these concepts overcome the diffraction limit by up to an order of magnitude (Huang et al. 2009). A straightforward method consists of digitally postprocessing an image sequence of stochastically blinking emitters acquired with a standard widefield fluorescence microscope. Densely packed single fluorophores can be distinguished in time by using highprecision localization algorithms, used for instance in photoactivation localization microscopy (PALM) (Betzig et al. 2006; Hess et al. 2006) and stochastic optical reconstruction microscopy (STORM) (Heilemann et al. 2008; Rust et al. 2006), or by analysing the statistics of the temporal fluctuations as exploited in superresolution optical fluctuation imaging (SOFI) (Dertinger et al. 2009; Dertinger et al. 2010). SOFI is based on a pixelwise auto or crosscumulant analysis, which yields a resolution enhancement growing with the cumulant order in all three dimensions (Dertinger et al. 2009). Uncorrelated noise, stationary background, as well as outoffocus light are greatly reduced by the cumulants analysis. While PALM and STORM are commonly based on a framebyframe analysis of images of individual fluorophores, SOFI processes the entire image sequence at once and therefore presents significant advantages in terms of the number of required photons per fluorophore and image, as well as in acquisition time (Geissbuehler et al. 2011). Localization microscopy requires a metastable dark state for imaging individual fluorophores (van de Linde et al. 2010). In contrast, SOFI relies solely on stochastic, reversible and temporally resolvable fluorescence fluctuations almost regardless of the onoff duty cycle (Geissbuehler et al. 2011). The main drawback of SOFI is the amplification of heterogeneities in molecular brightness and blinking statistics which limits the use of higherorder cumulants and therefore resolution. In this article, we revisited the original SOFI concept and propose a reformulation called balanced superresolution optical fluctuation imaging (bSOFI), which in addition to improving structural details opens a new functional dimension to stochastic switchingbased superresolution imaging. bSOFI allows the extraction of the superresolved spatial distribution of molecular statistics, such as the ontime ratio, the brightness and the concentration of fluorophores by combining several cumulant orders. Moreover, this information can be used to balance the image contrast in order to compensate for the nonlinear brightness and blinking response of conventional SOFI images. Consequently, bSOFI enables higherorder cumulants to be used and thereby achieves higher resolutions.
Methods
Theory and algorithm
SOFI is based on the computation of temporal cumulants or spatiotemporal crosscumulants. Cumulants are a statistical measure related to moments. Because cumulants are additive, the cumulant of a sum of independently fluctuating fluorophores corresponds to the sum of the cumulant of each individual fluorophore. This leads to a pointspread function raised to the power of the cumulant order n and therefore a resolution improvement of , respectively almost n with subsequent Fourier filtering (Dertinger et al. 2010). So far, SOFI has been used exclusively to improve structural details in an image (Dertinger et al. 2009; Dertinger et al. 2010). Information about the ontime ratio, the molecular brightness and the concentration has to our knowledge never been exploited before and therefore represents a new potential for superresolved imaging.
In the most general sense, the cumulant of order n of a pixel set with time lags can be calculated as (Leonov and Shiryaev 1959)
where stands for averaging over the time t. P runs over all partitions of a set , which means all possible divisions of into nonoverlapping and nonempty subsets or parts that cover all elements of . P denotes the number of parts of partition P and p enumerates these parts. is the intensity distribution measured over time on a detector pixel . We used the crosscumulant approach without repetitions to increase the pixel grid density and eliminate any bias arising from noise contributions in autocumulants (Geissbuehler et al. 2011). A 4x4 neighborhood around every pixel was considered to compute all possible npixel combinations excluding pixel repetitions. For computational reasons, we kept only a single combination featuring the shortest sum of distances with respect to the corresponding output pixel . For even better signaltonoise ratios, it would be beneficial to average over multiple combinations per output pixel. The heterogeneity in output pixel weighting arising from the different pixel combinations has been accounted for by the distance factor as described in (Dertinger et al. 2010).
Considering a sample composed of M independently fluctuating fluorophores and assuming a simple twostate blinking model (with characteristic lifetimes τ_{on},τ_{off}) with slowly varying molecular parameters compared to the size of the pointspread function (PSF), the cumulant of order n without timelags can be interpreted as
with the spatial distribution of the molecular brightness and the ontime ratio. is the system’s PSF and is the nth order cumulant of a Bernoulli distribution with probability ρ_{on}:
Assuming a uniform spatial distribution of molecules inside a detection volume V centered at , we may further approximate
where is the expectation value of or the nth moment of (see (Kask et al. 1997) for some examples) and denotes the number of molecules within the detection volume V . Finally, we can write
Based on at least three different cumulant orders and approximation (5), it is possible to extract the molecular parameter maps , and by solving an equation system, or by using a fitting procedure. For example, the cumulant orders two to four can be used to build the ratios
and to solve for the molecular brightness
the ontime ratio
and the molecular density
The spatial resolution of the estimation is limited by the lowest order cumulant, i.e. the second order in this case. However, the presented solution is not unique. Basically any three distinct cumulant orders could have provided a solution. Furthermore, the method is not limited to a twostate system; it can be extended to more states as long as the differences in fluorescence emission are detectable. Additional details on the analytical developments as well as a theoretical investigation of the estimation accuracy of the different parameters under different conditions are given in the Additional file 1.
Additional file
Addtional file 1. Additional details on the development of the theory, the algorithm, sample preparation protocols and a theoretical investigation of parameter estimation accuracies.
Format: PDF Size: 844KB Download file
This file can be viewed with: Adobe Acrobat Reader
In order to correct for the amplified brightness and blinking heterogeneities without compromising the resolution, the cumulants have to be deconvolved first. For this purpose, we used a LucyRichardson algorithm (Lucy 1974; Richardson 1972) implemented in MATLAB (deconvlucy, The MathWorks, Inc.), which is an iterative deconvolution without regularization that computes the most likely object representation given an image with a known PSF and assuming Poisson distributed noise. Apart from the estimate of the cumulant PSF and the specification of a maximum of 100 iterations, all arguments have been left at their default values. Assuming a perfect deconvolution without regularization, the result could be interpreted as
Taking then the nth root linearizes the brightness response without cancelling the resolution improvement of the cumulant. To reduce the amplified noise and masking residual deconvolution artefacts, small values (typically 15% of the maximum value) are truncated and the image is reconvolved with . This leads to a final resolution improvement of almost nfold compared to the diffractionlimited image, which is physically reasonable since the frequency support of the cumulantequivalent optical transfer function (OTF) is ntimes the support of the system’s OTF (cf. (Dertinger et al. 2010)). In contrast to Fourier reweighting (Dertinger et al. 2010), which is equivalent to a Wiener filter deconvolution and reconvolution with , we explicitly split these two steps and use an improved but computationally more expensive deconvolution algorithm that is appropriate for the subsequent linearization.
Since the cumulants are proportional to , which contains n roots for , there might still be hidden image features in these brightnesslinearized cumulants (result after deconvolution, nth root and reconvolution with ). However, using the onratio map , it is straightforward to identify the structural gaps around the roots of f_{n} and fill them in with the brightnesslinearized cumulant of order n1. To this end, the locations where f_{n} approaches zero are translated into a weighting mask with smoothed edges around these locations. The thresholds have been defined by computing the crossing points of and . This mask is then applied on the nth order brightnesslinearized cumulant and its negation is applied on order n1 (see Additional file 1 for further details). The result is a balanced cumulant image. It should be noted that the cancellation of by division is possible but not recommended, because it amplifies noisy structures in the vicinity of the roots of f_{n}. The combination of multiple cumulant orders in a balanced cumulant image results in a better overall image quality. However, it is also possible that the onratio varies only slightly throughout the field of view, such that a combination of multiple cumulant orders is not necessary. Figure 1 illustrates the different steps of the algorithm based on a simulation of randomly blinking fluorophores, arranged in a grid of increasing density from right to left, increasing brightness from left to right and increasing ontime ratio from top to bottom.
Figure 1. bSOFI algorithm. Flowchart to illustrate the different steps of the bSOFI algorithm. (a) Raw data. (b) Crosscumulant computation up to order n according to equation (1) without time lags. (c) Cumulant ratios K_{1} and K_{2} according to equation (6). (d) Deconvolved cumulant of order n. (e) Solution for the spatial distribution of the molecular brightness ε, ontime ratio ρ_{on}and density N using equations (79). (f) Balanced cumulant of order n, obtained by computing the nth root of the deconvolved cumulant, reconvolving with and filling in the structural gaps around the roots of f _{n} with a lowerorder cumulant. (g) Colorcoded molecular parameter maps overlaid with a balanced cumulant as a transparency mask.
Experiments
In order to verify the concept experimentally, we used a customdesigned objectivetype total internal reflection (TIR) fluorescence microscope with a highNA oilimmersion objective (Olympus, APON 60XOTIRFM, NA 1.49, used at 100x magnification), blue (490nm, 8mW, epiillumination) and red (632nm, 30mW, TIR illumination) laser excitation and an EMCCD detector (Andor Luca S). We imaged fixed HeLa cells with Alexa647labelled alpha tubulin and used a chemical buffer containing cysteamine and an oxygenscavenging system (Heilemann et al. 2008) (see Additional file 1 for further details) to generate reversible blinking and an increased bleaching stability. The blue laser excitation was used to accelerate the reactivation of dark fluorophores and to reduce the acquisition time. For data processing, 5000 images acquired at 69 frames per second (fps) were divided into 10 subsequences significantly shorter than the bleaching lifetime to avoid correlated dynamics among the fluorophores (Dertinger et al. 2010). The final bSOFI images are obtained by averaging over the processed subsequences.
Results and discussion
Figure 2 compares the performance of bSOFI with conventional SOFI and widefield fluorescence microscopy. The peak signaltonoise ratio (pSNR; noise measured on the background) in a single frame was 2023dB for the brightest molecules, which is rather low for performing localization microscopy, but more than sufficient for SOFI (Geissbuehler et al. 2011). Additionally, we observed significant readout noise at this acquisition speed (fixedpattern noise in the average image, Figure 2a,i), which was effectively removed in the crosscumulants analysis (Figure 2be and j,k). The estimated molecular ontime ratio (c,k), brightness (d) and density (e) are shown overlaid with the 5th order balanced cumulant as a transparency mask. Due to the overemphasis of slight heterogeneities in molecular brightness and blinking, the dynamic range of the conventional 5th order SOFI image (b and j), where values above 1% of the maximum are truncated, is too high to be represented meaningfully. Figures 2fh are the projected profiles of the widefield, SOFI, Fourier reweighted SOFI (Dertinger et al. 2010) and bSOFI images along the cuts 11’, 22’ and 33’, respectively. The second profile, with two microtubule structures separated by 80nm, illustrates a situation close to the Rayleigh criteria in the bSOFI case. This is consistent with the measured full width at half maximum (FWHM) of 78nm (Figure 2h). Although the Fourier reweighted SOFI features a FWHM of 75nm (Figure 2h), it does not resolve the two microtubules at 22’. Due to the nonlinear brightness response only a single one is visible (Figure 2g). When considering the effective width of the microtubule of 22nm as well as a 15nm linker length, this translates into a bSOFIequivalent PSF with 64nm FWHM, respectively a 4.6fold resolution improvement compared to widefield microscopy. The resolution improvement of the conventional 5th order SOFI image is close to the expected factor . With the red TIR illumination, the excitation intensity decreases exponentially along the optical axis. Assuming a homogeneous illumination in the xy plane, both the molecular brightness and the ontime ratio can be interpreted as a depth encoding because they are related to the illumination intensity (van de Linde et al. 2011). Obviously, a depth encoding based on molecular parameters can only be used as a qualitative impression of depth information rather than real 3D imaging, because it does not resolve additional structures in depth. Moreover, when looking at smaller scales (Figure 2k), the depth impression of colorcoded molecular parameters gets less evident, which can be explained by the influence of local differences in the chemical microenvironment or by the stochastic nature of individual fluorophores.
Figure 2. bSOFI demonstration. Experimental demonstration of bSOFI on fixed HeLa cells with Alexa647labelled microtubules. (a) Summed TIRF microscopy image [Widefield]. (b) Conventional 5th order crosscumulant SOFI [SOFI5]. (ce) Colorcoded molecular ontime ratio, brightness and density overlaid with the 5th order balanced cumulant [BC5]. (fh) Profiles along the cuts 11’, 22’ and 33’. In yellow we added the corresponding profiles when Fourier reweighting (cf. (Dertinger et al. 2010)) with a damping factor of 5% is applied on the 5th order crosscumulant SOFI image. (ik) Magnified views from the white insets highlighted in (ac). Scale bars:2μm(ae); 500nm (ik).
Although the used LucyRichardson deconvolution performed well on our measurements, it is not specifically adapted for cumulant images, because it assumes a Poissondistributed noise model and an underlying signal that is strictly positive. For the nth order cumulant, the signal on a single image can vary between positive and negative values according to the underlying onratios. Furthermore, initially Poissondistributed noise leads to a modified noise distribution in the cumulant image. In our experiments, the local onratio variations were small, which proves to be unproblematic for a deconvolution with a positivity constraint, when the negative and the positive parts are considered separately. However, a deconvolution algorithm specifically adapted for cumulant images using an appropriate noise model may improve the results of balanced cumulants in the future.
If the cumulants are computed for different sets of time lags and the acquisition rate oversamples the blinking rate, it is also possible to extract absolute estimates on the characteristic lifetimes of the different states. The temporal extent of the curve obtained by computing the secondorder crosscumulant as a function of time lag τ(corresponding to a centred secondorder crosscorrelation) before it approaches zero yields an estimate on the blinking period, provided that the timeframe of the measurement includes many blinking periods. In our case however, with only 10 to 20 blinking periods within a measurement window of 500 frames (@69fps) and a low onratio, the temporal extent of the correlation curve rather corresponds to the characteristic ontime. Figure 3a,b show the resulting on and offtime maps overlaid with a 5th order balanced cumulant as a transparency mask. The reported ontimes correspond to the time position where the curve decreased to e^{1} of the value at zero time lag. The offtime map is obtained by calculating . The offtime map hardly shows a dependency on the illumination intensity, which is in line with the deep penetration into the sample of the blue activation light. In the present case, the lifetime of the offstate is influenced mainly by the chemical composition of the microenvironment surrounding the probe (van de Linde et al. 2011).
Figure 3. On and offtimes. Spatial distribution of the estimated on (a) and offtimes (b) overlaid with a 5th order balanced cumulant as a transparency mask. The images correspond to an average over 10 subsequences of 500 frames each. (c) Secondorder crosscumulant with different time lags, averaged over the xyplane and 10 subsequences of 500 frames each. An exponential fit to the measured curve is shown in black. Scale bars:2μm.
For estimating the average ontime, we computed the secondorder crosscumulant as a function of time lag and averaged it over the xyplane and 10 subsequences of 500 frames (Figure 3c). The fitted exponential curve has a characteristic time constant of .
Conclusions
bSOFI is an extended version of SOFI and shares its advantages of simplicity, speed, rejection of noise and background, and compatibility with various blinking conditions. Since the bSOFIPSF shrinks in all three dimensions with increasing cumulant orders, bSOFI can easily be extended to the axial dimension by acquiring multiple depth planes and performing the analysis in three dimensions. In contrast to SOFI, the bSOFI response to brightness and blinking heterogeneities in the sample is nearly linear, which allows higher resolutions to be obtained by computing higher cumulant orders. The additional information on the spatial distribution of molecular statistics may be used to monitor static differences and/or dynamic changes of the probesurrounding microenvironments within cells and thus may enable functional superresolution imaging with minimum equipment requirements.
Competing interests
The authors declare that they have no competing interests.
Author’s contributions
SG and CD developed the theory and the algorithm, SG, NB, ML and TL conceived the study, NB and CB prepared the samples, SG and NB performed the experiments and analyzed data, ML and TL supervised the project and SG wrote the manuscript. All authors discussed the results and implications and commented on the manuscript at all stages.
Acknowledgements
This research was supported by the Swiss National Science Foundation (SNSF) under grants CRSII3125463/1 and 205321138305/1. The authors would like to thank Prof. Anne GrapinBotton for the provided infrastructures used for the preparation of the samples and acknowledge Arno Bouwens, Dr. Matthias Geissbuehler, and Dr. Erica MartinWilliams for their constructive comments on the manuscript.
References

Betzig E, Patterson G, Sougrat R, Lindwasser O, Olenych S, Bonifacino J, Davidson M, LippincottSchwartz J, Hess H (2006) Imaging Intracellular Fluorescent Proteins at Nanometer Resolution. Science 313(5793):1642 PubMed Abstract  Publisher Full Text

Dertinger T, Colyer R, Iyer G, Weiss S, Enderlein J (2009) Fast, backgroundfree, 3D superresolution optical fluctuation imaging (SOFI). Proc Nat Acad Sci USA 106(52):2228722292 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Dertinger T, Colyer R, Vogel R, Enderlein J, Weiss S (2010) Achieving increased resolution and more pixels with Superresolution Optical Fluctuation Imaging (SOFI). Opt Express 18(18):1887518885 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Dertinger T, Heilemann M, Vogel R, Sauer M, Weiss S (2010) Superresolution optical fluctuation imaging with organic dyes. Angew Chem Int Ed 49(49):94419443 Publisher Full Text

Geissbuehler S, Dellagiacoma C, Lasser T (2011) Comparison between SOFI and STORM. Biomed Opt Express 2(3):408420 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Heilemann M, Van De Linde S, Schüttpelz M, Kasper R, Seefeldt B, Mukherjee A, Tinnefeld P, Sauer M (2008) Subdiffractionresolution fluorescence imaging with conventional fluorescent probes. Angew Chem Int Ed 47(33):61726176 Publisher Full Text

Hess S, Girirajan T, Mason M (2006) Ultrahigh resolution imaging by fluorescence photoactivation localization microscopy. Biophys J 91(11):42584272 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Huang B, Bates M, Zhuang X (2009) SuperResolution Fluorescence Microscopy. Annu Rev Biochem 78:9931016 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Kask P, Gunther R, Axhausen P (1997) Statistical accuracy in fluorescence fluctuation experiments. Eur Biophys J Biophys Lett 25(3):163169 Publisher Full Text

Leonov V, Shiryaev A (1959) On a Method of Calculation of SemiInvariants. Theory Probability App 4(3):319329 Publisher Full Text

Lucy L (1974) Iterative Technique for Rectification of Observed Distributions. Astron J 79(6):745754

Richardson W (1972) BayesianBased Iterative Method of Image Restoration. J Opt Soc Am 62:55 Publisher Full Text

Rust M, Bates M, Zhuang X (2006) Subdiffractionlimit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 3(10):793795 PubMed Abstract  Publisher Full Text  PubMed Central Full Text

van de Linde S, Löschberger A, Klein T, Heidbreder M, Wolter S, Heilemann M, Sauer M (2011) Direct stochastic optical reconstruction microscopy with standard fluorescent probes. Nat Protoc 6(7):991 PubMed Abstract  Publisher Full Text

van de Linde S, Wolter S, Heilemann M, Sauer M (2010) The effect of photoswitching kinetics and labeling densities on superresolution fluorescence imaging. J Biotechnol 149(4):260266 PubMed Abstract  Publisher Full Text