Microtaper leaky-mode spectrometer with picometer resolution

Abstract

The wide application of optical spectroscopy makes miniaturized spectrometers with fundamental importance. The scalability, high-performance, low-cost, and small footprint are still contradicting each other and limiting the applicability of miniaturized spectrometer for practical application. Here we propose a compact spectrometer that satisfies the four advantages. The device uses a fiber taper tip to generate complex leaky mode patterns within 1 mm length. The unique correspondence between the pattern and wavelength operates effectively for hundreds of nanometers spectral range while providing a spectral resolution around ~ 1 pm. The integration of multiple taper tips enables hyperspectral imaging applications. The working range of our device can be further extended using different materials and detectors while keeping the similar architecture.

1 Introduction

The precision and elegance of obtaining abundant light-matter-interact information with a snapshot measurement makes optical spectroscopy indispensable elements for modern industries and scientific research [1]. The miniaturization of traditionally bulky spectrometers has been strongly motivated by the vast applications, including biomedical sensing [2, 3], material analysis [4], optical communication [5, 6] and light source characterization [7]. Researchers have been engineering spectrometers for lower cost, higher flexibilities, smaller size, better stabilities, and performances [8] for quite some time. But the existence of an inherent tradeoff between the above aspects are constraining this long-term theme of miniaturization from advancing. Generally, spectrometers with dispersive elements require extra spatial separations tend to leave a large footprint [9, 10]. Filters (including narrowband and reconstructive types) based designs suffer from power loss caused by either absorption or reflection as well as limited resolution and bandwidth due to limited channel numbers [11, 12]. On-chip spectrometers relying on nanofabrication usually have very low coupling efficiency for wideband operation [13, 14]. Multimode interferences could generate random speckles which are associated with spectra information [13]. However, most spectrometer designs based on it rely on random medium such as rough surface [15], multimode fiber [16, 17], integrating sphere [18], photonic crystals [19], etc., which usually require the companion of bulky or expensive equipment, such as a high-performance camera or even a microscope, to complete the measurement [13, 20]. Usually, optical fibers can generate random speckles, but the length of the fiber used are on the scale of meters. Long photonics crystal fibers or multimode fibers are vulnerable to environmental fluctuations thus weakening the stability of spectrometers based on them [21, 22]. The rest design requires structures with fancy characteristics that are difficult or expensive to fabricate [13, 23, 24]. Above all, a flexible low-cost tiny spectrometer with stable high performance is still elusive.

On the other hand, microfibers are ideal tools for manipulating light field for its tailorable dispersion and small footprint. But experiments usually use microfibers for confining light inside the fiber to propagate as long as possible. If a microfiber taper is drawn under non-adiabatic condition, the fiber geometry will force the guiding modes to evolve from single-mode to multimode and then couple to single-mode/few-mode again in a short distance. The latter process will generate leaky modes that normally being undesired for microfiber applications [20, 25,26,27,28]. But, by engineering the drawing conditions, we can maximize the leaky modes generation within a 1 mm taper region and make use of the leaked random interference speckles. The sharp angle of the taper makes it less susceptible to environmental disturbance compared with adiabatic fiber tapers. Here we demonstrate a low-cost, scalable spectrometer that has a picometer resolution and sub-millimeter footprint. The spectrometer utilizes complex leaky modes speckles projected from a curved microfiber taper tip that uniquely determines the wavelength of the input signal. By solid packaging with a complementary metal-oxide semiconductor (CMOS) imaging sensor (see Materials and Methods for details), the data acquisition of our spectrometer can be finished using a single snapshot with no external equipment needed. A light-weight vision transformer (ViT) network was used for analyzing the complex frames recorded by the CMOS image sensor (CIS). The correlation between the spectral information and leaky mode images can be easily constructed after training. In addition, this high-performance tiny device is fabricated with low-cost elements (the core components of the spectrometer cost less than $15) and could be used over long intervals while keeping the accuracy and reliability.

2 Results

Figure 1a shows the schematic of our spectrometer. The spectral signals lens-coupled into an in-line fiber polarizer (ILP) and a polarization maintaining fiber (PMF) to maintain the signal stability against coupling fluctuations and polarization drifts [20], and are then guided through a multimode fiber (MMF). The fusion spliced region between the PMF and the MMF are glued to a rigid glass substrate to ensure a stable coupling. After tapering a short part of the MMF using a homemade tapering machine, we use a three-dimensional stage to bring the taper into contact with the CIS (Sony IMX219). Ultraviolet sensitive glue was used to fix the position and gesture of the fiber taper tip (Fig. 1d, also see Additional file 1: Method for details). After fixation, the core structure of our spectrometer consists of a curved microfiber taper tip mounted on a CIS with both sides of the taper glued to the substrate became insensitive to common environmental vibrations and impacts. The reliability of our device enables us to capture incoming signals over a long time period (usually, the collection time for massive experimental data could exceed 8 h continuously).

Fig. 1

Anatomy of microtaper spectrometers. a Schematics of microtaper spectrometer. b, c Side-view and top-view of the leaky mode patterns from a microtaper shines on the substrate. The wavelength of the incident laser used here is 600 nm. d Photo of a fabricated microtaper spectrometer. e, f Scanning electron microscope images of the microtaper on the CIS. g Fiber transmission curves versus the taper parameter. The 4 fibers are tapered with elongation (x) of 2.72 mm, 3.20 mm, 4.64 mm and 7.76 mm respectively. By carefully control the shape of the microtaper, the incident power can be transported to the leaky mode pattern with near 100% efficiency. h Fiber transmission as a function of interaction length. The interaction length is the contact length of the tapered fiber waist region and the CIS. The 4 fibers are tapered with elongation (x) of 5 mm, 6 mm, 7 mm and 8 mm respectively. The power projected from the microtaper can also be tuned by changing the interaction length

The core-size-mismatch between PMF and MMF enables the excitation of multiple transverse modes. These modes will again couple to abundant transverse mode propagating inside the sharp upper taper region first [29, 30], then leaked at the predesigned lower taper region and project onto the CIS with a complex light field pattern as shown in Fig. 1b, c. The leakage only happens at certain position and orientation around the taper with sub-nanometer-scale surface roughness (Fig. 1e, f). The mode numbers of the taper, the taper length and the elongation show significant effect to the performance of the spectrometer (see Additional file 1: S9 for details). In general, a larger core diameter can help excite more modes to produce more complex pattern. Appropriately extending the taper length will also help us obtain more details of the patterns, which is expected to increase the accuracy of spectrum reconstruction. Besides, a larger elongation would produce a larger leaking area for leaky modes, thus more pixels can be used to record the spectral information, which may increase the bandwidth of the device [13]. Therefore, in the experiment, we selected a multimode fiber with a core diameter of 105 μm to excite more modes. The taper length and the curvature are fine tuned to reach a balance between small footprint and enough details of the patterns captured by the CIS. Using a homemade fiber tapering machine (Additional file 1), the size of the captured pattern can be shrunk down to an area within 300 μm * 600 μm while maintaining enough information. Furthermore, to enable a maximum detection efficiency, we also made sure the taper will leak almost all the power coupled-in by shape parameter tuning of the taper (Fig. 1g, h). The output of this ultracompact high-performance spectrometer is a complex interference pattern with ample fine structures that can be captured with a single shot within tens of milliseconds even when the power of the projected image is tens of microwatts.

Accurate reconstruction of the projected image to spectroscopy information relies heavily on the algorithms. To enable a fast and mobile friendly identification of the projected light pattern, we adopted a transformer-based model called MobileViT [31] for the image classification as shown in Fig. 2. The combination of convolutional neural networks (CNNs) and ViTs can enable a lightweight and low latency network that could be applicable for portable and field applications.

Fig. 2

Identifying the spectral information from captured leaky mode pattern images. a The transformer architecture used for spectra reconstruction. MobileViT units are used to introduce the benefits of both CNN and transformer. The 4 sets of images in the center are the activations of the first convolutional layer and the three mobileViT blocks, respectively. b Clustering of leaky mode patterns visualizes the high accuracy of the reconstructed results. Here 10 different clusters are used to show the identification results of 1000 patterns of different wavelengths. c Reconstruction of tested peak spectra. The black dashed lines indicate the ground truths, while the solid lines are the corresponding predicted spectra. The working bandwidth is limited by the performance of our silicon CIS used

The architecture of the network is shown in Fig. 2a. The images collected by the CIS is first transmitted to a 3 × 3 convolution layer (CNN) and 4 inverted residual blocks respectively to downscale the dimensions. Then the outputs are passed into 3 layers of inverted residual blocks and mobileViT blocks (see Method) alternatively. The 4 groups of images in the middle shows some examples of the outputs after passing the 3 layers respectively. As the network’s depth increases, the information inside the images is distillated gradually. We can visualize in the activation maps of our trained network (with Network Dissection; see Methods for details) and observed that, almost all the original fine features are preserved initially. But the following layers separate the main image characters into different channels, such as the overexposed area in the middle or the speckle area at the edges. As the images are processed during this process, the vision information is transformed into spectral information gradually. Finally, the data was sent to Dense layer to be activated by sigmoid function and retrieve the final reconstructed spectra.

The ViT network works excellently for image identifications which are easily capable of associating light pattern images with wavelength information. We performed a 2D multidimensional scaling (MDS) projection of the output of the last mobileViT layer. If leaky mode patterns are identified with similarities in the network, then the distance between their activation vectors will be less. Thus, the categorization process of subsequent network layers separating out the different images can be visualized clearly. Figure 2b shows the clustering of 1000 different leaky mode patterns (corresponding to different spectra varies in both intensity and wavelength) that are distinctly clustered by their spectral information extracted, which demonstrates the high accuracy of the reconstructed method.

Transfer-matrix-based reconstruction algorithms are susceptible to experimental noises in practice. But for neural networks, noises can be considered and reduced with proper training, since the training process can be fed by thousands of data that carries noise with different types and levels. Furthermore, the mobileViT network utilized here combines the advantages of both CNN and ViT. The convolution and pooling operation in CNN make the system architecturally invariant to translations thus immune to light intensity induced leaky mode pattern variation. And ViT could obtain the long-range information of the data, thus having better receptive field.

Utilizing these unique advantages, we tested the wavelength resolving ability of our spectrometer across a spectral range of 450–1000 nm (limited by the working bandwidth of our silicon CIS adopted in the experiments). As Fig. 2c shows, the spectrometer accurately recovers the positions of the tested spectra (generated by a supercontinuum light source (YSL Photonics SC-OEM) with an acousto-optic tunable filter (AOTF)) with an average peak signal-to-noise ratio (PSNR) of 46.7 dB. The reconstructed narrow-band lights from the proposed device (color solid line) and ground truth spectra from a commercial grating spectrometer (black dashed line, Ocean Optics, LEDPRO-50) shows great agreements. The center-wavelength error of the monochromatic light was approximately 0.0223%. The linewidth error is approximately 7.37%.

To evaluate the performance of our device, we further tested the bandwidth of spectra that can be identified with a single shot. Figure 3a shows some examples of continuous spectra that were reconstructed via our spectrometer. The full-width at half-maximum (FWHM) are around 30, 60, 90 nm respectively for the input signals generated by synthesizing spectrum. The black dashed lines show ground truths, which matches closely with the reconstructed spectra from our device. We find good agreement with the measured spectra at tens of nm bandwidth and qualitative agreement at near-hundred-nm bandwidth. For very broadband inputs, the superposition of the interference pattern will level out the fine structure and weaken the fine frequency reconstruction fidelity. But the coarse patterns left can still be used for the wide frequency range signal reconstruction when pre-feed with suitable training data. (See Additional file 1: Figure S5e, S5f).

Fig. 3

Characterization of microtaper spectrometers. a Reconstruction of continuous spectra with FWHM 30, 60, 90 nm. The black dashed line is ground truth. b Reconstruction of two narrow peaks with separation 1.53 pm, 6.89 pm, 18.95 pm. c Evaluation of resolution of three spectrometers made from tapers with different core diameters. The core diameter from left to right is 8.2 m$\upmu$, 62.5 m$\upmu$, 105 m$\upmu$. d Evaluation of resolution of three spectrometers made from tapers with different heater regions. The heater region length from left to right is 1.5 mm, 3.0 mm, 5.0 mm. Each column of data in the heatmap represents the estimated result of a set of measurement data (see method). The black dashed lines represent estimated resolution of each measurement data. The scale bar in c, d is shared by respective subfigures

The spectral resolution of a spectrometer can be estimated by identifying two nearby input wavelengths. We acquired a series of images while scanning a narrow linewidth laser (Matisse 2 TX-Light) from 753.01 to 753.15 nm in steps of around 0.95 pm to the trained MobileViT network. We get test speckle pattern using method in [16]. We gradually change the separation distance between wavelengths to find the reconstruction limit of our device. As shown in Fig. 3b, two peaks with wavelength difference around 1.53 pm can be identified in the NIR regime. The other cases with wavelength differences larger than that can be easily reconstructed. The recovery process still possesses high fidelity although the addition process will introduce higher readout noise and making the reconstruction process less stable, leading to possible jittering of the single peak FWHM.

The fiber taper tip parameter will also affect the performance of our spectrometer. Figure 3c shows an estimation of the resolution ability of three spectrometers with different fibers (differs in the core diameter) as the tapering fiber. The left, middle, right parts represent results for the standard single-mode fiber (core diameter = 8.2 μm) case, the standard multimode fiber (core diameter = 62.5 μm) case, the special multimode fiber (core diameter = 105 μm) case, respectively. The estimation was done by calculating spectral correlation function of speckle intensity, $C(\Delta \lambda )=\langle I(\lambda ,x)I(\lambda +\Delta \lambda ,x)\rangle /\langle I(\lambda ,x)\rangle \langle I(\lambda +\Delta \lambda ,x)\rangle -1$. The 105 μm-core-diameter multimode fiber case shows clearly better resolvability as a thicker fiber would yield more supporting modes and enriching the projected interference pattern for finer detail reconstructions. Similar trends can also be observed for fiber tapers with different shapes. As shown in Fig. 3d, three fiber taper spectrometers were fabricated using tapers with different heated region, which resulting in different tapering angles (see Additional file 1: Figure S6) [32]. From left to the right, the heated region on the same fibers is 1.5 mm, 3 mm, 5 mm. A larger taper angle will introduce more efficient mode coupling in the taper region and thus making the final pattern projected more complex and contain more information for the recovery process. On the other hand, using coupling fibers with larger diameter differences [33] or displace the fusion splice position [34] can further increase the mode numbers excited (see Additional file 1: Figure S7) to fully exploit the resolution ability of our device.

Due to the small footprint of our spectrometer, we can implement multiple fiber taper tips on one CIS and use it for hyperspectral imaging as shown in Fig. 4a. Here we use 20 tapers integrated on the CIS and arrange the fibers to be aligned in a row for scanning. As a proof-of-principle demonstration, we first use this hyperspectral imager to acquire an image for a projected target. Our target is formed by projecting the logo of Zhejiang University using a supercontinuum laser and an AOTF to generate spectra within 590–630 nm, as shown in Fig. 4b. The illuminated light spectra can be identified clearly using our device (Dashed lines and solid lines in Fig. 4c represent ground truth spectra and recovered spectra). A pseudo-colored image shows the hyperspectral imager can easily differentiate broadband light signals in an image even when they are almost identical to human eyes. We then test the applicability for real world samples. We use some color plastic sheets to test the color-resolving accuracy of our device. The actual look of the final hyperspectral image and the photo of the plastic sheet sample are not identical (Fig. 4d, e) because the spectrum below 450 nm is not recorded in our spectrometer (the AOTF we used does not work for light below 450 nm), while the photo is captured under direct sunlight. We reconstruct the actual spectra of the center point of each plastic sheets (Fig. 4f) and compares them with the results from a conventional spectrometer (Agilent Cary 5000). The results have very good agreements indicating the high accuracy of our spectrometer for real-world scenarios. The CIE color maps are also employed to compare the color consistency between the ground-truth color and our reconstruction results (Fig. 4g). The high agreements between data from our hyperspectral imager and data from the conventional spectrometer shows great potential of the ability of our design. The number of pixels available simultaneously (20 in this case) for our hyperspectral imager is limited by the pattern image size, resulting in the low-resolution of the recovered image (Fig. 4e). In practice, more pixels could be added efficiently by using more CIS with integrated tapers or a CIS with larger area. Although each spectrometer needs to be pre-calibrated before used effectively, it is realistic to construct a hyperspectral imager with thousands of “hyper-pixels” in each dimension by updating the current design.

Fig. 4

Hyperspectral imaging demonstration using our spectrometer. a 20-channel hyperspectral imager. b Reconstruction of a projected image. The RGB colors of the input and prediction are calculated from their spectra with color matching functions. A pseudo-colored image was constructed using the prediction color data to show the detail of the input image. c Spectra of the 11 colors used to assemble the image. The black dashed lines indicate the ground truths, while the solid lines are the corresponding predicted spectra. d Photo of a color film patch. e Reconstructed RGB image of d. The colors are calculated from the transmittance spectra which is predicted by our hyperspectral imager. f Spectra of the center point of each patch in e denoted by the markers. The black dashed lines are the ground truths, while the blue solid lines with markers are the corresponding predicted spectra. The purple shaded area indicates unreconstructed wavelength range (400–450 nm) due to the limitation of light source spectral range. g Color coordinates in CIE 1931 color space of the center point of each patch in e denoted by the markers

3 Discussion

Previous miniaturized spectrometers require high-precision fabrication or alignment of dispersive elements that are difficult to synthesize or maintain. However, this is not the case for the microtaper spectrometer. The fiber taper structure is very easy to fabricate [35] and can be integrated on the CIS [36] even with bare human hands. We can easily draw high quality fiber tapers with an alcohol burner (see Additional file 1: Figures S2, S3) and the integration process requires no alignment as long as the taper shed light on the CIS. This makes the microtaper spectrometer special from other congeners: A spectrometer with ultrahigh accuracy (~ 1 pm) and tiny footprint (~ 1 mm²) can be made easily at home for core components less than $15 with some simple tools. Further improvement of reconstruction quality can be achieved by using high performance image sensors (smaller pixel size, larger bit depth, lower dark current level, etc.), enlarging numbers of leaky modes and improving the device stability. Since the algorithms are trained on a computer with commercial graphic card with ease in our case, the scalability of this design is quite satisfactory. Ideally, since the silica fiber is transparent from 400 nm to almost 2000 nm, our spectrometer could be operational in this range with perfect performance with suitable CIS modules. Furthermore, if we consider the actual propagation length of the fiber taper is quite short (several centimeters long, the polarization maintaining part can be discarded when the whole device is used in a fixed scenario, including the input fiber for coupling.), the working range could be even wider (less absorption) to cover from 0.3 nm all the way to 3 $\mu$m. Such spectrometers may be applied to food inspection, drug identification, and to personalized health diagnostics with very low cost. Besides, if we consider polarization state in our calibration procedure, we could acquire both the spectrum and polarization state of unknown light.

Some issues need to be considered to further improve the performance of our spectrometer. For a broadband light input, speckle overlap will result in a decrease in image contrast, which leads to a trade-off between bandwidth and resolution of the spectrometer. Our spectrometer exhibits good results when analyzing coherent light. For incoherent light with different wavelengths, different mode distributions can be used to reconstruct the spectrum with a lower resolution, since reduced coherence will decrease the pattern complexity. In addition, although the spectrometer is easy to fabricate for bare hands, more efforts in the automation of the manufacture could lead to mass production of the spectrometers.

4 Methods

4.1 Micro taper spectrometer fabrication

The microfiber tapers are fabricated with a home-made fiber drawing machine (see Additional file 1: Figure S1). A commercial optical fiber was cleaved and put inside a 5-mm-long ceramic tube. The tube was heated to 1200℃ using a CO₂ laser. The two ends of the fiber were pulled by two linear servo motor stages (Yaskawa SGLF Series) fixed on a sliding rail. The flame-brushing technique was used to control the movement of the stages with the position of the ceramic tube fixed.

After tapering, the micro taper sample is fixed on a micro positioner. A CIS (Raspberry Pi Camera Module 2, Sony IM219) module is fixed directly under the taper after removing the lens and cover glasses. The micro positioner is adjusted slowly and gradually to move the taper into close contact with the CIS. A 650 nm laser was coupled into the taper to assist the process as an indicator for the contact condition. The taper is fixed with UV sensitive glue after the projected patterns observed from the CIS is optimized. The tilting angle of the taper is also finely tuned to make sure the leaking efficiency of the taper. Finally, we cover the device with a cover glass and seal the device with UV glue to ensure an air-tight capsulation.

The hyperspectral imaging sample was fabricated with very similar method. We fix two wedge structures on the substrate to elevate all the 20 fiber tapers. The relative position and direction of the 20 tapers are adjusted individually during the contact process mentioned above and fixed using UV glues to make sure the maximized utilization of the CIS chip area while preventing crosstalk of neighboring microtaper spectrometers.

4.2 MobileViT network training

We use a supercontinuum source and an AOTF to obtain monochromatic spectral signals at the wavelength range of 450–1000 nm of different intensities. The lasers are separated using a beam splitter before reaching the coupling lens of our spectrometer to be monitored by a conventional spectrometer (Ocean Optics LEDPRO-50) at the same time so that the spectrum of each signal is obtained. The leaky mode patterns corresponding to the signals were captured with the CIS respectively as training sets. We obtain 10-bits raw data from the CIS directly and crop the region of interest (ROI) with the size of 640 × 1400. Our dataset consists of images corresponding to random spectra that can be obtained by randomly synthesizing images of different intensities of different wavelengths. In order to obtain better training result, our training set usually has 10,000 images. We train our MobileViT network on a server with a NVIDIA 3090Ti graphic card. All the images are normalized to a 0 to 1 scale. The exposure time, gain of the CIS are kept constant during all the data collecting process. For the spectra reconstruction tasks, the Dense output layer is activated by the sigmoid function. The RMSprop method is used to optimize the MSE loss. The learning rate exponentially decays over time with the initial value of 5 × 10^–4. For the spectra classification tasks, the output layer is activated by the softmax function. The Adam method is used to optimize the categorical crossentropy with 0.1 label smoothing.

Inside the MobileViT, the input feature map is first convoluted with 3 × 3 kernel to learn the local representations by image-specific inductive biases of CNN, then projected to a high-dimensional space by the point-wise convolution. The output is further unfolded into a series of non-overlapping 2 × 2 patches, then fed to the L transformers. The self-attention-based transformers encode the inter-patch relationships to learn the global representations. After the extractions of local and long-range information, the output is folded back and fused with the original input. We train our MobileViT network on a server with a NVIDIA 3090Ti graphic card.

The clustering process shows the inherent connections between the leaky mode patterns. Since the patterns are correlated with the wavelength information, patterns associated with similar spectra will be more resembling. We divide the spectral range into multiple parts (In Fig. 2b we used 10 parts). And adjust the output of our network to classify the data. The activation function is switched to softmax and optimized with Adam optimizer. The distance between two different patterns is generated by calculating the pairwise distance between the activations of the two patterns. After we obtain the distance matrix, we use MDS to project the matrix into a two-dimensional distribution.

4.3 Resolution test

We use Matisse 2 TX-Light tunable laser to accomplish resolution test. The laser is coupled into a standard multimode fiber using a fiber collimator (THORLABS F810APC-850). After passing through the multimode fiber, the light is propagated to the standard single mode fiber or SMA fiber to input the light to the spectrometer. The fibers behind the in-line polarizer are fixed to stabilize the spatial polarization of the light. During the test, we tune the laser to change the wavelength gradually, at the same time, the Raspberry Pi stores the captured pictures. This data collection lasts for about 280 s and we can get about 2200 raw pictures. Matching the time of pictures captured with the recorded data of the wavelength meter, we can relate the wavelength of laser with its correspondence pictures captured. The estimated minimum resolution can be obtained by the spectral correlation function.

After acquiring a series of images to the trained MobileViT network, we characterize the spectral resolution of our microtaper spectrometer by testing its ability to distinguish two peaks ultra-narrow spectra. The pattern image for two peak ultra-narrow spectra were constructed by summing the images of the desired wavelengths sequentially.

4.4 Hyperspectral imaging data acquisition

The ZJU-logo images are projected using a supercontinuum source and an AOTF. The lasers are separated using a beam splitter before reaching the coupling lens of our spectrometer to be monitored by a conventional spectrometer (Ocean Optics LEDPRO-50) at the same time. The arbitrary colors of the ZJU logo are generated by combining the output of the 8 channels of the AOTF. All the spectrometers on the hyperspectral imager are pre-trained before the data collection using the method described above. The plastic sheet samples are also tested with the same equipment.