One of Superman's most famous abilities is his supervision, which allowed him to see atoms, see in the dark and over great distances, and see through objects. This ability is extremely rarely shown on the screens, but it is. In our reality, it is also possible to see through almost completely opaque objects by applying some scientific tricks. However, the resulting images were always black and white, until recently. Today we will get acquainted with a study in which scientists from Duke University (USA) were able to take a color picture of objects hidden behind an opaque wall using a single light exposure. What is this super technology, how does it work and in what areas can it be applied? The report of the research group will tell us about this. Go.
Research basis
Despite all the possible "goodies" of the technology for visualizing objects in scattering media, there are a number of problems in the implementation of this technology. The main one is the fact that the paths of photons passing through the scatterer vary greatly, which leads to random patterns. speckle* on the other side.
Speckle* is a random interference pattern formed by the mutual interference of coherent waves that have random phase shifts and / or a random set of intensities. Most often it looks like a set of light spots (dots) on a dark background.
In recent years, several imaging techniques have been developed to bypass scatter effects and extract object information from the speckle pattern. The problem with these techniques is their limitations - you need to have certain knowledge about the object, have access to a scattering medium or object, etc.
At the same time, according to scientists, there is a much more perfect method - visualization with the memory effect (ME). This method allows you to visualize an object without prior knowledge about itself or the scattering medium. Everyone has flaws, as we know, and the ME method is no exception. To obtain high-contrast speckle patterns and, accordingly, more accurate images, the illumination should be narrow-band, i.e. less than 1 nm.
It is also possible to outwit the limitations of the ME method, but, again, these tricks are related to accessing the optical source or object before the scatterer, or with direct measurement PS*.
PS* is a point spread function that describes the image that the formation system receives when observing a point source of light or a point object.
Researchers call these methods working, but not perfect, since PSF measurement is not always possible due, for example, to the scatterer's dynamism or its inaccessibility before the imaging procedure. In other words, there is work to be done.
In their work, the researchers propose a different approach. They show us a method for realizing multispectral imaging of objects through a scattering medium using a single speckle measurement with a monochrome camera. Unlike other techniques, this one does not require prior knowledge of the PSF system or source spectrum.
The new method makes it possible to create high-quality images of the target object in five well-separated spectral bands between 450 nm and 750 nm, which was confirmed by calculations. In practice, it has so far been possible to visualize three well-separated spectral channels between 450 nm and 650 nm and six adjacent spectral channels between 515 and 575 nm.
How the new method works
Image #1: lamp - spatial light modulator - diffuser (with iris) - coding aperture - prism - optical relay (1:1 visualization) - monochrome camera.
Researchers note the three main elements of any diffuser imaging: the object of interest (either lit from the outside or self-luminous), the diffuser, and the detector.
As in standard ME systems, this study considers an object whose angular size is located inside the ME field of view and at a distance u behind the scatterer. After interacting with the scatterer, the light travels a distance v before reaching the detector.
Conventional ME imaging uses standard cameras, while this method uses an encoding detector module consisting of an encoding aperture and a wavelength dependent optical element. The purpose of this element is to uniquely modulate each spectral channel before combining and transforming them in a monochrome detector.
Thus, instead of simply measuring a low-contrast speckle, the spectral channels of which are inextricably mixed, a WMD signal was recorded, which is well suited for separation.
The researchers emphasize once again that their method does not require any known characteristics or assumptions about the diffuser or light source.
After making preliminary measurements of the multiplexed speckle, the known TΞ» value (wavelength dependent coding pattern) was used to reconstruct the individual speckle in each spectral band.
In their work, at the stage of calculations and modeling, scientists applied certain machine learning methods that can help in the implementation of a previously unconsidered method. First of all, sparse matrix feature learning was used to represent the speckle.
Traits training* - allows the system to automatically find the representations necessary to identify the features of the original data.
As a result, a base was obtained trained on speckle images from various measurement configurations. This base is quite generalized and does not depend on specific objects and scatterers involved in the generation of the mask IΞ»x, y. In other words, the system is trained on the basis of a scatterer that is not used in the experimental configuration, i.e. the system does not have access to it, as the researchers wanted.
To obtain speckle images at each wavelength, the OMP algorithm was used (orthogonal matching pursuit).
Finally, by computing the autocorrelation of each spectral band independently and inverting the autocorrelation at each wavelength, images of the object were obtained. The resulting images at each wavelength are then combined to create a color image of the object.
Image #2: A step-by-step process for composing an image of an object.
This technique, according to its creators, does not make any assumptions about the correlations between spectral channels and only requires the assumption that the value of the wavelength is sufficiently random. In addition, this method only requires information about the encoding detector, relying on pre-calibration of the encoding aperture and a pre-trained data library. These characteristics make this imaging technique highly versatile and non-invasive.
Simulation results
First, let's look at the simulation results.
Image #3
The image above shows examples of a multispectral image of two objects taken through a diffuser. Top row on 3a contains an object of interest, consisting of several numbers, shown both in false color and broken down by spectral channel. False color plotting displays the intensity profile of each wavelength in CIE 1931 RGB space.
The reconstructed object (bottom row on 3a) both in false color and in terms of individual spectral channels, demonstrates that the technique provides excellent imaging and only minor crosstalk between spectral channels, which does not play a significant role in the process.
After receiving the reconstructed object, i.e. after rendering, it was necessary to assess the degree of accuracy by comparing the spectral intensity (averaged over all bright pixels) of the real object and the reconstructed (3b).
On the pictures 3c shows a real object (top row) and a reconstructed image (bottom row) for a cotton stalk cage, and 3d analysis of imaging accuracy is shown.
To assess the accuracy of visualization, it was necessary to calculate the values ββof the coefficient of structural similarity (SSIM) and the peak signal-to-noise ratio (pSNR) of the real object for each spectral channel.
The table above shows that each of the five channels has an SSIM coefficient of 0,8β0,9 and a PSNR greater than 20. It follows that despite the low contrast of the speckle signal, the superposition of five spectral bands with a width of 10 nm on the detector allows quite accurate reconstruction spatial-spectral properties of the object under study. In other words, the technique works, but these are only simulation results. For completeness of confidence in their work, scientists conducted a series of practical experiments.
Experiment Results
One of the most significant differences between modeling and real experiments is the environment, i.e. the conditions under which both are carried out. In the first case, there are controlled conditions; in the second, there are unpredictable ones, i.e. we'll see how it goes.
Three spectral channels 8-12 nm wide, centered at 450, 550 and 650 nm, were considered, which, in combination with different relative magnitudes, generate a wide range of colors.
Image #4
The image above shows a comparison between the real object (the multi-colored letter "H") and the reconstructed one. The exposure time (exposure, i.e. exposure) was set to 1800 s, which made it possible to obtain an SNR in the range of 60-70 dB. Such an indicator of SNR, according to scientists, is not extremely important for the experiment, but serves as an additional confirmation of the efficiency of their technique, especially in the case of complex objects. In reality, and not in laboratory conditions, this method can be an order of magnitude faster.
The top row of image #4 shows the object at each wavelength (left to right) and the actual full color object.
In order to obtain an image of a real object as a result of visualization, a machine vision camera with appropriate bandpass filters was used to directly display the spectral components and obtain a full-color image by summing the resulting spectral channels.
The second row of the image above shows the autocorrelation patterns of each reconstructed spectral channel forming the multiplexed measurements, which are input to the data processing step.
The third row is the reconstructed object in each spectral channel, as well as the reconstructed full-color object, i.e. end result of the rendering.
The full color image shows that the relative values ββbetween the spectral channels are also correct, since the color of the combined reconstructed image corresponds to the real value, and the SSIM coefficient reaches more than 0,92 for each channel.
The bottom row is a confirmation of this statement, showing a comparison of the intensity of a real object and a reconstructed one. The data of both coincide in all spectral ranges.
From this it follows that even the presence of noise and potential simulation errors did not prevent us from obtaining a high quality image, and the experimental results are in excellent agreement with the simulation results.
The above experiment was set up taking into account the separated spectral channels. Scientists conducted another experiment, but with adjacent channels, or rather with a continuous spectral range of 60 nm.
Image #5
The letter βXβ and the β+β sign acted as a real object (5a). The spectrum of the letter "X" is relatively uniform and continuous - between 515 and 575 nm, but "+" has a structured spectrum, mainly located between 535 and 575 nm (5b). For this experiment, the exposure was 120 s to achieve the desired (as before) SNR of 70 dB.
A 60 nm wide band pass filter was also used over the entire object and a low pass filter over the "+" sign. During the 60 nm reconstruction, the spectrum is split into 6 adjacent 10 nm wide channels (5b).
As we can see from the pictures 5s, the resulting images are in perfect agreement with the real object. This experiment showed that the presence or absence of spectral correlations in the measured speckle does not affect the efficiency of the imaging technique under study. Scientists themselves believe that a much greater role in the visualization process, or rather in its success, is played not so much by the spectral characteristics of the object as by the calibration of the system and the details of its encoding detector.
For more detailed information about the nuances of the study, I recommend looking at ΠΈ to him.
Finale
In this work, scientists described a new method for multispectral imaging through a scatterer. Wavelength-dependent speckle modulation with a coded aperture allowed one multiplexed measurement and speckle calculation using the machine learning-based OMP algorithm.
Using the multi-colored letter βHβ as an example, scientists have shown that focusing on five spectral channels corresponding to violet, green and three shades of red allows one to obtain an image reconstruction containing all the colors of the original (blue, yellow, etc.).
According to the researchers, their technique can be useful both in medicine and in astronomy. Color carries important information in both directions: in astronomy - the chemical composition of the objects under study, in medicine - the molecular composition of cells and tissues.
At this stage, scientists note only one problem that can cause visualization inaccuracies, these are modeling errors. Due to the rather long time required to complete the process, there may be changes in the environment that will make their own adjustments that were not taken into account at the preparation stage. However, in the future it is planned to find a way to level this problem, which will make the described imaging technique not only accurate, but also stable in any conditions.
Friday off-top:
Light, color, music and a trio of the most famous blue "weirdos" in the world (Blue Man Group).
Thanks for watching, stay curious, and have a great weekend everyone! π
Thank you for staying with us. Do you like our articles? Want to see more interesting content? Support us by placing an order or recommending to friends, 30% discount for Habr users on a unique analogue of entry-level servers, which was invented by us for you: (available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).
Dell R730xd 2 times cheaper? Only here in the Netherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $99! Read about
Source: habr.com