Smile and keystones are two types of optical aberrations that strongly impact the accuracy and the usability of pushbroom hyperspectral cameras. The smile is a spectral distortion and is primarily a property of the spectrograph, whereas the keystone is a spatial distortion and is mainly a property of the front objective.
Fig.1 below illustrates this:
Figure 1: Smile and keystone on a pushbroom hyperspectral sensor.
Source: www.yokoya1985.sakura.ne.jp
Smile
As illustrated in Fig.1, the smile can be seen as a spectral shift of the sensor over its entire field of view (FOV). If the user images a homogeneous target over its full FOV, the spectra measured at the middle and on the side of the FOV have an offset. In a sorting application, this may have dramatic consequences. If, for instance, a sorting model built with data from the middle of the “image” (i.e., middle of the conveyor belt), it will not necessarily work in other locations (i.e., at the side of the belt).
Fig.2 shows its effect on data acquired with a low performing spectrograph mounted on a camera detector with tiny pixels. We can see a spectral shift between the center and the side of the image of ca. 2 pixels.
Figure 2: Spectral shift due to smile distortion. The blue, red, and white tiles are spectrally similar, respectively, per group of color.
We designed Specim FX cameras for industrial applications, where robustness and model transferability are required. An Automatic Image Enhancement (AIE) algorithm corrects this smile effect real-time. This AIE correction reduces the smile distortion below ±15% of a pixel (see Fig.3). Those numbers are valid for all Specim FX cameras, regardless of their model (FX10, FX17, and FX50). We recommend using the FX10 with a spectral binning configuration x2, decreasing the effective smile by half.
Figure 3: Typical smile after AIE correction, here for a Specim FX10 (which contains 1024 spatial pixels)
Keystone
Analog with the smile, the keystone can be seen as spatial misregistration of the spectrum. It has a significant effect on spectral purity (the sensor’s ability to measure the spectrum of an object or single point only without being influenced by its surroundings). In practice, an optical system with a high keystone will measure spectra influenced by its environment. For a sorting application, building a robust model is challenging. A shell of nuts will not have its contour line sorted in the same way as its center, even if it would be homogenous. This may lead to a wrong interpretation of contamination.
In Fig.4 below, a homogenous white paper was measured with a low-quality spectrograph mounted on a camera detector with small pixels. On its side, a red line of ca. 2 pixels can be seen, revealing the presence of keystone.
Figure 4: effect of keystone on a sharp paper edge. Red, Green, and blue channels were allocated to 900, 550, and 450 nm, respectively.
To limit FX cameras’ keystone effects, we implemented a correction within the real-time AIE algorithm. It reduces the keystone of all FX cameras below 15% of a pixel (see Fig.5)
Figure 5: Typical Keystone after AIE correction, here for a Specim FX10 (which contains 448 spectral pixels)
Conclusions
Large smile and keystone distortions strongly affect the robustness, transferability, and efficiency of sorting models. To limit their effect, the level of smile and keystone should be well below the detector’s pixel size on which the hyperspectral system is based. Specim cameras correct and limit those aberrations in real-time; these effects are invisible at such a level.
