01/18/2022

# Smart residual sum of squares allows correction of systematic errors

Determining the true value of a physical or metrological quantity is of essential importance and the most important goal in statistics. In the absence of bias, the median or arithmetic mean can be excellent estimates of this true value.

Otherwise, there can be significant deviations. Fitting methods optimize the parameters of a given function in such a way that the deviations between measured and calculated curves are minimized.

The prerequisite for the application of fitting methods is therefore that the measurements are free of systematic errors. Scientists from the Leibniz Institute for Photonic Technologies (Leibniz-IPHT) in Jena, Germany, have developed an "intelligent" residual sum of squares that cannot only be used despite systematic errors, but can also correct them.

In spectroscopy, fitting methods based on the residual sum of squares are applied in order to adapt simulated to the measured spectra and to determine band positions, half-widths and areas ("band fitting"). However, the result of curve fitting can still look reasonable, even if the spectra contain systematic errors, for example due to thermal fluctuations. This leads to erroneous results and since the magnitude of the systematic error is usually unknown, a correction cannot be made. The approach recently developed at Leibniz-IPHT can solve this problem.

The researchers use a method that can identify correlations between different curves. This so-called 2D correlation analysis has proven itself to be invaluable in the field of spectroscopy and is used for many years with different types of spectroscopy, such as IR, UV/VIS and Raman spectroscopy. With 2D correlation spectroscopy, linear and non-linear variabilities of spectra based on systematic changes of boundary conditions are represented semi-quantitatively. In this way, for example, overlapping peaks can be identified and overlapping processes can be separated.

The scientists from Jena use 2D correlation analysis to compare series of measured and simulated IR spectra. Based on symmetry conditions, the smart residual sum of squares was derived from this comparison.

"Our original intention was to use 2D correlation analysis to quantitatively estimate the correlations between two series of spectra. Since we mainly deal with the quantitative analysis of IR spectra, it made sense to use this method to quantitatively compare series of measured and simulated spectra," explains Dr. Thomas Mayerhöfer from the Spectroscopy/Imaging department at Leibniz-IPHT.

At the Leibniz-IPHT, dispersion analysis is used for the quantitative evaluation of IR spectra. Dispersion analysis is based on wave optics and dispersion theory and allows not only the determination of the oscillator parameters, but also that of the functions of the optical constants. When dispersion analysis is combined with the smart loss function, unphysical reflectance or transmittance values greater than unity are ignored and the results agree with those obtained by the analysis of error-free spectra.

The method is not limited to the analysis of spectra and can be used wherever and whenever experimental curves are modeled with physical or mathematical functions. "We recently adapted the method of 2D correlation spectroscopy, in which a series consists of only two spectra, correspondingly so that one of the spectra represents the experimental spectrum and the other the simulated spectrum. This means that individual curves can also be fitted with the help of the corresponding smart residual sum of squares," Mayerhöfer continues. Since the conventional residual sum of squares is used in deep learning, interesting applications can also be foreseen in this area.

Source: Leibniz Institute of Photonic Technology (IPHT)