Semi-Physical Simulation Method for Stellar Maps with Color Temperature Information

1. Introduction

The star sensor—a star-sensitive device—is a key component of the spacecraft attitude determination and control system whose accuracy directly affects the performance of the navigation system [1]. The working conditions of the star-sensitive device [2] have become increasingly complex, and its navigation method has developed from the traditional astronomical angular measurement or ranging navigation method to emerging navigation technologies, such as astronomical spectral velocimetry navigation using stellar color temperature information [3,4,5]. Existing star simulators, which can only provide limited information on the color temperatures of the stars [6,7], fail to satisfy the ground-based calibration requirements of the star-sensitive device.

Recent efforts have been made to accurately simulate the multi-dimensional information of stellar maps in outer space. For example, Linghao Wu et al. [8] developed a star simulator with stray light suppression capability and simulated stellar map of stars as dim as magnitude 6. Schulz et al. [9] developed a stellar map generation method with controllable Gaussian background noise and simulated stellar map with an approximate sky background. Teague and Chahl [10] developed a simulation method for motion-blurred stellar map, effectively correcting the grayscale information of stellar map in stellar map display devices through color space conversion methods. Although this method only generated grayscale stellar map, it has significant implications for simulating stellar map with color temperature information using a single stellar map display device.

With the development of spectral simulation architectures from multi-LED mixing [11] to spatial light modulation-based dimming [12], stellar spectral simulation methods have been extensively investigated. Liu et al. [13] simulated a single star in the 3000–11000 K color temperature range using a digital micromirror device (DMD) and a fuzzy proportional–integral–derivative (PID) control algorithm. Yun et al. [14] extended the color temperature simulation range to 2000–12000 K by combining backpropagation (BP), a neural network, and a PID control algorithm. Although the above two methods can simulate the color temperature information of individual stars more realistically, they cannot simulate the real state of stars with different color temperature information in stellar map.

To improve the similarity between the stellar map and the actual state of outer space, Bin Zhao et al. [15] proposed a fast playback method in which the stellar maps were fused with cosmic background radiation information to improve the stellar maps’ refreshing speed. However, the simulation of stellar map with stellar color temperature information has still not been realized. Songzhou Yang et al. [16] developed a “series–parallel” hybrid, multi-source, information fusion, spatial target simulation system. In this system, three light sources (one LED light source and two blackbody light sources) and three spatial light modulation devices (one thin-film transistor liquid crystal display (TFT-LCD) and two DMDs) were used to simulate each star in the stellar map. The energy ratio of four spectral intervals (UV, visible, mid-wave infrared, and long-wave infrared) of each star in the stellar map could be adjusted. However, the system structure was complicated, and the system could not accurately simulate the color temperature.

In summary, the existing current stellar map simulators have various shortcomings, such as missing color temperature information, complex system structure, and inability to independently control the color temperature of stars. Therefore, this study aimed to address this research gap by developing a semi-physical stellar map simulation method with color temperature information. The stellar map simulator leveraged an OLED-based architecture. A stellar map simulation link with color temperature information was established, and a stellar map generation algorithm with color temperature information was proposed. We successfully controlled the color temperature of each star in the semi-physical simulation in the stellar map. This study provides new technical means and implementation approaches for the ground calibration and testing of star sensors. The developed star simulator has important theoretical and practical significance.

2. Framework for Semi-Physical Simulations of Stellar Maps with Color Temperature Information

When significantly distant stars are observed through optical payloads such as star sensitizers, they can be regarded as point light sources at infinity [17]. Currently, star simulators use liquid crystal on silicon (LCOS) [18], LCD [19], or DMD [20] as stellar map display devices placed at the focal plane and according to the working principle of the collimated optical system [21]. Combined with the stellar map illumination device [22], stellar maps are simulated at infinity by loading the grayscale images of stellar maps. However, the current stellar map illumination devices are mostly LED light sources [23], blackbody radiation sources [24], or specially designed illumination optical systems [25], which can only output illumination beams with fixed color temperature information. The illumination area of the stellar map illumination device covers the entire range of the stellar map display device. Therefore, the stars displayed in existing stellar map display devices all have the same color temperature information. However, the stars in actual space have different color temperature information, resulting in inconsistency between the simulated and actual color temperature information. Therefore, in this study, the architecture of the star simulator is simplified by replacing the traditional stellar map display device and stellar map illumination device with a self-luminous OLED device that enables independent and controllable color temperature information of each pixel. As such, the stellar map simulation with color temperature information is realized so that the stellar map simulated is closer to the actual state of stars in the universe.

The color temperature information of stars is primarily classified based on their surface temperatures [26], and the OLED devices adjust the color temperature by adjusting the ratio of the luminous intensity of the three primary colors of red (R), green (G), and blue (B) [27]. The light beam emitted from the stellar map displayed by the OLED device passes through the collimated optical system. Thus, the color temperature of the simulated stellar map deviates from that of the OLED device [28] and the spectral transmittance of the collimated optical system [29]. Compensating for this combined color temperature deviation is necessary. Therefore, a stellar map simulation algorithm linked with color temperature information was formulated to improve the ability of the OLED-based stellar map simulation architectures to accurately simulate stellar maps containing color temperature information. The stellar map simulation architecture and algorithm link with color temperature information are shown in Figure 1.

3. Algorithm for Stellar Map Simulation with Color Temperature Information

According to the architecture and algorithmic links, the stellar map simulation algorithm with color temperature information was divided into three modules: conversion of the stellar spectrum into the three primary colors of RGB, color temperature deviation calibration, and stellar map generation algorithm with color temperature information. The algorithmic flow is shown in Figure 2.

3.1. Stellar Spectral Type to RGB Triple-Color Conversion

Stellar spectral types are primarily classified into seven according to the Harvard spectral classification (HSC): O, B, A, F, G, K, and M. Each spectral type is further subdivided into ten subtypes ranging from 0 to 9 [30]. No formula has been established to convert stellar spectral types directly to RGB triplet colors owing to the complexity of the stellar spectra and the nonlinear relationship between spectral features and blackbody color temperature. Therefore, the stellar spectral types were numerically coded to facilitate the calculation and analysis, with 1 as the coding interval, and the 70 spectral subtypes from the O0 to M9 type were sequentially coded from 00 to 69 starting from 0. The coding rules are listed in Table 1.

Based on the discrete data of stellar spectral type and temperature provided by Harre and Heller [31], the spectral type of the star was converted to the corresponding stellar temperature by a polynomial fitting method [32] given by

$$f(s)=a_1\cdot s+a_2\cdot s^2+\cdots a_n\cdot s^n$$

(1)

where $f(s)$ is the stellar temperature, s is the spectral-type code, and a1, a2,…an are the polynomial coefficients corresponding to the nth spectral-type code. As stellar radiation approximates blackbody radiation at a certain temperature [33], according to Planck's formula for blackbody radiation, the spectral profile emitted by a star can be expressed as

$$E(\lambda ,f(s))={\displaystyle \frac{2h{c}^2}{{\lambda }^5}}\cdot {\displaystyle \frac{1}{e^{\frac{hc}{\lambda kf(s)}}-1}}$$

(2)

where λ is the wavelength, h is Planck's constant ($h=6.626\times {10}^{-34}J\cdot s$), c is the speed of light ( $c=3\times {10}^{8}m/s$), k is Boltzmann's constant ($k=1.381\times {10}^{-23}J/K$). According to the color matching functions, $\overline{x}(\lambda )$, $\overline{y}(\lambda )$, and $\overline{z}(\lambda )$ are defined by the International Commission on Illumination (CIE) [34]. The triple-stimulus XYZ values of the stellar spectral curve [35] were obtained as

$$\left\{\begin{array}{c}X={\displaystyle {\int }_{380}^{780}E(\lambda ,f(s))\cdot \overline{x}(\lambda )d\lambda }\\ Y={\displaystyle {\int }_{380}^{780}E(\lambda ,f(s))\cdot \overline{y}(\lambda )d\lambda }\\ Z={\displaystyle {\int }_{380}^{780}E(\lambda ,f(s))\cdot \overline{z}(\lambda )d\lambda }\end{array}\right.$$

(3)

The corresponding RGB triple-base color on the OLED was obtained through the RGB conversion matrix [36] as

$$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right]\cdot \left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]$$

(4)

where $a_{ij}$ is an element of the RGB transformation matrix representing the linear transformation coefficients from the XYZ to RGB color spaces.

3.2. Color Temperature Deviation Calibration

The color temperature deviation calibration was performed by loading the standard color card images of multiple color blocks on the OLED device and using a color temperature calibrated industrial camera to obtain the three chromatic colors of each color block after collimating the optical system. Thus, the color temperature deviation calibration matrix was obtained.

Let the number of color blocks in the standard color card be n. The theoretical tri-color of the standard color block numbered i can be expressed as ${\left[\begin{array}{ccc}{R}_i& G_i& B_i\end{array}\right]}^T$, where $i\in \left[1,n\right]$. Then, the theoretical RGB tri-color matrix of the standard color card is

$RGBP=\left[\begin{array}{cccc}{R}_1& R_2& \cdots & R_n\\ G_1& G_2& \cdots & G_n\\ B_1& B_2& \cdots & B_n\end{array}\right]$. The industrial camera can obtain the RGB tri-color of the corresponding standard color block as${\left[\begin{array}{ccc}{R^{\prime }}_i& {G^{\prime }}_i& {B^{\prime }}_i\end{array}\right]}^T$. Then, the RGB tri-color matrix of the standard color card obtained by the industrial camera is $RGBM=\left[\begin{array}{cccc}{R^{\prime }}_1& {R^{\prime }}_2& \cdots & {R^{\prime }}_n\\ {G^{\prime }}_1& {G^{\prime }}_2& \cdots & {G^{\prime }}_n\\ {B^{\prime }}_1& {B^{\prime }}_2& \cdots & {B^{\prime }}_n\end{array}\right]$. According to the principle of least squares, the color temperature deviation calibration matrix (CCM) can be expressed as

$$CCM={(RGB{M}^{T}RGBM)}^{-1}RGB{M}^{T}RGBP$$

(5)

According to Equation(1)–(5), the color temperature deviation-calibrated stellar spectral type corresponding to the three fundamental colors $R_i(s)$, $G_i(s)$, and $B_i(s)$ are

$$\left[\begin{array}{c}{R}_i(s)\\ G_i(s)\\ B_i(s)\end{array}\right]=CCM\cdot \left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right]\cdot \left[\begin{array}{c}{\displaystyle {\int }_{380}^{780}\frac{C_1}{{\lambda }^5(e^{\raisebox{1ex}{$C_2$}\!\left/ \!\raisebox{-1ex}{$\lambda f(s)$}\right.}-1)}\cdot \overline{x}(\lambda )d\lambda }\\ {\displaystyle {\int }_{380}^{780}\frac{C_1}{{\lambda }^5(e^{\raisebox{1ex}{$C_2$}\!\left/ \!\raisebox{-1ex}{$\lambda f(s)$}\right.}-1)}\cdot \overline{y}(\lambda )d\lambda }\\ {\displaystyle {\int }_{380}^{780}\frac{C_1}{{\lambda }^5(e^{\raisebox{1ex}{$C_2$}\!\left/ \!\raisebox{-1ex}{$\lambda f(s)$}\right.}-1)}\cdot \overline{z}(\lambda )d\lambda }\end{array}\right]\cdot {\displaystyle \frac{1}{{\displaystyle {\int }_{380}^{780}\frac{C_1}{{\lambda }^5(e^{\raisebox{1ex}{$C_2$}\!\left/ \!\raisebox{-1ex}{$\lambda f(s)$}\right.}-1)}\cdot (\overline{x}(\lambda )+\overline{y}(\lambda )+\overline{z}(\lambda ))d\lambda }}}$$

(6)

3.3. Algorithm for Generating Stellar Maps with Color Temperature Information

Based on the stellar spectral type to RGB three-color conversion law and the color temperature deviation calibration, the stellar map presents the attitude information of the celestial coordinate system [37] (azimuth $\phi$, pitch $\theta$, roll $\lambda$) to obtain the stellar map pointing unit vector ${\left[x,y,z\right]}^T$ as

$$\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{c}\cos \theta \cos \phi \\ \cos \theta \sin \phi \\ \sin \theta \end{array}\right]$$

(7)

The 3D coordinates of the star point [38] were screened according to the stellar map field of view (FOV), and the maximum coordinate values of the stellar map display device were set as $x_{\max }$ and $y_{\max }$. Then, the coordinates of the simulated star on the stellar map display device (x', y') were obtained as

$$\left\{\begin{array}{c}{x}^{\text{'}}=-{\displaystyle \frac{y\cdot x_{\max }}{2x\tan (FOV/2)}}\\ y^{\text{'}}=-{\displaystyle \frac{z\cdot y_{\max }}{2x\tan (FOV/2)}}\end{array}\right.$$

(8)

Subsequently, according to the stellar spectral type in the stellar catalog [39], Equation(5) and (6) were used to obtain the RGB triple colors of the OLED device corresponding to the star, thus completing the generation of the stellar map with color temperature information.

4. Experimentation and Discussion

4.1. Experimental Platform and Design

In this study, the stellar map simulation algorithm with color temperature information was implemented on the host computer using the Visual Studio 2022 development tool based on the Microsoft Foundation Class framework. OpenCV image processing and Eigen matrix manipulation libraries were introduced during the development process. The stellar map simulation device was constructed using Sony MicroOLED ECX331DB-6 with a resolution of 1024 × 768, an 8-bit color depth OLED device, and a collimated optical system with an FOV of 20° × 20° and a focal length of 35 mm. MV-UB1000C by MindVision, which has 10 million pixels, was used as an industrial camera. The semi-physical simulation platform for stellar maps with color temperature information is shown in Figure 3.

Based on the stellar map simulation algorithm with color temperature information, two experimental phases were involved in this study: pre-experiment and validation experiment. The specific experimental arrangement is summarized in Table 2.

4.2. Experimental Results of the Pre-Experimental Phase

The experiment on stellar map generation with color temperature information was divided into two parts: stellar spectral type to RGB triple-color conversion and color temperature deviation calibration.

4.2.1. Stellar Spectral Type to RGB Three-Color Conversion

Polynomial fits from 1 to 50 times were conducted to obtain the optimal polynomial fit of the spectral type of the star to the blackbody color temperature. The fit correlation coefficient [34] was used to evaluate the polynomial fit. The relationship between the polynomial counts and fit correlation coefficients is shown in Figure 4.

As shown in Figure 4, the fitting correlation coefficients were ˃0.99 for the fitting times in the interval from 3 to 37, and the 22nd polynomial fitted the best. sRGB is the most widely used color space [35], and its color gamut is sufficient to cover the range of stellar colors [36]. Therefore, the sRGB color space was selected. The stellar spectral type and RGB triple-base color scale relations were obtained (Figure 5) according to Equation(2)–(4).

4.2.2. Color Temperature Deviation Calibration

The 24-color standard color card was displayed on the OLED device (RGB three-color as $R_i$$,$ $G_i$$,$ and $B_i$). The industrial camera was used to obtain the image of the standard color card through the collimated optical system (RGB three-color ${R^{\prime }}_i$, ${G^{\prime }}_i$, and ${B^{\prime }}_i$), and according to Equation(5), the CCM was obtained as $\left[\begin{array}{rrr}\hfill 1.060828& \hfill 0.008814& \hfill -0.00608\\ \hfill -0.02595& \hfill 0.992383& \hfill -0.00513\\ \hfill -0.00894& \hfill -0.00811& \hfill 1.018448\end{array}\right]$. Then, the RGB three-color of the standard color card were set after calibrating the color temperature deviation to $R{C^{\prime }}_i$, $G{C^{\prime }}_i$, and $B{C^{\prime }}_i$. The color difference of the 24 standard colors before and after the calibration of $\Delta E_i$ and $\Delta E{C}_i$ was determined using

$$\left\{\begin{array}{c}\Delta E_i=\sqrt{{\left({R^{\prime }}_i-R_i\right)}^2+{\left({G^{\prime }}_i-G_i\right)}^2+{\left({B^{\prime }}_i-B_i\right)}^2}\\ \begin{array}{l}\Delta E{C}_i=\sqrt{{\left(R{C^{\prime }}_i-R_i\right)}^2+{\left(G{C^{\prime }}_i-G_i\right)}^2+{\left(B{C^{\prime }}_i-B_i\right)}^2}\\ S_{\Delta E}=\sqrt{{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^{24}{\left(\Delta E_i-\overline{\Delta E}\right)}^2}}{23}}}\\ S_{\Delta EC}=\sqrt{{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^{24}{\left(\Delta E{C}_i-\overline{\Delta EC}\right)}^2}}{23}}}\end{array}\end{array}\right.$$

(9)

Here, $\Delta E_i$ is the sum of squares of the three primary color deviations of the standard color card before calibration, $\overline{\Delta E}$ is the average value of $\Delta E_i$, $\Delta E{C}_i$ is the sum of squares of the three primary color deviations of the standard color card after calibration, and $\overline{\Delta EC}$ is the average value of $\Delta E{C}_i$. Using the standard deviation of $\Delta E_i$ and $\Delta E{C}_i$, $S_{\Delta E}$ and $S_{\Delta EC}$, which indicates the overall simulation accuracy of the OLED device on the 24 standard colors of the color blocks before and after calibration, was calculated as 6.98 and 4.34, respectively. The theoretical 24-color standard color card, as well as the standard color card image and the corresponding RGB three-color heat map acquired by the industrial camera before and after calibration, are shown in Figure 6(a,b), respectively. The color difference between the 24 standard color blocks before and after calibration is shown in Figure 6(c). The $S_{\Delta EC}$ was smaller than $S_{\Delta E}$, implying that the color emitted by the OLED after calibration was closer to the standard color card.

4.3. Experimental Results of the Validation Experiment Phase

The simulation accuracy of stellar map with color temperature information before and after calibrating the color temperature deviation was evaluated based on the mapping relationship between the stellar spectral type and RGB tri-color. This was achieved based on the following steps: 1) a mapping table of stellar spectral type ($S{P}_i$) and RGB tri-color ($R_i,G_i,B_i$) was established (notated as $Ta{b}_{SP-RGB}$), and 2) an algorithm for identifying the spectral type of a stellar star point based on the look-up-table method was designed. In the algorithm, the principle of least squares was used to determine the spectral-type position ($pos$) matching the simulated stellar RGB three-color values ($RS$, $GS$ and $BS$). Then, the corresponding spectral type of the star was obtained. The discriminative formula of the stellar star-point spectral type identification algorithm is given as

4.3.1. Stellar Map Image Generation with Color Temperature Information

Theoretical stellar map with color temperature information were selected from the commonly used SAO catalog (5091 stars) with magnitudes ranging from 0 Mv to +6.0 Mv and above [37]. Three additional stellar maps with the attitudes (0, 30, 8), (23, -80, 8), and (66, 34, 2) were randomly generated using the stellar map simulation algorithm with color temperature information to obtain the central attitude stellar map (0, 0, 0) and prevent inconsistent experimental results. The overall distribution of the maps and the four attitudes of the stellar maps are shown in Figure 7.

4.3.2. Semi-Physical Simulation of Stellar Maps with Color Temperature Information

Four theoretical stellar maps with color temperature information were loaded onto the OLED device. The stellar maps with uncalibrated and calibrated color temperature deviations were successively captured using the industrial camera, and the spectral type simulation effect of the four maps was obtained using the stellar star-point spectral-type recognition algorithm. The color temperature deviation-calibrated stellar maps are shown in Figure 8(a), and the spectral type simulation accuracy is shown in Figure 8(b).

The maximum, average, and standard deviations of the spectral type simulations of the four stellar maps before and after calibration are summarized in Table 3. The maximum deviation of the simulated spectral types of the four stellar maps after calibration was 9, which was smaller than the metric scale of ten minor spectral types within a major stellar spectral type. This deviation was reduced by a factor of 1.89–2.11, demonstrating significant improvement in the accuracy of the stellar maps spectral simulation. In addition, the average and standard deviations of the simulated spectral types of the four stellar maps after calibration were reduced by a factor of 1.29–1.73 and from 1.58 to 2.53, respectively. This indicated substantially improved accuracy of the spectral simulation of the stellar maps and improved dispersion of the simulation of the overall color temperature information of the stellar maps.

4.4. Discussion and Comparison

The comparison between the related studies on stellar maps simulation and the present study is summarized in Table 4. Linghao Wu et al. [8] successfully simulated fainter stars, Schulz et al. [9] and Bin Zhao et al. [15] simulated the superposition of stellar map and backgrounds, and Teague and Chahl [10] simulated stellar map of motion-blurred stars; however, none of these studies could simulate color temperature. The simulator proposed in [13] and [14] can simulate color temperature in a wide range. However, only a single stellar color temperature can be simulated. The simulators developed in [16] and this study can simulate the color temperature information of stars independently within the stellar map. However, in the simulator proposed in [16], only the energy ratio of four spectral intervals (ultraviolet, visible, mid-wave infrared, and long-wave infrared) can be adjusted, which is a weak simulation of the color temperature information. Conversely, the simulator developed in this study can cover the entire O–M types (the seven main spectral types) and the subtypes of 0–9 under each spectral type, with a total of ten subtypes, totaling 70 stellar spectral types.

In summary, to the best of the authors' knowledge, the proposed semi-physical simulation of stellar map with color temperature information achieved independent controllability of the color temperature information of each star within the stellar map. Concurrently, the simulation range of the stellar spectral types covers all 70 stellar spectral types of the HSC. However, this study presents a limitation in that the bit depth of the device is only 8 bits. Using a device with a higher bit depth would improve the correction ability of the proposed method. In future research, 16-bit color OLED display devices could be used to improve the degree of correction subdivision of the devices and obtain better calibration results.

5. Discussion and Conclusions

This study aimed to overcome the lack of independent simulation methods of stellar color temperature in stellar map and the complex structure of existing stellar map simulation systems with color temperature information. Therefore, a simplified stellar map simulation architecture based on OLED devices and a stellar map simulation algorithm with color temperature information was developed in this study. An experimental platform was developed, and two pre-experiments of stellar spectral type to RGB three-base color conversion and color temperature deviation calibration were performed. Subsequently, four pairs of stellar maps were used to verify the semi-physical simulation effect of stellar maps with color temperature information. According to the experimental results:

1.

The polynomial number of fits of the stellar spectral type to the blackbody color temperature ranged from 3 to 37 correlation coefficients above 0.99, with the 22nd polynomial fit being the best. The standard deviation of the simulated color difference of the 24 standard colors was reduced by a factor of 1.6 after calibration with a 24-color standard color card.

2.

The maximum deviation of the stellar spectral type simulations of the four stellar maps after color temperature bias calibration is better than 9, and the average deviation is better than 6.55, which are all smaller than the metric scale of 10 minor spectral types within a major stellar spectral type.

3.

After the color temperature deviation calibration, the maximum deviation of the spectral type simulation of the four stellar maps is reduced by a factor of 1.89 to 2.11, the average deviation by a factor of 1.29 to 1.73, and the standard deviation of the simulation deviation by a factor of 1.58 to 2.53, which indicates that the spectral simulation limiting accuracy of the stellar map, the overall correctness of the simulation of the color temperature information as well as the dispersion are significantly improved.

In addition, compared with international studies, the developed simulator realized independent control of the color temperature information of each star in the stellar map, and the spectral type simulation covered all 70 stellar spectral types of the HSC. The study findings provide a simulation method for the attitude determination algorithm using multi-dimensional information carried by the star sensor and establish a research basis for advancing the fields of remote sensing technology and spectral navigation.

In the future, using devices [40] with better performance can be further investigated to overcome the limitation of stellar spectral-type simulation accuracy. The cosmic radiation background can be increased, and the playback speed of the stellar map can be improved to make them closer to the actual stellar map of outer space. Color stellar map simulation technology provides strong technical support for the accuracy, reliability and adaptability of remote sensing technology by optimizing the performance of remote sensing equipment, supporting the calibration of radiation characteristics of remote sensing data and target recognition in complex environments. The innovative application of this technology not only promotes the development of remote sensing field, but also shows broad application prospects.