4.3.1. PlanetiQ-GFS Model Comparison
Figure 17 presents a comparison of the pressure, temperature, WVP, and refractivity profiles from the PlanetiQ and GFS models for the day of year (DOY) 2023.342, characterized by an average number of occultation events. The results exhibit noteworthy differences between the two models, as highlighted by key numerical metrics.
For WVP, the overall RMSE for the wet_avn scenario is 0.23, indicating a moderate level of deviation between models. Similarly, pressure and temperature differences display an overall RMSE of 0.58 mb and 1.54°C, respectively. The refractivity differences show the most significant deviation, with a total RMSE of 1.27.
The MAD further emphasizes the disparities, showcasing values of 0.08 mb, 0.35 mb, 0.5, and 1.07°C for WVP, pressure, refractivity, and temperature, respectively. These values demonstrate the magnitude of the differences between the mission profiles and the GFS model, with temperature being the highest absolute difference
Examining the altitude-specific analysis for the altitude range (5, 20) km, the mean differences highlight nuanced variations. Notably, WVP, pressure, and temperature exhibit small mean differences of -0.002 mb, 0.15 mb, and -0.02°C, respectively. However, refractivity displays a mean difference of 0.04, emphasizing a significant shift in this parameter. WVP, pressure, refractivity, and temperature all have altitude-specific RMSE values of 0.07 mb, 0.3 mb, 0.51, and 0.85°C, respectively, indicating model precision.
Figure 18 provides a complete comparison of various key metrics for pressure, refractivity, temperature, and WVP profiles between the PlanetIQ and ECMWF models throughout the study period. The Maximum Difference in pressure profiles is noted at 4.2 mb, with a corresponding maximum difference percentage of 0.6%. The mean difference for pressure is -0.19 mb, and the maximum RMSE is observed at 29.7 mb, occurring at an altitude of 3.0. The altitude of maximum mean difference for pressure is 14.2, and the mean difference percentage at maximum RMSE is -0.6%.
For refractivity, the maximum difference is 1.8, and the maximum difference percentage is 0.8%. The mean difference in refractivity is -0.07, while the maximum RMSE reaches 10.4, situated at an altitude of 3.0. The altitude of maximum mean difference for refractivity is 16.3, and the mean difference percentage at the maximum RMSE is -0.8%.
Temperature profiles reveal a maximum difference of 0.46°C, with a corresponding maximum difference percentage of 3.4%. The mean difference for temperature is -0.14°C, and the maximum RMSE is 3°C, occurring at an altitude of 40.0. The altitude of maximum mean difference in temperature is 19.9°C, and the mean difference percentage at maximum RMSE is 0.6%.
WVP profiles exhibit a Maximum Diff. of 0.1, along with a Maximum Diff.% of 26.8%. The Mean Diff. for WVP is -0.02 mb, and the Maximum RMSE is 1 mb, situated at an altitude of 3.0. The Altitude of Maximum Mean Diff. for WVP is 12.9, and Mean Diff.% at Maximum RMSE is -3.3%.
In
Figure 19, the distribution of diff.% between PlanetIQ and GFS profiles at 3 km altitude for various variables is depicted over the study period. Pressure differences remain generally within ±0.5%, except for certain land areas in the southern hemisphere where deviations are observed. Across the globe, diff.% at 3 km altitude for pressure does not exceed ±2%.
Temperature differences, given the relatively small values of measurements at 3 km altitude, may exhibit slightly higher diff.%, but still generally remain below ±2.5%. Notably, temperature diff.% increases around the equator and pole regions, with differences elsewhere not surpassing 0.4%.
WVP difference percentage typically is less than ±0.5%, except for specific distributed regions where it may reach up to ±3%, particularly near the equator. Refractivity difference percentage generally does not exceed ±4%, concentrated around the equator, and tends to decrease moving towards the poles.
In summary, the analysis reveals that the differences in the aforementioned variables between PlanetIQ and GFS profiles at 3 km altitude are generally within acceptable limits, with occasional exceptions in specific places. The findings provide valuable insights into the performance and agreement of the two datasets, highlighting areas of interest such as the equator and pole regions where variations are more pronounced.
4.3.3. PlanetiQ-ECMWF Model Comparison
Figure 20 presents a detailed examination of the profiles for pressure, temperature, WVP, and refractivity, comparing the PlanetIQ and GFS model on DOY 2023.342, characterized by an average number of occultation events. The results showcase substantial differences between the two models, with key numerical metrics offering insight into the extent of these variations.
For WVP, the overall RMSE for the wet_ech scenario is 0.18, signifying a relatively modest level of deviation between the mission profiles and the GFS model. Likewise, pressure and temperature differences exhibit overall RMSE values of 0.38 mb and 1.3°C, respectively. Refractivity differences demonstrate the most pronounced divergence, with an overall RMSE of 0.9 mb.
The MAD highlights the absolute magnitude of the disparities, presenting values of 0.05 mb, 0.24 mb, 0.35, and 0.9°C for WVP, pressure, refractivity, and temperature, respectively. Temperature differences are particularly noteworthy, indicating a substantial absolute difference between the two models.
Altitude-specific analysis for the range (5, 20) km reveals subtle mean differences. WVP, pressure, and temperature exhibit minor mean differences of -0.001 mb, -0.01 mb, and -0.05°C, respectively. Refractivity, however, displays a slightly larger mean difference of 0.0134, emphasizing a discernible shift in this parameter. WVP, pressure, refractivity, and temperature have altitude-specific RMSE values of 0.05 mb, 0.3 mb, 0.4, and 0.64°C, respectively, indicating model precision.
Figure 21 illustrates a comprehensive comparison of the critical variables: pressure, refractivity, temperature, and WVP, between the PlanetIQ and ECMWF model profiles for the entire study period. The results provide a full overview of the differences, expressed in mean difference, maximum difference, maximum difference percentage, and RMSE metrics.
For Pressure, the maximum difference is 3.0875, with a corresponding maximum difference percentage of 0.4%. The mean difference is -0.16, with a maximum RMSE reaches 25.6. The altitude of the maximum mean difference is 25.2 km, while the maximum RMSE occurs at an altitude of 3.0 km, constituting a mean difference percentage at maximum RMSE of -0.4%.
In terms of refractivity, the maximum difference is 1.0, accompanied by a maximum difference percentage of 0.5%. The mean difference is -0.04, with the maximum RMSE of 8.8284. The altitude of the largest mean difference is 12.5 km, while the maximum RMSE occurs at 3.0 km, with a mean difference percentage of -0.5% at the maximum RMSE.
Temperature differences exhibit a maximum difference of 0.2°C and a maximum difference percentage of 10.9%. The mean difference is -0.01, with a maximum RMSE of 2.7°C. The altitude of the maximum mean difference is 32.8 km, whereas the maximum RMSE occurs at 3.0 km, with a mean difference percentage of 10.9% at the maximum RMSE.
For WVP, the maximum difference is 0.04 mb, with a maximum difference percentage of 2.7%. The mean difference is -0.004 mb, with a maximum RMSE of 0.8 mb. The altitude of the maximum mean difference is 8.0 km, while the maximum RMSE occurs at 3.0 km, with a mean difference percentage at maximum RMSE of -1.2%.
Figure 22 illustrates the percentage difference between PlanetIQ and ECMWF profiles at 3 km altitude across various variables throughout the study duration. The pressure difference remains within ±0.2%, with minor deviations in limited areas globally. Notably, there is a marginal increase in the southern pole, indicating a more favorable agreement with the ECMWF model in terms of pressure differences compared to the GFS model.
Regarding temperature difference percentage, the ECMWF exhibits a slight escalation around the equator and poles. The maximum difference reaches ±2.5% in specific regions highlighted on the map, which is slightly higher value than the GFS temperature difference.
The WVP difference is normally less than ±0.5%, except for specific distributed areas where it may reach up to ±3%, particularly around the equator. This pattern is consistent with the GFS results.
The refractivity difference typically ranges between ±4%, with higher concentrations near the equator and decreasing toward the poles. This aligns with the distribution observed in the GFS results.
Figure 23 shows the boxplot for the RMSE across atmospheric variables to show the differences between the PlanetiQ profiles and GFS and ECMWF profiles. The RMSE values for differences between PlanetIQ observations and ECMWF/GFS models across meteorological parameters reveal notable distinctions. The median air pressure estimates for the ECMWF and GFS are around 1.67 mb and 1.82 mb, respectively. GFS has a slightly higher mean RMSE of 3.47 mb compared to ECMWF's 3.28 mb, indicating a larger overall discrepancy. The ECMWF has a moderate IQR of 0.14, while the GFS has a little lower IQR of 0.10 mb, with similar whisker min and max values.
The ECMWF and GFS have different median values for refractivity, with 0.53 and 0.61 respectively. The GFS has a slightly higher mean RMSE, 1.20, than the ECMWF's 1.11. The IQR for refractivity in GFS is notably smaller at 0.05, reflecting a more concentrated spread, whereas the ECMWF's IQR is 0.14. Whisker min and max values for both models fall within their respective IQRs.
For temperature, the ECMWF and GFS have median values of 1.05 and 1.29, respectively, with mean RMSE values of 1.21°C and 1.49°C. The ECMWF temperature data has a larger IQR of 0.60 compared to the GFS's 0.69°C. Whisker min and max values for both models extend beyond their respective IQRs, indicating the presence of outliers.
In case of WVP, the ECMWF and GFS have median values of 0.19 mb and 0.05 mb, respectively. The GFS has a lower mean RMSE (0.18 mb) than the ECMWF (0.25 mb). The IQR for WVP in ECMWF is 0.30 mb, reflecting a broader spread, while the GFS IQR is 0.26 mb. Whisker min and max values in WVP exhibit differences in distribution for both models.
In summary, the varying levels of agreement between PlanetIQ observations and ECMWF/GFS models across meteorological parameters are evident from the differences in mean RMSE, IQR, and whisker values.
To investigate the variance disparities between PlanetiQ profiles and GFS and ECMWF models, as well as the differences in observational profiles, an f-test and t-test were employed, respectively.
Table 2 presents the findings from these tests conducted on random samples of mission profiles, with six checkpoints spaced 25 to 50 days apart.
The f-test results show no significant differences in variance across all profile comparisons between PlanetiQ profiles and GFS and ECMWF models, with the exception of the GN02.2023.100.00.10.R20 WVP profile. This specific profile exhibits a noteworthy difference in variances between the profiles of the PlanetiQ and the two models.
Furthermore, the t-test findings for the six checkpoints indicate a lack of significant differences between PlanetiQ profile observations and model profiles, except for the WVP profile at the same checkpoint when compared to both models. It is crucial to note that the observed significant difference in WVP is not considered a limitation in the production of PlanetiQ WVP data. This discrepancy is attributed to small temporal and spatial variations. Given that NWP models generate spatial and temporal profiles from disparate observations, which are not necessarily in the same location and time, spatial and temporal interpolation in NWP models may introduce unreliability into the WVP profiles they produce.