Continuing this took longer than planned, so there has been another version release of HadISD (v126.96.36.199107p) in the meantime.
In the last post, we went through the changes that were made to the Climatological Outlier check as a result of the temperatures experienced in North America in June 2021. Since then, there have continued to be heatwave events across the world, with temperatures and impacts around the Mediterranean being the current focus (at time of writing). We will continue to use the North American heatwave for these changes for consistency, but note that of course changes to our QC will affect all stations and variables, and hence events.
Distributional Gap Check
In this check there are in fact two. The first uses monthly aggregated data, to look for asymmetries in the distribution, and we haven't changed that one. The second is what we delve into here, which uses all observations within a calendar month, and identifies gaps in this distribution to decide where to flag. We'll use the same station for our plots as in the previous post, 711130-99999 (Agazziz, BC, Canada).
As we use a very similar approach in this test, we also had the same issue where our diagnostic plots initially were not showing data from the incomplete calendar year. But that was an easy fix, see Figure 1.
As for the climatological check, we treated the complete and incomplete years separately, which meant that these observations were now tentatively flagged, which can be unset by the neighbour check (Figure 2).
The final thing that we wanted to change was the nature of the curve being used to fit the distribution. When putting this code together, we wanted to include skew and kurtosis, as the distributions were clearly non-gaussian. At the time, we used Gauss-Hermite polynomials to obtain the fit with these higher moments of the distribution. However, we have since found that these sometimes have artefacts which result in some "wiggles" in the distributions (see Figure 1). Although this approach is still useful for gauging where to start looking for gaps in the distribution, but we thought that this was an opportunity to see what else could be done. We tried using the same skewed distribution (no kurtosis) as for the climatological outlier check.
|Fig. 3: Same as Fig. 2, but now using a skewed-Gaussian ditribution for the fit rather than the Gauss-Hermite polynomials.|
For this month, it is a more sensible fit, and also has a co-benefit of moving the value from which this test starts searching for a gap to the right, and so includes all of the hot temperatures in June.