Wednesday, 30 September 2015

HadISD version 2

The paper describing HadISD version 2.0.0 has just appeared in the Discussions section of Climate of the Past:

There now follows 8 weeks of reviewing process. Two anonymous referees will be asked to make comments, which will appear online, and also anyone can make attributed comments (i.e. under their name) which will also appear.  After that we will have the chance to respond (these will also be published) and then the final paper will appear thereafter.  Once all that is done, then we can release the dataset and also the quality control code.  Hopefully this will all go through before the end of the year so that I can also run an update in January to v2.0.1.2015p.

For those of you who follow this blog, then a number of the sections in the paper will be familiar, however, the jist of the paper is expanding the time coverage of HadISD from 1973 back to 1931.  At the same time we've readdressed the way stations are selected and merged, and so v2.0.0.2014f has 8113 stations, with around 2000 of these being composite.

As part of the creation of HadISD.2.0.0, we have also re-written all code into Python for ease of use - and as such we were able to check and in some cases alter some of the QC tests to work a bit better.  We have also added new checks for wind speed and direction.

We believe that the result of these changes are that HadISD.2.0.0 is a more useful dataset for the study of extreme events, but also model validation, for ingestion into reanalyses and many other applications.

Update: January 2016

After some useful review comments from the referees and a discussion with the editorial team, it was suggested that we re-submit this paper to Geoscientific Instrumentation, Methods and Data Systems a partner journal of Climate of the Past.  As we are currently updating HadISD.1.0.4, we will do this once all the annual dataset updates are complete, and at the same time update HadISD.2.0.0 to include data from 2015.  We aim to resubmit this in early spring.