1. Introduction
The value of old historical weather observations in understanding the Earth's climate system cannot be overstated. These records, often dating back centuries, offer a unique and invaluable perspective on past weather patterns, climatic variability, and extreme events. The significance of these historical weather observations lies in their ability to complement modern climate data, providing crucial insights into long-term climate trends, variability, and the drivers behind climatic shifts.
Weather data has been collected on an unsystematic basis for many centuries, often attached to weather and climate events with considerable impact on economies and societies. However, systematic observations started only a few centuries ago [
1]. The history of coordinated weather observations by an observational network dates back to more than 200 years ago when, in 1781, the Societas Meteorologica Palatina in Europe began systematic and coordinated weather observations [
2]. Many observations have been taken since then, but only few meteorological observing stations have been operated from the same place over decades or centuries without disruption [e.g. 1 and references therein]. Such long-term observing stations represent a real heritage, and their time series of observational data represent unique sources of knowledge. There is no other source of systematic historic data for analysing and understanding the status, physical characteristics and spatiotemporal variability of the atmospheric elements of the climate system [
3].
Long-term observations from meteorological and climatological stations are therefore vital inputs to reanalysis [
4,
5,
6] as well as climate models. Historical weather observations also serve as a fundamental component in reconstructing past meteorological features and put them in the context of current conditions. These records, documented in various forms such as handwritten journals, logbooks, diaries, and early instrumental measurements, offer insights into weather conditions predating the establishment of standardized meteorological networks. They contribute essential information about temperature, precipitation, wind, pressure, marine conditions and specific meteorological phenomena.
Data rescue, and specifically marine data rescue, is a process by which data from original historic documents are converted to a machine-readable format. The process is many faceted, beginning with the finding and assessment of the original documents in the archives. This is followed by scanning or photography, after which the data are keyed, either directly or through crowdsourcing, or by Artificial Intelligence and optical character recognition in the future (once the necessary technology is perfected). The resulting output is then formatted, processed and quality controlled, before being made available to the scientific community. There are good practice guidelines available through the WMO [
7] and the Copernicus Climate Services portal (
https://datarescue.climate.copernicus.eu/best_practice_guidelines) for both imaging and keying original documents, and some key points are listed below. A joint recent effort aims at merging C3S and WMO guidelines on data rescue.
The main recommendations are:
1. All original documents should be imaged in their entirety.
2. Images of the original documents should be securely preserved and made easily accessible so that the provenance of every observation can be verified.
3. It should be clearly documented as to whether an original document has been imaged, keyed and processed as this will avoid needless duplication of effort in the future.
4. Ideally, all observations should be keyed from a document.
5. Where it is not possible to key all observations, due for instance to time or financial constraints, then this should be well documented and made clear.
6. All instrument and observing metadata should be keyed as well as metadata concerning the observing platform.
Although much valuable data rescue has been performed in the past, this has often been a component and part of the output of a research project. Such projects are usually narrowly defined, meaning that the data rescue component is constrained by time and financial boundaries, as well as the data needs of the project itself. It is therefore essential that marine and other data rescue are treated as projects in themselves or even as a programme, where the sole focus is the gathering of all observations recorded in collections. In addition, a good mixture of sail and steam vessels is needed to get comprehensive spatial coverage of observations across the world’s oceans.
By rescuing, digitizing and analyzing these records, researchers can extend climate datasets far beyond the era of modern instrumental observations [
1]. This extension facilitates a more comprehensive understanding of natural climate variability and trends. These records help also identify recurrent patterns, spatial variations, and regional climate sensitivities, enabling better preparation and adaptation to future extreme weather risks. The interdisciplinary nature of historical weather observations further amplifies their significance. These records often include qualitative descriptions of weather phenomena, ecological observations, agricultural records, and societal impacts of climatic changes. Such information helps understanding climate dynamics as well, linking the interactions between climate, ecosystems, and human societies across different time and space scales. Further, historical weather observations are indispensable for assessing the frequency, intensity and duration of extreme weather events. Studying past storms, droughts, heatwaves, and cold events provides crucial context for evaluating the changing behaviour of extreme events in a warming world.
Furthermore, historical weather observations play a crucial role in validating and refining climate models. By comparing model simulations with past weather patterns obtained from historical records, scientists can assess the models' accuracy in capturing known climatic variations. This validation process strengthens confidence in future climate projections and helps identify areas where models require refinement. A recent compilation of early instrumental data [
8], which additionally contains 13822 station years of newly digitized data, is now available for climate reconstructions. Complemented with proxy and documentary data, they allow new global data products based on data assimilation [
9] for several centuries back.
Supportive international activities
Long-term, high quality and reliable instrumental climate records are indispensable pieces of information required for undertaking robust and consistent studies to better understand, detect, predict and respond to global climate variability and change [
10]. As one example,
Figure 1 shows the annual mean temperature at Hohenpeissenberg, Germany, covering the period 1781 to 2022. Maintaining the operation of historically uninterrupted stations and observing systems has been acknowledged as one of the key principles of climate monitoring [
11,
12].
In 2013, the WMO Executive Council urged Members to sustain observation programmes in support of centennial observations (
Figure 2, exemplified with the Sonnblick Observatory, Austria) as an invaluable scientific heritage for future generations. The Council requested WMO Technical Commissions to investigate existing site certification mechanisms, network criteria and monitoring principles and to set up an appropriate WMO mechanism for the recognition of centennial observing stations, based on a minimum set of objective assessment criteria [
13].
Based on the outcomes of a WMO Scoping Meeting on a Potential WMO Recognition Mechanism for Centennial Observing Stations in June 2014, the 17th World Meteorological Congress decided to develop a recognition mechanism for long-term observing stations, including centennial observing stations, and the possibility of intermediate-level certification for 50 years and 75 years of observations [
14]. Following the successful conduct of a test phase, showing that 34 Members representing all six WMO regional associations had responded and submitted 79 candidate stations, Executive Council decided to endorse the mechanism and criteria for WMO recognition of (meteorological) long-term observing stations [
15]. The first set of 60 Centennial observing stations had been endorsed by WMO Executive Council in 2017 [
16], followed by a second set of 57 Centennial observing stations in 2018 [
17]. A dedicated WMO Website related to Centennial Observing Stations was implemented and has been updated regularly since then (
https://wmo.int/centennial-observing-stations). In 2019, WMO experts held a meeting to further develop the WMO recognition mechanism by analysing the experiences made so far. Consequently, the initial WMO recognition mechanism and its criteria had been refined in 2020 and 2021 [
18,
19] and the mechanism broadened in 2023 to include centennial marine and hydrological observing stations and a possibility to nationally recognize 75+ years stations [
20]. In parallel, another 293 centennial observing stations have been recognized by World Meteorological Congress and Executive Council [
18,
19,
21] and the first edition of a series of State of Recognition reports have been published in 2022 [
3]. All in all, 406 Centennial observing stations have been recognized by summer 2023 (10 centennial marine observing stations, 22 centennial hydrological observing stations and 372 centennial meteorological observing stations).
Long-term observations greatly contribute to WMO flagship products, such as the annual global and regional State of the Climate reports, which provide scientifically sound, reliable information for policymakers and decision makers. WMO has produced the annual State of the Global Climate report since 1993 (
https://wmo.int/publication-series/state-of-global-climate), which is now complemented by regional reports. Global estimates and analyses require both in situ data and historical observations provided by WMO Members. Among these records, historical marine data represent a treasure trove of information that has the potential to significantly enhance our understanding of climate dynamics.
Historical marine data encompass a rich array of information gathered from ships' logs, scientific expeditions, and marine observations dating back centuries. These data contain invaluable observations of sea surface temperatures, weather patterns, ocean currents, ice cover, biological phenomena, and more. However, much of this historical marine data remains scattered across archives, libraries, and repositories worldwide, often in fragile or deteriorating formats. The rescue and digitization of these invaluable records represent an urgent priority for climate researchers. By rescuing, digitizing, and standardizing these historical marine datasets, we can unlock a treasure trove of information, enabling scientists to extend climate records further back in time and expand spatial coverage. This effort holds immense promise in refining our understanding of past climate conditions and improving the accuracy of climate models used for future projections.
This contribution builds and expands on the recent publication of [
22] and others, highlighting some examples of new data sources, regional data activities and the need for good metadata, high standards and quality control of historical marine weather observations covering the past centuries. Much of this has been made possible by the international ACRE (Atmospheric Circulation Reconstructions over the Earth,
www.met-acre.onet) initiative and its specific ACRE Oceans chapter, with strong links to the International Comprehensive Ocean-Atmosphere Data Set (ICOADS), the Global Surface Air Temperature (GloSAT) (
https://www.glosat.org/) projects and Copernicus Climate Change Service (C3S) and the associated UK funding.
Examples of historical marine data efforts across the globe covering the past centuries
ICOADS is the prime data source for observations of marine air temperatures, sea-surface temperatures, sea level pressure and several other “essential climate variables (ECVs)”. The data coverage spans the globe and extends back in time to the late eighteenth century, albeit with increasingly sparse data coverage further back in time. ICOADS is the main source of observations for the marine component of atmospheric reanalyses. Its importance and utility cannot be overstated.
Although ICOADS holds a vast number of marine observations, these are not the entirety of what is available. There are many sets of original documents, such as ship logbooks, hydrographic reports, and a host of related material that is yet to be digitised and aggregated. These documents are to be found in naval and maritime libraries and museums, national archives, research institutes, and the archives of national weather services around the globe. Many such collections have been documented and catalogued but presently remain undigitized and thereby unavailable to the scientific community. Billions of observations are currently lost to science and need to be prioritized for their utility for different applications (see Table 5 in [
23]).
Furthermore, many of the collections and source datasets (termed as decks as many were derived from data stored on punch cards) that are already incorporated into ICOADS are incomplete or fragmented. This is due to the gradual accretion of observations in ICOADS over past decades, from a diverse range of marine datasets, produced by other agencies. These datasets may have been produced by researchers or agencies to answer specific scientific questions, rather than to gather a broad range of data. Thus, these datasets tend to be spatially specific, or only contain certain subsets of observations, whereas the source documents may be much broader in scope.
Recent advances in Historical Marine Data Rescue:
A prime example of this is ICOADS Deck 201, the UK Marine Data Bank 1850-1920. The observations in this deck are based on the collection of ships’ meteorological logbooks held by the UK National Meteorological Archive in Exeter. There are approximately 15,000 sets of logbooks covering this period and although not all of these logbooks are in Deck 201, many of them are. However, none of the more than 10 million sub daily pressure measurements in these logbooks have been keyed and are therefore not in ICOADS. Furthermore, air and sea temperatures have only been selectively keyed, conforming to spatially specific areas (see
Figure 3 showing data coverage from ICOADS deck 201 during the 1870s). The plot shows that large parts North Atlantic and Indian Ocean are devoid of observations and the South Atlantic has been omitted throughout the 1850-1920 period. Other parameters such as winds are similarly compromised, and observations of specific gravity and ice have not been keyed. Other ICOADS decks have not yet been subject to similar scrutiny or comparison with their original documents, but it is likely that other decks are also compromised in a similar way.
In addition, there is frequently an absence of good instrument and observing metadata. Sometimes the metadata are missing from the original documents but often metadata is lost through the use of compact data formats or because it was not thought to be important. The inadequacies are entirely an artefact arising from the original documents themselves or more often, the project that produced a particular ICOADS deck, and in no way reflect on the achievement of ICOADS itself. There are valuable lessons to be learned here. Current and future marine data rescue must, and is, ensuring that some of the issues raised above are addressed.
Some specific marine data sets and regional foci; convict and settler ships sailing to Australia and New Zealand
Convict Ships
From 1788 to 1868, some 806 ships sailed from England to Australia transporting male and female convicts. On board such vessels were Royal Navy surgeons and assistant surgeons who were required to compile and submit at the end of each voyage, their journals and diaries detailing the medical health, treatment and survival of the convicts during their journeys.
Some of the above surgeons also made, and documented in considerable detail, the various meteorological observations that they made during their passages to the Antipodes. Depending on the instruments they possessed themselves and their interests, these observations ranged from a few annotations of observed air temperatures and pressures in page margins, or between descriptions of medical activities (see example in
Figure 4), through to pages of tabulations of coarse monthly averages to detailed sub daily measurements of their ship’s latitude, longitude, course, wind direction and magnitude, internal hospital and/or deck air temperatures and barometric pressure. The latter that have survived, and passed quality control tests, are proving to be often the only records of instrumental weather at the time and at locations in the South Atlantic, southern Indian and southern Australian waters.
At the end of these voyages, land fall in Australia was made either at Port Jackson near Sydney in New South Wales or Hobart in Tasmania. In the latter years of transportation following the Crimean War (1853 to 1856), most voyages ended at the port of Perth at Fremantle in Western Australia.
In the above online scans, the international ACRE initiative was able to isolate instances of daily to sub daily instrumental meteorological observations made by ship surgeons on 158 convict ship voyages between 1817 and1868. Of the observations that were found and digitised, 90 ships made only air temperature and 53 ships made both air temperature and barometric pressure observations. The logs with the remaining observations, though said to have been scanned, have so far not been found.
An example of the once daily air temperature and barometric pressure observations made by a ship’s surgeon along the track of a convict vessel sailing to Australia from England, is shown in
Figure 5, in this case the ship Albion in 1828. Instances where strong dips in the barometric pressure were observed could be matched with written accounts of severe weather and storms in the surgeons’ journal, proving a basic initial check on the validity of the pressure record. On this occasion, no mismatches of observed barometric pressure falls and entries of severe storms were observed.
Settler Ships
In the British establishment of its colonies in Australia and New Zealand during the first half of the 19th century, what might be termed settler ships transported British migrants, who occasionally had some scientific, natural history or official administrative background and made daily to sub daily meteorological observations using thermometers and barometers. Two examples are shown below for HMS Buffalo, which transported the first European settlers to the state of South Australia in 1836 (
Figure 6), and the ship Tory, which carried early European settlers to New Zealand in 1839 (
Figure 7).
As with convict ships, the weather observations from settler ships often provide the only records of instrumental weather at the time and at locations in the South Atlantic, southern Indian and southern Australia waters. By the mid to later 19th century, as the first cruise ships begin to make journals between Europe and the Antipodes, weather observations, usually extracted from the ship’s logs during voyages, begin to appear in some of the on-board ship newspapers. These also begin to compliment the weather observations made on other, often regular, voyages of mail or packet ships, cable laying ships, and yachts travelling around the world.
The Mauritius Project: Historical weather observations extracted from ship logbooks
The Mauritius Project, which took nearly 8 years to come to fruition, and has involved the international ACRE initiative partnering with the Meteorological Society of Mauritius (in conjunction with the Mauritius Meteorological Services) in order to recover, scan/image, digitise, archive, and preserve old terrestrial and marine weather observations held in the National Archives of Mauritius and the Mauritius Meteorological Services. These are specifically:
- 1)
Observations extracted from ship logbooks in 188 volumes of Charles Meldrum's 'anemological' journals from 1853 to 1914.
- 2)
Ship logbooks from 1848 to 1874.
- 3)
Terrestrial weather observations for Mauritius, Le Réunion, Rodrigues, Seychelles and Diego Garcia Islands (including data from Colonel Lloyd's Colonial Observatory at Port Louis) from the late 18th to the early years of the 20th century.
The 'anemological' journals have been the initial focus of the project and contain important historical ship weather observations from vessels travelling around southern Africa on the old shipping routes through Mauritius to India, China, and Australia in the period 1853 to 1914. This material also contains Indian Ocean island station records from Mauritius, Le Réunion, Rodrigues, the Seychelles, and Diego Garcia in the second half of the 19th and early 20th centuries. The collection includes ship information, location data and a variety of meteorological parameters. These are once or twice daily records from vessels travelling across the Indian Ocean. A later focus on the ship logbooks from 1848 to1874, will add to the above.
The scanning and digitising effort from 2021-2023 was undertaken at the National Archives of Mauritius with funding from the UKMO Newton Fund Climate Science for Service Program (CSSP) China via ACRE to the Meteorological Society of Mauritius and the Mauritius Meteorological Service. A sample of the scan and digitised data from the day of the 2nd of February 1879 in the 'anemological' journals is shown in
Figure 8 and
Figure 9 respectively. Note that only the instrumental weather observations on the LH side of each daily journal entry were digitised.
Some of the data were digitised by ACRE/Copernicus Climate Change Service Data Rescue Service (C3S DRS)/UKMO Newton Fund Weather and Climate Science for Service Program (WCSSP) South Africa. With this funding, the weather observations in the ‘anemological' journals in some months of 1853, and for each year from 1859 to 1900, have been scanned, digitised and quality controlled (1876 data are still being finalised). The years 1854-1858 and 1901-1914 have yet to be completed due to the loss of funding after March 2023. The report on the project up to the end of March 2023, when the above funding finished can be found at
https://www.dropbox.com/scl/fi/vsygk3ovuiv6tqobcbmup/WCSSP_SA_End-of-Contract_Report-2023-c.docx?rlkey=iusume6qrferdw143h8674x2x&dl=0. There is also the potential to provide considerable additional information on the above ships using the listings of arrivals and departures of vessels at and from Port Louis on Mauritius in monthly tabulations in the Mauritian newspapers of the time. These detail ship names, nationality, tonnage, captain’s name, arrival date, where from, cargo, agents, departure date, where bound, cargo, agents, observations when in harbour (e.g., loading).
One particularly interesting finding in preliminary investigations of the digitised data, that was gleaned in conjunction with an examination of the cargo listed for each vessel in the Mauritian newspapers during the 1870s period, were ships sailing to the wider Indian Ocean, with a stop in Mauritius, which were involved in the Guano trade. There were some 50-60 ‘Guano’ vessels identified in this initial probing of the 1870s portion of the data set, that sailed from South America to Mauritius, travelling around Cape Horn across the southern Atlantic then around the Cape of Good Hope and South Africa. The portion of their route across the southern Atlantic Ocean is unlikely to have been traversed by any other vessels in a quasi-routine manner in such a period, making the observations made on such voyages extremely valuable in filling a significant gap in the data coverage at these times. This can be seen in the two examples shown below for January to February 1871 (
Figure 10) and July to October 1871 (
Figure 11), where each vessel’s passage is displayed on each map along with a plot of the daily air temperature and barometric pressure observations in the bottom LH side of each diagram. Passages of this nature at such mid to high latitudes around Cape Horn and the South Atlantic in Southern Hemisphere summer would have been taxing on the ship and crew, but doing so during the Southern Hemisphere winter would have been outright precarious. The time taken to make these similar voyages in distance is also indicative of open ocean weather conditions in each season - in summer the passage took just short of 2 months (55 days), while in winter the passage lasted over 2 and a half months (68 days). This work on the Guano ships will be extended to investigate such vessels in the full 1853-1914 journal data base.
Selected further ship logs
The National Archives (Kew, Richmond), The National Meteorological Archive (Exeter), The UK Hydrographic Office (Taunton), The Institute of Maritime History at Åbo Akademi University (Turku) and The Åland Maritime Museum (Mariehamn)
A plethora of logbooks from ships of the Royal Navy in the 19th century can be found in the maritime archive collections of a) The National Archives in Kew - Richmond, b) The National Meteorological Archive in Exeter and c) The UK Hydrographic Office in Taunton. These collections include a variety of ships’ logbooks, weather books, meteorological registers, private weather diaries, composite and individual remark books and miscellaneous papers. The ACRE/UKMO Newton Fund Weather and Climate Science for Service Program (WCSSP) South Africa facilitated the preservation of these archives with the scanning/imaging and digitization of the aforementioned logbooks, as well as the quality control of the digitized data. These logbooks cover the following time periods:
- 1)
The National Archives (134 completed logbooks) – from 1832 to 1833, from 1853 to 1880 and from 1898 to 1899
- 2)
The National Meteorological Archive (7 completed logbooks) – from 1849 to 1882
- 3)
The UK Hydrographic Office (46 completed logbooks) – 1816, from 1823 to 1825 and from 1844 to 1868
However, there are 9 logbooks from The National Archives (years 1856-1857, 1863-1866 and 1899-1901), 6 logbooks from The National Meteorological Archive (years 1856-1857, 1862, 1867-1868 and 1891-1892) and 24 logbooks from The UK Hydrographic Office (years 1862-1865) that have not been completed due to the loss of funding after March 2023.
Additionally, there is also an extensive archive of Finnish logbooks (written in Swedish) derived from The Institute of Maritime History at Åbo Akademi University in Turku and The Åland Maritime Museum in Mariehamn. Some of these logbooks have also been scanned/imaged and digitized. More specifically:
The Institute of Maritime History at Åbo Akademi University – 15 completed logbooks from 1850 to 1899 and 3 remaining logbooks (years 1862-1863, 1876-1877 and 1899-1901).
The Åland Maritime Museum – 2 completed logbooks (1853 and from 1880 to 1882)
These ships travelled from England to South Africa, China, Japan, Philippines, and Malaysia, as well as from Finland to South Africa. The duration of the voyages lasted from several months up to three years. During travelling the vessels’ crew recorded daily route information (longitude - latitude), remarks regarding the ship and the voyage (employment, deaths on board, ship damages and maintenance etc), meteorological parameters, observed weather and other events. However, the handwritten nature of the logbooks (calligraphy and different writings in the same logbook) made the recordings hardly readable. The meteorological observations usually refer to wind (speed and direction), barometric pressure, and air and sea temperature. During sailing the meteorological observations were performed hourly or every few hours, while when at anchor the observations were performed every two hours.
Figure 12,
Figure 13,
Figure 14 and
Figure 15 are examples of the vessel HMS Argus (The National Archives) that cruised in 1869 from Japan to England.
Old Weather-WW2 and Weather Rescue at Sea
Two projects which used citizen science to recover millions of marine weather observations are now discussed. Old Weather-WW2 rescued historical weather observations from United States Navy (USN) ships during World War 2 (WW2), and Weather Rescue at Sea (WRS) used UK naval logbooks to fill the gap in observational datasets in the 1860s. Both projects harnessed the cumulative power of crowd-sourced transcription to data-rescue historical observations.
Old Weather-WW2
All climate reconstructions show that the global oceans have warmed since the start of the 20th Century, but there is anomalous warmth in global mean SSTs during the WW2 period (between 1941 and 1945) when compared to the preceding and following 5-year periods (Chan and Huybers, 2021). Also, the uncertainty in the estimated anomaly for this period is several times larger than for more recent periods.
Several possible explanations have been put forward to account for this anomaly, referred to as the WW2 warm anomaly (WW2WA) by previous studies, such as the reduced number of observations [
24,
25] and changes in the types of SST measurement [
26,
27]. When WW2 commenced, trade routes were severely disrupted, limiting observations taken by voluntary observing merchant ships (VOS) which usually crisscross the global oceans. This caused a large drop (58%; [
24] in the number of marine observations available for the duration of WW2.
More crucially, poorly documented changes in the observing practices may have led to large biases and errors. For example, the preference for taking SST measurements from the inlet water pipes used to cool engines (known as Engine Room Intake, ERI), in contrast to hauling canvas/wooden buckets onboard, resulted in a warm bias in the aggregated SSTs [
28]. The rapid rate of these transitions is not always well documented and can be mis-labelled which impedes the correct adjustments being applied to the observations [
25]. Another practice changed during WW2 was that more observations were taken during daytime than night-time. Both of the above changes are assumed to be due to the need to reduce exposure to the enemy ships and avoid being detected [
25,
29]. Without additional data and documentation of prevailing practices, disentangling the reasons for the WW2WA is very difficult.
Most of the marine observations taken during WW2 were on board naval ships of various countries. However, many observations were destroyed as an act of war, or simply forgotten due to the length of time they were considered classified. To fill gaps in observational coverage and contribute to improving metadata regarding observing practices, the NOAA-funded project ‘Old Weather: World War 2’ gathered thousands of volunteers to transcribe weather observations from logbooks of US destroyers and other naval ships which were part of the US Pacific fleet based at Hawaii. These ships saw action in the Indo-Pacific and Far-East including the Pearl Harbour attack, taking observations at times and places where few or no other digitised observations exist.
In 2017, the National Declassification Center (NDC) at the National Archives and Records Administration (NARA) released nearly 200,000 pages of formerly classified U.S. Navy Command Files from the WW2 era. The files consisted primarily of records from the Pacific Theatre between 1941 and 1946. The files contain many kinds of documents, maps, ship logbooks, photographs etc. Here we focus on the ship logbooks containing meteorological observations (
Figure 16).
A dataset of more than 3.7 million observations has been rescued [
30]. The dataset has more than 630,000 unique records, where each record contains the date and time, positional information and one dry-bulb temperature (Tdry), wet-bulb temperature (Twet), Twater (SST=sea surface temperature), barometer-attached thermometer temperature (Baro At. therm.) and pressure observation. There are 611,223 observations of air pressure, 197,716 observations of Baro At. therm., 601,978 observations of dry bulb temperature (Tdry), 604,155 observations of wet bulb temperature (Twet), and 314,713 observations of SST. There are an average of 7000 records per ship per year, and each ship logbook has observations for around 300 days per year on average. All ship tracks are supported by documentary evidence about the ships’ movements from other sources (Cressman, 2000). Over the 5-year period, the various ships travelled across the Pacific, Indian and Atlantic oceans, providing a rich dataset all across the globe (
Figure 17).
As an example of the data available,
Figure 18 shows the track of USS Pennsylvania during the 1941-1945 period. During 1941 and 1942, the ship travelled between San Francisco and Pearl Harbor. In 1943, it made trips to the Aleutian Islands near Alaska, Marshall Islands, and Guam in the Pacific. For the year 1944 meteorological observations are present but navigation data is missing, hence the year is empty. In 1945, it travelled to Papua New Guinea and Philippines and other islands in the South China Sea from Pearl Harbour. It then reached Puget Sound Naval Shipyard in Washington towards the end of 1945. The meteorological observations of pressure and Tdry closely reflect the regions travelled.
Figure 18 also shows the track of USS Tennessee over the 1941-1945 period. During 1941, the ship travelled to Pearl Harbour from San Francisco, reaching Puget Sound Naval Shipyard in Washington at the end of the year. 1942 was spent completing various exercises off-California and in the seas around Hawaii. The years 1943, 1944 and 1945 were long-distance trips, first to Aleutian Islands, then Fiji, Marshall Islands, and Philippines. In 1945, it started from the Naval Shipyard in Washington and travelled to the southern coast of Japan via Hawaii, and also included multiple trips to the Chinese coast. Starting from Japan, the ship then visited Taiwan, Singapore, Sri Lanka, Cape Town, finally reaching New York, completing a circumnavigation.
Several studies have highlighted severe Dust Bowl droughts and heat waves in North America during the 1930s, followed by a strong 1939–1942 El Niño event which had significant impact over the globe. The El Niño during 1939-1942 led to extremes in global climate anomalies, including cold winters in Europe, warm winters in Alaska, wet springs in central Europe, and a drought in Australia. However, our understanding is partially complete due to severely limited coverage of observations for the WW2 period; the presented dataset in this study can help fill-in some of the gaps.
Weather Rescue at Sea
Observing and following the weather through the changing seasons was crucial to survival in the pre-industrial era. It was more so for those who spent long periods of time on-board ships travelling across the globe. In the age of sail, knowledge of winds and currents was crucial to reach their destinations safely and on-time. Out of practical necessity, gradually, maritime nations developed several weather observing instruments and procedures to record the weather encountered on long sea journeys. And, in 1854, a maritime conference of sea-faring nations tried to codify observational taking, and record keeping helping to standardise and share observations among themselves [
31]. That process amassed an enormous number of 'standard' logbooks containing detailed sub-daily weather observations at sea from around the globe.
There is a strong scientific interest in understanding the climate of the early industrial era against which our present climate could be measured, in order to assess anthropogenic impact on climate change. As large parts of the globe are covered in ocean, many previous studies have used historical marine observations to estimate these changes in the climate. The CLIWOC project [
32], a multinational study, systematically collected, extracted and analysed UK-Spain-Dutch ship logbooks before 1850. Brohan et al. [
33] produced a substantial number of historical data from English East India Company ship logbooks starting from 1789 and ending in 1834. They produced more than 200,000 records containing three meteorological variables (temperature, pressure and wind), giving unique insight into historical climate. This study provided further evidence that historical ship logbook observations can be used to study climate variability when land-based observational networks are not dense enough.
To further the development of reconstruction of past climate by enhancing the data available to them, the international ACRE initiative [
34] initiative coordinates various data-rescue efforts and communities. One of the narrowest bottlenecks of historical data extraction has been a lack of reliable and efficient automated processes to deal with hundreds of thousands of weather journals and ship logbooks which are written by hand. Many new archives have been located, catalogued and photographed by the data-rescue initiatives. However, there is at least as much data to be rescued as are currently available in digital archives for the period prior to 1950 [
22].
Data rescue (transcribing hand-written observations into computer readable digital format) of historical logbooks has been taking place for decades, but to manually transcribe an almost inexhaustible number of logbooks by individual researchers, would take thousands of human lifetimes. As a result, large gaps have remained in our knowledge of the climate, both in space and time. The 19th Century has fewer observations available than the 20th Century in the world's largest observation meteorological dataset, ICOADS version 3 (International Comprehensive Ocean-Atmosphere Data Set, [
24]). On closer inspection, the average number of monthly observations and percent of global coverage in the 1860s and 1870s is relatively poor compared to other decades after 1850.
For the volume of data contained in the collection described here, a traditional manual transcription approach would have taken many person-years of effort. Instead, the availability of scanned images of the ship logbooks enabled the creation of a science project that asked volunteers to transcribe the observations into digital form more efficiently.
The Zooniverse platform (
www.zooniverse.org) offers a flexible framework upon which various citizen science projects have been built. Many different themes are represented on the platform, from astronomy, biology, ecology and conservation to historical documents. The original Old Weather project was one of the first projects to extract historical weather observations contained in ship logbooks from an extended period around WW1. Since then, many projects have successfully used Zooniverse to digitise historical weather observations, e.g. WeatherRescue.org [
35,
36], RainfallRescue.org [
37], SouthernWeatherDiscovery.org [
38], Climate History Australia [
39], and Meteorologum ad Extremum Terrae [
40].
Within this context, the Weather Rescue At Sea (WRS) project has used the citizen science based Zooniverse platform to recover some of these observations and make them usable, with a focus on ships travelling through the Atlantic, Indian and Pacific ocean basins in the 1860s and 1870s. The focus has been on logbooks archived at the UKHO (UK Hydrographic Office) that are best suited to produce data in the targeted time period with global coverage (
Figure 19). Filling in the gaps in our knowledge will remove ambiguity in how the climate varied historically in many regions where observations are currently poor or non-existent. The data generated through this project will also help to fill many crucial gaps in the large climate datasets (e.g., ICOADS) which will be used to generate new estimates of the industrial and pre-industrial era baseline climate. But more generally, this data and data from other historical sources are currently used to improve the models and reanalysis systems used for climate and weather research.
So far, a total of 248 logbooks have been used in the project, totaling 25,000 images covering the 1860s and 1870s. More than 3000 volunteers contributed to the transcription process, the post-processing work of error corrections and consensus checking is still on-going. So far, we have processed ~44,000 records containing navigational and meteorological observations.
Figure 20 shows a snapshot of all ship tracks processed so far.
Finally, we highlight two of the main lessons learned from both the above Old Weather-WW2 and WRS projects. Firstly, the design of transcription workflows should reflect the structure of the logbook page. Providing context about the logbook pages, the purpose of the project, and where the data would be used, these all helped to motivate the volunteers. Secondly, information requiring transcription should be grouped together into workflows, e.g., positions, zones, dates and particular weather types (see [
30]).
National Institute of Water and Atmospheric Research (NIWA) activities
Initial work undertaken under ACRE focused on identifying marine observations from ships to corroborate early land-based pressure observations in New Zealand [
41]. Subsequently, support through the Deep South National Science Challenge was used to identify ship-based weather observations for the region south of New Zealand to Antarctica across the Southern Ocean that would improve the 20
th Century Reanalysis. This work was undertaken in a project called Southern Weather Discovery [
38], which had a primary focus of setting up data transcription workflows established by other leading data rescue projects (e.g. Weather Rescue), evaluating efficacy of AI for data transcription and determining optimal data keying replication standards for quality control. The latest data rescue efforts in New Zealand are currently focused on two fronts; securing digital surrogates of formal observation forms in National Institute of Water and Atmospheric Research (NIWA) archives and the recovery of ship log observations recorded on sub daily synoptic weather maps compiled by the New Zealand Meteorological Service. The work at NIWA archives was initiated in 2009 and is nearly completed, and the main document targets are first class climatological stations and third-class manual rainfall observation stations. The former is the highest priority as the sheets contain essential climate variables that are regularly reported on in terms of extremes and trends. Pressure observations from the first-class climatological stations have been aperiodically supplied to the International Surface Pressure Databank (ISPD) via the ACRE Pacific chapter. An exchange of materials held in UKMO (United Kingdom Meteorological Office) archives has recently helped to fill time gaps for the earliest official record in Auckland, which will overlap with several ships including the HMS Pandora, that undertook the first hydrographic survey of the colony. Critically, missing data sheets supplied by ACRE are now being used to test the validity of reported low pressure observations related to 19th century storms believed to be of ex-tropical origin, giving a longer historical context for recent impacts of tropical cyclone Gabrielle. The marine observations from this time period will be valuable for gap filling the land-based observation record which becomes more robust from the 1870s onward.
There are limited reports from ship logbooks that can provide a wider spatial context for the origins of former storms that impacted the southern mid latitudes. However, a recent trove of historical sub daily synoptic maps for New Zealand and the surrounding oceans were transferred from NIWA to the National Archives of New Zealand in Mangere, Auckland. These historical maps are in oversize format and bound in leather, requiring digital photography for capture. They are important because they contain observations that were sent by wireless telegraph from ships to the mainland, are not likely to be found in other sources, and therefore the most current extended reanalyses without radiosondes. The main interest in obtaining these historical maps, and the marine weather observations on them, is to evaluate the occurrence and origin of storms that were characterised by deep low pressures that impacted New Zealand prior to the 1940s.
Weather and Climate Science for Service Partnership South Africa (WCSSP South Africa)
The value of non-digitised marine data for producing reanalyses was demonstrated in two UKMO-Newton Fund WCSSP projects which aimed at digitising marine data in the Southern Hemisphere during two climatically important periods. The first project targeted the 1876-1878 period (
Figure 21, top), which was characterised by a very strong El Niño and was arguably one of the deadliest climate events in history. The project digitised climate data from the logs of 20 ships that cruised the South Atlantic and Indian Ocean during these years (Brugnara et al., 2023). Note the many curved ship tracks which are typical for sailing ships and provide a different coverage than the steam ships that are on a more linear track.
Using an offline data assimilation approach, it was then shown that assimilating these data into 20CRv3 would increase the skill of the product [
42]. The second project targeted a period around 1910, during which global temperature reached a decadal minimum. The causes of this anomaly are not well understood. Digitising data from 13 ships in the period 1902-1916 (
Figure 21, bottom) has contributed to a better understanding of this anomalous period, although many questions remain open (note the more linear tracks of the steamships during these years, also this period still saw a good coverage of the Southern Ocean, which changed rapidly after 1914 with the opening of the Panama Canal).
The data confirmed that the cold period is not an artefact of biased ship data but must be understood as an unusual combination of external factors (volcanic eruption of Santa Maria in 1902, perhaps Novarupta in 1911) and internal variability (La Niña, cold South Atlantic and Indian Ocean). Again, offline data assimilation was performed to demonstrate the usefulness of the data. The assimilation confirmed and strengthened the circulation anomalies that are already seen in 20CRv3 before assimilating the new data, namely a positive Southern Annular Mode [
43]. Much more data will be available for this period and awaits digitisation. This should lead to greatly improved reanalyses in future product cycles.