Obtaining Historical Weather Data from Qld Government DES Website

Quick Start

Install the latest stable version of cropgrowdays via CRAN with:

install.packages("cropgrowdays")

You can install the development version of cropgrowdays from GitLab with:

## if you don't have 'remotes' installed, automatically install it
if (!require("remotes")) {
  install.packages("remotes", repos = "http://cran.rstudio.com/")
  library("remotes")
}
install_gitlab("petebaker/cropgrowdays", build_vignettes = TRUE)

Overview

The cropgrowdays package provides functions to calculate agrometeorological quantities of interest for modelling crop data. Currently, functions are provided for calculating growing degree days, stress days, cumulative and daily means of weather data. Australian meteorological data can be obtained from Queensland Government’s Department of Environment and Science (DES) website. In addition, functions are provided to convert days of the year to dates, and vice-versa.

We recommend using the cropgrowdays package in conjunction with the tidyverse and lubridate packages. Additionally, we also recommend using the furrr package to speed up adding agrometeorological variables to large data frames. For this document, we only use the lubridate package as follows.

suppressMessages(library(lubridate))
library(cropgrowdays)

Note that if you are not familiar with the lubridate package, then in order to see which functions are provided and which functions conflict with other packages, initially it may best not to suppress messages using suppressMessages.

Obtaining Australian SILO Meteorological Data for One Site

You can use the get_silodata function to retrieve SILO weather data from the Queensland Government DES longpaddock website https://www.longpaddock.qld.gov.au.

The SILO (Scientific Information for Land Owners) is a database of Australian climate data hosted by the Science and Technology Division of the Queensland Government’s Department of Environment and Science. These datasets are constructed from Australian Bureau of Meteorology observations and provide national coverage with interpolated infills for missing data. Weather station data is the observed data while the gridded data is interpolated. Typically, for most variables, weather data can be obtained for the period 1 January 1889 to yesterday. Please see https://www.longpaddock.qld.gov.au/silo/about/overview/ for a more in-depth description.

SILO products are provided free of charge to the public for use under the Creative Commons Attribution 4.0 license. However, please note that this is a free service and so a fair-use limit is imposed even if exact limits are not specified.

The boonah dataset contains meteorological SILO data for the period 1 Jan 2019 to 31 May 2020 obtained from the Longpaddock Queensland Government DES web site https://www.longpaddock.qld.gov.au for Boonah which is located at -27.9927 S, 152.6906 E. The data is in APSIM format and contains temperature, rainfall, evaporation and radiation variables and the rows are consecutive days during the period. The weather data set was obtained using

boonah <-
   get_silodata(latitude = "-27.9927", longitude = "152.6906",
                email = "MY_EMAIL_ADDRESS", START = "20190101", FINISH = "20200531")

To obtain gridded data, which is what get_silodata assumes, you need to supply at least the site latitude and longitude as well as your email address by replacing MY_EMAIL_ADDRESS with your email address. The data is freely available under the Creative Commons 4.0 License. Note that SILO may be unavailable between 11am and 1pm (Brisbane time) each Wednesday and Thursday to allow for essential system maintenance. Also please note that, by default, apsim data are retrieved. Most, but not all, of the other formats are also available. See the help for get_silodata for details. You can obtain this help using ?cropgrowdays::get_silodata at the R Console prompt or using your favourite help system.

The data obtained is

## weather data object
print(boonah, n=5)
#: # A tibble: 517 × 10
#:    year   day  radn  maxt  mint  rain  evap    vp   code date_met  
#:   <int> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>  <int> <date>    
#: 1  2019     1  26.2  33.9  16.3     0   7.8  20.6 222222 2019-01-01
#: 2  2019     2  28.2  33.4  17.6     0   7.7  19.8 222222 2019-01-02
#: 3  2019     3  20.5  32.8  16.7     0   6.8  21.9 222222 2019-01-03
#: 4  2019     4  23    32.5  21       2   7.7  22   222222 2019-01-04
#: 5  2019     5  27    33.6  16.8     0   6    21.8 222222 2019-01-05
#: # ℹ 512 more rows

Finally, please note that by default, the APSIM format is returned and a date column called date_met is appended to the retrieved data. Many other formats are available and may need to be processed differently.

Obtaining SILO Data for Multiple Sites

Use the get_multi_silodata function to get SILO data for multiple sites. This is a simple wrapper to get_silodata. The main differences are that latitude and longitude are numerical vectors and the new parameter Sitenameis a character vector of site names or site labels.

A simple example is:

two_sites  <- get_multi_silodata(latitude = c(-27.00, -28.00),
                     longitude = c(151.00, 152.00),
                     Sitename = c("Site_1", "Site_2"),
                     START = "20201101", FINISH = "20201105",
                     email = "MY_EMAIL_ADDRESS")

The retrieved data are:

two_sites
#: # A tibble: 10 × 11
#:     year   day  radn  maxt  mint  rain  evap    vp   code date_met   Sitename
#:    <int> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>  <int> <date>     <chr>   
#:  1  2020   306  27.6  31.2  10.2     0   7.6   8.7 222222 2020-11-01 Site_1  
#:  2  2020   307  23    31.3  11.8     0   7.2  13.3 222222 2020-11-02 Site_1  
#:  3  2020   308  27.2  30.6  14       0   7.2  13   222222 2020-11-03 Site_1  
#:  4  2020   309  26.3  32.5  14.4     0   8.8  13.6 222222 2020-11-04 Site_1  
#:  5  2020   310  22.9  36.9  16.1     0  10    15.8 222222 2020-11-05 Site_1  
#:  6  2020   306  26.6  27.3   9.6     0   7    10.6 222222 2020-11-01 Site_2  
#:  7  2020   307  22.7  26.5  12.5     0   6.8  14.2 222222 2020-11-02 Site_2  
#:  8  2020   308  28.5  26.4  12.3     0   6.8  11.2 222222 2020-11-03 Site_2  
#:  9  2020   309  27.4  28.7  11.3     0   6.8  13.1 222222 2020-11-04 Site_2  
#: 10  2020   310  19    33.6  13.5     0   9    17   222222 2020-11-05 Site_2

Note that, to obtain gridded data, you need to supply at least each site’s latitude and longitude as well as your email address by replacing MY_EMAIL_ADDRESS with your email address. The data is freely available under the Creative Commons 4.0 License.

Other Australian weather data

The excellent R package bomrang (Adam H. Sparks et al. 2017; Adam H. Sparks et al. 2021), which used to provide forecast, bulletin and historic data (see https://github.com/ropensci-archive/bomrang/), was archived after the Australian Bureau of Meteorology appeared to ban scraping on it’s websites in March 2021. Adam Sparks and team have produced an R package weatherOz which also includes a SILO wrapper (see https://github.com/ropensci/weatherOz). Paid services for some forecast, current and historical weather data for areas of interest may be available but these are not discussed here.

References

Sparks, Adam H., Jonathan Carroll, James Goldie, Dean Marchiori, Paul Melloy, Mark Padgham, Hugh Parsonage, and Keith Pembleton. 2021. bomrang: Australian Government Bureau of Meteorology (BOM) Data Client. https://CRAN.R-project.org/package=bomrang.
Sparks, Adam H, Mark Padgham, Hugh Parsonage, and Keith Pembleton. 2017. “bomrang: Fetch Australian Government Bureau of Meteorology Weather Data.” The Journal of Open Source Software 2 (17). https://doi.org/10.21105/joss.00411.