Reproducing and validating regional NOAA CRW tiles

This workflow checks complete raster tiles, rather than selected points. It downloads a 90-day Florida Keys tile from the PacIOOS mirror of NOAA Coral Reef Watch v3.1, recalculates products from SST and NOAA’s official v3.1 climatology, and compares every cell. The region deliberately crosses islands and coastlines, so ocean, land and masked pixels are all exercised.

The companion script is data-raw/validate_noaa_regional_tile.R. Its inputs, download URLs, date range, bounding box, tolerances and output checksums are recorded in the output. Cached downloads make a repeated run deterministic; delete the cache to retrieve the source again.

Run the validation

With no climatology argument, the workflow downloads the official NOAA v3.1 climatology into the same external cache:

system2("Rscript", c(
  "data-raw/validate_noaa_regional_tile.R",
  "--cache=validation/noaa-florida-keys"
))

An existing climatology may instead be supplied with --climatology=/path/to/ct5km_climatology_v3.1.nc.

The script writes regional-validation.csv, regional-mask-audit.csv, and a machine-readable regional-validation.rds. It exits with an error if any product fails, which makes the same workflow suitable for a scheduled CI job. The network retrieval is intentionally not run while building the package vignette.

What is tested

For HotSpot, DHW, single-day BAA and 7-day maximum BAA, the script compares all values and the complete NA pattern. DHW and single-day BAA comparisons begin on day 84; the 7-day composite is compared on day 90, after seven independently calculated BAA days. Continuous fields allow 0.011 degrees C, just above NOAA’s published 0.01 degree C encoding increment; categorical fields require exact agreement.

NOAA’s HotSpot mask is counted by code across the full space-time tile. In addition, one valid ocean pixel is deliberately removed from every input day. The check requires that missingness propagate through HotSpot, DHW and BAA instead of being interpreted as zero heat stress. This controlled case ensures missing pixels are tested even if a future source tile happens to contain none.

Current BAA categories are checked by applying NOAA’s published post-15 December 2023 thresholds to the official HotSpot and DHW tile fields. The source’s archived categorical BAA is also compared and its mismatch count is reported separately, but is not a pass/fail target because parts of that archive retain the heritage Alert Level 2 ceiling. This prevents an obsolete archived category field from overriding the current NOAA methodology.

Published continuous reference values have already been rounded to 0.01 degrees C and 0.01 degree-heating-weeks. If a value lies within one encoding increment of a category threshold (for example, published DHW 12.00), the pre-encoding side of that threshold cannot be recovered. The report counts these as ambiguous_encoded_thresholds; only categorical differences outside the full rounding interval are failures. This avoids claiming false bit-level certainty while still requiring exact categories everywhere the published inputs uniquely determine them.

Post-December 2023 BAA fixtures

data-raw/build_noaa_baa_post_2023_reference.R rebuilds three small fixtures from official NOAA-derived HotSpot and DHW observations dated 10–16 December 2023. The final observations exercise Alert Levels 3, 4 and 5 (BAA codes 5, 6 and 7). Download URLs and response MD5 checksums are stored in the fixture; package tests use the committed RDS and therefore run offline.

NOAA-compatible output encoding

Calculations retain full precision. Set encoding = "noaa" to round only the requested final products and, when writing files, use NOAA-compatible storage:

products <- create_climatology(
  sst,
  climatology = "code/ct5km_climatology_v3.1.nc",
  products = c("hotspots", "dhw", "baa", "baa_7d"),
  encoding = "noaa",
  save_output = "outputs/florida"
)

Continuous products are rounded to 0.01 degrees C and stored as signed 16-bit integers with scale 0.01 and fill value -32768. BAA products are stored as unsigned bytes with fill value 251. encoding = "native" remains the default. File-level identity can still depend on container metadata and compression; decoded cell values, masks, scale, offset and fill value are the equivalence targets.