R client for the eolas.nz data API — 1,500+ official New Zealand datasets, geospatial layers included, plus OECD data for international comparisons, returned as tidy data frames (or sf objects for geospatial layers).
Coverage is New Zealand + OECD today. Australian sources are on the roadmap — not yet available; OECD data already includes Australia (and other OECD members) for cross-country comparisons.
From r-universe (recommended):
install.packages("eolas", repos = c("https://phildonovan.r-universe.dev", "https://cloud.r-project.org"))
r-universe ships a pre-built binary of eolas itself. On Linux its dependencies
(notably arrow and sf) may still compile from source — potentially several
minutes, and a C/C++ toolchain is required. To get binary dependencies too, point
R at Posit Public Package Manager (or use
r2u / your distro's r-cran-* packages)
before installing, e.g.:
options(repos = c(
eolas = "https://phildonovan.r-universe.dev",
P3M = "https://packagemanager.posit.co/cran/__linux__/jammy/latest",
CRAN = "https://cloud.r-project.org"
))
install.packages("eolas")
Or from GitHub (source build):
remotes::install_github("phildonovan/eolas-r")
Two one-off steps make every future R session frictionless:
1. Save your API key to the OS keyring (macOS Keychain / Windows Credential Manager / Linux Secret Service) so eolas_get_*() finds it automatically — no eolas_key() call needed:
install.packages("keyring") # one-off; on Linux: sudo apt install libsecret-1-dev first
eolas_key_save() # interactive masked prompt
Other key-management helpers: eolas_key_status(), eolas_key_clear().
2. Set a library directory so downloaded bulk files land somewhere permanent instead of the transient ~/.cache/eolas/ OS cache:
eolas_library_set("~/eolas-library") # writes to ~/.eolas/config.json
eolas_library_status() # show resolved path + source
eolas_library_clear() # revert to ~/.cache/eolas/ fallback
On first interactive use without a library configured, the package prompts you to choose one (skip with EOLAS_LIBRARY=… or by pre-calling eolas_library_set()).
Or use an env var (useful for CI, Shiny Server, systemd):
Sys.setenv(EOLAS_LIBRARY = "~/eolas-library")
After setting the library, eolas_get_local("nz_parcels") and the smart-routing in eolas_get("nz_parcels") will use ~/eolas-library/ automatically.
The OS keyring slot and config file (~/.eolas/config.json) are shared with the Python eolas-data client, so a key or library path set from R is immediately usable in Python and vice versa.
library(eolas)
library(ggplot2)
eolas_key("your_api_key") # or: eolas_key_save() once, or set EOLAS_API_KEY in .Renviron
# CPI index (monthly, RBNZ M1) — the usual Treasury/analyst choice
cpi <- eolas_get_rbnz("rbnz_m1_prices", start = "2020-01-01")
# OECD macro indicators (quarterly YoY % — not CPI index levels)
inflation <- eolas_get_oecd("nz_cpi", start = "2020-01-01")
gdp <- eolas_get_oecd("nz_gdp_growth")
# Discovery
all_datasets <- eolas_list()
nz_only <- eolas_list("Stats NZ")
eolas_search("cpi") # expands aliases; surfaces rbnz_m1_prices before nz_cpi
meta <- eolas_info("rbnz_m1_prices")
# Plot — use ggplot2 directly. eolas_get_* returns tidy data frames so
# `aes(date, value)` is usually all you need:
ggplot(inflation, aes(date, value)) + geom_line()
Get an API key at https://eolas.nz/signup. Free plan is 10 requests/month; Pro ($49/month) is unlimited.
One eolas_get_*() and eolas_list_*() per source:
Core: statsnz, statsnz_geo, oecd, rbnz, treasury, linz, lris, mbie, nzta, msd, police, acc, edcounts, worksafe, doc, geonet, pharmac, eeca, immigration, charities. Auckland: akl_council, akl_transport. Regional-council bundles: northland, bay_of_plenty, hawkes_bay, taranaki, manawatu_whanganui, wellington, top_of_south, west_coast, otago, southland, napier_whanganui, colab_waikato, ecan_canterbury.
Run eolas_list() for the full live catalogue — the set above is generated and grows; eolas_list() is always authoritative.
Generate ready-to-run connector configs for popular data-pipeline tools (Meltano, Fivetran, Azure Data Factory) directly from R:
# In-memory: inspect what the server would generate
result <- eolas_integration("meltano", c("nz_cpi", "nz_gdp_growth"))
names(result$files) # "meltano.yml", "README.md", ".env.example"
cat(result$files$meltano.yml)
# Or write straight to a directory ready for `meltano install`
eolas_integration(
"meltano",
c("nz_cpi", "nz_gdp_growth"),
output_dir = "./my-pipeline"
)
This is an Enterprise-plan feature. Non-Enterprise keys see the server's upgrade-pointer error message; the gating lives server-side so it's bypass-proof. See https://eolas.nz/#pricing.
Datasets with a geometry_wkt column auto-convert to sf objects (CRS WGS84) when the sf package is available:
install.packages("sf")
gdf <- eolas_get("nz_addresses") # sf object
df <- eolas_get("nz_addresses", as_sf = FALSE) # plain df, WKT preserved
The 5.4M-row linz.nz_parcels table allocates ~10 GB when materialised as an sf object. Pass as_arrow = TRUE to skip all geometry materialisation and get a zero-copy arrow::Table instead — geometry stays as character WKT until you need it:
# Zero-copy Arrow table — no sf allocation
tbl <- eolas_get_linz("nz_parcels", as_arrow = TRUE)
# Filter before materialising — dramatically cheaper than loading the full sf object
library(duckdb)
con <- dbConnect(duckdb())
result <- dbGetQuery(con, "
SELECT parcel_id, geometry_wkt
FROM tbl
WHERE ST_Within(ST_GeomFromText(geometry_wkt),
ST_GeomFromText('POLYGON((174.7 -41.3, 174.8 -41.3, 174.8 -41.4, 174.7 -41.4, 174.7 -41.3))'))
")
as_arrow = TRUE works on all datasets (geo or non-geo), on both the live eolas_get() path and eolas_get_local(), and on all eolas_get_*() source helpers. It cannot be combined with as_sf = TRUE.
eolas_get() (and every eolas_get_*()) automatically uses Apache Arrow as
the wire format when the arrow package is installed — typically 5–10×
faster end-to-end on large pulls, with no code change. It falls back to
JSON transparently if arrow isn't installed or the server doesn't support
it (you'll get a one-time hint about the speed-up).
install.packages("arrow") # one-off; everything else is unchanged
df <- eolas_get("nz_addresses") # now streamed via Arrow, same data frame
For a columnar file straight from the REST API:
curl -H "X-API-Key: $EOLAS_API_KEY" \
"https://api.eolas.nz/v1/datasets/nz_cpi/data?format=parquet" -o nz_cpi.parquet
See the R reference for the format benchmark.
eolas_get_local() for whole datasetseolas_get() hits the live /data endpoint (good for slices and small pulls). For whole datasets — especially large or geospatial layers — use eolas_get_local(). It syncs a CDN-cached Parquet/GeoParquet file to your library directory and reads from disk on subsequent calls.
# Whole-dataset path: nz_parcels from CDN-cached GeoParquet (seconds, not a 15-min Iceberg scan)
gdf <- eolas_get_local("nz_parcels") # sf object when sf is installed
df <- eolas_get_local("nz_cpi") # tidy data.frame from cached Parquet
# Live path: date slices, row limits, licence-restricted sources (e.g. OECD)
df <- eolas_get("nz_cpi", start = "2020-01-01")
df <- eolas_get("nz_cpi", limit = 100)
Use eolas_get_local() when you need to control cache_dir, format, or freshness:
# Explicit cache+sync with extra options
gdf <- eolas_get_local("nz_parcels")
df <- eolas_get_local("nz_cpi", cache_dir = "/data/eolas", format = "csv_gz")
df <- eolas_get_local("nz_parcels", as_sf = FALSE) # plain data.frame, no sf conversion
For advanced control over the sync lifecycle, use eolas_sync_bulk() directly. For one-shot downloads to a raw vector or file path, use eolas_download_bulk():
r <- eolas_sync_bulk("nz_cpi", path = "nz_cpi.parquet")
# r$status: "downloaded" | "unchanged" | "updated"; r$bytes_downloaded == 0 when unchanged
eolas_download_bulk("treasury_fiscal_spending", path = "t.parquet")
Progress bars: eolas_get_local() shows two phases in interactive sessions — a download byte bar while fetching from CDN, then a read spinner while Parquet/GeoParquet is loaded into a data frame or sf object (the read phase is often the slow part on multi-million-row geo datasets). Control with progress = TRUE (both), FALSE (neither), "download", or "read". Set EOLAS_NO_PROGRESS=1 to suppress both in batch scripts. Cached files skip the download bar and print an informative message instead.
Full docs: docs.eolas.nz/bulk-downloads/.
eolas_sync(name, path) keeps a local file current, automatically choosing how based on the dataset's CDC serving tier — you make the same call either way:
eolas_sync_bulk())./changes feed and pk-merge it into your file (eolas_sync_changes()).# Same call regardless of tier:
r <- eolas_sync("nz_building_outlines", path = "buildings.parquet")
r$status # "downloaded" (baseline) | "updated" | "unchanged"
r$sync_mode # "changelog" for changelog-tier datasets
r$ops_applied # number of change rows applied this run
r$current_seq # feed watermark after this sync
# First call baselines; subsequent calls apply only new changes:
r <- eolas_sync("nz_building_outlines", path = "buildings.parquet")
r$ops_applied # e.g. 1240
A sidecar at paste0(path, ".eolas-meta.json") records the snapshot id / feed watermark so the next call fetches only new data. For SCD2 datasets the merge keeps only the current rows (is_current = true), so buildings.parquet is always a clean current-state snapshot — the SCD2 history is handled for you. A 410 (watermark expired) self-heals by re-baselining.
The changelog sidecar (schema_version 2) is byte-compatible with the Python eolas-data client: a file synced from Python can be resumed from R and vice versa.
# unit tests (mocked HTTP — no API key needed)
Rscript -e 'devtools::test()'
# or from the monorepo root:
../scripts/test-clients.sh
R CMD check runs the same testthat suite via tests/testthat.R. CI runs check + coverage on every push/PR. A weekly workflow optionally hits the live API when EOLAS_API_KEY is configured as a repository secret.
MIT — applies to this client software only. Dataset use is subject to each source's licence and your eolas API plan.