Terroir

What we found along the way

Real government data comes with real problems. Here’s what broke, and how I caught each one.

1. 01

   A coordinate system that lied about itself

   One of the map data sources said it used WGS84, the coordinate system used worldwide (including by GPS) — but it was actually using New Zealand’s own local system, NZGD2000, instead. That mismatch went unnoticed until it caused two datasets to line up incorrectly, shifting parcels off their real-world location. I fixed it by checking every new data source’s coordinate system against its own documentation, rather than trusting the label it shipped with.

2. 02

   A join key that wasn’t actually unique

   The government dataset’s own ID field (parcel_id) looks like it should uniquely identify each piece of land, but it doesn’t: when several people legally share ownership of one block (cross-leases, unit titles), that same ID gets reused across all of their records. Combining datasets using that ID silently let one property’s data overwrite another’s, with no warning anything had gone wrong. I switched to a different ID field, and double-checked that this one really was unique before relying on it.

3. 03

   A macron silently deleted a whole district

   Ōpōtiki — like many NZ place names — carries a macron. The plain-ASCII text filter used to select territorial authorities didn’t match it. No error, no warning: Ōpōtiki District was simply absent from the analysis from day one. I only found it when I compared the filtered list against the full set of source districts and saw the gap.

4. 04

   An off-by-one that misfiled 19% of parcels

   The function that sorts parcels into Excellent/Good/Marginal tiers had a boundary error — a score sitting exactly on a tier edge landed one tier lower than it should have. It misclassified roughly 19% of scored parcels before I caught and corrected it.

5. 05

   A geographic scope that missed a growing region

   The initial scope covered three districts. I missed Whakatāne. I only realised later, remembering that I’d worked in Edgecumbe and that there are orchards in that district I hadn’t included. Rather than assuming the remaining neighbouring districts had no orchard land, that was confirmed empirically using the land-cover dataset itself — the evidence, not an assumption, closed the question.

6. 06
   Most significant

   The same plot of land, counted several times over

   While building a Power BI dashboard, I noticed thousands of rows sharing the exact same map location — one physical block of land appearing dozens of times in the data. Two separate issues were causing it: roads and waterways that had slipped into the dataset even though they were never farmland, and multiple ownership records for the same block of land (similar to the mix-up in finding 02) each being counted as if it were a separate property. Fixing both brought the total from 21,491 down to a more accurate 16,072 scored parcels, while still keeping track of how many owners each block has, instead of just throwing that detail away.