Small Numbers

Five became seven. That is not a 40% rise — it is five and seven.

New Zealand has 5.3 million people, so almost every number anyone argues about here sits on a denominator small enough that noise runs the show. A region of 32,000. A school of 200. A district with fifteen thousand ratepayers.

This page answers the two questions that follow: did it actually change, and is the worst area actually worst. The second answer is usually no — it is usually the smallest.

Nothing is uploaded. The arithmetic runs on the server for one request and nothing is written down.

Did it change?

Two counts, two periods. Exposures are optional — leave them at 1 if the two periods cover the same population for the same length of time.

5 → 7 is a ratio of 1.40× 95% interval 0.38× to 5.59×

Not distinguishable from no change.

With 5 before, 15 after would be the first count that means anything.

With 5 before, no later count at all — not even zero — would show a fall. That direction is unanswerable at this scale.

The interval is exact: conditional on the total, the later count is binomial, so no normal approximation is involved and there is no minimum count for the method to hold.

A league table

Region populations are rounded approximations, there for the shape of the spread — fifty-fold from Auckland to the West Coast. The event counts are simulated from a single common rate.

First, the preconditions

16 areasat least 3
331 events in totalat least 20 — below that the pooled rate itself is too vague to measure against
every row has a populationan exposure above zero
Nothing stands out

16 areas, 331 events, an overall rate of 6.5 per 100,000. Not one area escapes the 99.8% limits drawn for an area of its own size — so the ranking below is a ranking, not a finding.

The funnel

Rate against population, with the limits an area of each size would produce at the overall rate. The limits flare out on the left because small populations produce wild rates — that is arithmetic, not a story about those places. An area is only remarkable when it escapes the limits drawn for its own size.

100,0001.0m08.116overall 6.5 /100,000Tasman / Te Tai-o-Aorere — 6 / 58,000 = 10.3West Coast / Te Tai Poutini — 3 / 32,000 = 9.4Nelson / Whakatū — 5 / 55,000 = 9.1Wellington / Te Whanganui-a-Tara — 48 / 550,000 = 8.7Gisborne / Te Tairāwhiti — 4 / 52,000 = 7.7Waikato — 38 / 500,000 = 7.6Otago / Ōtākou — 16 / 250,000 = 6.4Taranaki — 8 / 125,000 = 6.4Bay of Plenty / Te Moana-a-Toi — 22 / 350,000 = 6.3Canterbury / Waitaha — 40 / 660,000 = 6.1Auckland / Tāmaki Makaurau — 102 / 1.7m = 6.0Marlborough / Te Tauihu-o-te-waka — 3 / 52,000 = 5.8Northland / Te Tai Tokerau — 11 / 200,000 = 5.5Manawatū-Whanganui — 13 / 260,000 = 5.0Southland / Murihiku — 5 / 103,000 = 4.9Hawke's Bay / Te Matau-a-Māui — 7 / 180,000 = 3.9population (log scale) →

The limits are stepped because counts are whole numbers. A smooth curve would imply a precision the arithmetic does not have.

#AreaEventsPopulationRate /100,00095% rangeExpected count
1Tasman / Te Tai-o-Aorere658,000 10.33.8 – 22.50 – 8
2West Coast / Te Tai Poutini332,000 9.41.9 – 27.40 – 5
3Nelson / Whakatū555,000 9.13.0 – 21.20 – 8
4Wellington / Te Whanganui-a-Tara48550,000 8.76.4 – 11.623 – 48
5Gisborne / Te Tairāwhiti452,000 7.72.1 – 19.70 – 7
6Waikato38500,000 7.65.4 – 10.421 – 44
7Otago / Ōtākou16250,000 6.43.7 – 10.48 – 24
8Taranaki8125,000 6.42.8 – 12.62 – 14
9Bay of Plenty / Te Moana-a-Toi22350,000 6.33.9 – 9.513 – 32
10Canterbury / Waitaha40660,000 6.14.3 – 8.329 – 56
11Auckland / Tāmaki Makaurau1021.7m 6.04.9 – 7.389 – 131
12Marlborough / Te Tauihu-o-te-waka352,000 5.81.2 – 16.90 – 7
13Northland / Te Tai Tokerau11200,000 5.52.7 – 9.85 – 20
14Manawatū-Whanganui13260,000 5.02.7 – 8.68 – 25
15Southland / Murihiku5103,000 4.91.6 – 11.31 – 12
16Hawke's Bay / Te Matau-a-Māui7180,000 3.91.6 – 8.04 – 19
All 16 areas are inside the limits

About 0.8 of them would be expected outside the 95% band by chance alone, and 0 are. Nothing here needs explaining.

The worst area is not distinguishable from average

Tasman / Te Tai-o-Aorere tops the table at 10.3 per 100,000 — but on 6 events over 58,000 people its own interval runs from 3.8 to 22.5, which contains the overall rate of 6.5. It is at the top because it is small, not because it is worse.

West Coast / Te Tai Poutini has the widest interval on the chart

3 events over 32,000 people is a rate of 9.4 per 100,000 — and anything from 1.9 to 27.4. That is not a precise number with error bars on it. It is a range, and the middle of it is not more true than the ends.

What it does not know

It does not know why. An area outside the limits has something to explain — a different population, a different way of recording, or a real difference. This page says the number is unusual, and nothing at all about which.

Recording is not the same everywhere. A district that records more carefully will look worse. On small counts that difference alone can move an area from the middle to the top.

The populations matter as much as the counts. A rate is a fraction, and half of it comes from a denominator somebody else estimated. If the population is a projection rather than a count, that uncertainty is not in any interval on this page.

One year is one year. The area at the top this year is usually not the area at the top next year, and that is not improvement — it is the same arithmetic running again.