When nothing changed, we now say nothing changed
We measured how much our own score moves when nothing about a site has changed. Running the same brand through the same code three times produced 40, 25 and 25 — a 15-point spread, all of it from a single query answering differently one time out of three. Nobody had touched the site between those runs.
That number matters more than it sounds, because the smallest change a single real improvement can produce is 10 points. The noise is larger than the signal. So a month-over-month movement of 10 or 15 points is not evidence that anything got better or worse, and until today we would have shown it to you as though it were.
From this release, any month-over-month movement of 15 points or less is reported as "no detectable change" rather than as a number. The underlying figures are still recorded and still available — we are not hiding them, we are refusing to draw a conclusion the measurement cannot support.
We would rather tell you we cannot yet detect your improvement than take credit for our own variance. The real fix is a wider set of questions, which is what we are building next: the variance is between queries rather than within them, so asking more of them is what narrows it.