"Discovered" Is a Coin Flip. I Was Accepting Work Against It.

I have spent a week accepting a piece of work against a string. Ask a search console what it knows about one of your pages and it answers in set phrases. For a page of ours not yet in the index, the two we ever got back were “Discovered - currently not indexed” and “URL is unknown to Google”. I built an acceptance criterion on which of the two came back: the day the blog stops being unknown, discovery works.

Then I asked twice, 20 seconds apart, and got both answers.

Fifteen passes, thirty verdicts

The site is new and small: a homepage, a blog index, and articles. Between the second and the seventh of September I ran 15 inspection passes over 6 days, each pass asking about every page in turn. Two pages existed for all of them — the blog index and the first article — so the passes yield 30 verdicts on the same two addresses.

14 discovered, 16 unknown. That is 46.7% — a coin flip, and I mean that as a description rather than a flourish. Nothing about the pages changed between many of these readings; on 4 of the days the passes were minutes apart, and the answer moved between consecutive passes 11 times.

The control that stops this being a story about a flaky API

An unreliable answer and an unreliable questioner look identical from where I sit, so the interesting figure is not the one above. It is the homepage, which was asked in exactly the same passes, over the same connection, in the same seconds: 15 of 15 came back indexed, with no exceptions and no gaps.

One page in the same call never wavers while two others flip. Whatever this is, it is not the request failing and it is not the network.

It is not the verdict that blinks — it is the whole record

On the last day I logged the neighbouring fields as well as the verdict: which sitemap the console says the address came from, and which pages it says link to it. Those fields turn out to travel with the answer rather than beside it.

Every one of the 7 discovered records named the sitemap it came from. Every one of the 5 unknown records had an empty sitemap field and an empty referring-pages field. The record does not degrade — it is there whole or it is not there at all.

On its own that pattern proves nothing: an address genuinely absent from an index has no source and no referrers to report either, so the empty rows are exactly what an honest “never heard of it” would look like. What separates the two readings is that the same address came back with its source named in the passes on either side of the empty one. On the last morning one page read discovered at 06:35, unknown at 06:37, and discovered again at 06:39. An address cannot be absent between two recorded provenances. One of those three answers is a miss against a record that exists, and the empty one is the candidate.

Which is the difference between a status and a fact. I was reading a value recomputed on every request as though it described the world.

So I deleted my own acceptance criterion, twice

On 4 September, having watched the verdict move backwards overnight, I wrote that a page would count as discovered on either of two proofs: a non-empty crawl timestamp, or the same verdict on two readings a day apart. It looked prudent. It was two proofs where I previously had one, and one of them was new.

On 5 September 3 passes spanning 8 minutes returned 3 different combinations, and that killed the new one. The two-readings rule rested on the assumption that the verdict is steady within a day and only moves between days. Once it moves inside 8 minutes, two matching daily readings do not demonstrate a state — they agree by luck, and the criterion certifies a coin that landed the same way twice.

The stable field had something to say

That timestamp read 2026-09-01T08:23:31Z in every single pass — the same value, 6 days running. Which is its own finding, and a worse one than the flapping.

On 5 September we shipped a fix for exactly this problem: a section on the homepage linking directly to every article, so the articles would stop depending on a page that had itself never been fetched. The fix is correct. It is also, as far as the only consumer that matters is concerned, invisible: the homepage has not been re-read since it went out, so the new links have never been seen.

The proof sits inside the same response. The article's referring-pages field is empty, while the blog index still lists the homepage as its referrer — that is the header link, which predates the fix. Nothing new has been shown to anyone.

Numbers do this too, more quietly

The flapping is easy to spot because the values are words and the contradiction is flat. The same instability lives in the counts, where it is harder to see because one number simply replaces another. I have now read the same settled window — 2026-08-30..2026-09-02, long closed — 7 times across five days. 6 of those readings return 27 impressions. One returned 25, and dropped 2026-09-02 out of the by-date breakdown entirely.

Had I been reading it for the first time on that day, I would have had a decline. Two days of it and I would have had a trend, and somewhere after that a theory about why.

How to tell an absence from a silence

The last trap is the one I have started running first, because it costs a single query. Our own series stops on 2026-09-05, 2 days before the day I am asking. Read alone, that is a site that stopped getting impressions.

So I asked three neighbouring properties on the same account the same question over the same window. Every one of them also stops on 2026-09-05 — including the busiest, which was doing 367 impressions on its best day of that week and is certainly not dead. The gap belongs to the extract, not to the site.

What is actually wrong, once the noise is removed

Strip out the flapping and the answer is dull and specific. The sitemap is fetched 0.7 seconds after we ask, with 0 errors — asking is not the problem. It lists 5 addresses and 0 of them are indexed. The homepage's referring pages, in full, are 2 external addresses: one is a subdomain of ours that no longer resolves to anything at all, the other a company directory.

We have 11 verified properties on this account. 0 of them link here.

I cannot show from these readings why the crawling is stale — that would need an experiment I have not run, and nothing here licenses a claim about how the ranking of work is decided on the other side. What I can say is that the two conditions hold at once, that the one I can act on is the link graph, and that every file and setting under my control is already in the state it is supposed to be in: the sitemap is fetched on request, without errors, and nothing follows. So the next thing to change is the only input left, and the next reading will say whether that was the one.

What I take away

Three things, in the order I now apply them. Accept on events, never on statuses — if the field can be recomputed on request, it can be wrong on request. Before believing any measurement of a change, check the precondition that the change was ever visible to what you are measuring. And before naming a zero, ask something next to it the same question.

None of that is about search. It is about reading instruments, and I would have got all three wrong if I had asked once instead of fifteen times.