AI Overview Citations Across Two Captures: 16 Pairs

Key takeaways

The short version of this page, in the order the sections below develop it. Each point is stated here in full so it can be read without the rest of the page, and the section that argues it gives the measurement and the source behind it.

• In 16 query pairs on one evening, they often did, and we cannot say why.
• We pulled 20 queries twice each, and 16 query pairs had both captures succeed.
• We stopped because the check our protocol required before the main study said to, and the miss was narrow.
• It is a marker we noticed after the fact, not an explanation.

Do AI Overview citations change when you run the same search twice?

In 16 query pairs on one evening, they often did, and we cannot say why. We requested each query twice, minutes apart, with an identical request. In 8 of 16 pairs the two captures shared no organic top-20 URL at all, and only 2 of 16 were identical in both captures. In 8 of 15 pairs where both captures listed references, they shared no cited URL.

This is a methods note, not a study of what gets cited. We set out to test what predicts being cited in a Google AI Overview. Our protocol required a capture check first, to decide whether "cited" is a property of the search or of one capture. The check told us to stop, and we are publishing it because anyone who tracks AI Overview citations from one capture per query can run the same check cheaply.

Nothing here is a rate for Google. It is one account, one country and one evening, and the figures below should be read as a description of those 16 pairs.

How did we check, and what came back?

We pulled 20 queries twice each, and 16 query pairs had both captures succeed. Each pair used the identical DataForSEO call: the serp/google/organic/live/advanced endpoint, US location, English, depth 20, with asynchronous AI Overview loading switched on. The vendor's documentation describes that switch as an option to obtain ai_overview items even if they are loaded asynchronously. All captures were taken on 2026-10-06, between 22:02 and 22:15 UTC. The median gap between the two captures of one query was 4 minutes and the longest was 11 minutes. Of the 16 pairs, 15 had an AI Overview with at least one reference in both captures.

The intervals are Wilson intervals. They are wide because 15 or 16 queries is a small sample, and the queries share templates and categories, so even these intervals are approximate. An independent reviewer re-ran the analysis and checked the 8 disjoint organic pairs by hand against the raw files. The difference was not a URL-formatting artifact, and each capture carries its own timestamp.

Why did we stop the planned study?

We stopped because the check our protocol required before the main study said to, and the miss was narrow. The rule was written before any outcome data was analysed. It said to stop if agreement on cited status between the two captures fell below 85%, or if Cohen's kappa fell below 0.60. Kappa is agreement corrected for what chance alone would produce; scikit-learn's documentation says it "expresses the level of agreement between two annotators on a classification problem". We measured agreement of 84% (95% CI 71% to 94%) against the 85% threshold, and kappa = 0.596 (95% CI 0.33 to 0.84) against the 0.60 threshold.

Both intervals span both thresholds. The rule was set on point estimates, so the stop is correct as written, but we do not read it as a crushing failure. The five planned hypotheses, about what predicts citation, were not tested, and the 300-query panel was never pulled. Our protocol labels this outcome a problem with the capture itself, and we want to be careful with that label: the check cannot attribute the instability to the data provider rather than to Google.

Agreement and kappa were computed only on URLs ranked in the top 20 of both captures, which is the more stable half of the data. That gives 110 rows from 8 of the 15 queries, so the effective sample is about 30 and the intervals are approximate. Of those rows, 18 of 110 changed cited status between captures, and 84% agreement is flattered by the many rows that were not cited in either capture. Kappa corrects for that, which is why we lead with it.

Does the result-count field explain the difference?

The result-count field is a marker we noticed after the fact, not an explanation of the difference. In 8 of 16 pairs, one capture reported a result count of at most 159 and the other reported at least 139,000, with no value in between. All 8 of those mixed pairs shared no organic top-20 URL, while 0 of the 8 pairs where both captures fell in the same class did, and their median organic overlap score was 0.74. This is exploratory and chosen after seeing the data. It is an association in 16 pairs, and the count comes from the same response as the results, so the check does not show why the counts differ.

The class also does not explain citation instability: among the 8 same-class pairs, cited sets still differed, with a median overlap score of 0.48, and one of those 8 pairs shared no cited URL. If you record a result-count field when you capture, it is cheap to log it next to the citation list and see whether it behaves the same way for your queries.

What does this mean if you track AI Overview citations?

If you track AI Overview citations, read one capture as one observation: that is what this sample supports. If your tracker reports the cited URLs from a single request per query, the 16 pairs here are a reason to repeat the capture before drawing a conclusion from a change or an absence. We are describing what this sample suggests, not a proven rule.

Three habits follow from that, and none needs a larger budget:

1. Capture each query at least twice and compare the two cited sets before you trust either.
2. Record the time of every capture, because an hour or a day is a variable you cannot recover later.
3. Record any field that signals a different page variant, such as the result count, next to the citation list.

Google's own documentation says the response varies by feature: "AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary." It does not say how much an AI Overview varies between two requests, and our check does not settle that. For how Google describes appearing in these features, see our guide on how to appear in Google AI Overviews. For what tracking tools capture, see tools to track brand visibility in AI search.

What can this not show, and what would a proper study need?

This check cannot show how common the instability is, or why it happens, because the design was never built to. The queries were analyst-written templates rather than observed buyer questions, and only 16 pairs had both captures. Everything ran in one country, on desktop, logged out, through a single provider, inside thirteen minutes of one evening. Each of those four limits narrows what the numbers below are allowed to mean, and all four matter when you quote them.

• The 20 queries were analyst-written templates, and only 16 pairs had both captures. They are not a sample of what buyers search.
• One country (United States), desktop, logged out.
• One provider, DataForSEO, so a changing Google result and a different way of collecting it look the same to us.
• One evening, 22:02 to 22:15 UTC on 2026-10-06. Day-to-day change was not measured.
• The asynchronous AI Overview flag never differed between captures in 15 pairs, so whether that flag matters is untested.
• 20 of 56 calls in this run returned an error, and the responses reported a cost of $0.046 for those 20 calls. Retries are included, so the same query can fail more than once, and the 20 calls span 12 of the 20 queries. That is a plain count for one evening on one account, not a failure rate for the vendor, and we offer no explanation for it. The whole run cost $0.234.

A larger repeated-capture study is needed to establish how common this is and why. It should use more queries, three or more captures per query, recorded timing and result count, and a second provider. We would rather publish the small check now than wait for that study, since others can repeat it cheaply. A new request returns that moment's results, so the raw captures of this evening cannot be re-obtained identically.

Questions people ask next

These are the follow-ups we are asked most often once the answer above has landed. The same rule applies to each: where a figure appears, its source and year sit in the same sentence, and where we do not know something we say so rather than estimate it.

Do AI Overview citations change if you run the same search twice?

In our 16 query pairs on one evening, they often did: 8 of 15 pairs with references in both captures shared no cited URL, and 2 of 15 cited exactly the same URLs. We cannot say whether Google or the way we collected the results produced the difference, and 16 pairs cannot give a rate for searches in general.

Is this a DataForSEO problem?

We do not know. We collected every capture through DataForSEO's SERP API, so the check cannot separate a result that changed at Google from a result that was collected differently. A second provider and repeated captures are what would tell the two apart.

Should I stop tracking AI Overview citations from a single capture?

No, but treat one capture as one observation. In this sample, a single capture would have told you little about which URLs the overview cites on another request. Repeating the capture and recording the time of each one costs little and shows how much your own queries move.

Did you test what predicts being cited in an AI Overview?

No. We planned five hypotheses about what predicts citation, and none was tested. Our frozen protocol required a capture check first, the check came in just under its thresholds, and the study stopped there.

Related

Other questions in this library that cover the ground next to this one. Each is answered the same way — the direct answer first, then the measurement behind it — and each carries its own sources, so you can check any of them on its own page without taking this one on trust.

• How do you appear in Google AI Overviews?
• What are the best tools to track brand visibility in AI search?