Study 01 · AI Crawler Access · Methodology Version History

C01 version history

The AI Crawler Access Study evolved its methodology openly. This page records the transition from the original binary reading (v1.2) to the five-class functional-purpose taxonomy (v1.5) that is the current research.

Versions

From binary to functional taxonomy

Version

Summary

Status

v1.2

Original binary methodology. Reported a domain as “blocking AI” or not, with a pooled rate of ~40% across four markets. Now understood to over-read the underlying policies by counting any disallowed path — including operational and secondary paths — as blocking.

Superseded

v1.5

Five-class functional-purpose taxonomy. Classifies each restricted path by what it is, and reports strict (whole-site), meaningful (primary content), and expanded (any restriction) measures separately. Four independently frozen country volumes plus a synthesis flagship. Adds Australia as an independent volume.

Current

Why both versions remain available. The programme does not silently rewrite the published record. v1.2 is preserved with a superseded notice so the methodological evolution is legible and the change in headline is explained rather than hidden. The transition is documented in GDR-002 — C01 Version Transition (v1.2 → v1.5).

Current research (v1.5)

The four-market series

Cross-Market Flagship

The synthesis. Binary overstates meaningful restriction.

United States

808 domains · strict 3.8% · meaningful 10.7%

Australia

409 domains · strict 5.0% · meaningful 11.7%

United Kingdom

754 domains · strict 4.1% · meaningful 9.6%

Singapore

268 domains · nine sectors · strict 4.0% · meaningful 8.0%

Historical record (v1.2)

Archived binary-methodology pages

These pages reflect the superseded v1.2 binary methodology and remain available as part of the published record.

Cross-market (v1.2) · United States (v1.2) · United Kingdom (v1.2) · Singapore (v1.2)