PTODA-ResearchBot

PTODA-ResearchBot/1.0 (+https://ptoda.org/researchbot)

An academic research crawler operated by Digital Dominator Pty Ltd under the Periodic Table of Digital Authority™ (PTODA) research programme. It collects observable, publicly available machine-readable signals from organisation websites for published benchmark research. It does not scrape content for training, resale, or commercial enrichment.

01Identity & operator

User-Agent: PTODA-ResearchBot/1.0 (+https://ptoda.org/researchbot)
Operator: Digital Dominator Pty Ltd (ABN 28 616 931 116), Australia
Programme: Periodic Table of Digital Authority™ research programme — ptoda.org
Purpose category: Academic Research
Study register: PTODA-VC-2026-001 (verified-crawl study series; individual runs carry timestamped run identifiers)
Contact: [email protected]

02Cryptographic verification

PTODA-ResearchBot signs its requests using Web Bot Auth (HTTP Message Signatures, RFC 9421), so its identity can be verified cryptographically rather than trusted from the User-Agent string alone.

Signature-Agent: https://ptoda.org
Key directory: /.well-known/http-message-signatures-directory
Algorithm: Ed25519

Requests presenting this User-Agent without a valid signature verifiable against the key directory above should not be attributed to the PTODA research programme. Suspected impersonation can be reported to [email protected].

03Crawl conduct

robots.txt is honoured. Disallow rules for PTODA-ResearchBot or * are obeyed, including crawl-delay directives.
Honest identification. The crawler always presents its true User-Agent and signed identity. It never masquerades as a browser or another bot.
Conservative request rates. Collection runs use fixed, throttled request budgets per host, documented in each study's frozen methodology.
Refusals are recorded, not circumvented. HTTP 403/429 responses, challenges, and access controls are logged as observations. The crawler does not retry aggressively, rotate identities, or attempt to evade publisher preferences.
Public pages only. No authentication barriers are crossed and no personal data is deliberately collected. Observations concern organisation-level, publicly served technical signals.

04Opting out

To exclude a site from PTODA research collection, add the following to robots.txt:

User-agent: PTODA-ResearchBot
Disallow: /

The directive takes effect from the crawler's next visit. Site operators may also request exclusion, data about their domain, or removal of prior observations by emailing [email protected].

05What the data is used for

Observations feed the PTODA Global Digital Authority Benchmark Series — published, methodology-frozen studies of how organisations' public web infrastructure presents to AI and search systems. Published research and methodology documents are available at ptoda.org. Results are reported at population level under each study's disclosure rules.