Visibility-first checklist. A site should be accessible to documented search/retrieval bots, indexable by major search engines, readable without client-side-only content, and clear enough to quote without guessing.

Platform matrix

ChatGPT / OpenAI

Allow OAI-SearchBot. Keep public content available without authentication or JavaScript-only rendering. GPTBot is a separate training control.

Gemini / Google

Prioritize Googlebot and Google Search indexing. Google-Extended is a separate control for Gemini model development and grounding, not a Search ranking signal.

Claude

Allow Claude-SearchBot for search discovery and Claude-User for user-directed retrieval. ClaudeBot is the separate model-development crawler.

Perplexity

Allow PerplexityBot. Publish authorship, corrections and first-party expertise — Perplexity says its source review looks at domain-level trust signals.

Grok / xAI

Grok searches the broader web and X with citations. xAI does not currently publish a site-owner robots token comparable to OAI-SearchBot, so broad crawlability and strong web/X entity signals matter.

DeepSeek

DeepSeek offers server-side web search but does not currently publish a dedicated site-owner crawler token. Keep pages accessible to standard search/retrieval systems.

Alice / Yandex

Get the page indexed by Yandex and do not block YandexAdditional/YandexAdditionalBot. Yandex says structured, informative pages are more likely to support Alice answers.

GigaChat

GigaChat offers internet search, but no dedicated webmaster crawler token is currently documented. Strong Russian-language indexing, clear facts and public source pages are the practical foundation.

On-page checklist

  1. One canonical URL per page and clean EN/RU hreflang.
  2. Descriptive title, H1 and first paragraph that answer the page’s actual question.
  3. Visible author and update date on expert material.
  4. Primary source links where facts depend on external platforms.
  5. Stable Organization and Person entity IDs in Schema.org.
  6. Service pages that explain scope, deliverables, limitations and evidence.
  7. Server logs monitored for crawler access and unexpected 403/429/5xx responses.

Do not optimize for a hallucinated crawler

If a vendor has not documented a crawler token, adding a guessed User-agent name does not create visibility. The fallback is standards-compliant crawlability plus strong indexing in the search ecosystems the agent can access.