User-agent: * Allow: / # OpenAI User-agent: GPTBot Allow: / # Google Gemini User-agent: Google-Extended Allow: / # Anthropic Claude User-agent: ClaudeBot Allow: / # xAI Grok User-agent: xAI-Grok Allow: / # Perplexity AI User-agent: PerplexityBot Allow: / # Common Crawl User-agent: CCBot Allow: / # Meta AI User-agent: FacebookBot Allow: / # Apple AI User-agent: Applebot Allow: / # Twitter / X User-agent: Twitterbot Allow: /