Sitemap: https://www.be-market.com/sitemap.xml User-agent: * Disallow: # Allow AI search and agent use (traen tráfico/citan la fuente) User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: PerplexityBot User-agent: FirecrawlAgent User-agent: AndiBot User-agent: ExaBot User-agent: PhindBot User-agent: YouBot User-agent: Claude-User User-agent: Claude-SearchBot Allow: / # Disallow AI training data collection (scrapers de entrenamiento) User-agent: GPTBot User-agent: CCBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: Bytespider User-agent: Meta-ExternalAgent User-agent: Meta-ExternalFetcher User-agent: Amazonbot User-agent: Diffbot User-agent: cohere-ai User-agent: Timpibot Disallow: /