# mine robots.txt User-agent: ClaudeBot User-agent: DataForSeoBot User-agent: Amazonbot User-agent: GPTBot User-agent: meta-externalagent User-agent: dotbot User-agent: AhrefsBot User-agent: Bytespider User-agent: SemrushBot Disallow: /