User-agent: * Disallow: /api/internal/ Disallow: /admin/ Disallow: /metrics Allow: /