User-agent: * Disallow: /api/ Disallow: /ajax/ Disallow: /search/ Disallow: /cms/ Disallow: /my_account/