User-agent: Amazonbot Disallow: / # User-agent: * # Disallow: /ScriptResource.axd # Disallow: /WebResource.axd # Disallow: /*.axd$ #Comment to make sure this is not synced 8-10-2017 User-agent: * #Disallow: / Crawl-delay: 10 Disallow: /ScriptResource.axd Disallow: /WebResource.axd Disallow: /*.axd$ Sitemap: https://www.vault.com/Master-sitemap.xml #Crawl-Delay: 10 User-agent: SemanticScholarBot User-agent: BUbiNG User-agent: Gluten Free Crawler User-agent: AhrefsBot User-Agent: MJ12bot User-agent: MauiBot User-agent: SEMrushBot User-agent: Quora-Bot User-agent: YandexBot User-Agent: serpstatbot Disallow: /