Issues with Crawling a Domain

Whitelist SplitSignalBot:

To whitelist the bot, contact your webmaster or hosting provider and ask them to whitelist SplitSignalBot.

The bot’s IP subnetworks are: 85.208.98.32/28 and 85.208.98.48/28

The bot is using standard 80 HTTP and 443 HTTPS ports to connect.

If you use any plugins (Wordpress, for example) or CDNs (content delivery networks) to manage your site, you will have to whitelist the bot IP within those as well.

For whitelisting on Wordpress, contact Wordpress support.

Common CDNs that block our crawler include:

• Cloudflare - read how to whitelist here.
• Incapsula - read how to whitelist here (add Semrush as a “Good bot”).
• ModSecurity - read how to whitelist here.
• Sucuri - read how to whitelist here.

Please note: If you have shared hosting, it is possible that your hosting provider may not allow you to whitelist any bots or edit the Robots.txt file.

Hosting Providers

Below is a list of some of the most popular hosting providers on the web and how to whitelist a bot on each or reach their support team for assistance: 

• Siteground - whitelisting instructions 
• 1&1 IONOS - whitelisting instructions 
• Bluehost* - whitelisting instructions 
• Hostgator* - whitelisting instructions 
• Hostinger - whitelisting instructions 
• GoDaddy - whitelisting instructions 
• GreenGeeks - whitelisting instructions 
• Big Commerce - Must contact support 
• Liquid Web - Must contact support 
• iPage - Must contact support
• InMotion - Must contact support
• Glowhost - Must contact support
• A2 Hosting - Must contact support
• DreamHost - Must contact support

* Please note: these instructions work for HostGator and Bluehost if you have a website on VPS or Dedicated Hosting.