How long does it take to crawl a website? It appears that my audit is stuck.
The amount of time it takes for our bot to crawl a website varies depending on your crawl-delay setting and the size of your audit (how many pages we need to crawl).
In some cases, you can experience a delay in the Site Audit after the crawling pages counter stops. Usually, this happens in the last stage of the audit. At this time Site Audit bot checks on the external links and images and saves the information. If your website has a lot of external links and images, it is not unusual to see a delay, and it can take some time for your audit to finish.
Most of the time, there is nothing to worry about, and you can stop crawling yourself if needed:
If you stop and save results in the middle of a crawl, the audit will present the new results of its unfinished crawl as your up-to-date audit.
If you discard results, Site Audit will not crawl or audit your pages and your crawl budget will be saved.
However, if your audit is stuck for more than 24 hours, and you cannot stop it yourself, please contact our Support team.
- What Issues Can Site Audit Identify?
- How many pages can I crawl in a Site Audit?
- How do I audit a subdomain?
- Can I manage the automatic Site Audit re-run schedule?
- Can I set up a custom re-crawl schedule?
- How is Site Health Score calculated in the Site Audit tool?
- How Does Site Audit Select Pages to Analyze for Core Web Vitals?
- How do you collect data to measure Core Web Vitals in Site Audit?
- Why is there a difference between GSC and Semrush Core Web Vitals data?
- Why are only a few of my website’s pages being crawled?
- Why do working pages on my website appear as broken?
- Why can’t I find URLs from the Audit report on my website?
- Why does Semrush say I have duplicate content?
- Why does Semrush say I have an incorrect certificate?
- What are unoptimized anchors and how does Site Audit identify them?
- What do the Structured Data Markup Items in Site Audit Mean?
- Can I stop a current Site Audit crawl?
- How to Disable JS Rendering and Inspect a Page
- Configuring Site Audit
- Troubleshooting Site Audit
- Site Audit Overview Report
- Site Audit Thematic Reports
- Reviewing Your Site Audit Issues
- Site Audit Crawled Pages Report
- Site Audit Statistics
- Compare Crawls and Progress
- Exporting Site Audit Results
- How to Optimize your Site Audit Crawl Speed
- How To Integrate Site Audit with Zapier
- JS Impact Report