About The Tool
The Screaming Frog SEO Spider allows you to quickly crawl, analyse and audit a sites’ onsite SEO. It can be used to crawl both small and very large websites, where manually checking every page would be extremely labour intensive (or impossible!) and where you can easily miss a redirect, meta refresh or duplicate page issue. You can view, analyse and filter the crawl data as it’s gathered and updated continuously in the program’s user interface.
The SEO Spider allows you to export key onsite SEO elements (URL, page title, meta description, headings etc) to a spread sheet, so it can easily be used as a base for SEO recommendations. Check our out demo video above.
Crawl 500 URLs For Free
For just £149 per year you can purchase a licence, which removes the 500 URL crawl limit, allows you to save crawls, and opens up the spider’s configuration options and advanced features.
Alternatively hit the ‘buy a licence’ button in the SEO Spider to buy a licence after downloading and trialing the software.
FAQ & User Guide
The SEO Spider crawls sites like Googlebot discovering hyperlinks in the HTML using a breadth-first algorithm. It uses a configurable hybrid storage engine, able to save data in RAM and disk to crawl large websites. By default it will only crawl the raw HTML of a website, but it can also render web pages using headless Chromium to discover content and links.
For more guidance and tips on our to use the Screaming Frog SEO crawler –
Keep updated with future releases of the by subscribing to the our RSS feed or following us on Twitter @screamingfrog.
Support & Feedback
If you have any technical problems, feedback or feature requests for the SEO Spider, then please just contact us via our support. We regularly update the SEO Spider and currently have lots of new features in development!