How to Crawl a Sub-Folder Only in Screaming Frog [2025]

For example – to crawl mysite.com/buyers-guides/

To crawl only URLs in the /buyers-guides/ folder on networldsports.co.uk using Screaming Frog, follow these steps:

1. Open Screaming Frog SEO Spider and enter the URL: https://www.mysite.com/buyers-guides/
2. Go to Configuration > Spider > Crawl
3. Under “Crawl Limits,” select “Limit Crawl Depth” and set it to 1 to crawl only the specified folder
4. In the “Include” box, add this pattern: https://www.mysite.com/buyers-guides/.*
5. Start the crawl by clicking the “Start” button

This configuration will ensure Screaming Frog only crawls URLs within the /buyers-guides/ folder on networldsports.co.uk, excluding other sections of the website.