How to Review Crawler Warnings
Decide what to do with URLs the crawler flags prior to delivery.
Choose whether to check first
The optional pre-check looks for reachability and technical indexing problems, such as HTTP errors, access restrictions, robots rules, noindex signals or an unexpected canonical URL. It doesn't replace a Google Index Check.
Choose Check URLs and skip blocked pages to apply this filter. Use all URLs bypasses it; it doesn't repair a blocked or unavailable page.
Review a flagged URL
Open the project and read the explanation in Crawler / decision. Where available, inspect the HTTP status, final URL and check time.
- Continue: allow this URL to proceed despite the warning and queue its delivery.
- Reject: exclude this URL from delivery while retaining its record in the project.
- Security block: continuation is unavailable. Review the supplied address or contact support.
Eligible URLs continue independently; one problematic URL doesn't have to hold up the rest.
Confirm several warnings
Continue all warnings (N) applies to all eligible warnings across the project, such as rows on other pages or hidden by your current filters. Review the whole set prior to confirming. Security blocks remain excluded.
Where you control the page, fix the cause first. Where it belongs to a publisher, ask the publisher to review the issue. Continuing a warning doesn't change the page or guarantee indexing.
See TOK and billing for service costs; don't treat a crawler warning as a confirmed billing credit.
Need help with a particular project?
Contact support