Crawler hang via wildcard regex backtracking
Published Aug 28, 2026 · Updated Aug 28, 2026
Regular-expression denial of service in Scrapy Protego before 0.6.2 allows remote attackers to hang crawlers through crafted robots.txt rules. _URLPattern._prepare_pattern_for_regex converts every asterisk in an Allow or Disallow value to a lazy regex wildcard, producing exponential backtracking when can_fetch tests a near-miss URL. Exploitation requires a crawler to retrieve and parse attacker-controlled robots.txt content and then test such a URL, monopolizing its matching thread.
Summary
What happened
Regular-expression denial of service in Scrapy Protego before 0.6.2 allows remote attackers to hang crawlers through crafted robots.txt rules. _URLPattern._prepare_pattern_for_regex converts every asterisk in an Allow or Disallow value to a lazy regex wildcard, producing exponential backtracking when can_fetch tests a near-miss URL. Exploitation requires a crawler to retrieve and parse attacker-controlled robots.txt content and then test such a URL, monopolizing its matching thread.
The record
- CVE
- CVE-2026-55520
- Published
- Aug 28, 2026
- Updated
- Aug 28, 2026
- Vendor
- Scrapy
- Product
- Protego
- Classifications
- CWE-400, CWE-1333, T1499.004
- Attack vector
- network
- Privileges
- unauthenticated
Timeline
How it unfolded
- Aug 28, 2026CVE publishedPublication date reported by the CVE source.
- Aug 28, 2026Record updatedLatest update available in the CVE record.
Exploitability
Present is not the same as exploitable
Compare your product and version with the public record. A matching version still requires validation against your environment.
Is a vulnerable build present?
Compare these published version ranges with your installed build and any vendor patches.
- Affected versionversion=< 0.6.2
What conditions does exploitation require?
What is affected?
Published CVSS scores
CVSS describes severity. EPSS estimates exploitation probability.
Attacks
What attackers are doing with it
Daily unique IPs observed by Shadowserver honeypots for known exploited vulnerabilities (KEVs). Missing observations do not establish an absence of attacks.
Weakness, pattern, technique
Public exploit references
- Upstream Protego ReDoS proof of conceptproof of concept · demonstrated
Labels summarize the accepted research assessment. They do not indicate a test against your environment.
Technologies
Your stack
See the directory against your own environment.
Your stack
Check the software in your environment
Book a demo to see how Hinoki identifies affected software and validates exploitability in your environment.
Book a demo