Baseline Labs wiki / Bot
Scrapy
Bot in AI search and SEO
Stub
The default user agent of Scrapy, the most widely used open-source scraping framework. Anything from academic research to industrial AI data collection ships under this name, so it is a tool signature rather than an operator identity.
Verification
Operator documentation: https://scrapy.org
A user agent is a claim, not a proof: any client can send any name. Where an operator publishes IP ranges, a reverse lookup is the only way to tell a real visit from an impersonation.
Seen by Baseline
Requests, 30 days
...
Last 7 days
...
First seen
...
Last seen
...
Requests from this bot across sites tracked by Baseline, last 30 days. Aggregated over - sites, never reported per site.
Last updated 2026-09-04. Written and maintained by Baseline Labs.
