Baseline Labs wiki / Bot
ApifyWebsiteContentCrawler
Bot in AI search and SEO
Stub
The training scraper ApifyWebsiteContentCrawler is operated by Apify. It collects text in bulk for model training, which is a separate question from whether an AI product can cite the site today. It appears in server logs as apifywebsitecontentcrawler.
Verification
There is no documentation page or IP range list from Apify that we can point at for this token.
A user agent is a claim, not a proof: any client can send any name. Where an operator publishes IP ranges, a reverse lookup is the only way to tell a real visit from an impersonation.
Seen by Baseline
Requests, 30 days
...
Last 7 days
...
First seen
...
Last seen
...
Requests from this bot across sites tracked by Baseline, last 30 days. Aggregated over - sites, never reported per site.
Last updated 2026-09-04. Written and maintained by Baseline Labs.
