← AI agent directory

Crawlspace · Model training

Crawlspace

Operator
Crawlspace
Purpose
Model training (Plumb's classification)
Verification
None
Follows robots.txt
Not documented

01Identify and verify

How it shows up in your logs

User-agent contains

  • Crawlspace

Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.

Is it really Crawlspace?

The operator publishes no ranges or keys, so any request using this name is unverified.

02Publishers

How major publishers treat it

Its robots.txt token is not documented, so the leaderboard cannot grade it.

03robots.txt

robots.txt token not documented

Crawlspace does not document a robots.txt token for Crawlspace, and the leaderboard does not grade one, so Plumb offers no rule to paste. A guessed name would match nothing and block nothing. To stop it, block its user agent at your CDN or firewall.

Crawlspace (Crawlspace): what it is and how to block it · Plumb