← AI agent directory

ByteDance · Model training

Bytespider

Operator
ByteDance
Purpose
Model training (as the leaderboard grades it)
Verification
None
Follows robots.txt
Not documented

01Identify and verify

How it shows up in your logs

User-agent contains

  • Bytespider

Matching is case-insensitive. Anyone can send this string, so a match alone says only what the request claims to be.

Is it really Bytespider?

The operator publishes no ranges or keys, so any request using this name is unverified.

Bytespider publishes no ranges.

02Publishers

How major publishers treat it

56%of 319 publishers on the leaderboard block Bytespider outright (180). 97 more close some sections. 177 name it explicitly in their robots.txt.

From the scan of 21 November 2026 · see every publisher

03robots.txt

A rule you can paste

The group names Bytespider exactly. A group for a different token from the same company does not cover it.

ByteDance does not document whether this agent follows robots.txt. Plumb can tell you whether it does on your site.

# Bytespider (ByteDance)
User-agent: Bytespider
Disallow: /

04Same operator

ByteDance's other agents

Bytespider (ByteDance): what it is and how to block it · Plumb