This question is almost always argued on principle — either "they are taking our work" or "you cannot fight the future" — and almost never with the two numbers that decide it.
Here are the numbers, and then the cases where each answer is right.
What blocking actually costs
Referral traffic you currently receive. Measurable today. Every visit from an assistant is a visit you lose if you become unreadable to it. For most sites this is small in absolute terms and growing; for some it is already meaningful.
Presence in answers where you are not cited. Harder. Assistants routinely synthesise from sources without linking, and being part of that synthesis still shapes whether your product or view is mentioned at all. Blocking removes you from the conversation as well as from the citation, and only one of those was ever going to show up in your analytics.
Future discovery. Increasing numbers of people begin research inside an assistant rather than a search box. Being unreadable to those surfaces is, over a long enough period, similar to having been unreadable to Google in 2005.
What blocking actually saves
Bandwidth and origin load. Real, and occasionally the whole reason. Some crawlers are badly behaved and fetch aggressively. If you are paying per gigabyte or per request, this is a line item you can compute.
Content you would rather not train a competitor. Legitimate, especially if your material is your product — a paid course, a research corpus, a reference work people pay to access.
Leverage. Several publishers have blocked and then licensed. That only works at a scale where somebody wants to negotiate with you.
Who should block
Publishers whose product is the article. If people pay for your writing, or your advertising revenue depends on people reading it on your site, an assistant that answers the question without the click is a direct substitute. Block training, and consider blocking retrieval.
Anyone whose content is the paid product. Courses, reference works, research. Straightforward.
Sites with a real cost problem. If crawlers are a measurable share of your hosting bill, that is a sufficient reason on its own and needs no philosophy behind it.
Who should not block
Anyone selling something the content markets. If your articles exist to bring people who might buy, being cited by an assistant is distribution. Blocking is turning off a channel because you dislike its business model — a position that costs you and inconveniences nobody else.
Documentation and developer tools. Being the source an assistant answers from is now a genuine acquisition channel. When somebody asks how to do a thing and the answer is your library with your syntax, that is worth more than the pageview would have been.
Local and service businesses. Assistants increasingly answer "who does X near me". You want to be in that answer.
Anyone with no measurement in place. Not a category of site, but the most common actual situation. Blocking without knowing what you get from these crawlers is a decision made blind, and it is reversible only in the sense that recovery takes months.
The middle position
The distinction most people miss is that you can block training and allow retrieval, because the crawlers are separate agents with separate tokens. GPTBot is training; OAI-SearchBot and ChatGPT-User are how you get read and cited. Disallowing the first and allowing the others is coherent, easy to express in robots.txt, and is what most sites should probably do.
There is also a partial block by path. Your documentation open, your paid course closed. Your blog open, your archive closed. Robots.txt supports this and hardly anyone uses it.
Decide with data
Before changing anything, get four weeks of two numbers: how many pages each crawler fetched, and how many visits each assistant sent back. The ratio is the entire argument, and it is different for every site.
If you cannot see either number, that is the first problem to solve — and it is solvable. A decision this consequential should not be made from a blog post's opinion, including this one.
Common questions
Does blocking AI crawlers hurt my Google rankings?
Blocking GPTBot, ClaudeBot, PerplexityBot or CCBot has no effect on Google Search rankings — they are unrelated crawlers. Google-Extended is also safe: it governs Gemini training and grounding only, not indexing. The one thing that would hurt you is blocking Googlebot itself, which is a different token and should stay allowed.
Can I block training but still appear in AI answers?
Yes, and for most sites it is the right position. Training and retrieval use separate crawlers with separate robots.txt tokens. Disallowing GPTBot, ClaudeBot and CCBot while allowing OAI-SearchBot, ChatGPT-User, PerplexityBot and Perplexity-User keeps you out of future training runs and visible in answers people are asking for now.
How much traffic do AI assistants actually send?
For most sites it is currently low single digits as a share of visits, and growing steadily. The distribution is very uneven: technical documentation, how-to content and reference material see far more than brochure sites. Because the range across sites is so wide, published averages are close to useless — your own measurement is the only figure worth acting on.