Crawler documentation
itsbydavidBot
If you found this in your access log, this page is the answer. It is a diagnostic scanner, it runs one page at a time at someone’s request, and blocking it takes one line.
What it is
A diagnostic, not a crawler
itsbydavidBot checks whether AI systems can read a website. Someone pastes a URL into a free tool, and it fetches that one page, plus robots.txt, sitemap.xml and llms.txt, to report what a crawler would receive.
It does not spider. It does not follow your links, it does not queue up your site, and it does not come back on a schedule. One request set, when a person asks for it.
It is run by David Monzón, an independent consultant in Guatemala. There is no company behind it and no data resale.
The part that looks alarming
Why it sends other crawlers’ names
The scanner reports whether a site actually enforces the crawler policy it declares. Measuring that means asking for the page the way GPTBot would ask, and seeing whether the server answers differently. There is no way to measure it without sending a bot-shaped request.
So the user agent carries a probing-token. It never impersonates. The bot’s own name comes first, this URL comes second, and the token being tested is labelled as what it is:
Mozilla/5.0 (compatible; itsbydavidBot/1.0; +https://itsbydavid.com/tools/itsbydavidbot/; probing-token=GPTBot)A rule matching on "GPTBot" still fires, which is the point. But anyone reading the log sees who actually made the request, and can tell it apart from the real GPTBot, which arrives from OpenAI’s own address ranges.
Robots policy
It respects robots.txt, and it respects being told no
A Disallow aimed at itsbydavidBot is honoured, always. If your robots.txt names this bot and says no, it stops: it reads your policy and reports it, but runs no live requests against your pages.
To block it by name
User-agent: itsbydavidBot
Disallow: /That is the whole thing. No form, no account, nothing to email. If you would rather it never touched your server again, this is enough.
Limits
What it costs you
- At most about fifteen requests for a single scan, all at once, none repeated.
- One scan per domain per hour, globally. A second person asking about the same domain within that window is served the cached result rather than a new round of requests.
- Per-IP limits on the requesting side, so the tool cannot be pointed at someone else’s server repeatedly.
- A five second timeout per request and a two megabyte ceiling on what it reads.
Contact
A real person answers
If this bot did something you did not expect, or you want it to stop and blocking it is not convenient, write to me and I will handle it.