Analyzer
Free User-Agent Lookup
Identify the crawler, bot, SEO tool or browser behind a User-Agent string.
Crawlers visiting? Invite them in
IndexBoost submits your URLs to search engines so the right bots show up sooner.
Get started freeContext
Trusting a user agent
- 01
Anyone can claim to be Googlebot
Verify with a reverse DNS lookup and a forward confirm, or the operator's published IP ranges.
- 02
AI crawlers have separate tokens
Training, search and user-triggered fetches use different names. Match the one you intend to allow or block.
- 03
Mobile Googlebot contains a phone UA
The string includes Android and Chrome tokens. Look for 'Googlebot' to tell them apart.