| Unidentified Indexer/Scraper |
26,613 |
224 |
| Scrapy/2.11.2 |
19,167 |
0 |
| Unidentified WordPress Searcher |
18,534 |
0 |
| AhrefsBot/7.0 |
9,923 |
357 |
| Amazonbot/0.1 |
8,021 |
126 |
| GPTBot/1.2 |
4,551 |
358 |
| SemrushBot/7~bl |
3,184 |
671 |
| Googlebot/2.1 |
3,047 |
435 |
| Anonymous Crawler 1 |
2,980 |
0 |
| ClaudeBot/1.0 |
2,829 |
123 |
| Bytespider |
2,759 |
140 |
| facebookexternalhit/1.1 |
2,145 |
2,130 |
| MJ12bot/v1.4.8 |
1,925 |
131 |
| AwarioBot/1.0 |
1,800 |
77 |
| BLEXBot/1.0 |
1,623 |
53 |
| bingbot/2.0 |
1,294 |
153 |
| Sogou web spider/4.0 |
1,174 |
19 |
| Unidentified PHP Searcher |
920 |
0 |
| serpstatbot/2.1 |
803 |
4 |
| meta-externalagent/1.1 |
730 |
0 |
| netEstate NE Crawler |
675 |
20 |
| Unidentified Configuration Searcher |
609 |
1 |
| Unidentified Image Searcher |
596 |
0 |
| YandexBot/3.0 |
589 |
279 |
| Mozlila/5.0 |
568 |
0 |
| DreamHost Data Team |
567 |
0 |
| DotBot/1.2 |
552 |
255 |
| Unidentified Directory Searcher |
517 |
0 |
| OAI-SearchBot/1.0 |
483 |
394 |
| PerplexityBot/1.0 |
360 |
33 |
| GoogleOther |
352 |
0 |
| ImagesiftBot |
349 |
73 |
| Unidentified Demandware Searcher |
201 |
0 |
| ChatGPT-User/1.0 |
193 |
47 |
| Anonymous Crawler 2 |
175 |
0 |
| SEOkicks |
141 |
3 |
| Turnitin |
135 |
14 |
| DuckDuckBot/1.1 |
134 |
0 |
| DataForSeoBot/1.0 |
123 |
27 |
| MarketGoo/2.1 |
121 |
4 |
| SeznamBot/4.0 |
109 |
49 |
| Applebot/0.1 |
107 |
45 |
| MJ12bot/v2.0.0 |
105 |
49 |
| CCBot/2.0 |
103 |
27 |
| Timpibot/0.9 |
102 |
0 |
| Barkrowler/0.9 |
94 |
16 |
| MJ12bot/v2.0.2 |
94 |
47 |
| Tumblr/14.0.835.186 |
87 |
0 |
| TikTokSpider |
74 |
0 |
| SemrushBot-BA |
71 |
34 |
| panscient.com |
59 |
3 |
| ALittle Client |
57 |
0 |
| Dataprovider.com |
56 |
3 |
| PetalBot |
47 |
41 |
| Unidentified Zip File Searcher |
33 |
0 |
| Googlebot-Image/1.0 |
31 |
0 |
| CensysInspect/1.1 |
24 |
0 |
| wpbot/1.3 |
24 |
12 |
| Chrome Privacy Preserving Prefetch Proxy |
21 |
0 |
| MJ12bot/v2.0.1 |
20 |
10 |
| Qwantbot/1.0 |
20 |
10 |
| Unidentified SQL File Searcher |
19 |
0 |
| BW/1.3; rb.gy/qyzae5 |
15 |
3 |
| fluid/0.0 |
15 |
15 |
| CMS-Checker/1.0 |
14 |
0 |
| GPTBot/1.0 |
14 |
13 |
| InternetMeasurement/1.0 |
13 |
0 |
| ZoominfoBot |
12 |
12 |
| ips-agent |
11 |
3 |
| MixrankBot |
11 |
0 |
| FeedBurner/1.0 |
10 |
0 |
| Grammarly/1.0 |
10 |
0 |
| Let's Encrypt validation server |
10 |
0 |
| MojeekBot/0.11 |
10 |
6 |
| YandexFavicons/1.0 |
10 |
0 |
| 2ip bot/1.1 |
9 |
0 |
| AliyunSecBot/Aliyun |
9 |
4 |
| SemanticScholarBot |
9 |
6 |
| BacklinksExtendedBot |
8 |
4 |
| linkbot 1.0 |
8 |
8 |
| mithril-crawler/1.0 |
8 |
6 |
| Linkbot 1.0 |
7 |
0 |
| BitSightBot/1.0 |
6 |
0 |
| DomainStatsBot/1.0 |
6 |
4 |
| Scrapy/2.12.0 |
6 |
0 |
| YisouSpider |
6 |
3 |
| CheckMarkNetwork/1.0 |
5 |
3 |
| Feedly/1.0 |
5 |
0 |
| NetcraftSurveyAgent/1.0 |
5 |
0 |
| SurdotlyBot/1.0 |
5 |
1 |
| Baiduspider/2.0 |
4 |
0 |
| GeedoProductSearch |
4 |
2 |
| Pandalytics/2.0 |
4 |
2 |
| Pinterestbot/1.0 |
4 |
2 |
| Timpibot/0.8 |
4 |
0 |
| DuckAssistBot/1.2 |
3 |
0 |
| ShellBot 2.0 |
3 |
0 |
| Website-info.net-Robot |
3 |
2 |
| Claude-SearchBot/1.0 |
2 |
1 |
| DNSScanner/1.0 |
2 |
0 |
| PriEcoBot/1.0 |
2 |
0 |
| Screaming Frog SEO Spider/20.2 |
2 |
0 |
| SEMrushBot |
2 |
0 |
| URLCheckerBot |
2 |
0 |
| VsuSearchSpider/1.0 |
2 |
1 |
| WireReaderBot/1.0 |
2 |
1 |
| YandexImages/3.0 |
2 |
0 |
| AGU/0.8.6 |
1 |
0 |
| archive.org_bot |
1 |
0 |
| KStandBot/1.0 |
1 |
0 |
| ScannerBot/1.0 |
1 |
0 |
| trafilatura/2.0.0 |
1 |
0 |
| Twitterbot/1.0 |
1 |
0 |
| yacybot |
1 |
1 |