Activity by Crawler SemanticScholarBot

Crawler

Name SemanticScholarBot
Type Indexer/Scraper
Type Description Identifies HTML pages for indexing and/or scraping
Is Identified Yes
Web Page https://www.semanticscholar.org/crawler
Email Address
Note
User Agent Strings 1
IP Addresses 1
Resources 2
Requests 18
Requests for robots.txt 12
Earliest Request 2025-04-02 14:49:05
Latest Request 2025-08-18 21:04:49

User Agent String ▼

1 User Agent String
ID⭡ User Agent String⭥ Requests⭥
1496 Mozilla/5.0 (compatible) SemanticScholarBot (+https://www.semanticscholar.org/crawler) 18

IP Address ▼

1 IP Address–Crawler Combination
IP Address⭡ Host Crawler⭥ Requests⭥
35.160.27.221 ec2-35-160-27-221.us-west-2.compute.amazonaws.com SemanticScholarBot 18

Resources ▼

Domain sphaerula.com ▼

Resources Found ▼

1 Resource–Status Code Combination, 6 Requests
Resource Resource Type Status Code Requests
/robots.txt Text 200 6

Resources Refused or Not Found ▼

2 Resource–Status Code Combinations, 12 Requests
Resource Resource Type Status Code Requests
/popular-science/a-series-of-fortunate-events/ Nonexistent 404 6
/robots.txt Text 404 6

Requests ▼

18 Requests
ID⭥ Method⭥ Domain⭥ Resource⭥ Referrer⭥ Status⭥ IP Address⭥ User Agent String ID⭥ Timestamp⭣ Crawler⭥
181575 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-02 14:49:05 SemanticScholarBot
181576 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-02 14:49:15 SemanticScholarBot
181577 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-04-02 14:49:28 SemanticScholarBot
185012 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-15 17:02:53 SemanticScholarBot
185013 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-15 17:03:08 SemanticScholarBot
185014 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-04-15 17:03:22 SemanticScholarBot
189235 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-28 20:29:01 SemanticScholarBot
189236 GET sphaerula.com /robots.txt 404 35.160.27.221 1496 2025-04-28 20:29:08 SemanticScholarBot
189237 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-04-28 20:29:23 SemanticScholarBot
199631 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-07-14 13:20:19 SemanticScholarBot
199632 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-07-14 13:20:31 SemanticScholarBot
199633 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-07-14 13:20:42 SemanticScholarBot
294498 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-08-06 03:31:53 SemanticScholarBot
294499 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-08-06 03:32:07 SemanticScholarBot
294500 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-08-06 03:32:21 SemanticScholarBot
307621 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-08-18 21:04:21 SemanticScholarBot
307622 GET sphaerula.com /robots.txt 200 35.160.27.221 1496 2025-08-18 21:04:34 SemanticScholarBot
307623 GET sphaerula.com /popular-science/a-series-of-fortunate-events/ 404 35.160.27.221 1496 2025-08-18 21:04:49 SemanticScholarBot