Activity by Crawler mithril-crawler/1.0
Crawler
| Name | mithril-crawler/1.0 |
|---|---|
| Type | Indexer/Scraper |
| Type Description | Identifies HTML pages for indexing and/or scraping |
| Is Identified | Yes |
| Web Page | https://498-search-engine.github.io/website/ |
| Email Address | mithril498@umich.edu |
| Note | |
| User Agent Strings | 1 |
| IP Addresses | 2 | Resources | 2 |
| Requests | 8 |
| Requests for robots.txt | 6 |
| Earliest Request | 2025-04-04 20:30:32 |
| Latest Request | 2025-04-08 20:00:21 |
User Agent String ▼
| ID⭡ | User Agent String⭥ | Requests⭥ |
|---|---|---|
| 1503 | mithril-crawler/1.0 (mithril498@umich.edu; +https://498-search-engine.github.io/website/) | 8 |
IP Addresses ▼
| IP Address⭡ | Host | Crawler⭥ | Requests⭥ |
|---|---|---|---|
| 3.17.27.93 | ec2-3-17-27-93.us-east-2.compute.amazonaws.com | mithril-crawler/1.0 | 5 |
| 18.118.171.47 | ec2-18-118-171-47.us-east-2.compute.amazonaws.com | mithril-crawler/1.0 | 3 |
Resources ▼
Domain sphaerula.com ▼
Resources Refused or Not Found ▼
| Resource | Resource Type | Status Code | Requests |
|---|---|---|---|
| /popular-science/a-series-of-fortunate-events/ | Nonexistent | 404 | 2 |
| /robots.txt | Text | 404 | 6 |
Requests ▼
| ID⭥ | Method⭥ | Domain⭥ | Resource⭥ | Referrer⭥ | Status⭥ | IP Address⭥ | User Agent String ID⭥ | Timestamp⭣ | Crawler⭥ |
|---|---|---|---|---|---|---|---|---|---|
| 182095 | GET | sphaerula.com | /robots.txt | 404 | 3.17.27.93 | 1503 | 2025-04-04 20:30:32 | mithril-crawler/1.0 | |
| 182174 | GET | sphaerula.com | /robots.txt | 404 | 3.17.27.93 | 1503 | 2025-04-04 21:55:22 | mithril-crawler/1.0 | |
| 182387 | GET | sphaerula.com | /robots.txt | 404 | 3.17.27.93 | 1503 | 2025-04-05 16:04:16 | mithril-crawler/1.0 | |
| 182388 | GET | sphaerula.com | /robots.txt | 404 | 3.17.27.93 | 1503 | 2025-04-05 16:16:41 | mithril-crawler/1.0 | |
| 182389 | GET | sphaerula.com | /popular-science/a-series-of-fortunate-events/ | 404 | 3.17.27.93 | 1503 | 2025-04-05 16:17:03 | mithril-crawler/1.0 | |
| 182708 | GET | sphaerula.com | /robots.txt | 404 | 18.118.171.47 | 1503 | 2025-04-06 15:05:48 | mithril-crawler/1.0 | |
| 183390 | GET | sphaerula.com | /robots.txt | 404 | 18.118.171.47 | 1503 | 2025-04-08 20:00:19 | mithril-crawler/1.0 | |
| 183391 | GET | sphaerula.com | /popular-science/a-series-of-fortunate-events/ | 404 | 18.118.171.47 | 1503 | 2025-04-08 20:00:21 | mithril-crawler/1.0 |