Activity by Crawler NetcraftSurveyAgent/1.0

Crawler

Name NetcraftSurveyAgent/1.0
Type Indexer/Scraper
Type Description Identifies HTML pages for indexing and/or scraping
Is Identified Yes
Web Page
Email Address info@netcraft.com
Note
User Agent Strings 1
IP Addresses 28
Resources 4
Requests 37
Requests for robots.txt 0
Earliest Request 2025-04-04 17:57:45
Latest Request 2025-08-11 07:01:16

User Agent String ▼

1 User Agent String
ID⭡ User Agent String⭥ Requests⭥
65 Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com) 37

IP Addresses ▼

28 IP Address–Crawler Combinations
IP Address⭡ Host Crawler⭥ Requests⭥
3.249.82.25 ec2-3-249-82-25.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.249.246.88 ec2-3-249-246-88.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.252.25.4 ec2-3-252-25-4.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 2
3.252.29.114 ec2-3-252-29-114.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.252.37.79 ec2-3-252-37-79.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.252.68.17 ec2-3-252-68-17.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.253.83.223 ec2-3-253-83-223.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.253.118.49 ec2-3-253-118-49.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.253.132.166 ec2-3-253-132-166.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.254.3.139 ec2-3-254-3-139.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
3.255.184.170 ec2-3-255-184-170.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
18.201.31.175 ec2-18-201-31-175.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.241.161.111 ec2-34-241-161-111.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 3
34.242.3.178 ec2-34-242-3-178.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 3
34.242.29.128 ec2-34-242-29-128.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.245.53.184 ec2-34-245-53-184.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.245.71.223 ec2-34-245-71-223.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.245.221.90 ec2-34-245-221-90.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 2
34.251.49.205 ec2-34-251-49-205.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.253.97.177 ec2-34-253-97-177.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
34.254.234.58 ec2-34-254-234-58.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
52.212.228.85 ec2-52-212-228-85.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
52.215.88.135 ec2-52-215-88-135.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
54.78.96.33 ec2-54-78-96-33.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
54.170.122.118 ec2-54-170-122-118.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 2
54.171.199.84 ec2-54-171-199-84.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 2
54.229.136.21 ec2-54-229-136-21.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 1
54.247.193.229 ec2-54-247-193-229.eu-west-1.compute.amazonaws.com NetcraftSurveyAgent/1.0 2

Resources ▼

Domain conradhalling.com ▼

Resources Found ▼

1 Resource–Status Code Combination, 4 Requests
Resource Resource Type Status Code Requests
/blog/ HTML 200 4

Resources Refused or Not Found ▼

2 Resource–Status Code Combinations, 14 Requests
Resource Resource Type Status Code Requests
/ Redirect 302 6
/ Redirect 429 8

Domain sphaerula.com ▼

Resources Found ▼

1 Resource–Status Code Combination, 3 Requests
Resource Resource Type Status Code Requests
/blog/ HTML 200 3

Resources Refused or Not Found ▼

2 Resource–Status Code Combinations, 16 Requests
Resource Resource Type Status Code Requests
/ Redirect 301 2
/ Redirect 302 14

Requests ▼

37 Requests
ID⭥ Method⭥ Domain⭥ Resource⭥ Referrer⭥ Status⭥ IP Address⭥ User Agent String ID⭥ Timestamp⭣ Crawler⭥
2739 GET conradhalling.com / 302 54.171.199.84 65 2025-04-04 17:57:45 NetcraftSurveyAgent/1.0
2740 GET conradhalling.com /blog/ 200 54.171.199.84 65 2025-04-04 17:57:49 NetcraftSurveyAgent/1.0
136875 GET sphaerula.com / 302 54.247.193.229 65 2025-04-05 08:54:52 NetcraftSurveyAgent/1.0
136876 GET sphaerula.com /blog/ 200 54.247.193.229 65 2025-04-05 08:54:54 NetcraftSurveyAgent/1.0
183018 GET sphaerula.com / 302 18.201.31.175 65 2025-04-07 03:09:52 NetcraftSurveyAgent/1.0
183032 GET sphaerula.com / 302 3.249.82.25 65 2025-04-07 07:27:36 NetcraftSurveyAgent/1.0
5772 GET conradhalling.com / 302 3.249.246.88 65 2025-04-07 07:31:59 NetcraftSurveyAgent/1.0
5775 GET conradhalling.com / 429 34.242.29.128 65 2025-04-07 07:58:03 NetcraftSurveyAgent/1.0
40877 GET conradhalling.com / 302 54.170.122.118 65 2025-05-03 00:15:30 NetcraftSurveyAgent/1.0
40878 GET conradhalling.com /blog/ 200 54.170.122.118 65 2025-05-03 00:15:33 NetcraftSurveyAgent/1.0
247321 GET sphaerula.com / 302 3.253.132.166 65 2025-05-10 19:48:08 NetcraftSurveyAgent/1.0
47864 GET conradhalling.com / 429 3.253.118.49 65 2025-05-10 20:11:26 NetcraftSurveyAgent/1.0
247324 GET sphaerula.com / 302 3.254.3.139 65 2025-05-10 20:18:37 NetcraftSurveyAgent/1.0
47883 GET conradhalling.com / 429 34.245.71.223 65 2025-05-10 21:17:40 NetcraftSurveyAgent/1.0
71754 GET conradhalling.com / 302 3.252.25.4 65 2025-06-01 03:59:36 NetcraftSurveyAgent/1.0
71755 GET conradhalling.com /blog/ 200 3.252.25.4 65 2025-06-01 03:59:40 NetcraftSurveyAgent/1.0
159737 GET sphaerula.com / 301 34.242.3.178 65 2025-06-01 05:45:16 NetcraftSurveyAgent/1.0
208900 GET sphaerula.com / 302 34.242.3.178 65 2025-06-01 05:45:19 NetcraftSurveyAgent/1.0
208901 GET sphaerula.com /blog/ 200 34.242.3.178 65 2025-06-01 05:45:22 NetcraftSurveyAgent/1.0
215759 GET sphaerula.com / 302 52.215.88.135 65 2025-06-07 03:37:40 NetcraftSurveyAgent/1.0
82123 GET conradhalling.com / 302 3.252.37.79 65 2025-06-07 09:18:25 NetcraftSurveyAgent/1.0
215970 GET sphaerula.com / 302 54.229.136.21 65 2025-06-07 09:19:56 NetcraftSurveyAgent/1.0
82139 GET conradhalling.com / 429 54.78.96.33 65 2025-06-07 10:04:41 NetcraftSurveyAgent/1.0
195367 GET sphaerula.com / 302 34.245.53.184 65 2025-07-07 12:46:29 NetcraftSurveyAgent/1.0
195377 GET sphaerula.com / 302 52.212.228.85 65 2025-07-07 14:00:23 NetcraftSurveyAgent/1.0
118850 GET conradhalling.com / 429 34.254.234.58 65 2025-07-07 14:44:49 NetcraftSurveyAgent/1.0
119123 GET conradhalling.com / 429 3.252.68.17 65 2025-07-07 17:18:50 NetcraftSurveyAgent/1.0
289838 GET sphaerula.com / 301 34.241.161.111 65 2025-08-07 11:01:22 NetcraftSurveyAgent/1.0
295280 GET sphaerula.com / 302 34.241.161.111 65 2025-08-07 11:01:25 NetcraftSurveyAgent/1.0
295281 GET sphaerula.com /blog/ 200 34.241.161.111 65 2025-08-07 11:01:28 NetcraftSurveyAgent/1.0
281248 GET conradhalling.com / 302 34.245.221.90 65 2025-08-07 11:57:43 NetcraftSurveyAgent/1.0
281249 GET conradhalling.com /blog/ 200 34.245.221.90 65 2025-08-07 11:57:47 NetcraftSurveyAgent/1.0
297136 GET sphaerula.com / 302 3.253.83.223 65 2025-08-10 23:41:57 NetcraftSurveyAgent/1.0
297148 GET sphaerula.com / 302 3.252.29.114 65 2025-08-11 00:33:42 NetcraftSurveyAgent/1.0
297194 GET sphaerula.com / 302 34.253.97.177 65 2025-08-11 03:47:20 NetcraftSurveyAgent/1.0
286306 GET conradhalling.com / 429 3.255.184.170 65 2025-08-11 04:55:46 NetcraftSurveyAgent/1.0
286408 GET conradhalling.com / 429 34.251.49.205 65 2025-08-11 07:01:16 NetcraftSurveyAgent/1.0