www.wired.com Open in urlscan Pro
151.101.2.194 Public Scan

Go To Rescan
Add Verdict Report

Submitted URL:
https://link.wired.com/click/27786158.1493579/aHR0cHM6Ly93d3cud2lyZWQuY29tL3N0b3J5L3RoZS1vbmxpbmUtc3BpZGVyLW1hcmtldC1pc...
Effective URL:
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d...
Submission: On May 20 via api (May 20th 2022, 10:58:46 pm UTC) from GB — Scanned from GB

Summary HTTP 129 Redirects Links 33 Behaviour Indicators

Summary

This website contacted 45 IPs in 5 countries across 40 domains to perform 129 HTTP transactions. The main IP is 151.101.2.194, located in United States and belongs to FASTLY, US. The main domain is www.wired.com. The Cisco Umbrella rank of the primary domain is 18964.
TLS certificate: Issued by GlobalSign Atlas R3 DV TLS CA 2020 on July 22nd 2021. Valid for: a year.

This is the only time www.wired.com was scanned on urlscan.io!

urlscan.io Verdict: No classification

Domain & IP information

	IP Address	AS Autonomous System
1 10	151.101.0.239 151.101.0.239	54113 (FASTLY) (FASTLY)
1 22	151.101.2.194 151.101.2.194	54113 (FASTLY) (FASTLY)
9	2606:4700::68... 2606:4700::6810:9440	13335 (CLOUDFLAR...) (CLOUDFLARENET)
3	172.217.16.130 172.217.16.130	15169 (GOOGLE) (GOOGLE)
2	2.20.86.213 2.20.86.213	16625 (AKAMAI-AS) (AKAMAI-AS)
2	108.156.255.177 108.156.255.177	16509 (AMAZON-02) (AMAZON-02)
1	23.35.236.247 23.35.236.247	16625 (AKAMAI-AS) (AKAMAI-AS)
1	2a00:1450:400... 2a00:1450:4001:828::200e	15169 (GOOGLE) (GOOGLE)
2	2a00:1450:400... 2a00:1450:4001:801::200e	15169 (GOOGLE) (GOOGLE)
1	2a04:4e42::282 2a04:4e42::282	54113 (FASTLY) (FASTLY)
3	151.101.64.239 151.101.64.239	54113 (FASTLY) (FASTLY)
2	2606:4700:10:... 2606:4700:10::6814:b844	13335 (CLOUDFLAR...) (CLOUDFLARENET)
1	18.233.44.90 18.233.44.90	14618 (AMAZON-AES) (AMAZON-AES)
3	18.65.159.128 18.65.159.128	16509 (AMAZON-02) (AMAZON-02)
1	2a00:1450:400... 2a00:1450:4001:802::2008	15169 (GOOGLE) (GOOGLE)
2	2606:4700:440... 2606:4700:4400::ac40:9b40	13335 (CLOUDFLAR...) (CLOUDFLARENET)
5	151.101.192.239 151.101.192.239	54113 (FASTLY) (FASTLY)
1	54.77.221.60 54.77.221.60	16509 (AMAZON-02) (AMAZON-02)
7	151.101.128.239 151.101.128.239	54113 (FASTLY) (FASTLY)
8	2a00:1450:400... 2a00:1450:4001:813::200e	15169 (GOOGLE) (GOOGLE)
1	143.204.209.161 143.204.209.161	16509 (AMAZON-02) (AMAZON-02)
1	151.139.128.11 151.139.128.11	20446 (STACKPATH...) (STACKPATH-CDN)
1	108.157.4.21 108.157.4.21	16509 (AMAZON-02) (AMAZON-02)
1	65.9.58.150 65.9.58.150	16509 (AMAZON-02) (AMAZON-02)
3	35.201.67.47 35.201.67.47	15169 (GOOGLE) (GOOGLE)
2	35.190.91.160 35.190.91.160	15169 (GOOGLE) (GOOGLE)
1	35.190.59.101 35.190.59.101	15169 (GOOGLE) (GOOGLE)
1	72.21.195.65 72.21.195.65	16509 (AMAZON-02) (AMAZON-02)
1	2a00:1450:400... 2a00:1450:4001:812::200e	15169 (GOOGLE) (GOOGLE)
1	65.9.61.60 65.9.61.60	16509 (AMAZON-02) (AMAZON-02)
1	2a00:1450:400... 2a00:1450:400c:c08::9d	15169 (GOOGLE) (GOOGLE)
1	108.157.4.38 108.157.4.38	16509 (AMAZON-02) (AMAZON-02)
6	143.204.215.26 143.204.215.26	16509 (AMAZON-02) (AMAZON-02)
1	2a00:1450:400... 2a00:1450:4001:813::2004	15169 (GOOGLE) (GOOGLE)
1	2a00:1450:400... 2a00:1450:4001:82b::2003	15169 (GOOGLE) (GOOGLE)
1	54.155.18.159 54.155.18.159	16509 (AMAZON-02) (AMAZON-02)
1	52.51.233.122 52.51.233.122	16509 (AMAZON-02) (AMAZON-02)
3	2a00:1450:400... 2a00:1450:4001:800::200a	15169 (GOOGLE) (GOOGLE)
4	108.138.2.156 108.138.2.156	16509 (AMAZON-02) (AMAZON-02)
2	34.232.146.139 34.232.146.139	14618 (AMAZON-AES) (AMAZON-AES)
1	2a00:1450:400... 2a00:1450:4001:80f::2006	15169 (GOOGLE) (GOOGLE)
1	2a03:2880:f02... 2a03:2880:f02d:100:face:b00c:0:3	32934 (FACEBOOK) (FACEBOOK)
3	65.9.58.194 65.9.58.194	16509 (AMAZON-02) (AMAZON-02)
1	2a00:1450:400... 2a00:1450:4001:827::2002	15169 (GOOGLE) (GOOGLE)
129	45

10 151.101.0.239 (United States) 1 redirects

ASN54113 (FASTLY, US)

link.wired.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
media.wired.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.newyorker.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

22 151.101.2.194 (United States) 1 redirects

ASN54113 (FASTLY, US)

www.wired.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

9 2606:4700::6810:9440 (United States)

ASN13335 (CLOUDFLARENET, US)

cdn.cookielaw.org	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 172.217.16.130 (United States)

ASN15169 (GOOGLE, US)
PTR: fra15s46-in-f2.1e100.net

securepubads.g.doubleclick.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 2.20.86.213 (Milan, Italy)

ASN16625 (AKAMAI-AS, US)
PTR: a2-20-86-213.deploy.static.akamaitechnologies.com

z.moatads.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
px.moatads.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 108.156.255.177 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-108-156-255-177.dus51.r.cloudfront.net

c.amazon-adsystem.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 23.35.236.247 (Frankfurt am Main, Germany)

ASN16625 (AKAMAI-AS, US)
PTR: a23-35-236-247.deploy.static.akamaitechnologies.com

js-sec.indexww.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:828::200e (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

apis.google.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 2a00:1450:4001:801::200e (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

news.google.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a04:4e42::282 (United States)

ASN54113 (FASTLY, US)

polyfill.io	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 151.101.64.239 (United States)

ASN54113 (FASTLY, US)

journey.wired.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.allure.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.teenvogue.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 2606:4700:10::6814:b844 (United States)

ASN13335 (CLOUDFLARENET, US)

geolocation.onetrust.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 18.233.44.90 (Ashburn, United States)

ASN14618 (AMAZON-AES, US)
PTR: ec2-18-233-44-90.compute-1.amazonaws.com

infinityid.condenastdigital.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 18.65.159.128 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-18-65-159-128.nrt51.r.cloudfront.net

player.cnevids.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:802::2008 (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

www.googletagmanager.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 2606:4700:4400::ac40:9b40 (United States)

ASN13335 (CLOUDFLARENET, US)

cookies-data.onetrust.io	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

5 151.101.192.239 (United States)

ASN54113 (FASTLY, US)

api.condenast.io	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.cntraveler.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
pitchfork.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.self.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.vanityfair.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 54.77.221.60 (Dublin, Ireland)

ASN16509 (AMAZON-02, US)
PTR: ec2-54-77-221-60.eu-west-1.compute.amazonaws.com

mb.moatads.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

7 151.101.128.239 (United States)

ASN54113 (FASTLY, US)

www.architecturaldigest.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.bonappetit.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.epicurious.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.glamour.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.gq.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.them.us	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
www.vogue.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

8 2a00:1450:4001:813::200e (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

www.google-analytics.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 143.204.209.161 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-143-204-209-161.fra53.r.cloudfront.net

z-na.associates-amazon.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 151.139.128.11 (United States)

ASN20446 (STACKPATH-CDN, US)

s.skimresources.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 108.157.4.21 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-108-157-4-21.dus51.r.cloudfront.net

static.hotjar.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 65.9.58.150 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-65-9-58-150.fra56.r.cloudfront.net

d1z2jf7jlzjs58.cloudfront.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 35.201.67.47 (Kansas City, United States)

ASN15169 (GOOGLE, US)
PTR: 47.67.201.35.bc.googleusercontent.com

t.skimresources.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 35.190.91.160 (Kansas City, United States)

ASN15169 (GOOGLE, US)
PTR: 160.91.190.35.bc.googleusercontent.com

p.skimresources.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 35.190.59.101 (Kansas City, United States)

ASN15169 (GOOGLE, US)
PTR: 101.59.190.35.bc.googleusercontent.com

r.skimresources.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 72.21.195.65 (Ashburn, United States)

ASN16509 (AMAZON-02, US)
PTR: s3-1.amazonaws.com

assoc-na.associates-amazon.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:812::200e (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

ampcid.google.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 65.9.61.60 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-65-9-61-60.fra56.r.cloudfront.net

cdn.parsely.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:400c:c08::9d (Brussels, Belgium)

ASN15169 (GOOGLE, US)

stats.g.doubleclick.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 108.157.4.38 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-108-157-4-38.dus51.r.cloudfront.net

script.hotjar.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

6 143.204.215.26 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-143-204-215-26.fra53.r.cloudfront.net

vars.hotjar.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools
dp8hsntg6do36.cloudfront.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:813::2004 (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

www.google.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:82b::2003 (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

www.google.co.uk	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 54.155.18.159 (Dublin, Ireland)

ASN16509 (AMAZON-02, US)
PTR: ec2-54-155-18-159.eu-west-1.compute.amazonaws.com

p1.parsely.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 52.51.233.122 (Dublin, Ireland)

ASN16509 (AMAZON-02, US)
PTR: ec2-52-51-233-122.eu-west-1.compute.amazonaws.com

in.hotjar.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 2a00:1450:4001:800::200a (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

imasdk.googleapis.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

4 108.138.2.156 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-108-138-2-156.fra56.r.cloudfront.net

d2c8v52ll5s99u.cloudfront.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

2 34.232.146.139 (Ashburn, United States)

ASN14618 (AMAZON-AES, US)
PTR: ec2-34-232-146-139.compute-1.amazonaws.com

capture.condenastdigital.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:80f::2006 (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

s0.2mdn.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a03:2880:f02d:100:face:b00c:0:3 (Frankfurt am Main, Germany)

ASN32934 (FACEBOOK, US)

connect.facebook.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

3 65.9.58.194 (United States)

ASN16509 (AMAZON-02, US)
PTR: server-65-9-58-194.fra56.r.cloudfront.net

dwgyu36up6iuz.cloudfront.net	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

1 2a00:1450:4001:827::2002 (Frankfurt am Main, Germany)

ASN15169 (GOOGLE, US)

pagead2.googlesyndication.com	Domain lookup VirusTotal SecurityTrails crt.sh Censys Domaintools

	Apex Domain Subdomains	Transfer
32	wired.com 2 redirects link.wired.com — Cisco Umbrella Rank: 478831 www.wired.com — Cisco Umbrella Rank: 18964 media.wired.com — Cisco Umbrella Rank: 39949 journey.wired.com — Cisco Umbrella Rank: 107367	2 MB
13	cloudfront.net d1z2jf7jlzjs58.cloudfront.net d2c8v52ll5s99u.cloudfront.net dwgyu36up6iuz.cloudfront.net dp8hsntg6do36.cloudfront.net	1 MB
9	cookielaw.org cdn.cookielaw.org — Cisco Umbrella Rank: 501	188 KB
8	google-analytics.com www.google-analytics.com — Cisco Umbrella Rank: 37	21 KB
7	skimresources.com s.skimresources.com — Cisco Umbrella Rank: 3078 t.skimresources.com — Cisco Umbrella Rank: 3071 p.skimresources.com — Cisco Umbrella Rank: 4136 r.skimresources.com — Cisco Umbrella Rank: 2873	15 KB
5	google.com apis.google.com — Cisco Umbrella Rank: 100 news.google.com — Cisco Umbrella Rank: 4610 ampcid.google.com — Cisco Umbrella Rank: 1833 www.google.com — Cisco Umbrella Rank: 7	88 KB
4	hotjar.com static.hotjar.com — Cisco Umbrella Rank: 645 script.hotjar.com — Cisco Umbrella Rank: 896 vars.hotjar.com — Cisco Umbrella Rank: 989 in.hotjar.com — Cisco Umbrella Rank: 1730	70 KB
4	doubleclick.net securepubads.g.doubleclick.net — Cisco Umbrella Rank: 187 stats.g.doubleclick.net — Cisco Umbrella Rank: 92	154 KB
3	googleapis.com imasdk.googleapis.com — Cisco Umbrella Rank: 407	363 KB
3	cnevids.com player.cnevids.com — Cisco Umbrella Rank: 18925	37 KB
3	condenastdigital.com infinityid.condenastdigital.com — Cisco Umbrella Rank: 28172 capture.condenastdigital.com — Cisco Umbrella Rank: 16610	1 KB
3	moatads.com z.moatads.com — Cisco Umbrella Rank: 374 mb.moatads.com — Cisco Umbrella Rank: 569 px.moatads.com — Cisco Umbrella Rank: 405	75 KB
2	parsely.com cdn.parsely.com — Cisco Umbrella Rank: 2689 p1.parsely.com — Cisco Umbrella Rank: 2118	21 KB
2	associates-amazon.com z-na.associates-amazon.com — Cisco Umbrella Rank: 6159 assoc-na.associates-amazon.com — Cisco Umbrella Rank: 3314	4 KB
2	onetrust.io cookies-data.onetrust.io — Cisco Umbrella Rank: 4103	86 B
2	onetrust.com geolocation.onetrust.com — Cisco Umbrella Rank: 824	810 B
2	amazon-adsystem.com c.amazon-adsystem.com — Cisco Umbrella Rank: 288	40 KB
1	googlesyndication.com pagead2.googlesyndication.com — Cisco Umbrella Rank: 95	13 KB
1	facebook.net connect.facebook.net — Cisco Umbrella Rank: 146	27 KB
1	2mdn.net s0.2mdn.net — Cisco Umbrella Rank: 264	17 KB
1	google.co.uk www.google.co.uk — Cisco Umbrella Rank: 3062	501 B
1	vogue.com www.vogue.com — Cisco Umbrella Rank: 23493	1 KB
1	vanityfair.com www.vanityfair.com — Cisco Umbrella Rank: 25557	1 KB
1	them.us www.them.us — Cisco Umbrella Rank: 39762	722 B
1	teenvogue.com www.teenvogue.com — Cisco Umbrella Rank: 35856	1 KB
1	self.com www.self.com — Cisco Umbrella Rank: 35362	1 KB
1	pitchfork.com pitchfork.com — Cisco Umbrella Rank: 27281	1 KB
1	newyorker.com www.newyorker.com — Cisco Umbrella Rank: 14237	1 KB
1	gq.com www.gq.com — Cisco Umbrella Rank: 14658	1 KB
1	glamour.com www.glamour.com — Cisco Umbrella Rank: 35060	1 KB
1	epicurious.com www.epicurious.com — Cisco Umbrella Rank: 35291	1 KB
1	cntraveler.com www.cntraveler.com — Cisco Umbrella Rank: 34569	1 KB
1	bonappetit.com www.bonappetit.com — Cisco Umbrella Rank: 16994	1 KB
1	architecturaldigest.com www.architecturaldigest.com — Cisco Umbrella Rank: 33603	1 KB
1	allure.com www.allure.com — Cisco Umbrella Rank: 34554	1 KB
1	condenast.io api.condenast.io — Cisco Umbrella Rank: 23893	7 KB
1	googletagmanager.com www.googletagmanager.com — Cisco Umbrella Rank: 71	147 KB
1	polyfill.io polyfill.io — Cisco Umbrella Rank: 1408	397 B
1	indexww.com js-sec.indexww.com — Cisco Umbrella Rank: 658	15 KB
0	getpublica.com Failed pbs.getpublica.com Failed
129	40

	Domain	Requested by
22	www.wired.com	1 redirects www.wired.com
9	cdn.cookielaw.org	www.wired.com cdn.cookielaw.org
8	www.google-analytics.com	www.googletagmanager.com www.google-analytics.com
8	media.wired.com	www.wired.com
5	dp8hsntg6do36.cloudfront.net	d2c8v52ll5s99u.cloudfront.net
4	d2c8v52ll5s99u.cloudfront.net	player.cnevids.com d2c8v52ll5s99u.cloudfront.net
3	dwgyu36up6iuz.cloudfront.net	d2c8v52ll5s99u.cloudfront.net
3	imasdk.googleapis.com	player.cnevids.com imasdk.googleapis.com
3	t.skimresources.com	www.wired.com s.skimresources.com
3	player.cnevids.com	www.wired.com player.cnevids.com
3	securepubads.g.doubleclick.net	www.wired.com securepubads.g.doubleclick.net
2	capture.condenastdigital.com
2	p.skimresources.com	www.wired.com
2	cookies-data.onetrust.io	cdn.cookielaw.org
2	geolocation.onetrust.com	cdn.cookielaw.org
2	news.google.com	www.wired.com news.google.com
2	c.amazon-adsystem.com	www.wired.com c.amazon-adsystem.com
1	pagead2.googlesyndication.com	srcdoc
1	connect.facebook.net	d2c8v52ll5s99u.cloudfront.net
1	s0.2mdn.net	imasdk.googleapis.com
1	in.hotjar.com	script.hotjar.com
1	p1.parsely.com	www.wired.com
1	www.google.co.uk	www.wired.com
1	www.google.com	www.wired.com
1	vars.hotjar.com	static.hotjar.com
1	script.hotjar.com	static.hotjar.com
1	stats.g.doubleclick.net	www.google-analytics.com
1	cdn.parsely.com	d1z2jf7jlzjs58.cloudfront.net
1	ampcid.google.com	www.google-analytics.com
1	assoc-na.associates-amazon.com	z-na.associates-amazon.com
1	r.skimresources.com	s.skimresources.com
1	d1z2jf7jlzjs58.cloudfront.net	www.wired.com
1	static.hotjar.com	www.wired.com
1	px.moatads.com	www.wired.com
1	s.skimresources.com	www.wired.com
1	z-na.associates-amazon.com	www.wired.com
1	www.vogue.com	www.wired.com
1	www.vanityfair.com	www.wired.com
1	www.them.us	www.wired.com
1	www.teenvogue.com	www.wired.com
1	www.self.com	www.wired.com
1	pitchfork.com	www.wired.com
1	www.newyorker.com	www.wired.com
1	www.gq.com	www.wired.com
1	www.glamour.com	www.wired.com
1	www.epicurious.com	www.wired.com
1	www.cntraveler.com	www.wired.com
1	www.bonappetit.com	www.wired.com
1	www.architecturaldigest.com	www.wired.com
1	www.allure.com	www.wired.com
1	mb.moatads.com	z.moatads.com
1	api.condenast.io	www.wired.com
1	www.googletagmanager.com	www.wired.com
1	infinityid.condenastdigital.com	www.wired.com
1	journey.wired.com	www.wired.com
1	polyfill.io	www.wired.com
1	apis.google.com	www.wired.com
1	js-sec.indexww.com	www.wired.com
1	z.moatads.com	www.wired.com
1	link.wired.com	1 redirects
0	pbs.getpublica.com Failed	player.cnevids.com
129	61

This site contains links to these domains. Also see Links.

Domain
subscribe.wired.com
events.wired.com
www.facebook.com
twitter.com
www.nature.com
www.scifac.hku.hk
www.ncbi.nlm.nih.gov
www.youtube.com
arachnoboards.com
www.science.org
www.standardspeaker.com
www.aphis.usda.gov
news.vin.com
wired.com
pinterest.com
instagram.com
www.tiktok.com
www.condenast.com
subscriptions.wired.com
www.condenaststore.com
www.cnspotlight.com

Subject Issuer	Validity	Valid
*.admagazine.fr GlobalSign Atlas R3 DV TLS CA 2020	`2021-07-22` - `2022-08-23`	a year	crt.sh
cookielaw.org Cloudflare Inc ECC CA-3	`2022-05-01` - `2023-05-01`	a year	crt.sh
*.g.doubleclick.net GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
moatads.com DigiCert SHA2 Secure Server CA	`2021-11-27` - `2022-11-29`	a year	crt.sh
c.amazon-adsystem.com Amazon	`2021-07-06` - `2022-06-27`	a year	crt.sh
san.casalemedia.com GeoTrust RSA CA 2018	`2021-12-12` - `2022-12-13`	a year	crt.sh
*.apis.google.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
*.news.google.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
polyfill.io GlobalSign Atlas R3 DV TLS CA 2022 Q1	`2022-03-08` - `2023-04-09`	a year	crt.sh
onetrust.com Cloudflare Inc ECC CA-3	`2022-01-12` - `2023-01-12`	a year	crt.sh
conde.io Amazon	`2021-06-30` - `2022-07-29`	a year	crt.sh
*.cnevids.com Amazon	`2021-09-18` - `2022-10-17`	a year	crt.sh
*.google-analytics.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
onetrust.io Cloudflare Inc ECC CA-3	`2022-05-04` - `2023-05-04`	a year	crt.sh
*.google.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
*.moatads.com DigiCert TLS RSA SHA256 2020 CA1	`2021-05-25` - `2022-06-25`	a year	crt.sh
z-na.associates-amazon.com Amazon	`2021-05-21` - `2022-06-19`	a year	crt.sh
*.skimresources.com DigiCert SHA2 Secure Server CA	`2021-09-27` - `2022-10-28`	a year	crt.sh
*.hotjar.com Amazon	`2021-11-25` - `2022-12-23`	a year	crt.sh
*.cloudfront.net Amazon	`2022-02-01` - `2023-01-31`	a year	crt.sh
assoc-na.associates-amazon.com Amazon	`2021-10-25` - `2022-09-30`	a year	crt.sh
*.parsely.com Amazon	`2021-07-05` - `2022-08-03`	a year	crt.sh
www.google.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
www.google.co.uk GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
upload.video.google.com GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
*.doubleclick.net GTS CA 1C3	`2022-05-04` - `2022-07-27`	3 months	crt.sh
*.facebook.com DigiCert SHA2 High Assurance Server CA	`2022-02-27` - `2022-05-28`	3 months	crt.sh

This page contains 6 frames:

Primary Page: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Frame ID: 82D6C75181DAB930DAF15E8A2DC0F68A
Requests: 105 HTTP requests in this frame

Frame: https://t.skimresources.com/api/v2/robots.txt?__skimjs_preflight__please_ignore__=true&rnd=0.47958488805356403
Frame ID: B5358C9D35E852B0170458060D0E647C
Requests: 1 HTTP requests in this frame

Frame: https://vars.hotjar.com/box-4924254a9ce4dc9b959b6e4a9b662d60.html
Frame ID: 1EDADADC29E4E1A0DF1BB7E8B8BBB765
Requests: 1 HTTP requests in this frame

Frame: https://imasdk.googleapis.com/js/sdkloader/ima3.js
Frame ID: 74EB80973EECAA988235E1317698CD9E
Requests: 19 HTTP requests in this frame

Frame: https://imasdk.googleapis.com/js/core/bridge3.516.0_en.html
Frame ID: 1E2292ACFFB8B02A22390E4747D5265B
Requests: 1 HTTP requests in this frame

Frame: https://pagead2.googlesyndication.com/omsdk/releases/live/omweb-v1.js
Frame ID: 6D1DDE976EBC65EC5A54E197A331EA27
Requests: 1 HTTP requests in this frame

Screenshot
Live screenshot

Full Image

Page Title

The Online Spider Market Is Massive—and Crawling With Issues | WIREDMenuStory SavedCloseStory SavedCloseSearchFacebookTwitterEmailSave StoryFacebookTwitterEmailSave StoryFacebookTwitterPinterestYouTubeInstagramTiktok

Page URL History Show full URLs

https://link.wired.com/click/27786158.1493579/aHR0cHM6Ly93d3cud2lyZWQuY29tL3N0b3J5L3RoZS1vbmxpbmUtc... HTTP 302
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues?utm_term=... HTTP 301
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=604... Page URL

Detected technologies

Google Sign-in (Social logins) Expand

Overall confidence: 100%
Detected patterns

apis\.google\.com/js/platform\.js

AMP (JavaScript frameworks) Expand

Overall confidence: 100%
Detected patterns

<link rel="amphtml"

DoubleClick Campaign Manager (DCM) (Advertising Networks) Expand

Overall confidence: 100%
Detected patterns

2mdn\.net

Facebook (Widgets) Expand

Overall confidence: 100%
Detected patterns

//connect\.facebook\.([a-z]+)/[^/]*/[a-z]*\.js

Google AdSense (Advertising Networks) Expand

Overall confidence: 100%
Detected patterns

googlesyndication\.com/
2mdn\.net

Google Analytics (Analytics) Expand

Overall confidence: 100%
Detected patterns

google-analytics\.com/(?:ga|urchin|analytics)\.js

Google Plus (Widgets) Expand

Overall confidence: 100%
Detected patterns

apis\.google\.com/js/[a-z]*\.js

Google Tag Manager (Tag Managers) Expand

Overall confidence: 100%
Detected patterns

googletagmanager\.com/gtm\.js

Hotjar (Analytics) Expand

Overall confidence: 100%
Detected patterns

//static\.hotjar\.com/

Moat (Analytics) Expand

Overall confidence: 100%
Detected patterns

moatads\.com

OneTrust (Cookie compliance) Expand

Overall confidence: 100%
Detected patterns

cdn\.cookielaw\.org
otSDKStub\.js

Polyfill (JavaScript Libraries) Expand

Overall confidence: 100%
Detected patterns

/polyfill\.min\.js

Page Statistics

129
Requests

96 %
HTTPS

36 %
IPv6

40
Domains

61
Subdomains

45
IPs

5
Countries

4797 kB
Transfer

12082 kB
Size

56
Cookies

33 Outgoing links

These are links going to different origins than the main page.

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_GLOBAL_NAV_CTA_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: SUBSCRIBE

Search URL Search Domain Scan URL

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_DESKTOP_NAV_ROLLOVER_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: Get WIRED + a free tote SUBSCRIBE

Search URL Search Domain Scan URL

URL: https://events.wired.com/home
Title: Events

Search URL Search Domain Scan URL

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_DESKTOP_PAYWALL_METER_ARTICLE_1_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: Subscribe now.

Search URL Search Domain Scan URL

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_TABLET_PAYWALL_METER_ARTICLE_1_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: Subscribe now.

Search URL Search Domain Scan URL

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_MOBILE_PAYWALL_METER_ARTICLE_1_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: Subscribe now.

Search URL Search Domain Scan URL

URL: https://www.facebook.com/dialog/feed?&display=popup&caption=&app_id=719405864858490&link=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Futm_source%3Dfacebook%26utm_medium%3Dsocial%26utm_campaign%3Donsite-share%26utm_brand%3Dwired%26utm_social-type%3Dearned
Title: Facebook

Search URL Search Domain Scan URL

URL: https://twitter.com/intent/tweet/?url=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Futm_source%3Dtwitter%26utm_medium%3Dsocial%26utm_campaign%3Donsite-share%26utm_brand%3Dwired%26utm_social-type%3Dearned&text=&via=wired
Title: Twitter

Search URL Search Domain Scan URL

URL: https://www.nature.com/articles/s42003-022-03374-0
Title: new report

Search URL Search Domain Scan URL

URL: https://www.scifac.hku.hk/page/detail/7696
Title: Alice Hughes

Search URL Search Domain Scan URL

URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6551028/
Title: antimicrobial, painkilling, and cancer-fighting compounds

Search URL Search Domain Scan URL

URL: https://www.youtube.com/c/thetarantulacollective
Title: Tarantula Collective

Search URL Search Domain Scan URL

URL: https://arachnoboards.com/
Title: Arachnoboards

Search URL Search Domain Scan URL

URL: https://www.science.org/doi/10.1126/science.363.6430.914
Title: report in the journal Science

Search URL Search Domain Scan URL

URL: https://www.standardspeaker.com/news/business/new-legislation-could-hamper-transport-of-exotic-animals/article_70070f25-9a43-52c3-886b-dda9455f4cfe.html
Title: ban the sale of non-native exotic pets

Search URL Search Domain Scan URL

URL: https://www.aphis.usda.gov/aphis/ourfocus/planthealth/import-information/lacey-act/lacey-act
Title: Lacey Act

Search URL Search Domain Scan URL

URL: https://news.vin.com/default.aspx?pid=210&Id=10811745
Title: veterinary groups

Search URL Search Domain Scan URL

URL: https://wired.com/gallery/best-trail-running-shoes-round-up/?itm_campaign=BottomRelatedStories&itm_content=footer-recirc
Title: shoes

Search URL Search Domain Scan URL

URL: https://subscribe.wired.com/subscribe/wired/145192?source=JNY_WIR_GLOBAL_FOOTER_0_GROWLER_SAVINGS_CONTROL_2022_ZZ
Title: One year for $29.99 $10 Get WIRED SUBSCRIBE

Search URL Search Domain Scan URL

URL: https://www.facebook.com/wired/
Title: Facebook

Search URL Search Domain Scan URL

URL: https://twitter.com/wired/
Title: Twitter

Search URL Search Domain Scan URL

URL: https://pinterest.com/wired/
Title: Pinterest

Search URL Search Domain Scan URL

URL: https://www.youtube.com/user/wired/
Title: YouTube

Search URL Search Domain Scan URL

URL: https://instagram.com/wired/
Title: Instagram

Search URL Search Domain Scan URL

URL: https://www.tiktok.com/@wired?lang=en
Title: Tiktok

Search URL Search Domain Scan URL

URL: https://www.condenast.com/brands/wired
Title: Advertise

Search URL Search Domain Scan URL

URL: https://subscriptions.wired.com/pubs/N3/WIR/Register.jsp?cds_page_id=175371&amp;cds_mag_code=WIR&amp;id=1423757547774&amp;lsid=50431012277019467&amp;vid=1
Title: Customer Care

Search URL Search Domain Scan URL

URL: https://www.condenaststore.com/
Title: Condé Nast Store

Search URL Search Domain Scan URL

URL: https://www.cnspotlight.com/
Title: Condé Nast Spotlight

Search URL Search Domain Scan URL

URL: https://www.condenast.com/user-agreement/
Title: User Agreement

Search URL Search Domain Scan URL

URL: http://www.condenast.com/privacy-policy#privacypolicy
Title: Privacy Policy and Cookie Statement

Search URL Search Domain Scan URL

URL: http://www.condenast.com/privacy-policy#privacypolicy-california
Title: Your California Privacy Rights.

Search URL Search Domain Scan URL

URL: http://www.condenast.com/privacy-policy#privacypolicy-optout
Title: Ad Choices

Search URL Search Domain Scan URL

Page URL History

This captures the URL locations of the websites, including HTTP redirects and client-side redirects via JavaScript or Meta fields.

https://link.wired.com/click/27786158.1493579/aHR0cHM6Ly93d3cud2lyZWQuY29tL3N0b3J5L3RoZS1vbmxpbmUtc3BpZGVyLW1hcmtldC1pcy1tYXNzaXZlLWFuZC1jcmF3bGluZy13aXRoLWlzc3Vlcz91dG1fdGVybT1QMSZ1dG1fc291cmNlPW5sJnV0bV9icmFuZD13aXJlZCZ1dG1fbWFpbGluZz1XSVJfRGFpbHlfMDUyMDIyJnV0bV9jYW1wYWlnbj1hdWQtZGV2JnV0bV9tZWRpdW09ZW1haWwmdXRtX2NvbnRlbnQ9V0lSX0RhaWx5XzA1MjAyMiZieGlkPTYwNDQ4ODNkOGYzZGEzNzFlZTZlYjNkNiZjbmRpZD02NDAzNTI4NSZlc3JjPWdyb3dsMi1yZWdHYXRlLTExMjAmc291cmNlPUVEVF9XSVJfTkVXU0xFVFRFUl8wX0RBSUxZX1paJm1iaWQ9bWJpZCUzRENSTVdJUjAxMjAxOSUwQSUwQQ/6044883d8f3da371ee6eb3d6Bcc618f74 HTTP 302
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues?utm_term=P1&utm_source=nl&utm_brand=wired&utm_mailing=WIR_Daily_052022&utm_campaign=aud-dev&utm_medium=email&utm_content=WIR_Daily_052022&bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&mbid=mbid%3DCRMWIR012019%0A%0A HTTP 301
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1 Page URL

Redirected requests

There were HTTP redirect chains for the following requests:

129 HTTP transactions
0 data transactions

Method
Protocol Status Resource
Path Size
x-fer Time
Latency Type
MIME-Type IP
Location

GET
H2

200

Primary Request / Show response
www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/
Redirect Chain

https://link.wired.com/click/27786158.1493579/aHR0cHM6Ly93d3cud2lyZWQuY29tL3N0b3J5L3RoZS1vbmxpbmUtc3BpZGVyLW1hcmtldC1pcy1tYXNzaXZlLWFuZC1jcmF3bGluZy13aXRoLWlzc3Vlcz91dG1fdGVybT1QMSZ1dG1fc291cmNlPW5...
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues?utm_term=P1&utm_source=nl&utm_brand=wired&utm_mailing=WIR_Daily_052022&utm_campaign=aud-dev&utm_medium=email...
https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source...

611 KB
151 KB

588ms
588ms

Document
text/html

151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

2cdbd73d559e8a310d85e46c4c0c1341feee9b6d9ff8b8c105d937d68395dc3d

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;

Strict-Transport-Security

max-age=31536000; preload

Request headers

Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
accept-language: en-GB,en;q=0.9

Response headers

accept-ranges: none
age: 0
apple-news-services-handled: false
apple-news-services-host: www.wired.com
apple-news-services-parsed-url: /story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
apple-news-services-request-url: /story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
cache-control: no-cache
content-encoding: gzip
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
content-type: text/html; charset=utf-8
date: Fri, 20 May 2022 22:58:39 GMT
payment: free
strict-transport-security: max-age=31536000; preload
vary: accept-encoding, Accept-Encoding, cn-experiments, X-UA-Device, Verso, bypass-verso-payment
verso: true
via: 1.1 varnish
x-cache: MISS
x-cache-hits: 0
x-esi: on
x-served-by: cache-lcy19261-LCY
x-timer: S1653087519.192450,VS0,VE555
x-ua-device: desktop

Redirect headers

accept-ranges: bytes
age: 2951
apple-news-services-handled: false
apple-news-services-host: www.wired.com
apple-news-services-parsed-url: /story/the-online-spider-market-is-massive-and-crawling-with-issues?utm_term=P1&utm_source=nl&utm_brand=wired&utm_mailing=WIR_Daily_052022&utm_campaign=aud-dev&utm_medium=email&utm_content=WIR_Daily_052022&bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&mbid=mbid%3DCRMWIR012019%0A%0A
apple-news-services-request-url: /story/the-online-spider-market-is-massive-and-crawling-with-issues?utm_term=P1&utm_source=nl&utm_brand=wired&utm_mailing=WIR_Daily_052022&utm_campaign=aud-dev&utm_medium=email&utm_content=WIR_Daily_052022&bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&mbid=mbid%3DCRMWIR012019%0A%0A
cache-control: no-cache
content-length: 0
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
content-type: text/html; charset=utf-8
date: Fri, 20 May 2022 22:58:39 GMT
location: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
strict-transport-security: max-age=31536000; preload
vary: cn-experiments, X-UA-Device, Verso, bypass-verso-payment
verso: true
via: 1.1 varnish
x-cache: HIT
x-cache-hits: 1
x-esi: on
x-served-by: cache-lcy19261-LCY
x-timer: S1653087519.160571,VS0,VE1
x-ua-device: desktop

GET
H/1.1 200
OK TA_science_Spider1-(1).jpg
media.wired.com/photos/628697bbc3e9412350f63f47/master/w_1600,c_limit/ 301 KB
302 KB 105ms
28ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/628697bbc3e9412350f63f47/master/w_1600,c_limit/TA_science_Spider1-(1).jpg
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: a566a49a53a5b9c184eb5094a82c3b35e68a25b08bf76215dbf2e375cfe6d9f9

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:39 GMT
Connection: keep-alive
Age: 99639
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=2942643 idim=2400x1600 ifmt=jpeg ofsz=308522 odim=1600x1067 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kiad7000161-IAD, cache-lhr7343-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087520.889436,VS0,VE2
Etag: "E7er39G7CHS6QL07LwyLGOYjzqVwlo86gcEbdvRQ3Hc"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 308522
timing-allow-origin: *
X-Cache-Hits: 1, 1

GET
H2 200 otSDKStub.js Show response
cdn.cookielaw.org/scripttemplates/ 20 KB
7 KB 103ms
45ms Script
application/javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/scripttemplates/otSDKStub.js

Requested by

Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

03aa6fcac2902227e1b66a01b87824692f708bbf9bfe441784f8ed22d677f6de

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: +q2Bd0SvXowDeesSOf+0yw==
age: 681
vary: Accept-Encoding
content-length: 6782
x-ms-lease-status: unlocked
last-modified: Tue, 17 May 2022 16:31:29 GMT
server: cloudflare
etag: 0x8DA3822B23D0DE9
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/javascript
access-control-allow-origin: *
x-ms-request-id: d4700742-d01e-00d7-241e-6a92c1000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8ada73ff975c6-LHR

GET
H2 200 otCCPAiab.js Show response
cdn.cookielaw.org/opt-out/ 22 KB
6 KB 102ms
45ms Script
application/javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/opt-out/otCCPAiab.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

a7b7120dffd25546c93c1367b9c86a3dc87e71d2c89ebb39163a71eb3b659f01

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: ERttG9+iQk1LCPjR495NRw==
age: 4672
vary: Accept-Encoding
x-ms-lease-status: unlocked
last-modified: Tue, 22 Feb 2022 22:01:18 GMT
server: cloudflare
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/javascript
access-control-allow-origin: *
x-ms-request-id: 5ac9baf6-b01e-0029-5355-28aea4000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
cf-ray: 70e8ada73ffb75c6-LHR

GET
H2 200 gpt.js Show response
securepubads.g.doubleclick.net/tag/js/ 82 KB
29 KB 155ms
50ms Script
text/javascript 172.217.16.130
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://securepubads.g.doubleclick.net/tag/js/gpt.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

172.217.16.130 , United States, ASN15169 (GOOGLE, US),

Reverse DNS

fra15s46-in-f2.1e100.net

Software

sffe /

Resource Hash

9ff1f6746123f8bb8d719e2f1b804a96f3513586f36d2b7c1eb878433be2d6a0

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
x-content-type-options: nosniff
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 28589
x-xss-protection: 0
server: sffe
etag: "1221 / 824 of 1000 / last-modified: 1653084304"
vary: Accept-Encoding
report-to: {"group":"ads-gpt-scs","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-gpt-scs"}]}
content-type: text/javascript
cache-control: private, max-age=900, stale-while-revalidate=3600
timing-allow-origin: *
cross-origin-opener-policy-report-only: same-origin; report-to="ads-gpt-scs"
expires: Fri, 20 May 2022 22:58:39 GMT

GET
H2 200 moatheader.js Show response
z.moatads.com/condenastprebidheader987326845656/ 213 KB
75 KB 185ms
55ms Script
application/x-javascript 2.20.86.213
AKAMAI-AS

General

Check archive.org

Show headers Download Go to

Full URL: https://z.moatads.com/condenastprebidheader987326845656/moatheader.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_256_GCM
Server: 2.20.86.213 Milan, Italy, ASN16625 (AKAMAI-AS, US),
Reverse DNS: a2-20-86-213.deploy.static.akamaitechnologies.com
Software: AmazonS3 /
Resource Hash: be51d69b63e7d47d25cfd9bedf139d053f4bd56157527005c3fdea76a4e9c9ed

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
last-modified: Thu, 12 May 2022 16:24:31 GMT
server: AmazonS3
x-amz-request-id: 2FVDWSB7Q0DR5RK1
etag: "9943e299aa9bbd31facc0a8d57e52db5"
vary: Accept-Encoding
content-type: application/x-javascript
cache-control: max-age=14557
accept-ranges: bytes
content-length: 75991
x-amz-id-2: wj/Emn9i8cR7YGsFNktzHY9aPCUEDncZQEjorq5FYVdl6jnGSevOGloENRWPBGS77wyTJvGmKNk=

GET
H2 200 apstag.js Show response
c.amazon-adsystem.com/aax2/ 135 KB
37 KB 154ms
50ms Script
application/javascript 108.156.255.177
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://c.amazon-adsystem.com/aax2/apstag.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 108.156.255.177 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-156-255-177.dus51.r.cloudfront.net
Software: Server /
Resource Hash: 1909b2a83fd41494d94862c4323944d9d0aa1f1e653f252ea5a73fc5944308b0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-amz-version-id: STlSjRvyyTgJyl_raxUeHIFBn6F5DqB3
content-encoding: gzip
etag: 4abd427e43cd6822329a2c05539e321f
age: 695
x-cache: Hit from cloudfront
server: Server
x-amz-rid: 0AC57EPPC79BHJP7SJ47
date: Fri, 20 May 2022 22:47:12 GMT
vary: Accept-Encoding
content-type: application/javascript
via: 1.1 ddd7d19501f4b19d560bfedbdd9b13ce.cloudfront.net (CloudFront)
cache-control: public, max-age=900
x-amz-cf-pop: DUS51-P2
accept-ranges: bytes
timing-allow-origin: *
x-amz-cf-id: J33_Vx_fmITc85Bq9mdNLkOLrqoMk_GcSIcgehaS09Xe6JigbJiOew==

GET
H2 200 prebid.min.js Show response
www.wired.com/hotzones/esi/wired/ 273 KB
81 KB 27ms
26ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/hotzones/esi/wired/prebid.min.js

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

befa0a7f28e5b50c7af0e063611c12a999d86005f5ab6501f6c623eba45cbac9

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
vary: accept-encoding, cn-experiments, Verso, bypass-verso-payment
age: 26108
x-cache: HIT
hz-zone: 1
content-encoding: gzip
content-length: 82568
x-served-by: cache-lcy19261-LCY
strict-transport-security: max-age=31536000; preload
content-type: application/javascript; charset=utf-8
cache-control: max-age=86400
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
apple-news-services-host: www.wired.com
x-cache-hits: 22

GET
H/1.1 200
OK 183973-93942139695505.js Show response
js-sec.indexww.com/ht/p/ 43 KB
15 KB 186ms
39ms Script
text/javascript 23.35.236.247
AKAMAI-AS

General

Check archive.org

Show headers Download Go to

Full URL: https://js-sec.indexww.com/ht/p/183973-93942139695505.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_256_GCM
Server: 23.35.236.247 Frankfurt am Main, Germany, ASN16625 (AKAMAI-AS, US),
Reverse DNS: a23-35-236-247.deploy.static.akamaitechnologies.com
Software: Apache /
Resource Hash: 88082e2436305c53b9849eab602898e4d5b728b68c2439cbfad581846fd32cdf

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:39 GMT
Content-Encoding: gzip
Last-Modified: Fri, 20 May 2022 22:46:32 GMT
Server: Apache
ETag: "da3f00-ada8-5df79430cc6c5"
Vary: Accept-Encoding
P3P: policyref="/w3c/p3p.xml", CP="NOI DSP COR DEVa TAIa OUR BUS UNI"
Cache-Control: max-age=3100
Connection: keep-alive
Accept-Ranges: bytes
Content-Type: text/javascript
Content-Length: 14812
Expires: Fri, 20 May 2022 23:50:19 GMT

GET
H2 200 platform.js Show response
apis.google.com/js/ 52 KB
21 KB 143ms
48ms Script
text/javascript 2a00:1450:4001:828::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://apis.google.com/js/platform.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:828::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

bb05bc3976691db38d1c403ede59d1d2181fb64a6227b6ee1d849beba8f8c35d

Security Headers

Name	Value
Content-Security-Policy	require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/gapi-team
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

content-security-policy: require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/gapi-team
content-encoding: gzip
x-content-type-options: nosniff
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 20362
x-xss-protection: 0
server: sffe
cross-origin-opener-policy: same-origin; report-to="gapi-team"
date: Fri, 20 May 2022 22:58:39 GMT
vary: Accept-Encoding
report-to: {"group":"gapi-team","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/gapi-team"}]}
content-type: text/javascript
access-control-allow-origin: *
cache-control: private, max-age=1800, stale-while-revalidate=1800
etag: "9fd532fc5ee3ba82"
accept-ranges: bytes
timing-allow-origin: *
expires: Fri, 20 May 2022 22:58:39 GMT

GET
H2 200 swg.js Show response
news.google.com/swg/js/v1/ 145 KB
45 KB 145ms
39ms Script
text/javascript 2a00:1450:4001:801::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://news.google.com/swg/js/v1/swg.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:801::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

c2457ed91b16d9dbf7a21917fd4f8c47da415054f1c93354749ed075ad211fb4

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:42:07 GMT
content-encoding: gzip
x-content-type-options: nosniff
age: 992
content-security-policy-report-only: require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/news-frontend
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 45793
x-xss-protection: 0
last-modified: Wed, 18 May 2022 18:04:52 GMT
server: sffe
cross-origin-opener-policy: same-origin; report-to="news-frontend"
vary: Accept-Encoding
report-to: {"group":"news-frontend","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/news-frontend"}]}
content-type: text/javascript
cache-control: public, max-age=3000
accept-ranges: bytes
expires: Fri, 20 May 2022 23:32:07 GMT

GET
H2 200 logo-header.a7598835a549cb7d5ce024ef0710935927a034f9.svg
www.wired.com/verso/static/wired/assets/ 1 KB
2 KB 40ms
39ms Image
image/svg+xml 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.wired.com/verso/static/wired/assets/logo-header.a7598835a549cb7d5ce024ef0710935927a034f9.svg

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

1d097e69ff47df9414a0ec07dfc70401084f4599617045a3a3edc7661ff76f3f

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
age: 2151476
via: 1.1 varnish
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/wired/assets/logo-header.a7598835a549cb7d5ce024ef0710935927a034f9.svg
last-modified: Mon, 25 Apr 2022 23:50:50 GMT
verso: true
content-length: 600
x-amz-id-2: ID5pvDXYbFf8GlvIuXRTmRuhO814ctXXHWwLm+Jb3WDoJwgJMLRiw+2Wd/ZbaUMjvKT2/hROll4=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.815407,VS0,VE1
apple-news-services-request-url: /verso/static/wired/assets/logo-header.a7598835a549cb7d5ce024ef0710935927a034f9.svg
etag: W/"ce65105f89c50c0f92e35be389684c24"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: RZBEVT1JACD85BM9
access-control-allow-origin: *
expires: Wed, 26 Apr 2023 01:20:44 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: image/svg+xml
apple-news-services-host: www.wired.com
x-cache-hits: 41

GET
H2 200 BreveText-Book.woff
www.wired.com/verso/static/assets/fonts/ 35 KB
35 KB 40ms
37ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/BreveText-Book.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

b9daef61d4b711f1d28c3eef6bd8d522b8df518e833767512ad79502cc605dbd

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 2746887
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/BreveText-Book.woff
last-modified: Mon, 18 Apr 2022 22:15:40 GMT
verso: true
content-length: 35492
x-amz-id-2: 89tgebckMDR4U2xUz8eAjQoSR3RIM1iqil15WAVRwa1JuC+hwcYZUKaZFhZmgRDwFKurTzt6v4c=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.818491,VS0,VE0
apple-news-services-request-url: /verso/static/assets/fonts/BreveText-Book.woff
etag: "97629378bc029d817ccbb382c7245bd6"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: RSY4EDJ9CJW4GY4D
access-control-allow-origin: *
expires: Wed, 19 Apr 2023 03:57:14 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 42

GET
H2 200 WiredMono-Bold.woff
www.wired.com/verso/static/assets/fonts/ 21 KB
22 KB 41ms
39ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/WiredMono-Bold.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

26ed5f6aa822d65b6ea7df8d13f1a217d7a933376a824b7ef52af0e6f64f1a44

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 5168045
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/WiredMono-Bold.woff
last-modified: Tue, 22 Mar 2022 02:00:24 GMT
verso: true
content-length: 21196
x-amz-id-2: Jwew/TYDx83MlxUEHEh5PhgSvVQCu5dnNT85E0lzPLePQEGAJNm7Vu8nlqtn59t+9JpRdxG8pvI=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.818814,VS0,VE0
apple-news-services-request-url: /verso/static/assets/fonts/WiredMono-Bold.woff
etag: "9bae9bc16cf9e407e25b7c9c4ae30e1f"
vary: Verso
strict-transport-security: max-age=31536000; preload
x-amz-request-id: TV4MSG2PEKQ6JG7E
access-control-allow-origin: *
expires: Wed, 22 Mar 2023 03:24:34 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 44

GET
H2 200 LabGrotesque-Black.woff2
www.wired.com/verso/static/assets/fonts/ 47 KB
47 KB 41ms
40ms Font
application/x-font-woff2 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/LabGrotesque-Black.woff2

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

c41dc37fea212372d1f53109304ebae695e644f9ce083dcab08d5978c8c3020f

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 3203521
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/LabGrotesque-Black.woff2
last-modified: Wed, 13 Apr 2022 21:06:03 GMT
verso: true
content-length: 47924
x-amz-id-2: M4lpv8Iv5WPQO3YYVwL5wAwe24PL3O6VqzAAQYCWGPk9N6QlEYSF2KhpcdsfHg3In2sqT5QwG9Q=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.818990,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/LabGrotesque-Black.woff2
etag: "44b6bf0cd9f1d027a6ca723b2024925c"
vary: Verso
strict-transport-security: max-age=31536000; preload
x-amz-request-id: VYK64VDMJA0ZMEJE
access-control-allow-origin: *
expires: Thu, 13 Apr 2023 21:06:38 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff2
apple-news-services-host: www.wired.com
x-cache-hits: 136

GET
H2 200 WiredMono-Regular.woff2
www.wired.com/verso/static/assets/fonts/ 18 KB
19 KB 50ms
49ms Font
application/x-font-woff2 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/WiredMono-Regular.woff2

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

c8c9128b649afff93f89f77eb2aa5a4bbbb1443bebc5156d0f697780c8beaa26

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 3356066
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/WiredMono-Regular.woff2
last-modified: Tue, 12 Apr 2022 02:38:39 GMT
verso: true
content-length: 18912
x-amz-id-2: adKZ/mZ2JrgRKyVus5Zjv0jQtFOl2UjfHiYMMtJhosqrN0+0iuvd4cl3nBCZO1GDCEnDAkLrLo0=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.819241,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/WiredMono-Regular.woff2
etag: "e755d282ae1120887b3b1d207bb930ce"
vary: Verso
strict-transport-security: max-age=31536000; preload
x-amz-request-id: X8YANZK4NKD3BQJE
access-control-allow-origin: *
expires: Wed, 12 Apr 2023 02:44:15 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff2
apple-news-services-host: www.wired.com
x-cache-hits: 43

GET
H2 200 ProximaNova-Regular.woff
www.wired.com/verso/static/assets/fonts/ 27 KB
28 KB 47ms
47ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/ProximaNova-Regular.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

9e0843dbf1dc0d65a75182a82b945a9373557932e61934c27679c357f20c33a9

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 5610866
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/ProximaNova-Regular.woff
last-modified: Wed, 16 Mar 2022 21:16:52 GMT
verso: true
content-length: 27408
x-amz-id-2: R/VnssqNDkzmZuSz6XrC+s/invDnrHUAif96OZib4gaFmDRLVItJr/wolX2/OeaqabOf/GqboAc=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.819196,VS0,VE0
apple-news-services-request-url: /verso/static/assets/fonts/ProximaNova-Regular.woff
etag: "57acd3677d276ad405bb6838d2b120d5"
vary: Verso
strict-transport-security: max-age=31536000; preload
x-amz-request-id: JP10MGE6R310PWE6
access-control-allow-origin: *
expires: Fri, 17 Mar 2023 00:24:13 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 42

GET
H2 200 ProximaNova-Bold.woff2
www.wired.com/verso/static/assets/fonts/ 22 KB
22 KB 49ms
48ms Font
application/x-font-woff2 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/ProximaNova-Bold.woff2

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

6a4dac260dffc284594d633859fb508b2fcfade38b61c8af9cd55eb23adf9e89

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 1547957
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/ProximaNova-Bold.woff2
last-modified: Tue, 03 May 2022 00:03:36 GMT
verso: true
content-length: 22500
x-amz-id-2: jlfU6ZhUCyWqUatJkB5bRjVdbBvVkJDslOl3cbvZyZ84Hd0J7YO9IOHEYw8VRP36/ynYwKAHpfA=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.819421,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/ProximaNova-Bold.woff2
etag: "2ee806e52a1e28138bd67a5113c99949"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: NW8418R7D97ZYAQN
access-control-allow-origin: *
expires: Wed, 03 May 2023 00:59:22 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff2
apple-news-services-host: www.wired.com
x-cache-hits: 43

GET
H2 200 LabGrotesque-Bold.woff2
www.wired.com/verso/static/assets/fonts/ 47 KB
47 KB 30ms
30ms Font
application/x-font-woff2 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/LabGrotesque-Bold.woff2

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

2c667cdbe90922576bac69bbd0fa8f61d0c410748bf29b5bccea09b21123f1a0

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 933912
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/LabGrotesque-Bold.woff2
last-modified: Tue, 10 May 2022 03:12:28 GMT
verso: true
content-length: 47856
x-amz-id-2: EyzrzP/QE4uVXBklSUI4DgIt0NDV3pCjQzHpr1XPIhRtcFveDsAgQ1iZbhcXfJSbWZyEWmVS4nU=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.839831,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/LabGrotesque-Bold.woff2
etag: "181b7a06e7a0586c230d9b6282d73532"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: 7XFTKEEPPQYAN3NG
access-control-allow-origin: *
expires: Wed, 10 May 2023 03:33:28 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff2
apple-news-services-host: www.wired.com
x-cache-hits: 33

GET
H2 200 logo-reverse.548f3a7478ee71f618044082aa222dd05f31249c.svg
www.wired.com/verso/static/wired/assets/ 1 KB
2 KB 27ms
26ms Image
image/svg+xml 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.wired.com/verso/static/wired/assets/logo-reverse.548f3a7478ee71f618044082aa222dd05f31249c.svg

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

af29cacfb5125f85da0f1557bb56456abcc1556dbd3094bb56e569890348c984

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
age: 4485020
via: 1.1 varnish
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/wired/assets/logo-reverse.548f3a7478ee71f618044082aa222dd05f31249c.svg
last-modified: Tue, 29 Mar 2022 23:19:54 GMT
verso: true
content-length: 610
x-amz-id-2: LQKz1LKgpgsg5r3RLDNWWq7cNV6bJoOtnJUKCpNPmd7UoOZStOVtb408laPLrH/AbLPZqxgUe00=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.857959,VS0,VE1
apple-news-services-request-url: /verso/static/wired/assets/logo-reverse.548f3a7478ee71f618044082aa222dd05f31249c.svg
etag: W/"2cba2fa9380ed2b50927ed9d520aaa3c"
vary: Verso
strict-transport-security: max-age=31536000; preload
x-amz-request-id: Z0GZ025Q1W3X2EVB
access-control-allow-origin: *
expires: Thu, 30 Mar 2023 01:08:19 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: image/svg+xml
apple-news-services-host: www.wired.com
x-cache-hits: 220

GET
H2 200 polyfill.min.js Show response
polyfill.io/v3/ 72 B
397 B 137ms
40ms Script
text/javascript 2a04:4e42::282
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL: https://polyfill.io/v3/polyfill.min.js?version=3.103.0&features=Object.assign%2CIntersectionObserver%2CPromise%2Cfetch%2CIntl.Locale%2CIntl.getCanonicalLocales%2CIntl.ListFormat%2CIntl.ListFormat.%7Elocale.en-GB%2CIntl.ListFormat.%7Elocale.en-US%2CIntl.ListFormat.%7Elocale.en-IN%2CIntl.ListFormat.%7Elocale.fr%2CIntl.ListFormat.%7Elocale.es
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 2a04:4e42::282 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: aaecd144d2b8763b2fa5c91f09778294363cef363c10504205f4203922644d11

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: br
last-modified: Mon, 25 Apr 2022 20:56:30 GMT
age: 0
vary: User-Agent, Accept-Encoding
access-control-allow-methods: GET,HEAD,OPTIONS
content-type: text/javascript; charset=utf-8
access-control-allow-origin: *
cache-control: public, s-maxage=31536000, max-age=604800, stale-while-revalidate=604800, stale-if-error=604800
useragent_normaliser: chrome/101.0.0
server-timing: cache-lcy19247, PASS, fastly;desc="Edge time";dur=15
accept-ranges: bytes
content-length: 74

GET
H2 200 chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~e5a0131a.4df54eca8dd78de2595d.js Show response
www.wired.com/verso/static/ 1 MB
371 KB 28ms
27ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~e5a0131a.4df54eca8dd78de2595d.js

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

2236efed5f46e344722474597cff28f2dbb8c5cd134c79ab744d87a4f9b6581e

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
age: 12010
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~e5a0131a.4df54eca8dd78de2595d.js
last-modified: Fri, 20 May 2022 19:22:22 GMT
verso: true
content-length: 379391
x-amz-id-2: 5wbBo+cuuhZdui0UubYbWgvrXqKWwY7oglGKnIpgNCwnxIswfCHLGAdJ0CqtUFFVOIV1Sxt1YTc=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.864469,VS0,VE1
apple-news-services-request-url: /verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~e5a0131a.4df54eca8dd78de2595d.js
etag: W/"775f1c5866c17e97bc8a3b615598d8c9"
vary: cn-experiments, Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: 59MNFD1E9G68R5HP
via: 1.1 varnish
expires: Sat, 20 May 2023 19:38:29 GMT
cache-control: max-age=86400
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/javascript
apple-news-services-host: www.wired.com
x-cache-hits: 2

GET
H2 200 chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~fe219ec7.2aae932dc3e9768bba1b.js Show response
www.wired.com/verso/static/ 679 KB
152 KB 30ms
30ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~fe219ec7.2aae932dc3e9768bba1b.js

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

e0a6819d2a0186738790df0b93af04cb893c3040c74a5ffcde80b0eed215e3ba

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
age: 12010
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~fe219ec7.2aae932dc3e9768bba1b.js
last-modified: Fri, 20 May 2022 19:22:22 GMT
verso: true
content-length: 154325
x-amz-id-2: n+/FG7dBDrn8hUlRFiov3ROp5UhA5kj3hbnQWaySc1E896w6cy+BGGGsj5qXxllkxPZPSzUPzEA=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.864757,VS0,VE0
apple-news-services-request-url: /verso/static/chunk.vendors~presenter-account-bookmarks~presenter-account-linking~presenter-account-sign-in-help~present~fe219ec7.2aae932dc3e9768bba1b.js
etag: W/"b20e4eb456b083e4f90cd1e6e30a76fd"
vary: cn-experiments, Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: 59MZGDV67FMEP05R
via: 1.1 varnish
expires: Sat, 20 May 2023 19:38:29 GMT
cache-control: max-age=86400
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/javascript
apple-news-services-host: www.wired.com
x-cache-hits: 2

GET
H2 200 presenter-articles.f2770f6e6dd98bbb80d4.js Show response
www.wired.com/verso/static/ 2 MB
491 KB 30ms
30ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

fd829117a8525d07eb88964288d6e9401c14e5db450532eae36ab9185ec8349d

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
content-encoding: gzip
age: 54465
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js
last-modified: Fri, 20 May 2022 07:48:19 GMT
verso: true
content-length: 502186
x-amz-id-2: ZW4Dg+jBl4tcl024pXWRbitPtRHNmQiwKDu+8D3UlAUIl/JFJeWdCGa3NgrJzE1YUQcRNKUEwYM=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.864731,VS0,VE0
apple-news-services-request-url: /verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js
etag: W/"2bcb9fc9da4e7b65bd5a2da7dd035012"
vary: cn-experiments, Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: S54JQF9KRKCRXRKZ
via: 1.1 varnish
expires: Sat, 20 May 2023 07:50:54 GMT
cache-control: max-age=86400
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/javascript
apple-news-services-host: www.wired.com
x-cache-hits: 2

GET
H2 200 pixelpropagate.js Show response
www.wired.com/hotzones/src/ 3 KB
1 KB 75ms
75ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/hotzones/src/pixelpropagate.js?cb=10229

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

4de3f44f9af02f0a9ac1366998ed8d04b85caee3bccd4552c04edd8dcd926bee

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
vary: accept-encoding, cn-experiments, Verso, bypass-verso-payment
age: 25071
x-cache: HIT
hz-zone: 1
content-encoding: gzip
content-length: 1188
x-served-by: cache-lcy19261-LCY
strict-transport-security: max-age=31536000; preload
content-type: application/javascript; charset=utf-8
cache-control: max-age=86400
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
apple-news-services-host: www.wired.com
x-cache-hits: 58

GET
H2 200 user-context Show response
www.wired.com/ 1 KB
1 KB 165ms
165ms Script
application/javascript 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/user-context?referrer=&verso=true&paymentForm=free&location=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6414975cf1d0285b683085f648dc5dc2efbdee90406182ad244c30a7a7fa4446

Security Headers

Name	Value
Content-Security-Policy	default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://.hotjar.com wss://.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
Strict-Transport-Security	max-age=31536000; preload
X-Content-Type-Options	nosniff
X-Frame-Options	DENY
X-Xss-Protection	1; mode=block

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
via: 1.1 varnish
x-content-type-options: nosniff
x-cache: MISS
server-timing: geo;desc="continent=EU;country=GB;pop=LCY"
content-encoding: gzip
vary: origin,accept-encoding, Accept-Encoding
x-xss-protection: 1; mode=block
x-served-by: cache-lcy19261-LCY
expires: 0
server: nginx/1.15.8
x-frame-options: DENY
x-download-options: noopen
strict-transport-security: max-age=31536000; preload
content-type: application/javascript; charset=utf-8
access-control-expose-headers: WWW-Authenticate,Server-Authorization
cache-control: no-cache
access-control-allow-credentials: true
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: none
timing-allow-origin: *
apple-news-services-host: user-context.condenastdigital.com
x-cache-hits: 0

GET
H/1.1 200
OK build-c734366b5f4995c51ce8c9a49fb2c95f.js Show response
journey.wired.com/ 287 KB
44 KB 110ms
26ms Script
text/javascript 151.101.64.239
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL: https://journey.wired.com/build-c734366b5f4995c51ce8c9a49fb2c95f.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.64.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: AmazonS3 /
Resource Hash: 4041b947acd8eb0e87e3577c9121ddf40336ce3d2bb5033262ab74bb80032682

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:39 GMT
Content-Encoding: gzip
Age: 9550
X-Cache: HIT
server-timing: geo;desc="continent=EU;country=GB;pop=LCY"
Content-Length: 44518
x-amz-id-2: wZRzyzBl1tM1FoLu4ydLIq/CZbwTcqtpbXJvOnxKoFmodBc6FD68Uri002RR6Fyvy9dzJ2xDwoQ=
X-Served-By: cache-lcy19249-LCY
timing-allow-origin: *
Last-Modified: Fri, 20 May 2022 20:18:45 GMT
Server: AmazonS3
Cache-Control: public, max-age=31536000
X-Timer: S1653087520.961867,VS0,VE0
ETag: "c734366b5f4995c51ce8c9a49fb2c95f"
Vary: Accept-Encoding
x-amz-request-id: 767K8XEZ6JT9NGSD
Via: 1.1 varnish
Connection: keep-alive
Accept-Ranges: bytes
Content-Type: text/javascript
X-Cache-Hits: 14

GET
H2 200 ProximaNova-RegularItalic.woff
www.wired.com/verso/static/assets/fonts/ 27 KB
28 KB 65ms
64ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/ProximaNova-RegularItalic.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

8392b864ce606bf8ec20cf749f148dd7134d770200dd81df9b7adbc33a88978e

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 1405327
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/ProximaNova-RegularItalic.woff
last-modified: Wed, 04 May 2022 16:35:52 GMT
verso: true
content-length: 27564
x-amz-id-2: 5Zhwj1AHb+V5npUcnIQRIEYnK0ybDtml4LWYErlC4sbWPpQsOPLD+8lRYump/ymv4KWzwW9FXLo=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.889010,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/ProximaNova-RegularItalic.woff
etag: "37e0e76d0baa901390de58544942384d"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: CB299JDBWD7P6TQQ
access-control-allow-origin: *
expires: Thu, 04 May 2023 16:36:32 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 134

GET
H2 200 WiredMono-Light.woff
www.wired.com/verso/static/assets/fonts/ 21 KB
22 KB 64ms
64ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/WiredMono-Light.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

7ff758ced20d4139eb5f580cfc93d1161ed5e19a4a2d4020728143855d17adeb

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 332179
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/WiredMono-Light.woff
last-modified: Tue, 17 May 2022 01:50:53 GMT
verso: true
content-length: 21256
x-amz-id-2: bFO7pF3LccraamSPuelXOa4tGw66NMCLBOUzicG1EtnHNNTTnxYR28qXotDp58BrUh2ZMFjOIn4=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.889269,VS0,VE0
apple-news-services-request-url: /verso/static/assets/fonts/WiredMono-Light.woff
etag: "86ae3ff7d521e7ad53923c387fddc7b8"
vary: cn-experiments, Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: YP9N25FDDN61FV9G
access-control-allow-origin: *
expires: Wed, 17 May 2023 02:42:20 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 28

GET
H2 200 BreveText-BookItalic.woff
www.wired.com/verso/static/assets/fonts/ 35 KB
35 KB 64ms
64ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/BreveText-BookItalic.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

c34a9420de5b381d4f2459193c70e4d84d5aaa0928a67907a57e2a9e9d3a8852

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:39 GMT
via: 1.1 varnish
age: 933912
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/BreveText-BookItalic.woff
last-modified: Tue, 10 May 2022 03:12:22 GMT
verso: true
content-length: 35744
x-amz-id-2: bgL0BbFJXjq+zSTbq2UGxwBK8be9mFst5xpTMnE5C85FNcPyrlZhIOmc+SN/T5ZBjY4t0UiJ5KI=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087520.889239,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/BreveText-BookItalic.woff
etag: "f8b4eded3c32119b3384de37a8c158a6"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: BCGQSB1N4FJDP1AH
access-control-allow-origin: *
expires: Wed, 10 May 2023 03:33:28 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 6

GET
H2 200 dnsfeed Show response
geolocation.onetrust.com/cookieconsentpub/v1/geo/location/ 163 B
377 B 99ms
43ms Script
text/javascript 2606:4700:10::6814:b844
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://geolocation.onetrust.com/cookieconsentpub/v1/geo/location/dnsfeed

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/opt-out/otCCPAiab.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700:10::6814:b844 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

b29dfe7b6e22fffd693378fa14676d1d27a38b0f9c9a88b9e0793af1f34ddf48

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
server: cloudflare
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
vary: Accept-Encoding
content-type: text/javascript
strict-transport-security: max-age=31536000; includeSubDomains; preload
cf-ray: 70e8ada85a89775c-LHR

GET
H2 200 93ddfe0c-4b21-4ad5-8191-612d2a67aad3.json Show response
cdn.cookielaw.org/consent/93ddfe0c-4b21-4ad5-8191-612d2a67aad3/ 4 KB
2 KB 90ms
35ms XHR
application/x-javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/consent/93ddfe0c-4b21-4ad5-8191-612d2a67aad3/93ddfe0c-4b21-4ad5-8191-612d2a67aad3.json

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/otSDKStub.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

29cf1db1cac70e1421ddad6a60111ad50afea460acc234bff478aa8072a6aad5

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: zwo3roU11ROZKTcwOwSq3w==
age: 823
vary: Accept-Encoding
content-length: 1603
x-ms-lease-status: unlocked
last-modified: Thu, 21 Apr 2022 22:01:02 GMT
server: cloudflare
etag: 0x8DA23E26D1D4B84
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/x-javascript
access-control-allow-origin: *
x-ms-request-id: 647a681e-101e-002f-7acb-5559dc000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Cache-Control,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: public, max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8ada85d767714-LHR
expires: Sat, 21 May 2022 02:58:40 GMT

GET
H/1.1 200
OK beacon
infinityid.condenastdigital.com/infinityid/ 35 B
934 B 461ms
102ms Image
image/gif 18.233.44.90
AMAZON-AES

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://infinityid.condenastdigital.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 18.233.44.90 Ashburn, United States, ASN14618 (AMAZON-AES, US),
Reverse DNS: ec2-18-233-44-90.compute-1.amazonaws.com
Software: nginx/1.15.8 /
Resource Hash: 6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Server: nginx/1.15.8
vary: origin
Content-Type: image/gif
cache-control: no-cache, no-store, must-revalidate
Connection: keep-alive
accept-ranges: bytes
Content-Length: 35
expires: 0

GET
H2 200 aps_csm.js Show response
c.amazon-adsystem.com/bao-csm/aps-comm/ 6 KB
3 KB 129ms
43ms XHR
application/javascript 108.156.255.177
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://c.amazon-adsystem.com/bao-csm/aps-comm/aps_csm.js
Requested by: Host: c.amazon-adsystem.com
URL: https://c.amazon-adsystem.com/aax2/apstag.js
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 108.156.255.177 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-156-255-177.dus51.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: 06b99248a163333e36980a6cfb756f1a7de60fa49517162b87b1a44d5d48f844

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-amz-version-id: aaJeHz3g2a7aWr9hYquBq.aDaObnNoK3
content-encoding: gzip
etag: W/"a4d296427fc806b21335359e398c025c"
age: 66799
x-cache: Hit from cloudfront
access-control-max-age: 3000
access-control-allow-origin: *
last-modified: Thu, 28 Apr 2022 01:41:20 GMT
server: AmazonS3
date: Fri, 20 May 2022 04:27:55 GMT
vary: Origin
access-control-allow-methods: GET
content-type: application/javascript
via: 1.1 5db4f6b1c04035a37ba6548e89b362be.cloudfront.net (CloudFront)
cache-control: public, max-age=86400
x-amz-cf-pop: DUS51-P2
x-amz-cf-id: 0XehMQnqaV8ZAgtcYuwEdPQNmigauRizPznwU8eDNtmHDTTnJy3Z1A==

GET
H2 200 pubads_impl_2022051801.js Show response
securepubads.g.doubleclick.net/gpt/ 367 KB
125 KB 40ms
39ms Script
text/javascript 172.217.16.130
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://securepubads.g.doubleclick.net/gpt/pubads_impl_2022051801.js?cb=31067704

Requested by

Host: securepubads.g.doubleclick.net
URL: https://securepubads.g.doubleclick.net/tag/js/gpt.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

172.217.16.130 , United States, ASN15169 (GOOGLE, US),

Reverse DNS

fra15s46-in-f2.1e100.net

Software

sffe /

Resource Hash

71f7b22f7b615b6a6cb2240ba7516fb2e83d2028607d5983fd64d1b755fd11f5

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 15:03:41 GMT
content-encoding: gzip
x-content-type-options: nosniff
age: 28499
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 127552
x-xss-protection: 0
last-modified: Wed, 18 May 2022 08:34:35 GMT
server: sffe
vary: Accept-Encoding
report-to: {"group":"ads-gpt-scs","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-gpt-scs"}]}
content-type: text/javascript
cache-control: public, immutable, max-age=31536000
accept-ranges: bytes
timing-allow-origin: *
cross-origin-opener-policy-report-only: same-origin; report-to="ads-gpt-scs"
expires: Sat, 20 May 2023 15:03:41 GMT

GET
H3 200 ppub_config Show response
securepubads.g.doubleclick.net/pagead/ 320 B
175 B 127ms
48ms XHR
application/json 172.217.16.130
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://securepubads.g.doubleclick.net/pagead/ppub_config?ippd=www.wired.com

Requested by

Host: securepubads.g.doubleclick.net
URL: https://securepubads.g.doubleclick.net/tag/js/gpt.js

Protocol

Security

QUIC, , AES_128_GCM

Server

172.217.16.130 , United States, ASN15169 (GOOGLE, US),

Reverse DNS

fra15s46-in-f2.1e100.net

Software

cafe /

Resource Hash

1f90db0a2498c14a33ddf6dc7e65b81adeded4d160e4d60740e03b9e11ba829d

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

timing-allow-origin: *
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
server: cafe
p3p: policyref="https://googleads.g.doubleclick.net/pagead/gcn_p3p_.xml", CP="CURa ADMa DEVa TAIo PSAo PSDo OUR IND UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR"
access-control-allow-origin: *
cache-control: private, max-age=3600, stale-while-revalidate=3600
cross-origin-resource-policy: cross-origin
content-disposition: attachment; filename="f.txt"
content-type: application/json; charset=UTF-8
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 150
x-xss-protection: 0
expires: Fri, 20 May 2022 22:58:40 GMT

GET
H/1.1 200
OK wired.js Show response
player.cnevids.com/interlude/ 37 KB
11 KB 974ms
437ms Script
text/javascript 18.65.159.128
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL

https://player.cnevids.com/interlude/wired.js?verso=true&onReady=CNE_onReady_16530875202810&hasExcludedEmbed=false&embeddedVideos=&rightRail=true&interludeOverrideId=583dce7afd2e614a9500000d

Requested by

Host: www.wired.com
URL: https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

18.65.159.128 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-18-65-159-128.nrt51.r.cloudfront.net

Software

nginx/1.18.0 /

Resource Hash

bfa1a0373c6e9fb398ea2d490254d5c33353f5ab88491a63cc94a16463432c7b

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	1; mode=block

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:41 GMT
Content-Encoding: gzip
X-Content-Type-Options: nosniff
X-Permitted-Cross-Domain-Policies: none
X-Amz-Cf-Pop: NRT51-P2
X-Cache: Miss from cloudfront
Status: 200 OK
Connection: keep-alive
Content-Length: 10304
X-XSS-Protection: 1; mode=block
X-Request-Id: e2eb3bfc-8fa5-485f-962d-c63fe0e761fb
X-Runtime: 0.007284
X-Backend-Node: 10.110.40.13
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.18.0
ETag: W/"f535e92076b5c37602db6c53042f230c"
X-Download-Options: noopen
Vary: Origin,Accept-Encoding
Content-Type: text/javascript; charset=utf-8
Via: 1.1 10669864d9506e9be77be128d971849a.cloudfront.net (CloudFront)
Cache-Control: max-age=0, private, must-revalidate
X-Amz-Cf-Id: lRepamxudO6WGsGdL9N1NjmpqAHV6UxPmOjG9d6l850wuvsPbIud_A==

GET
H2 200 gtm.js Show response
www.googletagmanager.com/ 569 KB
147 KB 174ms
89ms Script
application/javascript 2a00:1450:4001:802::2008
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.googletagmanager.com/gtm.js?id=GTM-5HBJC2K&l=dataLayer

Requested by

Host: www.wired.com
URL: https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:802::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Google Tag Manager /

Resource Hash

91afb6158e33ea43973ffd3f04b6caf266c5684a6b180a4acf1b001d2afaae78

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: br
vary: Accept-Encoding
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 150467
x-xss-protection: 0
last-modified: Fri, 20 May 2022 22:23:31 GMT
server: Google Tag Manager
strict-transport-security: max-age=31536000; includeSubDomains
content-type: application/javascript; charset=UTF-8
access-control-allow-origin: *
cache-control: private, max-age=900
access-control-allow-credentials: true
access-control-allow-headers: Cache-Control
expires: Fri, 20 May 2022 22:58:40 GMT

OPTIONS
H2 200 domaingroupcheck
cookies-data.onetrust.io/bannersdk/v1/ Frame 0
0 91ms
35ms Preflight
application/json 2606:4700:4400::ac40:9b40
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL: https://cookies-data.onetrust.io/bannersdk/v1/domaingroupcheck
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 2606:4700:4400::ac40:9b40 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software: cloudflare /
Resource Hash

Request headers

Accept: */*
Access-Control-Request-Headers: domainid,location,url
Access-Control-Request-Method: GET
Origin: https://www.wired.com
Sec-Fetch-Mode: cors
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

access-control-allow-headers: domainId, url, location, Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With
access-control-allow-methods: GET, HEAD, OPTIONS
access-control-allow-origin: *
cf-ray: 70e8adab0e3c7695-LHR
content-length: 0
content-type: application/json
date: Fri, 20 May 2022 22:58:40 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
vary: Accept-Encoding

GET
H2 200 domaingroupcheck Show response
cookies-data.onetrust.io/bannersdk/v1/ 17 B
86 B 43ms
43ms XHR
application/json 2606:4700:4400::ac40:9b40
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL: https://cookies-data.onetrust.io/bannersdk/v1/domaingroupcheck
Requested by: Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/otSDKStub.js
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 2606:4700:4400::ac40:9b40 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software: cloudflare /
Resource Hash: d1aabe1212b2f9cb8f6a547454bd4e5f4773485e3e001b327e501ba3e0e77cc7

Request headers

location: cdn.cookielaw.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
url: www.wired.com
accept-language: en-GB,en;q=0.9
domainId: 93ddfe0c-4b21-4ad5-8191-612d2a67aad3

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
server: cloudflare
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
vary: Accept-Encoding
access-control-allow-methods: GET, HEAD, OPTIONS
content-type: application/json
access-control-allow-origin: *
cf-ray: 70e8adab4e7d7695-LHR
access-control-allow-headers: Content-Type
content-length: 17

GET
H2 200 location Show response
geolocation.onetrust.com/cookieconsentpub/v1/geo/ 154 B
433 B 97ms
44ms XHR
application/json 2606:4700:10::6814:b844
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://geolocation.onetrust.com/cookieconsentpub/v1/geo/location

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/otSDKStub.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700:10::6814:b844 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

2fa8c45d309615057c3ae6c240e1f9eb9c7f15e16d5ef3b6470467aa8d3b6c0a

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload

Request headers

accept: application/json
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
server: cloudflare
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
vary: Accept-Encoding
access-control-allow-methods: GET, OPTIONS
content-type: application/json
access-control-allow-origin: *
strict-transport-security: max-age=31536000; includeSubDomains; preload
cf-ray: 70e8adab0c4e75c9-LHR
access-control-allow-headers: Content-Type

GET
H/1.1 200
OK recommendations Show response
api.condenast.io/v1/ 26 KB
7 KB 231ms
176ms Fetch
application/json 151.101.192.239
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL: https://api.condenast.io/v1/recommendations?applicationID=wired-right-rail&brand=wired&url=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F&exclude%5Bcategory%5D=functional-tags%2Fno-river&filter%5BnumberOfDays%5D=-30&filter%5Bstrategy%5D=POPULAR&filter%5Blanguage%5D=en-US&page%5Bsize%5D=4&xid=346f4b04-1d97-480c-b8c0-21b9cbba4cd5&filter%5BcontentType%5D=ARTICLE&filter%5BcontentType%5D=REVIEW&filter%5BcontentType%5D=GALLERY
Requested by: Host: www.wired.com
URL: https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.192.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: nginx/1.15.8 /
Resource Hash: e5f4543004148e64beaa3dd965ef70153f0dc50dc4d0adc63c2bbac600754ba0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
X-Backend: 2SrKDXXFWNz87LdtRpzPzK--F_api_eu_central_1_condenast_io
access-control-allow-origin: https://www.wired.com
transfer-encoding: chunked
X-Cache: MISS
Connection: keep-alive
X-Served-By: cache-lhr7383-LHR
Server: nginx/1.15.8
X-Timer: S1653087521.553607,VS0,VE150
Vary: origin,accept-encoding, Accept-Encoding, Origin
Content-Type: application/json;charset=utf-8
Via: 1.1 varnish
access-control-expose-headers: WWW-Authenticate,Server-Authorization
cache-control: private, max-age=0
access-control-allow-credentials: true
Accept-Ranges: bytes
X-Cache-Hits: 0

GET
H3 200 swg-button.css
news.google.com/swg/js/v1/ 21 KB
21 KB 120ms
40ms Stylesheet
text/css 2a00:1450:4001:801::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://news.google.com/swg/js/v1/swg-button.css

Requested by

Host: news.google.com
URL: https://news.google.com/swg/js/v1/swg.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:801::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

bde06a0400c168573473e2de967d842eec383f2f755aef4ec017b2f333e7ff85

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:13:16 GMT
x-content-type-options: nosniff
age: 2724
content-security-policy-report-only: require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/news-frontend
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 21972
x-xss-protection: 0
last-modified: Wed, 12 Jan 2022 22:09:41 GMT
server: sffe
cross-origin-opener-policy: same-origin; report-to="news-frontend"
vary: Accept-Encoding
report-to: {"group":"news-frontend","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/news-frontend"}]}
content-type: text/css
cache-control: public, max-age=3000
accept-ranges: bytes
expires: Fri, 20 May 2022 23:03:16 GMT

GET
H2 200 v2 Show response
mb.moatads.com/yi/ 313 B
487 B 182ms
46ms Script
text/html 54.77.221.60
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://mb.moatads.com/yi/v2?ol=0&qn=%604%7BZEYwoqI%24%5BK%2BdLLU)%2CMm~tM!90vv9L%24%2FoDb%2FG)lKr%23l9jmUdTfN%5Bqir1fcSC%3AU%3FWOvTh%7CzFK%3F%5B%22l!j%3F%5DV%22%3BU!%2FBwj%5DUG0U20!9%3Am%5EG..%2C*%5D%407%25rxaxcpaO%2BZ%5EhG%22%3ExZq%224%7CQjw%60.%7Bi%3F%5DQZ%2CA2%2BNhloI%40s1%7CZ5*%3FVl%3Fe3%7CqL5%40J%3D%5B98bjmPghU%3C4Y%24%26%7Cu9.aS%3B4oD%7D%60%3Fjc!L2LmqMs%3Cex1bxNTK7%2BuCTpY%3CZ.T%5B%2B%22gbzbSSr1r4YvKUntB&tf=1_nMzjG---CSa7H-1SJH-bW7qhB-LRwqH-nMzjG-&vi=111111&rb=2-py0NafZqZuc7FwBUS30oB3JRuQXyveKl6wEaW7S4UrVWTx6jqrPoFzc%3D&rs=1-H141M1%2F1Kg7M0g%3D%3D&sc=1&os=1-EQ%3D%3D&qp=10000&is=BBBBB2BBEYBvGl2BBCBBtUTE1RmsqbKW8BsrBu0rCFE48CRBeeBS2hWTMBBQeQBBn2soYggyUig0CBlWZ0uBBCCCCCCOgRBBiOfnE6Bkg7Oxib8MxOtJYHCBdm5kBhIcC9Y8oBXckXBR76iUUsJBCBBBBBBBBBWBBBj3BBBZeGV2BBBCMciUBBBjgEBBBBBB94UMgTdJMtEcpMBBBQBBBniOccBBBBBB47kNwxBbBBBBBBBBBhcjG6BBJM2L4Bk8BwCBQmIoRBBCzBz1BBCTClBBrbBBC4ehueB57NG9aJeRzBqBBiuwBBBB&iv=8&qt=0&gz=0&hh=0&hn=0&tw=&qc=0&qd=0&qf=1600&qe=1200&qh=1600&qg=1200&qm=0&qa=1600&qb=1200&qi=1600&qj=1200&to=000&po=1-0020002000002120&vy=ot%24b%5Bh%40%22oD~T_Gr1%3E%3AB%40NVt7%3BY%3EhyMmxNXJZPV8t6%3D%3Dh_GW3r4Aj!L%3E%2BbK0pH%23H&qr=0&url=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&pcode=condenastprebidheader987326845656&rx=114366975064&callback=MoatNadoAllJsonpRequest_76788085
Requested by: Host: z.moatads.com
URL: https://z.moatads.com/condenastprebidheader987326845656/moatheader.js
Protocol: H2
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 54.77.221.60 Dublin, Ireland, ASN16509 (AMAZON-02, US),
Reverse DNS: ec2-54-77-221-60.eu-west-1.compute.amazonaws.com
Software: Microsoft-IIS/6.0 /
Resource Hash: 27e55b238d002105e17ff50848818d1a4fa2656f650d0d5ca1cd40aa95b2c6ec

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
cache-control: max-age=900
server: Microsoft-IIS/6.0
timing-allow-origin: *
etag: "60bf9181a4ac2b77e25757f1a33d1cd03a678676"
content-length: 313
content-type: text/html; charset=UTF-8

GET
H/1.1 200
OK beacon
www.allure.com/infinityid/ 35 B
1 KB 177ms
105ms Image
image/gif 151.101.64.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.allure.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.64.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Vary: origin, cn-experiments, Verso
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lcy19264-LCY
Server: nginx/1.15.8
X-Timer: S1653087521.695512,VS0,VE80
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.architecturaldigest.com/infinityid/ 35 B
1 KB 177ms
105ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.architecturaldigest.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Server: nginx/1.15.8
Connection: keep-alive
Vary: origin
X-Cache: MISS
Content-Type: image/gif
expires: 0
cache-control: no-cache
X-Cache-Hits: 0
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
Strict-Transport-Security: max-age=7776000; preload
accept-ranges: bytes
Content-Length: 35
X-Served-By: cache-lcy19244-LCY

GET
H/1.1 200
OK beacon
www.bonappetit.com/infinityid/ 35 B
1 KB 178ms
106ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.bonappetit.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; frame-src https: data: blob:; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Vary: origin, cn-experiments, Verso
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lcy19280-LCY
Server: nginx/1.15.8
X-Timer: S1653087521.696238,VS0,VE81
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; frame-src https: data: blob:; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.cntraveler.com/infinityid/ 35 B
1 KB 183ms
108ms Image
image/gif 151.101.192.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.cntraveler.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.192.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Vary: origin, cn-experiments, Verso
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lhr7376-LHR
Server: nginx/1.15.8
X-Timer: S1653087521.700733,VS0,VE82
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.epicurious.com/infinityid/ 35 B
1 KB 163ms
107ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.epicurious.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Vary: origin, cn-experiments, Verso
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lcy19266-LCY
Server: nginx/1.15.8
X-Timer: S1653087521.681382,VS0,VE82
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.glamour.com/infinityid/ 35 B
1 KB 168ms
107ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.glamour.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Server: nginx/1.15.8
Connection: keep-alive
Vary: origin, cn-experiments, Verso
X-Cache: MISS
Content-Type: image/gif
expires: 0
Cache-Control: no-cache
X-Cache-Hits: 0
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
Strict-Transport-Security: max-age=7776000; preload
accept-ranges: bytes
Content-Length: 35
X-Served-By: cache-lcy19237-LCY

GET
H/1.1 200
OK beacon
www.gq.com/infinityid/ 35 B
1 KB 163ms
108ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.gq.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
X-Cache: MISS
X-UA-Device: desktop
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lcy19232-LCY
Verso: false
Server: nginx/1.15.8
X-Timer: S1653087521.682148,VS0,VE82
Vary: origin, cn-experiments, Verso
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.newyorker.com/infinityid/ 35 B
1 KB 177ms
105ms Image
image/gif 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.newyorker.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.0.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=86400; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
Via: 1.1 varnish
Connection: keep-alive
Date: Fri, 20 May 2022 22:58:40 GMT
Vary: origin
X-Cache: MISS
Content-Type: image/gif
expires: 0
cache-control: no-cache
X-Cache-Hits: 0
Strict-Transport-Security: max-age=86400; preload
accept-ranges: bytes
Content-Length: 35
X-Served-By: cache-lhr7371-LHR

GET
H/1.1 200
OK beacon
pitchfork.com/infinityid/ 35 B
1 KB 169ms
107ms Image
image/gif 151.101.192.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://pitchfork.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.192.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lhr7372-LHR
Verso: false
Server: nginx/1.15.8
X-Timer: S1653087521.802793,VS0,VE81
Vary: Accept-Encoding, X-Format, Verso
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.self.com/infinityid/ 35 B
1 KB 185ms
111ms Image
image/gif 151.101.192.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.self.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.192.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Vary: origin, cn-experiments, Verso
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lhr7333-LHR
Server: nginx/1.15.8
X-Timer: S1653087521.863763,VS0,VE85
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
X-FC-Vary-Parameters: acceptencoding
expires: 0

GET
H/1.1 200
OK beacon
www.teenvogue.com/infinityid/ 35 B
1 KB 181ms
108ms Image
image/gif 151.101.64.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.teenvogue.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.64.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
X-Cache: MISS
X-Cache-Hits: 0
Connection: keep-alive
Content-Length: 35
X-Served-By: cache-lcy19256-LCY
X-Fastly-Backend: XID_BEACON
Server: nginx/1.15.8
X-Timer: S1653087521.863525,VS0,VE83
Vary: origin, cn-experiments, Verso
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
Cache-Control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: bytes
expires: 0

GET
H/1.1 200
OK beacon
www.them.us/infinityid/ 35 B
722 B 172ms
108ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://www.them.us/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.128.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: nginx/1.15.8 /
Resource Hash: 6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Server: nginx/1.15.8
Vary: origin, Accept-Encoding, cn-experiments, Verso
X-Cache: MISS
Content-Type: image/gif
expires: 0
Cache-Control: no-cache
transfer-encoding: chunked
X-Cache-Hits: 0
Connection: keep-alive
accept-ranges: none
content-encoding: gzip
X-Served-By: cache-lcy19228-LCY

GET
H/1.1 200
OK beacon
www.vanityfair.com/infinityid/ 35 B
1 KB 179ms
107ms Image
image/gif 151.101.192.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.vanityfair.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.192.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Via: 1.1 varnish
Vary: origin, Accept-Encoding, cn-experiments, Verso
transfer-encoding: chunked
X-Cache: MISS
Connection: keep-alive
content-encoding: gzip
X-Served-By: cache-lhr7328-LHR
Server: nginx/1.15.8
X-Timer: S1653087521.874704,VS0,VE81
Strict-Transport-Security: max-age=7776000; preload
Content-Type: image/gif
expires: 0
cache-control: no-cache
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests
accept-ranges: none
apple-news-services-host: infinityid.condenastdigital.com
X-Cache-Hits: 0

GET
H/1.1 200
OK beacon
www.vogue.com/infinityid/ 35 B
1 KB 163ms
109ms Image
image/gif 151.101.128.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.vogue.com/infinityid/beacon?id=346f4b04-1d97-480c-b8c0-21b9cbba4cd5

Requested by

Protocol

HTTP/1.1

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.128.239 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

nginx/1.15.8 /

Resource Hash

6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992

Security Headers

Name

Value

Content-Security-Policy

default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; upgrade-insecure-requests

Strict-Transport-Security

max-age=7776000; preload

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Server: nginx/1.15.8
Varnish-X-Cache: MISS
Connection: keep-alive
Vary: origin, cn-experiments, Verso
X-Cache: MISS
Content-Type: image/gif
expires: 0
Cache-Control: no-cache
X-Cache-Hits: 0
Content-Security-Policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: data: blob: android-webview-video-poster:; media-src https: data: blob:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; upgrade-insecure-requests
Strict-Transport-Security: max-age=7776000; preload
accept-ranges: bytes
Content-Length: 35
X-Served-By: cache-lcy19228-LCY

GET
H2 200 otBannerSdk.js Show response
cdn.cookielaw.org/scripttemplates/6.32.0/ 335 KB
79 KB 35ms
35ms Script
application/javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/otSDKStub.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

f8bd28fee94c800df636a486d42ed91d2df89db1fd3e223d5e89ce3d9dd107fe

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: ryfZhYsqLisJEnBsOqgVsQ==
age: 729
vary: Accept-Encoding
content-length: 81095
x-ms-lease-status: unlocked
last-modified: Fri, 18 Mar 2022 16:29:23 GMT
server: cloudflare
etag: 0x8DA08FC76466F7A
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/javascript
access-control-allow-origin: *
x-ms-request-id: 856eacc6-301e-0077-03f7-3a5da7000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8adabfca275c6-LHR

GET
H2 200 analytics.js Show response
www.google-analytics.com/ 49 KB
20 KB 122ms
38ms Script
text/javascript 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/analytics.js

Requested by

Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtm.js?id=GTM-5HBJC2K&l=dataLayer

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

a1925038db769477ab74b4df34350c35688a795bb718727b0f4292a4a78a6210

Security Headers

Name	Value
Strict-Transport-Security	max-age=10886400; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

strict-transport-security: max-age=10886400; includeSubDomains; preload
content-encoding: gzip
x-content-type-options: nosniff
last-modified: Wed, 13 Apr 2022 21:02:38 GMT
server: Golfe2
age: 5034
date: Fri, 20 May 2022 21:34:46 GMT
vary: Accept-Encoding
content-type: text/javascript
cache-control: public, max-age=7200
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 20006
expires: Fri, 20 May 2022 23:34:46 GMT

GET
H2 200 v2 Show response
z-na.associates-amazon.com/onetag/ 11 KB
4 KB 130ms
39ms Script
text/javascript 143.204.209.161
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://z-na.associates-amazon.com/onetag/v2?MarketPlace=US&instanceId=11cb70e4-21b3-453f-834c-1463094700df
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.209.161 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-209-161.fra53.r.cloudfront.net
Software: Server /
Resource Hash: 8cbeee80c026a82a6a5e3b67ee444bf9f42a36b51e512dd9afc0da3220d54bcf

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 07:44:37 GMT
content-encoding: gzip
accept-charset: UTF-8
server: Server
age: 54843
x-amz-rid: X95B25MJQZ000QGSGP9H
vary: accept-encoding,Accept-Encoding
x-cache: Hit from cloudfront
content-type: text/javascript
access-control-allow-origin: *
permissions-policy: interest-cohort=()
x-amz-cf-pop: FRA53-C1
x-amz-cf-id: Bb6pxFTviAVuxLgSt1NFkUYUqzLCGeFfrJqm4qYOEEj3DWXH4gEdTQ==
via: 1.1 1cc446ef4692d8e752b16c07f2f58a58.cloudfront.net (CloudFront)

GET
H2 200 100099X1555751.skimlinks.js Show response
s.skimresources.com/js/ 35 KB
14 KB 102ms
29ms Script
application/octet-stream 151.139.128.11
STACKPATH-CDN

General

Check archive.org

Show headers Download Go to

Full URL: https://s.skimresources.com/js/100099X1555751.skimlinks.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 151.139.128.11 , United States, ASN20446 (STACKPATH-CDN, US),
Reverse DNS
Software: AmazonS3 /
Resource Hash: ed98b083c5cb0ed2d72279f5fa70a2c33badb8cd1b51475efad0c97d99ec29ed

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
last-modified: Tue, 15 Mar 2022 10:52:15 GMT
server: AmazonS3
x-amz-request-id: 3FHZPF0W7S0N9GJ9
etag: "d91e9cbd0c81a19ca71d1cbac83c0957"
x-hw: 1653087520.cds291.lo4.hn,1653087520.cds255.lo4.c
content-type: application/octet-stream
cache-control: max-age=3600
accept-ranges: bytes
content-length: 13760
x-amz-id-2: UQME1kU5MqUcUhNJNBOmmoqhvE7zvObQiwk1bt5Qi6+ffmmfU16tN1c/A4CiwsTl/nY+0bqvFjE=

GET
H2 200 en.json Show response
cdn.cookielaw.org/consent/93ddfe0c-4b21-4ad5-8191-612d2a67aad3/5f81f934-1e1b-4cab-a804-fd3cfc6afeb6/ 173 KB
30 KB 36ms
35ms Fetch
application/x-javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/consent/93ddfe0c-4b21-4ad5-8191-612d2a67aad3/5f81f934-1e1b-4cab-a804-fd3cfc6afeb6/en.json

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

480bcf79f3ffb2d1f72bba3877edda3483c37e75e4db201770f9c962b881e744

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: vOxN1fIL4B4u8ecYHcFqzw==
age: 13528
vary: Accept-Encoding
content-length: 30611
x-ms-lease-status: unlocked
last-modified: Thu, 21 Apr 2022 22:01:03 GMT
server: cloudflare
etag: 0x8DA23E26DCEA838
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/x-javascript
access-control-allow-origin: *
x-ms-request-id: 913c571e-901e-0058-5dcb-55dc9d000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Cache-Control,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: public, max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8adac7bff7714-LHR
expires: Sat, 21 May 2022 02:58:40 GMT

GET
H2 200 iab2Data.json Show response
cdn.cookielaw.org/vendorlist/ 297 KB
41 KB 34ms
33ms Fetch
application/x-javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/vendorlist/iab2Data.json

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

61a0a971698a6b0901df941b1c31c46a41053e7833ce6ab080962ffaf49e5027

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: vE5pU+6UEUfV50+ckTNPzw==
age: 5646
vary: Accept-Encoding
content-length: 41875
x-ms-lease-status: unlocked
last-modified: Fri, 20 May 2022 19:23:44 GMT
server: cloudflare
etag: 0x8DA3A9641669DEC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/x-javascript
access-control-allow-origin: *
x-ms-request-id: 91cc2148-001e-0152-018a-6c8341000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Cache-Control,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: public, max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8adac7c037714-LHR

GET
H2 200 otTCF.js Show response
cdn.cookielaw.org/scripttemplates/6.32.0/ 68 KB
15 KB 32ms
32ms Script
application/javascript 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/scripttemplates/6.32.0/otTCF.js

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

1f94185bf320b088eb3c40b75de95ac8516680f4036bd287131b34f9c058146a

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: mHxiAQQ1IvoJwolLJhbn4g==
age: 10179
vary: Accept-Encoding
content-length: 15011
x-ms-lease-status: unlocked
last-modified: Fri, 18 Mar 2022 16:29:22 GMT
server: cloudflare
etag: 0x8DA08FC75744999
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/javascript
access-control-allow-origin: *
x-ms-request-id: 2d09e78d-c01e-00c8-7700-3b49d1000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8adac7d3475c6-LHR

GET
H/1.1 200
OK Gear-Pom-by-Dame---Source-Dame-2.jpg
media.wired.com/photos/5ed0830175fe02f511141a7a/1:1/w_320,h_350,c_limit/ 5 KB
6 KB 28ms
27ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/5ed0830175fe02f511141a7a/1:1/w_320,h_350,c_limit/Gear-Pom-by-Dame---Source-Dame-2.jpg
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: 8ce81789e3ba1b78f99aac5cc82d35c652244c072a5cc8df0fd99225ea18fbd9

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Connection: keep-alive
Age: 902628
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=85505 idim=682x511 ifmt=jpeg ofsz=5446 odim=320x320 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kiad7000077-IAD, cache-lhr7343-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087521.912026,VS0,VE1
Etag: "4/Q8w2iHJ4+JKEfGvp5Vl2aovmb9j6B/Xar9EDPWCtM"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 5446
timing-allow-origin: *
X-Cache-Hits: 1, 2

GET
H/1.1 200
OK Induction-Cooktops-Gear.jpg
media.wired.com/photos/627d9ed6d2cc3583cf5c8fcb/1:1/w_320,h_350,c_limit/ 17 KB
18 KB 51ms
28ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/627d9ed6d2cc3583cf5c8fcb/1:1/w_320,h_350,c_limit/Induction-Cooktops-Gear.jpg
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: f700de8771b792ff0d89ef4213de13fb344b45ac7e41cf0af724886d8ee1763b

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Connection: keep-alive
Age: 687596
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=1367921 idim=2400x1800 ifmt=jpeg ofsz=17278 odim=320x320 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kiad7000032-IAD, cache-lhr7343-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087521.940240,VS0,VE2
Etag: "N87w1fzVAj3gr+Xp5lTpBKjywdpzWROx9tcmjyK3g2A"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 17278
timing-allow-origin: *
X-Cache-Hits: 1, 1

GET
H/1.1 200
OK Red6-Berkut-Flight-Military-Metaverse-Business.jpg
media.wired.com/photos/627d8a8bd2cc3583cf5c8fc9/1:1/w_320,h_350,c_limit/ 6 KB
7 KB 62ms
27ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/627d8a8bd2cc3583cf5c8fc9/1:1/w_320,h_350,c_limit/Red6-Berkut-Flight-Military-Metaverse-Business.jpg
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: 58a5f8bddb693b55eb999460bbe2d671132ae787eec709202724aed802b51c76

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Connection: keep-alive
Age: 225785
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=401141 idim=2400x1600 ifmt=jpeg ofsz=6634 odim=320x320 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kjyo7100166-IAD, cache-lhr7331-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087521.969510,VS0,VE2
Etag: "F4kLHTxgopAmGbmIQHWC37eomnGT6Iy8hYDVoEDG+ss"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 6634
timing-allow-origin: *
X-Cache-Hits: 1, 1

GET
H/1.1 200
OK New-Honda-Facility-Design-Business.jpg
media.wired.com/photos/628431090c454f12d5142777/1:1/w_320,h_350,c_limit/ 19 KB
19 KB 61ms
27ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/628431090c454f12d5142777/1:1/w_320,h_350,c_limit/New-Honda-Facility-Design-Business.jpg
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: ad3b5971ebec97a3b507722b499de667292078966cb9dd3b121d7a9646794179

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:40 GMT
Connection: keep-alive
Age: 104068
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=1022223 idim=2400x1350 ifmt=jpeg ofsz=19302 odim=320x320 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kjyo7100122-IAD, cache-lhr7343-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087521.971374,VS0,VE1
Etag: "cEya/DF+jg90qp+J5ov1OiOehF8UC2/vOX5mJQtwkg4"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 19302
timing-allow-origin: *
X-Cache-Hits: 1, 3

GET
H2 200 pixel.gif
px.moatads.com/ 43 B
253 B 54ms
53ms Image
image/gif 2.20.86.213
AKAMAI-AS

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://px.moatads.com/pixel.gif?e=24&d=data%3Adata%3Adata%3Adata&i=CONDENAST_PREBID_HEADER1&ac=1&k=SyntaxError%20in%20closure%20%28cb%29%3A%20Failed%20to%20execute%20%27measure%27%20on%20%27Performance%27%3A%20The%20mark%20%27ATP-Init%27%20does%20not%20exist.%2C%20stack%3DError%3A%20Failed%20to%20execute%20%27measure%27%20on%20%27Performance%27%3A%20The%20mark%20%27ATP-Init%27%20does%20not%20exist.%0A%20%20%20%20at%20Object.setMoatLoad%20%5Bas%20moatLoad%5D%20%28https%3A//www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%3A1230%3A328356%29%0A%20%20%20%20at%20emitBoomPixel%20%28https%3A//www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%3A1230%3A329466%29%0A%20%20%20%20at%20handleMoatYieldIntelligenceReady%20%28https%3A//www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%3A1230%3A391173%29%0A%20%20%20%20at%20window.moatYieldReady%20%28https%3A//www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%3A1230%3A391431%29%0A%20%20%20%20at%20https%3A//z.moatads.com/condenastprebidheader987326845656/moatheader.js%3A61%3A335%0A%20%20%20%20at%20h%20%28https%3A//z.moatads.com/condenastprebidheader987326845656/moatheader.js%3A379%3A192%29%0A%20%20%20%20at%20https%3A//z.moatads.com/condenastprebidheader987326845656/moatheader.js%3A392%3A250%0A%20%20%20%20at%20a.xa.sode.a.xa.sode.zaxs%20%28https%3A//z.moatads.com/condenastprebidheader987326845656/moatheader.js%3A104%3A125%29%0A%20%20%20%20at%20g.%3Ccomputed%3E%20%28https%3A//z.moatads.com/condenastprebidheader987326845656/moatheader.js%3A317%3A223%29%0A%20%20%20%20at%20https%3A//mb.moatads.com/yi/v2%3Fol%3D0%26qn%3D%25604%257BZEYwoqI%2524%255BK%252BdLLU%29%252CMm%7EtM%2190vv9L%2524%252FoDb%252FG%29lKr%2523l9jmUdTfN%255Bqir1fcSC%253AU%253FWOvTh%257CzFK%253F%255B%2522l%21j%253F%255DV%2522%253BU%21%252FBwj%255DUG0U20%219%253Am%255EG..%252C*%255D%25407%2525rxaxcpaO%252BZ%255EhG%2522%253ExZq%25224%257CQjw%2560.%257Bi%253F%255DQZ%252CA2%252BNhloI%2540s1%257CZ5*%253FVl%253Fe3%257CqL5%2540J%253D%255B98bjmPghU%253C4Y%2524%2526%257Cu9.aS%253B4oD%257D%2560%253Fjc%21L2LmqMs%253Cex1bxNTK7%252BuCTpY%253CZ.T%255B%252B%2522gbzbSSr1r4YvKUntB%26tf%3D1_nMzjG---CSa7H-1SJH-bW7qhB-LRwqH-nMzjG-%26vi%3D111111%26rb%3D2-py0NafZqZuc7FwBUS30oB3JRuQXyveKl6wEaW7S4UrVWTx6jqrPoFzc%253D%26rs%3D1-H141M1%252F1Kg7M0g%253D%253D%26sc%3D1%26os%3D1-EQ%253D%253D%26qp%3D10000%26is%3DBBBBB2BBEYBvGl2BBCBBtUTE1RmsqbKW8BsrBu0rCFE48CRBeeBS2hWTMBBQeQBBn2soYggyUig0CBlWZ0uBBCCCCCCOgRBBiOfnE6Bkg7Oxib8MxOtJYHCBdm5kBhIcC9Y8oBXckXBR76iUUsJBCBBBBBBBBBWBBBj3BBBZeGV2BBBCMciUBBBjgEBBBBBB94UMgTdJMtEcpMBBBQBBBniOccBBBBBB47kNwxBbBBBBBBBBBhcjG6BBJM2L4Bk8BwCBQmIoRBBCzBz1BBCTClBBrbBBC4ehueB57NG9aJeRzBqBBiuwBBBB%26iv%3D8%26qt%3D0%26gz%3D0%26hh%3D0%26hn%3D0%26tw%3D%26qc%3D0%26qd%3D0%26qf%3D1600%26qe%3D1200%26qh%3D1600%26qg%3D1200%26qm%3D0%26qa%3D1600%26qb%3D1200%26qi%3D1600%26qj%3D1200%26to%3D000%26po%3D1-0020002000002120%26vy%3Dot%2524b%255Bh%2540%2522oD%7ET_Gr1%253E%253AB%2540NVt7%253BY%253EhyMmxNXJZPV8t6%253D%253Dh_GW3r4Aj%21L%253E%252BbK0pH%2523H%26qr%3D0%26url%3Dhttps%253A%252F%252Fwww.wired.com%252Fstory%252Fthe-online-spider-market-is-massive-and-crawling-with-issues%252F%253Fbxid%253D6044883d8f3da371ee6eb3d6%2526cndid%253D64035285%2526esrc%253Dgrowl2-regGate-1120%2526mbid%253Dmbid%25253DCRMWIR012019%25250A%25250A%2526source%253DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%2526utm_brand%253Dwired%2526utm_campaign%253Daud-dev%2526utm_content%253DWIR_Daily_052022%2526utm_mailing%253DWIR_Daily_052022%2526utm_medium%253Demail%2526utm_source%253Dnl%2526utm_term%253DP1%26pcode%3Dcondenastprebidheader987326845656%26rx%3D114366975064%26callback%3DMoatNadoAllJsonpRequest_76788085%3A1%3A1%2C%20%0Acb%3D%28%29%3D%3E%7BhandleMoatYieldIntelligenceReady%28e%29%7D%0A&ar=8d467bec877-clean&iw=9f80626&bq=11&j=&cs=1653087520795
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_256_GCM
Server: 2.20.86.213 Milan, Italy, ASN16625 (AKAMAI-AS, US),
Reverse DNS: a2-20-86-213.deploy.static.akamaitechnologies.com
Software: Apache /
Resource Hash: cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:40 GMT
last-modified: Fri, 20 May 2016 15:16:00 GMT
server: Apache
etag: "ad4b0f606e0f8465bc4c4c170b37e1a3:1463757360"
content-type: image/gif
cache-control: max-age=0, no-cache, no-store
accept-ranges: bytes
content-length: 43
expires: Fri, 20 May 2022 22:58:40 GMT

GET
H2 200 hotjar-1537182.js Show response
static.hotjar.com/c/ 15 KB
5 KB 143ms
46ms Script
application/javascript 108.157.4.21
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL

https://static.hotjar.com/c/hotjar-1537182.js?sv=6

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

108.157.4.21 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-108-157-4-21.dus51.r.cloudfront.net

Software

Resource Hash

2eaad22c1c05e82825717d920a7e69250da0a9117a1ecfbab0e7e2e4d08dee79

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:28 GMT
content-encoding: br
x-content-type-options: nosniff
cache-control: max-age=60
age: 45
etag: W/696f4d1bae29550a410a77c958d5210b
vary: Accept-Encoding
x-cache: Hit from cloudfront
content-type: application/javascript; charset=UTF-8
access-control-allow-origin: *
x-cache-hit: 1
cross-origin-resource-policy: cross-origin
x-amz-cf-pop: DUS51-P2
x-amz-cf-id: W8y8moyB9MVJuEMa1_5CWzNlYUo5doj7fRS_t0gTX--VAYDGbEPfiA==
via: 1.1 b1dc6a0d7547e8d4ab339f8c4caf9ea8.cloudfront.net (CloudFront)

GET
H2 200 otCenterRounded.json Show response
cdn.cookielaw.org/scripttemplates/6.32.0/assets/ 9 KB
3 KB 35ms
35ms Fetch
application/json 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/scripttemplates/6.32.0/assets/otCenterRounded.json

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

1da25cf82f065474aadce40a8c7369f6df48e1847cb95c21bd9083488eff7390

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: 1/jyRD3NFPea58Neu+wVyA==
age: 4711
vary: Accept-Encoding
content-length: 2585
x-ms-lease-status: unlocked
last-modified: Fri, 18 Mar 2022 16:29:15 GMT
server: cloudflare
etag: 0x8DA08FC7156585E
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: application/json
access-control-allow-origin: *
x-ms-request-id: 19bbb77e-801e-012e-537b-431e74000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Content-Encoding,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
accept-ranges: bytes
cf-ray: 70e8adad8da57714-LHR

GET
H2 200 otCommonStyles.css Show response
cdn.cookielaw.org/scripttemplates/6.32.0/assets/ 21 KB
4 KB 34ms
34ms Fetch
text/css 2606:4700::6810:9440
CLOUDFLARENET

General

Check archive.org

Show headers Download Go to

Full URL

https://cdn.cookielaw.org/scripttemplates/6.32.0/assets/otCommonStyles.css

Requested by

Host: cdn.cookielaw.org
URL: https://cdn.cookielaw.org/scripttemplates/6.32.0/otBannerSdk.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2606:4700::6810:9440 , United States, ASN13335 (CLOUDFLARENET, US),

Reverse DNS

Software

cloudflare /

Resource Hash

8edbd08b9bb87f815ad871e44aae03af609fc44b1961d608e94eff3f4e010375

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-ms-blob-type: BlockBlob
date: Fri, 20 May 2022 22:58:40 GMT
content-encoding: gzip
x-content-type-options: nosniff
cf-cache-status: HIT
content-md5: SHFDtZO2nDZuiPDW83p1IQ==
age: 10326
vary: Accept-Encoding
x-ms-lease-status: unlocked
last-modified: Fri, 18 Mar 2022 16:29:27 GMT
server: cloudflare
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=31536000; includeSubDomains; preload
content-type: text/css
access-control-allow-origin: *
x-ms-request-id: 1dd5245f-d01e-0032-237b-438036000000
access-control-expose-headers: x-ms-request-id,Server,x-ms-version,Content-Type,Last-Modified,ETag,Content-MD5,x-ms-lease-status,x-ms-blob-type,Content-Length,Date,Transfer-Encoding
cache-control: max-age=14400
x-ms-version: 2009-09-19
cf-ray: 70e8adad8da67714-LHR

GET
H/1.1 200
OK p.js Show response
d1z2jf7jlzjs58.cloudfront.net/ 930 B
1 KB 130ms
39ms Script
application/javascript 65.9.58.150
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://d1z2jf7jlzjs58.cloudfront.net/p.js
Requested by: Host: www.wired.com
URL: https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 65.9.58.150 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-65-9-58-150.fra56.r.cloudfront.net
Software: nginx /
Resource Hash: 62f586be8571b23584eb4a60a45a3157ff7c8388b1b1e3b4e8890e243b3e47de

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 02:01:56 GMT
Via: 1.1 106758604a7f1ae0fa6678cd3d828d62.cloudfront.net (CloudFront)
Age: 75405
X-Cache: Hit from cloudfront
Connection: keep-alive
Content-Length: 930
Pragma: public
Last-Modified: Wed, 06 May 2020 20:19:48 GMT
Server: nginx
ETag: "5eb31be4-3a2"
Content-Type: application/javascript
Cache-Control: max-age=86400, public
X-Amz-Cf-Pop: FRA56-C1
Accept-Ranges: bytes
X-Amz-Cf-Id: 0L9rI1lMVAqPsmVMIb5_zzoow56AFBXXul2OWIgp7d2_YoNk560RIw==
Expires: Sat, 21 May 2022 02:01:56 GMT

GET
H2 200 BreveText-Bold.woff
www.wired.com/verso/static/assets/fonts/ 35 KB
36 KB 29ms
28ms Font
application/x-font-woff 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/BreveText-Bold.woff

Requested by

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

394e311f0ce57b37ce616f3891c36dd3ff97aab169aa16ead4d3cad4ac6a409b

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:40 GMT
via: 1.1 varnish
age: 1541043
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/BreveText-Bold.woff
last-modified: Tue, 03 May 2022 02:48:30 GMT
verso: true
content-length: 35968
x-amz-id-2: GwAqQQt5dCPkcgrb7YHoL8TSHKfCZn7uJfQXS1W6WlsxHfNEqfi05GQiCGpVxcjQLj9yQSsJofM=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087521.926533,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/BreveText-Bold.woff
etag: "853fddd53ec1c4a69a95444822d3c086"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: R98W9CD9QEA40PS6
access-control-allow-origin: *
expires: Wed, 03 May 2023 02:54:38 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff
apple-news-services-host: www.wired.com
x-cache-hits: 33

GET
H2 206 robots.txt
t.skimresources.com/api/v2/ Frame B535 0
134 B 108ms
36ms Image
text/plain 35.201.67.47
GOOGLE

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://t.skimresources.com/api/v2/robots.txt?__skimjs_preflight__please_ignore__=true&rnd=0.47958488805356403
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 35.201.67.47 Kansas City, United States, ASN15169 (GOOGLE, US),
Reverse DNS: 47.67.201.35.bc.googleusercontent.com
Software: Python/3.7 aiohttp/3.5.4 /
Resource Hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language: en-GB,en;q=0.9
Referer
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
via: 1.1 google
server: Python/3.7 aiohttp/3.5.4
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length: 0
content-type: text/plain charset=UTF-8

GET
H2 200 px.gif
p.skimresources.com/ 43 B
102 B 99ms
34ms Image
image/gif 35.190.91.160
GOOGLE

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://p.skimresources.com/px.gif?ch=1&rn=6.352090755704672
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 35.190.91.160 Kansas City, United States, ASN15169 (GOOGLE, US),
Reverse DNS: 160.91.190.35.bc.googleusercontent.com
Software: Skimlinks Pixel 1.0 /
Resource Hash: dcecab1355b5c2b9ecef281322bf265ac5840b4688748586e9632b473a5fe56b

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
via: 1.1 google
server: Skimlinks Pixel 1.0
p3p: policyref="http://skimlinks.com/w3c/p3p.xml", CP="NOI DSP COR CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length: 43
content-type: image/gif

GET
H2 200 px.gif
p.skimresources.com/ 43 B
276 B 95ms
33ms Image
image/gif 35.190.91.160
GOOGLE

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://p.skimresources.com/px.gif?ch=2&rn=6.352090755704672
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 35.190.91.160 Kansas City, United States, ASN15169 (GOOGLE, US),
Reverse DNS: 160.91.190.35.bc.googleusercontent.com
Software: Skimlinks Pixel 1.0 /
Resource Hash: dcecab1355b5c2b9ecef281322bf265ac5840b4688748586e9632b473a5fe56b

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
via: 1.1 google
server: Skimlinks Pixel 1.0
p3p: policyref="http://skimlinks.com/w3c/p3p.xml", CP="NOI DSP COR CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length: 43
content-type: image/gif

POST
H2 200 / Show response
r.skimresources.com/api/ 162 B
378 B 94ms
34ms XHR
application/json 35.190.59.101
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://r.skimresources.com/api/

Requested by

Host: s.skimresources.com
URL: https://s.skimresources.com/js/100099X1555751.skimlinks.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

35.190.59.101 Kansas City, United States, ASN15169 (GOOGLE, US),

Reverse DNS

101.59.190.35.bc.googleusercontent.com

Software

openresty/1.11.2.5 /

Resource Hash

1b4482365e2a58facf50f2495c776b236aef0626f0ae276b8459e1e3cec6c126

Security Headers

Name	Value
Strict-Transport-Security	max-age=31536000
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-type: application/x-www-form-urlencoded

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
content-encoding: gzip
x-content-type-options: nosniff
server: openresty/1.11.2.5
strict-transport-security: max-age=31536000
content-type: application/json
access-control-allow-origin: https://www.wired.com
vary: Accept-Encoding
access-control-allow-credentials: true
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
via: 1.1 google

GET
H/1.1 200
OK andoncord Show response
assoc-na.associates-amazon.com/onetag/ 16 B
413 B 491ms
106ms XHR
application/json 72.21.195.65
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://assoc-na.associates-amazon.com/onetag/andoncord
Requested by: Host: z-na.associates-amazon.com
URL: https://z-na.associates-amazon.com/onetag/v2?MarketPlace=US&instanceId=11cb70e4-21b3-453f-834c-1463094700df
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 72.21.195.65 Ashburn, United States, ASN16509 (AMAZON-02, US),
Reverse DNS: s3-1.amazonaws.com
Software: Server /
Resource Hash: c462d460eab61de19f36cc384c99666e5bf65eaeba0c12b8f594c5410c01f220

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:41 GMT
Server: Server
x-amz-rid: MV6FAN3CF285QFWF0RPP
Vary: Content-Type,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
Content-Type: application/json
Access-Control-Allow-Origin: https://www.wired.com
Access-Control-Allow-Credentials: true
Permissions-Policy: interest-cohort=()
Connection: keep-alive
Content-Length: 16

GET
H3 200 linkid.js Show response
www.google-analytics.com/plugins/ua/ 2 KB
884 B 116ms
39ms Script
text/javascript 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/plugins/ua/linkid.js

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

92fca55833f48b4289ac8f1cedd48752b580fce4ec4b5d81670b8193d6e51b54

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 21:59:34 GMT
content-encoding: gzip
x-content-type-options: nosniff
age: 3547
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 859
x-xss-protection: 0
last-modified: Tue, 22 Oct 2019 18:15:00 GMT
server: sffe
vary: Accept-Encoding
report-to: {"group":"static-on-bigtable","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/static-on-bigtable"}]}
content-type: text/javascript
cache-control: public, max-age=3600
accept-ranges: bytes
cross-origin-opener-policy-report-only: same-origin; report-to="static-on-bigtable"
expires: Fri, 20 May 2022 22:59:34 GMT

POST
H2 200 publisher:getClientId Show response
ampcid.google.com/v1/ 3 B
460 B 131ms
46ms XHR
application/json 2a00:1450:4001:812::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://ampcid.google.com/v1/publisher:getClientId?key=AIzaSyA65lEHUEizIsNtlbNo-l2K18dT680nsaM

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:812::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

ESF /

Resource Hash

ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Frame-Options	SAMEORIGIN
X-Xss-Protection	0

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
content-encoding: gzip
x-content-type-options: nosniff
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
server: ESF
x-frame-options: SAMEORIGIN
content-type: application/json; charset=UTF-8
access-control-allow-origin: https://www.wired.com
access-control-expose-headers: vary,vary,vary,content-encoding,date,server,content-length
cache-control: private
access-control-allow-credentials: true
vary: Origin, X-Origin, Referer
content-length: 23
x-xss-protection: 0

POST
H2 200 page Show response
t.skimresources.com/api/v2/ 22 B
339 B 37ms
36ms XHR
application/javascript 35.201.67.47
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://t.skimresources.com/api/v2/page

Requested by

Host: s.skimresources.com
URL: https://s.skimresources.com/js/100099X1555751.skimlinks.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

35.201.67.47 Kansas City, United States, ASN15169 (GOOGLE, US),

Reverse DNS

47.67.201.35.bc.googleusercontent.com

Software

Python/3.7 aiohttp/3.5.4 /

Resource Hash

fcc83a5b6aef86420c1ad553167106df96bd0ff4192ffe52b1647599948edbcf

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-type: text/plain

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
via: 1.1 google
x-content-type-options: nosniff
server: Python/3.7 aiohttp/3.5.4
access-control-allow-headers: Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token
access-control-allow-methods: GET, POST, OPTIONS
content-type: text/plain; charset=utf-8, application/javascript
access-control-allow-origin: https://www.wired.com
cache-control: no-store, no-cache, must-revalidate
access-control-allow-credentials: true
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length: 22

POST
H2 200 link Show response
t.skimresources.com/api/v2/ 22 B
114 B 41ms
41ms XHR
application/javascript 35.201.67.47
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://t.skimresources.com/api/v2/link

Requested by

Host: s.skimresources.com
URL: https://s.skimresources.com/js/100099X1555751.skimlinks.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

35.201.67.47 Kansas City, United States, ASN15169 (GOOGLE, US),

Reverse DNS

47.67.201.35.bc.googleusercontent.com

Software

Python/3.7 aiohttp/3.5.4 /

Resource Hash

fcc83a5b6aef86420c1ad553167106df96bd0ff4192ffe52b1647599948edbcf

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-type: text/plain

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
via: 1.1 google
x-content-type-options: nosniff
server: Python/3.7 aiohttp/3.5.4
access-control-allow-headers: Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token
access-control-allow-methods: GET, POST, OPTIONS
content-type: text/plain; charset=utf-8, application/javascript
access-control-allow-origin: https://www.wired.com
cache-control: no-store, no-cache, must-revalidate
access-control-allow-credentials: true
warning: 299 - "Deprecated API"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length: 22

GET
H2 200 p.js Show response
cdn.parsely.com/keys/wired.com/ 56 KB
21 KB 168ms
44ms Script
application/javascript 65.9.61.60
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://cdn.parsely.com/keys/wired.com/p.js
Requested by: Host: d1z2jf7jlzjs58.cloudfront.net
URL: https://d1z2jf7jlzjs58.cloudfront.net/p.js
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 65.9.61.60 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-65-9-61-60.fra56.r.cloudfront.net
Software: nginx /
Resource Hash: e4312f2f537cbd6e153d8426ab167e1537c2aa53c8be4f62f05de9e666155d13

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

pragma: public
date: Fri, 20 May 2022 05:26:14 GMT
content-encoding: gzip
last-modified: Mon, 17 Jan 2022 22:10:23 GMT
server: nginx
age: 63231
etag: W/"61e5e94f-e009"
x-cache: Hit from cloudfront
content-type: application/javascript
via: 1.1 f2fa38e6635ded6d22a69d089217bc90.cloudfront.net (CloudFront)
cache-control: max-age=86400, public
x-amz-cf-pop: FRA56-C1
x-amz-cf-id: MozOGh_Pvhm6F005c3s-3VR3D8musE4RQBNulU-9inbdUoQHvnWiJg==
expires: Sat, 21 May 2022 05:24:50 GMT

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 46ms
46ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

POST
H2 200 collect Show response
stats.g.doubleclick.net/j/ 7 B
444 B 98ms
32ms XHR
text/plain 2a00:1450:400c:c08::9d
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://stats.g.doubleclick.net/j/collect?t=dc&aip=1&_r=3&v=1&_v=j96&tid=UA-8293713-27&cid=1914554227.1653087521&jid=2104130837&gjid=1768948176&_gid=124107926.1653087521&_u=aGBAiUAjBAQCAE~&z=2119389652

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:400c:c08::9d Brussels, Belgium, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

3c25b077a6d92cd9d3576660b68c4c0bd135b78b3cd3b66491ff2c7aa0eeaad3

Security Headers

Name	Value
Strict-Transport-Security	max-age=10886400; includeSubDomains; preload
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain

Response headers

pragma: no-cache
strict-transport-security: max-age=10886400; includeSubDomains; preload
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
date: Fri, 20 May 2022 22:58:41 GMT
content-type: text/plain
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 7
expires: Fri, 01 Jan 1990 00:00:00 GMT

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 46ms
46ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 45ms
45ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 46ms
45ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

GET
H2 200 modules.f31ba00513b7ef8234d1.js Show response
script.hotjar.com/ 243 KB
63 KB 141ms
48ms Script
application/javascript 108.157.4.38
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL

https://script.hotjar.com/modules.f31ba00513b7ef8234d1.js

Requested by

Host: static.hotjar.com
URL: https://static.hotjar.com/c/hotjar-1537182.js?sv=6

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

108.157.4.38 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-108-157-4-38.dus51.r.cloudfront.net

Software

Resource Hash

5475ef880793a875564088fea38154cee107eede5a2af036f3774a3dec5e48e6

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 16:09:07 GMT
content-encoding: br
x-content-type-options: nosniff
age: 24574
x-cache: Hit from cloudfront
cross-origin-resource-policy: cross-origin
content-length: 63712
access-control-allow-origin: *
last-modified: Fri, 20 May 2022 16:08:12 GMT
etag: "bb85a92d3aefdabfa0ed466815889fc6"
vary: Accept-Encoding
content-type: application/javascript
via: 1.1 9f88eecf68d9192420b110f5f3f14fd6.cloudfront.net (CloudFront)
cache-control: max-age=31536000
x-amz-cf-pop: DUS51-P2
accept-ranges: bytes
x-robots-tag: none
x-amz-cf-id: BlDHVN7XC5rJVTgiDXQiSJl-VSxPHgqySinh_uzZahOCevxcLuP3Og==

GET
H2 200 box-4924254a9ce4dc9b959b6e4a9b662d60.html Show response
vars.hotjar.com/ Frame 1EDA 2 KB
1 KB 139ms
42ms Document
text/html 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://vars.hotjar.com/box-4924254a9ce4dc9b959b6e4a9b662d60.html
Requested by: Host: static.hotjar.com
URL: https://static.hotjar.com/c/hotjar-1537182.js?sv=6
Protocol: H2
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: /
Resource Hash: 67f8c7fd7353ad063da1f3115924c458c494cb134f4d87de4407a132842c9bc9

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
accept-language: en-GB,en;q=0.9

Response headers

accept-ranges: bytes
age: 3240575
cache-control: max-age=31536000
content-encoding: br
content-length: 1044
content-type: text/html
cross-origin-embedder-policy: require-corp
cross-origin-resource-policy: cross-origin
date: Wed, 13 Apr 2022 10:49:06 GMT
etag: "1635635016e428baa170305e9282c34a"
last-modified: Wed, 13 Apr 2022 10:48:29 GMT
vary: Accept-Encoding
via: 1.1 1f5757b46371746e677236d4fc67d364.cloudfront.net (CloudFront)
x-amz-cf-id: _ofJ4KeRGVe1HlRtc5-i-QxgXzDikXG7jBpDJJj9SOQzArOuivba8w==
x-amz-cf-pop: FRA53-C1
x-cache: Hit from cloudfront
x-robots-tag: none

GET
H2 200 ga-audiences
www.google.com/ads/ 42 B
501 B 133ms
49ms Image
image/gif 2a00:1450:4001:813::2004
GOOGLE

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.google.com/ads/ga-audiences?t=sr&aip=1&_r=4&slf_rd=1&v=1&_v=j96&tid=UA-8293713-27&cid=1914554227.1653087521&jid=2104130837&_u=aGBAiUAjBAQCAE~&z=1951369414

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:813::2004 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

cafe /

Resource Hash

ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
server: cafe
timing-allow-origin: *
p3p: policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC"
cache-control: no-cache, no-store, must-revalidate
cross-origin-resource-policy: cross-origin
content-type: image/gif
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 42
x-xss-protection: 0
expires: Fri, 01 Jan 1990 00:00:00 GMT

GET
H2 200 ga-audiences
www.google.co.uk/ads/ 42 B
501 B 134ms
50ms Image
image/gif 2a00:1450:4001:82b::2003
GOOGLE

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://www.google.co.uk/ads/ga-audiences?t=sr&aip=1&_r=4&slf_rd=1&v=1&_v=j96&tid=UA-8293713-27&cid=1914554227.1653087521&jid=2104130837&_u=aGBAiUAjBAQCAE~&z=1951369414

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:82b::2003 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

cafe /

Resource Hash

ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:41 GMT
x-content-type-options: nosniff
server: cafe
timing-allow-origin: *
p3p: policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC"
cache-control: no-cache, no-store, must-revalidate
cross-origin-resource-policy: cross-origin
content-type: image/gif
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 42
x-xss-protection: 0
expires: Fri, 01 Jan 1990 00:00:00 GMT

GET
H/1.1 200
OK 583dce7afd2e614a9500000d.js Show response
player.cnevids.com/script/video/ 64 KB
22 KB 415ms
414ms Script
text/javascript 18.65.159.128
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL

https://player.cnevids.com/script/video/583dce7afd2e614a9500000d.js?autoplay=1&muted=1&continuousPlay=1&hasCompanion=false&hideHoverTitle=1&hidePosterTitle=1&interludeOverride=true&onReady=setupInterlude1&playerType=interlude&recAlgorithm=copilotIdOverride&recStrategy=copilotIdOverride&showPlaylistBar=false&isRightRail=true&onIframeReady=onIframeReady16530875212660

Requested by

Host: www.wired.com
URL: https://www.wired.com/verso/static/presenter-articles.f2770f6e6dd98bbb80d4.js

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

18.65.159.128 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-18-65-159-128.nrt51.r.cloudfront.net

Software

nginx/1.18.0 /

Resource Hash

2576bfa8494eebe395580bcae1c994c50b1872f2f5a17b805652ff125d26b935

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	1; mode=block

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:41 GMT
Content-Encoding: gzip
X-Content-Type-Options: nosniff
X-Permitted-Cross-Domain-Policies: none
X-Amz-Cf-Pop: NRT51-P2
X-Cache: Miss from cloudfront
Status: 200 OK
Connection: keep-alive
Content-Length: 21725
X-XSS-Protection: 1; mode=block
X-Request-Id: 51216533-0b6c-4e22-ad84-d1debf97e2e8
X-Runtime: 0.005326
X-Backend-Node: 10.110.40.13
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.18.0
ETag: W/"1b825fdf651d90dc62de3231697848b3"
X-Download-Options: noopen
Vary: Origin,Accept-Encoding
Content-Type: text/javascript; charset=utf-8
Via: 1.1 10669864d9506e9be77be128d971849a.cloudfront.net (CloudFront)
Cache-Control: max-age=0, private, must-revalidate
X-Amz-Cf-Id: psFxCIu8JtO19mrC7gvZVVVga-pYopOx8_OcG56tDZVuasRL51ggaA==

GET
H/1.1 200
OK /
p1.parsely.com/plogger/ 43 B
257 B 164ms
36ms Image
image/gif 54.155.18.159
AMAZON-02

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://p1.parsely.com/plogger/?rand=1653087521280&plid=15315987&idsite=wired.com&url=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&urlref=&screen=1600x1200%7C1600x1200%7C24&data=%7B%22plan%22%3A%22Not+Active%22%7D&sid=1&surl=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&sref=&sts=1653087521275&slts=0&title=The+Online+Spider+Market+Is+Massive%E2%80%94and+Crawling+With+Issues+%7C+WIRED&date=Fri+May+20+2022+22%3A58%3A41+GMT%2B0000+(GMT)&action=pageview&pvid=40616143&u=pid%3D1b3e2fa0b3149ecb5c2dab237069b24e
Requested by: Host: www.wired.com
URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 54.155.18.159 Dublin, Ireland, ASN16509 (AMAZON-02, US),
Reverse DNS: ec2-54-155-18-159.eu-west-1.compute.amazonaws.com
Software: nginx /
Resource Hash: cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:41 GMT
Cache-Control: no-cache
Last-Modified: Friday, 20-May-2022 22:58:41 GMT
Server: nginx
Connection: keep-alive
Content-Length: 43
Content-Type: image/gif

POST
H2 200 visit-data Show response
in.hotjar.com/api/v2/client/sites/1537182/ 147 B
322 B 138ms
41ms XHR
application/json 52.51.233.122
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://in.hotjar.com/api/v2/client/sites/1537182/visit-data?sv=6
Requested by: Host: script.hotjar.com
URL: https://script.hotjar.com/modules.f31ba00513b7ef8234d1.js
Protocol: H2
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 52.51.233.122 Dublin, Ireland, ASN16509 (AMAZON-02, US),
Reverse DNS: ec2-52-51-233-122.eu-west-1.compute.amazonaws.com
Software: /
Resource Hash: 3e8e76a70b5ec0a97f60491364274ab39aebd8f949b6a310a174633b015d4738

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain; charset=UTF-8

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
content-encoding: br
vary: Accept-Encoding
content-type: application/json
access-control-allow-origin: *
access-control-max-age: 86400
cache-control: no-cache, no-store
access-control-allow-credentials: true

GET onetag
assoc-na.associates-amazon.com/ 0
0

GET
H/1.1 200
OK embed-api.json Show response
player.cnevids.com/ 10 KB
4 KB 999ms
471ms Fetch
application/json 18.65.159.128
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL

https://player.cnevids.com/embed-api.json?videoId=583dce7afd2e614a9500000d&interludeOverride=true&playerType=interlude&embedLocation=wired

Requested by

Host: player.cnevids.com
URL: https://player.cnevids.com/script/video/583dce7afd2e614a9500000d.js?autoplay=1&muted=1&continuousPlay=1&hasCompanion=false&hideHoverTitle=1&hidePosterTitle=1&interludeOverride=true&onReady=setupInterlude1&playerType=interlude&recAlgorithm=copilotIdOverride&recStrategy=copilotIdOverride&showPlaylistBar=false&isRightRail=true&onIframeReady=onIframeReady16530875212660

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

18.65.159.128 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-18-65-159-128.nrt51.r.cloudfront.net

Software

nginx/1.18.0 /

Resource Hash

155f28c98e64816b163401a271a70611552b8f142643d4151f6bd08310ad08e9

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	1; mode=block

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:42 GMT
Content-Encoding: gzip
X-Content-Type-Options: nosniff
X-Permitted-Cross-Domain-Policies: none
X-Amz-Cf-Pop: NRT51-P2
X-Cache: Miss from cloudfront
Status: 200 OK
Access-Control-Max-Age: 1728000
Connection: keep-alive
Content-Length: 3484
X-XSS-Protection: 1; mode=block
X-Request-Id: f012a51f-5416-4423-85b7-210b781ef217
X-Runtime: 0.021784
X-Backend-Node: 10.110.9.48
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.18.0
ETag: W/"ce6737b24a12798da0ef82c9eee39d75"
X-Download-Options: noopen
Vary: Origin,Accept-Encoding
Access-Control-Allow-Methods: GET, OPTIONS
Content-Type: application/json; charset=utf-8
Via: 1.1 a74f1fa8dd14a282972d3bd81c744d8e.cloudfront.net (CloudFront)
Access-Control-Expose-Headers
Cache-Control: max-age=300, public
Access-Control-Allow-Origin: *
X-Amz-Cf-Id: I0PEKVUBW5FR7Y6pCgaMrXZaXydcSBdUACP2ShtmJ95mBkG01-flpg==

GET
H2 200 ima3.js Show response
imasdk.googleapis.com/js/sdkloader/ Frame 74EB 377 KB
126 KB 166ms
81ms Script
text/javascript 2a00:1450:4001:800::200a
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://imasdk.googleapis.com/js/sdkloader/ima3.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:800::200a Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

c92ee2460b4063f46ccd0ad0e0a68d212c6b756c4a0ef3a7fdf0afe0989781b1

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:41 GMT
content-encoding: gzip
x-content-type-options: nosniff
server: sffe
cross-origin-opener-policy: same-origin; report-to="ads-doubleclick-instream-static"
vary: Accept-Encoding
report-to: {"group":"ads-doubleclick-instream-static","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-doubleclick-instream-static"}]}
content-type: text/javascript
cache-control: private, max-age=900, stale-while-revalidate=3600
cross-origin-resource-policy: cross-origin
accept-ranges: bytes
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 128852
x-xss-protection: 0
expires: Fri, 20 May 2022 22:58:41 GMT

GET
H2 200 gpt_proxy.js Show response
imasdk.googleapis.com/js/sdkloader/ 85 KB
31 KB 123ms
39ms Script
text/javascript 2a00:1450:4001:800::200a
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://imasdk.googleapis.com/js/sdkloader/gpt_proxy.js

Requested by

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:800::200a Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

92bcf815f419b05328f7002e2c5f304632455fa2b0f3d86a16cb47f877e7ae5a

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:56:08 GMT
content-encoding: gzip
x-content-type-options: nosniff
age: 153
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 31598
x-xss-protection: 0
last-modified: Tue, 10 May 2022 20:29:23 GMT
server: sffe
cross-origin-opener-policy: same-origin; report-to="ads-doubleclick-instream-static"
vary: Accept-Encoding
report-to: {"group":"ads-doubleclick-instream-static","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-doubleclick-instream-static"}]}
content-type: text/javascript
cache-control: public, max-age=900
accept-ranges: bytes
expires: Fri, 20 May 2022 23:11:08 GMT

GET
H/1.1 200
OK player-style-b53d54db9671e9d427bd.css
d2c8v52ll5s99u.cloudfront.net/player/ Frame 74EB 90 KB
13 KB 135ms
44ms Stylesheet
text/css 108.138.2.156
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://d2c8v52ll5s99u.cloudfront.net/player/player-style-b53d54db9671e9d427bd.css
Requested by: Host: player.cnevids.com
URL: https://player.cnevids.com/script/video/583dce7afd2e614a9500000d.js?autoplay=1&muted=1&continuousPlay=1&hasCompanion=false&hideHoverTitle=1&hidePosterTitle=1&interludeOverride=true&onReady=setupInterlude1&playerType=interlude&recAlgorithm=copilotIdOverride&recStrategy=copilotIdOverride&showPlaylistBar=false&isRightRail=true&onIframeReady=onIframeReady16530875212660
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 108.138.2.156 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-138-2-156.fra56.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: 6c26965b3758255462804203b738f7b446de4a66ec96fa8095a0dcc52b334918

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Mon, 16 May 2022 15:48:48 GMT
Content-Encoding: gzip
Age: 371394
X-Cache: Hit from cloudfront
x-amz-replication-status: COMPLETED
Connection: keep-alive
Content-Length: 12888
Last-Modified: Wed, 11 May 2022 18:53:17 GMT
Server: AmazonS3
ETag: "fa4f9cf006104ef494127316840b2ea8"
x-amz-version-id: DdN5Y84Atx5yhBQvA82BKCNxeoeewWoR
Via: 1.1 ab68583a58d574d6a9e5fca1fb1e6316.cloudfront.net (CloudFront)
Cache-Control: max-age=63072000, public
X-Amz-Cf-Pop: FRA56-P6
Accept-Ranges: bytes
Content-Type: text/css; charset=utf-8
X-Amz-Cf-Id: Ii9mI9pWIRCtAvgEZpSu9hs-gSUDnoAv5pgrkmGXtHlYD3W_UhY7Kw==
Expires: Tue, 01 Jan 2030 00:00:00 GMT

GET
H/1.1 200
OK main-fbb87acc700ec3dfa85d.js Show response
d2c8v52ll5s99u.cloudfront.net/player/ Frame 74EB 826 KB
225 KB 136ms
45ms Script
application/javascript 108.138.2.156
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js
Requested by: Host: player.cnevids.com
URL: https://player.cnevids.com/script/video/583dce7afd2e614a9500000d.js?autoplay=1&muted=1&continuousPlay=1&hasCompanion=false&hideHoverTitle=1&hidePosterTitle=1&interludeOverride=true&onReady=setupInterlude1&playerType=interlude&recAlgorithm=copilotIdOverride&recStrategy=copilotIdOverride&showPlaylistBar=false&isRightRail=true&onIframeReady=onIframeReady16530875212660
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 108.138.2.156 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-138-2-156.fra56.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: 6ded4fb8e38deab4abb8dedbe990d41a33177bf001810013926a196ed73487b6

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Mon, 16 May 2022 15:48:48 GMT
Content-Encoding: gzip
Age: 371394
X-Cache: Hit from cloudfront
x-amz-replication-status: COMPLETED
Connection: keep-alive
Content-Length: 230065
Last-Modified: Wed, 11 May 2022 18:53:17 GMT
Server: AmazonS3
ETag: "81b1f66eaf684d9175a40cd89dadb3c7"
x-amz-version-id: v8_YjbyxuR0wzsv6G20h4zokXRk_edOi
Via: 1.1 fb5610ec56d3f427bcbcfdd851770614.cloudfront.net (CloudFront)
Cache-Control: max-age=63072000, public
X-Amz-Cf-Pop: FRA56-P6
Accept-Ranges: bytes
Content-Type: application/javascript
X-Amz-Cf-Id: _zQslPnrzszChSyUkbLXLNV9xGR3xrmCC8JAhs4DeC2LmU19Ug-jOQ==
Expires: Tue, 01 Jan 2030 00:00:00 GMT

GET
BLOB 200
OK 67ba5a73-4353-4c65-ad6b-29217ce3369a
https://www.wired.com/ Frame 74EB 31 B
0 Other
application/javascript

General

Check archive.org

Show headers Download Go to

Full URL: blob:https://www.wired.com/67ba5a73-4353-4c65-ad6b-29217ce3369a
Protocol: BLOB
Server: -, , ASN (),
Reverse DNS
Software: /
Resource Hash: 7725538fe2f71147bffeba7452b434c826aef9009666cb4360c605d0b2a91d1d

Request headers

accept-language: en-GB,en;q=0.9
Referer
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Content-Length: 31
Content-Type: application/javascript

GET s2s-hb
pbs.getpublica.com/v1/ 0
0

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 46ms
46ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:43 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

GET
H/1.1 200
OK background-img.png
media.wired.com/photos/5e421bd83130b10008e82ca9/master/pass/ 2 KB
3 KB 27ms
27ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/5e421bd83130b10008e82ca9/master/pass/background-img.png
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: 3e2c74520955923ebd7093beb2cf2a2fd6b1dabba81de3948e4712620159e61f

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:43 GMT
Connection: keep-alive
Age: 3941170
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=148842 idim=940x140 ifmt=png ofsz=2234 odim=940x140 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kcgs7200077-IAD, cache-lhr7343-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087523.134021,VS0,VE1
Etag: "e/BIdoIoxWGiYzJLIFTMCqKlNJbzbdd4WpWHogtDO2E"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 2234
timing-allow-origin: *
X-Cache-Hits: 1, 6

GET
H/1.1 200
OK background-img%403x.jpg
media.wired.com/photos/5dc1c39d774e490008209a29/master/pass/ 8 KB
8 KB 28ms
27ms Image
image/webp 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/5dc1c39d774e490008209a29/master/pass/background-img%403x.jpg
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: 7475b0e4ec6aaefe03e26dee8683fe8dbb06a19d4443265e59aae73cb9f1e261

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:43 GMT
Connection: keep-alive
Age: 2035813
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=55588 idim=900x750 ifmt=jpeg ofsz=7716 odim=900x750 ofmt=webp
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kiad7000068-IAD, cache-lhr7331-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087523.155685,VS0,VE1
Etag: "M/mkmbs9rl9E75PnmRszaDodeeuPe4h+bNlXbhapVRY"
vary: accept
Content-Type: image/webp
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 7716
timing-allow-origin: *
X-Cache-Hits: 3, 5

GET
H/1.1 200
OK track
capture.condenastdigital.com/ 48 B
48 B 408ms
100ms Image
image/gif 34.232.146.139
AMAZON-AES

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://capture.condenastdigital.com/track?_o=cne&app=playerservice&cCh=videos%2Fshow&pHr=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&pID=&sID=&uId=&xid=346f4b04-1d97-480c-b8c0-21b9cbba4cd5&_ts=2022-05-20T22%3A58%3A43.267Z&_c=error&_t=PrebidError&dim1=%7B%22errorData%22%3A%7B%22timeout%22%3A500%7D%7D&dim3=Timeout
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 34.232.146.139 Ashburn, United States, ASN14618 (AMAZON-AES, US),
Reverse DNS: ec2-34-232-146-139.compute-1.amazonaws.com
Software: /
Resource Hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Access-Control-Allow-Origin: *
Date: Fri, 20 May 2022 22:58:43 GMT
Connection: keep-alive
Content-Length: 48
Content-Type: image/gif

GET
H/1.1 200
OK sf-ui-display-medium-webfont.woff2
d2c8v52ll5s99u.cloudfront.net/assets/fonts/ Frame 74EB 29 KB
30 KB 497ms
417ms Font
application/font-woff2 108.138.2.156
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://d2c8v52ll5s99u.cloudfront.net/assets/fonts/sf-ui-display-medium-webfont.woff2
Requested by: Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/player-style-b53d54db9671e9d427bd.css
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 108.138.2.156 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-138-2-156.fra56.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: d281c9d6bae645f3da6d2f0769a2cf0668709fd28e2021ce74821cdd8c7117b3

Request headers

Referer: https://d2c8v52ll5s99u.cloudfront.net/player/player-style-b53d54db9671e9d427bd.css
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-amz-version-id: null
Content-Encoding: gzip
ETag: "7d18db04f980971f2a9c5026bbc34bed"
X-Amz-Cf-Pop: FRA56-P6
X-Cache: RefreshHit from cloudfront
Access-Control-Max-Age: 3000
Connection: keep-alive
Content-Length: 29632
Access-Control-Allow-Origin: *
Last-Modified: Mon, 26 Jun 2017 15:24:42 GMT
Server: AmazonS3
Date: Fri, 20 May 2022 22:58:44 GMT
Vary: Origin,Access-Control-Request-Headers,Access-Control-Request-Method
Access-Control-Allow-Methods: GET
Content-Type: application/font-woff2
Via: 1.1 6c2674fb15c38f5458794dd680986b8e.cloudfront.net (CloudFront)
Cache-Control: max-age=63072000, public
Accept-Ranges: bytes
X-Amz-Cf-Id: DDhZlgTamSRkeHNX1KvIXNSXASCRT2LUrNABhnsHzOtcMO3Ex8MPZw==
Expires: Tue, 01 Jan 2030 00:00:00 GMT

GET
H3 200 bridge3.516.0_en.html Show response
imasdk.googleapis.com/js/core/ Frame 1E22 634 KB
205 KB 114ms
39ms Document
text/html 2a00:1450:4001:800::200a
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://imasdk.googleapis.com/js/core/bridge3.516.0_en.html

Requested by

Host: imasdk.googleapis.com
URL: https://imasdk.googleapis.com/js/sdkloader/ima3.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:800::200a Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

066fc3ce045361bba8240f583393178cdedced02f8d4bb917c2d3f0520032564

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
accept-language: en-GB,en;q=0.9

Response headers

accept-ranges: bytes
age: 376228
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
cache-control: public, max-age=31536000
content-encoding: gzip
content-length: 210041
content-type: text/html
cross-origin-opener-policy: same-origin; report-to="ads-doubleclick-instream-static"
cross-origin-resource-policy: cross-origin
date: Mon, 16 May 2022 14:28:15 GMT
expires: Tue, 16 May 2023 14:28:15 GMT
last-modified: Tue, 10 May 2022 20:24:29 GMT
report-to: {"group":"ads-doubleclick-instream-static","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-doubleclick-instream-static"}]}
server: sffe
vary: Accept-Encoding
x-content-type-options: nosniff
x-xss-protection: 0

GET
H2 200 client.js Show response
s0.2mdn.net/instream/video/ Frame 74EB 44 KB
17 KB 134ms
49ms Script
text/javascript 2a00:1450:4001:80f::2006
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://s0.2mdn.net/instream/video/client.js

Requested by

Host: imasdk.googleapis.com
URL: https://imasdk.googleapis.com/js/sdkloader/ima3.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:80f::2006 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

d0bffc7261df1454c5e05475cda7d9e6647318dc6c3936767e1252bfe8849c54

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:43 GMT
content-encoding: gzip
x-content-type-options: nosniff
server: sffe
vary: Accept-Encoding
report-to: {"group":"ads-doubleclick-media","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-doubleclick-media"}]}
content-type: text/javascript
cache-control: private, max-age=900
cross-origin-resource-policy: cross-origin
accept-ranges: bytes
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 16746
x-xss-protection: 0
cross-origin-opener-policy-report-only: same-origin; report-to="ads-doubleclick-media"
expires: Fri, 20 May 2022 22:58:43 GMT

GET
H2 200 fbevents.js Show response
connect.facebook.net/en_US/ Frame 74EB 99 KB
27 KB 126ms
39ms Script
application/x-javascript 2a03:2880:f02d:100:face:b00c:0:3
FACEBOOK

General

Check archive.org

Show headers Download Go to

Full URL

https://connect.facebook.net/en_US/fbevents.js

Requested by

Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a03:2880:f02d:100:face:b00c:0:3 Frankfurt am Main, Germany, ASN32934 (FACEBOOK, US),

Reverse DNS

Software

Resource Hash

acbe6770b0fc8b621a9d4f7068b241fb403fe999ea33270931ee59ec4cfdf3f1

Security Headers

Name	Value
Content-Security-Policy	default-src facebook.net .facebook.net fbcdn.net .fbcdn.net fbsbx.com .fbsbx.com data: blob: 'self';script-src .fbcdn.net .facebook.net 'unsafe-inline' 'unsafe-eval' blob: data: 'self';style-src data: blob: 'unsafe-inline' facebook.net .facebook.net fbcdn.net .fbcdn.net fbsbx.com .fbsbx.com;connect-src .fbcdn.net .facebook.net wss://*.fbcdn.net attachment.fbsbx.com blob: 'self';block-all-mixed-content;upgrade-insecure-requests;report-uri https://www.facebook.com/csp/reporting/?m=c&minimize=0;
Strict-Transport-Security	max-age=31536000; preload; includeSubDomains
X-Content-Type-Options	nosniff
X-Frame-Options	DENY
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

content-security-policy: default-src facebook.net *.facebook.net fbcdn.net *.fbcdn.net fbsbx.com *.fbsbx.com data: blob: 'self';script-src *.fbcdn.net *.facebook.net 'unsafe-inline' 'unsafe-eval' blob: data: 'self';style-src data: blob: 'unsafe-inline' facebook.net *.facebook.net fbcdn.net *.fbcdn.net fbsbx.com *.fbsbx.com;connect-src *.fbcdn.net *.facebook.net wss://*.fbcdn.net attachment.fbsbx.com blob: 'self';block-all-mixed-content;upgrade-insecure-requests;report-uri https://www.facebook.com/csp/reporting/?m=c&minimize=0;
content-encoding: gzip
x-content-type-options: nosniff
document-policy: force-load-at-top
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=86400,h3-29=":443"; ma=86400
content-length: 26310
x-xss-protection: 0
pragma: public
x-fb-debug: 8vzTRcg1lDZqIow+Ttw7HZkowApV5gKSp0Q6zBhZS6VvK5VzB7cjtb1NpJh8y4DvN+USc5L8Ka2i0xJkT3/4tA==
x-fb-trip-id: 917726464
x-frame-options: DENY
date: Fri, 20 May 2022 22:58:43 GMT
strict-transport-security: max-age=31536000; preload; includeSubDomains
content-type: application/x-javascript; charset=utf-8
vary: Accept-Encoding
cache-control: public, max-age=1200
x-fb-rlafr: 0
priority: u=3,i
expires: Sat, 01 Jan 2000 00:00:00 GMT

GET
H/1.1 200
OK track
capture.condenastdigital.com/ Frame 74EB 48 B
48 B 399ms
100ms Image
image/gif 34.232.146.139
AMAZON-AES

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://capture.condenastdigital.com/track?_o=cne&_ts=2022-05-20T22%3A58%3A43.271Z&_c=&_t=Player%20Requested&pHr=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 34.232.146.139 Ashburn, United States, ASN14618 (AMAZON-AES, US),
Reverse DNS: ec2-34-232-146-139.compute-1.amazonaws.com
Software: /
Resource Hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Access-Control-Allow-Origin: *
Date: Fri, 20 May 2022 22:58:43 GMT
Connection: keep-alive
Content-Length: 48
Content-Type: image/gif

GET
H/1.1 200
OK wired_absurd-creature-jumping-spider-tk.jpg
dwgyu36up6iuz.cloudfront.net/heru80fdn/image/upload/c_fill,d_placeholder_wired.png,fl_progressive,g_face,h_450,q_80,w_800/v1480451541/ Frame 74EB 25 KB
26 KB 156ms
65ms Image
image/jpeg 65.9.58.194
AMAZON-02

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://dwgyu36up6iuz.cloudfront.net/heru80fdn/image/upload/c_fill,d_placeholder_wired.png,fl_progressive,g_face,h_450,q_80,w_800/v1480451541/wired_absurd-creature-jumping-spider-tk.jpg

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

65.9.58.194 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-65-9-58-194.fra56.r.cloudfront.net

Software

Cloudinary /

Resource Hash

e853373f24fc82d9ddb21a9c0e592f8eb5771e168ab81418fba7e7bcee617d63

Security Headers

Name	Value
Strict-Transport-Security	max-age=604800
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Strict-Transport-Security: max-age=604800
Via: 1.1 2fc0d20914c32e5cd76477ed042298d0.cloudfront.net (CloudFront)
X-Content-Type-Options: nosniff
X-Amz-Cf-Pop: FRA56-C1
X-Cache: RefreshHit from cloudfront
Date: Fri, 20 May 2022 22:58:43 GMT
Server-Timing: fastly;dur=1;cpu=0;start=2022-05-19T08:28:04.472Z;desc=hit,rtt;dur=0
Content-Length: 25338
Last-Modified: Sun, 12 Nov 2017 00:20:43 GMT
Server: Cloudinary
Cache-Control: public, no-transform, max-age=300
ETag: "1f09a329ca606b834e4450bee455840b"
Content-Type: image/jpeg
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Length,ETag,Server-Timing,X-Content-Type-Options
Connection: keep-alive
Accept-Ranges: bytes
Timing-Allow-Origin: *
X-Amz-Cf-Id: IX6b6NyD8YrSOdT8xC86dewdwSG4Oi0Gi_lxb-wjKx0FsCIJEEO5Ug==

GET
H2 200 omweb-v1.js Show response
pagead2.googlesyndication.com/omsdk/releases/live/ Frame 6D1D 37 KB
13 KB 119ms
37ms Script
text/javascript 2a00:1450:4001:827::2002
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://pagead2.googlesyndication.com/omsdk/releases/live/omweb-v1.js

Requested by

Host: srcdoc
URL: about:srcdoc

Protocol

Security

TLS 1.3, , AES_128_GCM

Server

2a00:1450:4001:827::2002 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

sffe /

Resource Hash

e2511b147f3cf95f742758d3e2062eac98f5265a859dc07959eb8a32f0a2f528

Security Headers

Name	Value
X-Content-Type-Options	nosniff
X-Xss-Protection	0

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:08:34 GMT
content-encoding: gzip
x-content-type-options: nosniff
age: 3009
content-security-policy-report-only: require-trusted-types-for 'script'; report-uri https://csp.withgoogle.com/csp/omsdk-team-release-policy
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 12861
x-xss-protection: 0
last-modified: Tue, 26 Oct 2021 20:08:54 GMT
server: sffe
cross-origin-opener-policy: same-origin; report-to="omsdk-team-release-policy"
vary: Accept-Encoding
report-to: {"group":"omsdk-team-release-policy","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/omsdk-team-release-policy"}]}
content-type: text/javascript
cache-control: public, max-age=3600
accept-ranges: bytes
expires: Fri, 20 May 2022 23:08:34 GMT

GET
H/1.1 206
Partial Content 370882a1-1d6b-45b8-8cfc-d98e928c2086thumbs.mp4
dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/ Frame 74EB 24 KB
0 147ms
41ms Media
video/mp4 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/370882a1-1d6b-45b8-8cfc-d98e928c2086thumbs.mp4
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: AmazonS3 /
Resource Hash

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Accept-Encoding: identity;q=1, *;q=0
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Range: bytes=0-

Response headers

Date: Fri, 20 May 2022 22:31:02 GMT
Via: 1.1 85dc19f43b2a0bd8840fdf8baf07d762.cloudfront.net (CloudFront)
Last-Modified: Tue, 29 Nov 2016 20:23:51 GMT
Server: AmazonS3
Age: 14705
ETag: "c840703cd2df566205591c2174d46f5e"
X-Cache: Hit from cloudfront
Content-Type: video/mp4
Content-Range: bytes 0-180235/180236
Connection: keep-alive
X-Amz-Cf-Pop: FRA53-C1
Accept-Ranges: bytes
Content-Length: 180236
X-Amz-Cf-Id: H1wKUq41KOP-8VFEsov6FPr4nyrbGul3kcx1eUTU15TuWTglnVLITQ==

GET
H/1.1 206
Partial Content 370882a1-1d6b-45b8-8cfc-d98e928c2086thumbs.mp4
dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/ Frame 74EB 74 KB
0 150ms
42ms Media
video/mp4 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/370882a1-1d6b-45b8-8cfc-d98e928c2086thumbs.mp4
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: AmazonS3 /
Resource Hash

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Accept-Encoding: identity;q=1, *;q=0
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Range: bytes=0-

Response headers

Date: Fri, 20 May 2022 22:31:02 GMT
Via: 1.1 b16802a1e349d80b7688070778305ae2.cloudfront.net (CloudFront)
Last-Modified: Tue, 29 Nov 2016 20:23:51 GMT
Server: AmazonS3
Age: 14705
ETag: "c840703cd2df566205591c2174d46f5e"
X-Cache: Hit from cloudfront
Content-Type: video/mp4
Content-Range: bytes 0-180235/180236
Connection: keep-alive
X-Amz-Cf-Pop: FRA53-C1
Accept-Ranges: bytes
Content-Length: 180236
X-Amz-Cf-Id: 6O9UczRgFZ-lBr_Yoat-PnleX7wDy2h4nMw7sH0-w9nAiyysjCIiww==

GET
H/1.1 200
OK comscore-min.js Show response
d2c8v52ll5s99u.cloudfront.net/player/ Frame 74EB 38 KB
11 KB 40ms
40ms Script
text/javascript 108.138.2.156
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://d2c8v52ll5s99u.cloudfront.net/player/comscore-min.js
Requested by: Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 108.138.2.156 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-108-138-2-156.fra56.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: 833a86642252016b29f08dd45ffd27f9e00ca237f28d8c5f0147a6e15d009377

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

x-amz-version-id: null
Content-Encoding: gzip
Vary: Accept-Encoding
Last-Modified: Wed, 24 May 2017 18:19:15 GMT
Server: AmazonS3
Age: 117
ETag: W/"054acb6fbd2b2a6c1ac561705bffb0cc"
Transfer-Encoding: chunked
X-Cache: Hit from cloudfront
Content-Type: text/javascript
Via: 1.1 fb5610ec56d3f427bcbcfdd851770614.cloudfront.net (CloudFront)
Connection: keep-alive
Date: Fri, 20 May 2022 22:56:51 GMT
X-Amz-Cf-Pop: FRA56-P6
X-Amz-Cf-Id: 9n_LnSqeE7izVNIzsfWyW6l14tKzfcpQD60J3w4QpIo67M4rcoEGog==

GET
H/1.1 200
OK 370882a1-1d6b-45b8-8cfc-d98e928c2086manifest-ios.m3u8 Show response
dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/ Frame 74EB 918 B
1 KB 137ms
41ms XHR
application/x-mpegurl 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/370882a1-1d6b-45b8-8cfc-d98e928c2086manifest-ios.m3u8?videoIndex=0&requester=oo
Requested by: Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: 738d35cb217fbd63b59fb51803ea8e491cd8114db5f292358695afe6a77c1802

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 08:32:32 GMT
Via: 1.1 9d27077cd67d98c0474b05ec9d68df4a.cloudfront.net (CloudFront)
Vary: Origin
Age: 51972
X-Cache: Hit from cloudfront
Connection: keep-alive
Content-Length: 918
Last-Modified: Sat, 21 Jan 2017 15:00:15 GMT
Server: AmazonS3
ETag: "7538be757367bd513982ad33b9bd1a30"
Access-Control-Max-Age: 3000
Access-Control-Allow-Methods: GET, HEAD
Content-Type: application/x-mpegURL
Access-Control-Allow-Origin: *
X-Amz-Cf-Pop: FRA53-C1
Accept-Ranges: bytes
X-Amz-Cf-Id: v76z7uI3g5jAdI7brK1PeyCVthUBF36223eALQfzD5iYLqJa5EvV4w==

GET
BLOB 200
OK f4d6a14d-160d-4431-80f9-f11836564220
https://www.wired.com/ Frame 74EB 5 KB
0 Other
application/javascript

General

Check archive.org

Show headers Download Go to

Full URL: blob:https://www.wired.com/f4d6a14d-160d-4431-80f9-f11836564220
Protocol: BLOB
Server: -, , ASN (),
Reverse DNS
Software: /
Resource Hash: 173d7c7e266672df75c4e048a934c55ee24d9a9028a87fd2957e74d1bd6a8d08

Request headers

accept-language: en-GB,en;q=0.9
Referer
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Content-Length: 5409
Content-Type: application/javascript

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://dwgyu36up6iuz.cloudfront.net/heru80fdn/image/upload/c_fill,d_placeholder_wired.png,fl_progressive,g_face,h_450,q_80,w_800/v1480451541/wired_absurd-creature-jumping-spider-tk.jpg

Requested by

Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

65.9.58.194 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-65-9-58-194.fra56.r.cloudfront.net

Software

Cloudinary /

Resource Hash

e853373f24fc82d9ddb21a9c0e592f8eb5771e168ab81418fba7e7bcee617d63

Security Headers

Name	Value
Strict-Transport-Security	max-age=604800
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Strict-Transport-Security: max-age=604800
Via: 1.1 cc763905c39a59494c951c09271b0422.cloudfront.net (CloudFront)
X-Content-Type-Options: nosniff
X-Amz-Cf-Pop: FRA56-C1
X-Cache: Hit from cloudfront
Date: Fri, 20 May 2022 22:58:43 GMT
Server-Timing: fastly;dur=1;cpu=0;start=2022-05-19T08:28:04.472Z;desc=hit,rtt;dur=0
Content-Length: 25338
Last-Modified: Sun, 12 Nov 2017 00:20:43 GMT
Server: Cloudinary
Cache-Control: public, no-transform, max-age=300
ETag: "1f09a329ca606b834e4450bee455840b"
Content-Type: image/jpeg
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Length,ETag,Server-Timing,X-Content-Type-Options
Connection: keep-alive
Accept-Ranges: bytes
Timing-Allow-Origin: *
X-Amz-Cf-Id: yRQJf9f98WWPOrJ6vy_x6f1UipFbs8SE7nxy8tETAhVsj9VPIj1uYA==

General

Check archive.org

Show headers Download Go to Show image

Full URL

https://dwgyu36up6iuz.cloudfront.net/heru80fdn/image/upload/c_fill,d_placeholder_wired.png,fl_progressive,g_face,h_450,q_80,w_800/v1480451541/wired_absurd-creature-jumping-spider-tk.jpg

Protocol

HTTP/1.1

Security

TLS 1.3, , AES_128_GCM

Server

65.9.58.194 , United States, ASN16509 (AMAZON-02, US),

Reverse DNS

server-65-9-58-194.fra56.r.cloudfront.net

Software

Cloudinary /

Resource Hash

e853373f24fc82d9ddb21a9c0e592f8eb5771e168ab81418fba7e7bcee617d63

Security Headers

Name	Value
Strict-Transport-Security	max-age=604800
X-Content-Type-Options	nosniff

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Strict-Transport-Security: max-age=604800
Via: 1.1 2fc0d20914c32e5cd76477ed042298d0.cloudfront.net (CloudFront)
X-Content-Type-Options: nosniff
Age: 0
X-Cache: Hit from cloudfront
Date: Fri, 20 May 2022 22:58:43 GMT
Server-Timing: fastly;dur=1;cpu=0;start=2022-05-19T08:28:04.472Z;desc=hit,rtt;dur=0
Content-Length: 25338
Last-Modified: Sun, 12 Nov 2017 00:20:43 GMT
Server: Cloudinary
Cache-Control: public, no-transform, max-age=300
ETag: "1f09a329ca606b834e4450bee455840b"
Content-Type: image/jpeg
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Length,ETag,Server-Timing,X-Content-Type-Options
Connection: keep-alive
X-Amz-Cf-Pop: FRA56-C1
Accept-Ranges: bytes
Timing-Allow-Origin: *
X-Amz-Cf-Id: 6ddTbzD3NOjjQndjyVOiO7Mp6sSShsp2vOa2mbPaK9C00ICHh5ePVA==

GET
H/1.1 200
OK 370882a1-1d6b-45b8-8cfc-d98e928c2086file-1422k-128-48000-768.m3u8 Show response
dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/ Frame 74EB 818 B
1 KB 41ms
41ms XHR
application/x-mpegurl 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/370882a1-1d6b-45b8-8cfc-d98e928c2086file-1422k-128-48000-768.m3u8?requester=oo
Requested by: Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: abb8f643c4c3c538b113f051044746b5e5bf19273ccf217807546d1c3eee64f8

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:48:43 GMT
Via: 1.1 9d27077cd67d98c0474b05ec9d68df4a.cloudfront.net (CloudFront)
Vary: Origin
Age: 10688
X-Cache: Hit from cloudfront
Connection: keep-alive
Content-Length: 818
Last-Modified: Sat, 21 Jan 2017 15:00:12 GMT
Server: AmazonS3
ETag: "eb858d64cea2670234d5d54d094f3dd4"
Access-Control-Max-Age: 3000
Access-Control-Allow-Methods: GET, HEAD
Content-Type: application/x-mpegURL
Access-Control-Allow-Origin: *
X-Amz-Cf-Pop: FRA53-C1
Accept-Ranges: bytes
X-Amz-Cf-Id: i75OuZu8hpcDm3Rm0GUok-60W1BQLrh07MWHf4TaLsHLVFsl2auJGQ==

GET
BLOB 200
OK ebb09bd0-bbc2-473b-846a-b71b4f337dd4
https://www.wired.com/ Frame 74EB 63 KB
0 Other
application/javascript

General

Check archive.org

Show headers Download Go to

Full URL: blob:https://www.wired.com/ebb09bd0-bbc2-473b-846a-b71b4f337dd4
Protocol: BLOB
Server: -, , ASN (),
Reverse DNS
Software: /
Resource Hash: 87b2408523892f375c00a9d521c67f6eb516ecac25c479a7b15705bfab08fcd3

Request headers

accept-language: en-GB,en;q=0.9
Referer
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Content-Length: 64606
Content-Type: application/javascript

GET
H/1.1 200
OK 370882a1-1d6b-45b8-8cfc-d98e928c2086file-1422k-128-48000-768-00001.ts Show response
dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/ Frame 74EB 951 KB
952 KB 42ms
41ms XHR
video/mp2t 143.204.215.26
AMAZON-02

General

Check archive.org

Show headers Download Go to

Full URL: https://dp8hsntg6do36.cloudfront.net/583dce7afd2e614a9500000d/370882a1-1d6b-45b8-8cfc-d98e928c2086file-1422k-128-48000-768-00001.ts?requester=oo
Requested by: Host: d2c8v52ll5s99u.cloudfront.net
URL: https://d2c8v52ll5s99u.cloudfront.net/player/main-fbb87acc700ec3dfa85d.js
Protocol: HTTP/1.1
Security: TLS 1.3, , AES_128_GCM
Server: 143.204.215.26 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS: server-143-204-215-26.fra53.r.cloudfront.net
Software: AmazonS3 /
Resource Hash: e690c27c97f5a802d795964cff40f2ee90b90d87ec3e8043489840404d1ce6eb

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:48:43 GMT
Via: 1.1 9d27077cd67d98c0474b05ec9d68df4a.cloudfront.net (CloudFront)
Vary: Origin
Age: 11192
X-Cache: Hit from cloudfront
Connection: keep-alive
Content-Length: 973840
Last-Modified: Tue, 29 Nov 2016 20:24:10 GMT
Server: AmazonS3
ETag: "b948992cf510cd5c74621347956622cb"
Access-Control-Max-Age: 3000
Access-Control-Allow-Methods: GET, HEAD
Content-Type: video/mp2t
Access-Control-Allow-Origin: *
X-Amz-Cf-Pop: FRA53-C1
Accept-Ranges: bytes
X-Amz-Cf-Id: nVUKKClx3-ro3UQQHqZtePcjyp6wyizgy3YmAVf8qFBAuRPWUGPgNQ==

GET
H2 200 LabGrotesque-Medium.woff2
www.wired.com/verso/static/assets/fonts/ 45 KB
46 KB 27ms
27ms Font
application/x-font-woff2 151.101.2.194
FASTLY

General

Check archive.org

Show headers Download Go to

Full URL

https://www.wired.com/verso/static/assets/fonts/LabGrotesque-Medium.woff2

Protocol

Security

TLS 1.2, ECDHE_RSA, AES_128_GCM

Server

151.101.2.194 , United States, ASN54113 (FASTLY, US),

Reverse DNS

Software

Resource Hash

8575c83bcd649c28701830bc98d0051adba45ba685776092d2417c9240ee0c67

Security Headers

Name

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
Origin: https://www.wired.com
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

date: Fri, 20 May 2022 22:58:44 GMT
via: 1.1 varnish
age: 2143584
x-cache: HIT
x-ua-device: desktop
apple-news-services-parsed-url: /verso/static/assets/fonts/LabGrotesque-Medium.woff2
last-modified: Mon, 25 Apr 2022 23:50:36 GMT
verso: true
content-length: 46416
x-amz-id-2: QgEiW6Jl6IlyMgR3GtLkxbHmbvpphGA4JsXZcTKioy9K5JdD8tDMIyDlmfmM/ldB3a1isPwVVuw=
x-served-by: cache-lcy19261-LCY
apple-news-services-handled: false
x-timer: S1653087524.356964,VS0,VE1
apple-news-services-request-url: /verso/static/assets/fonts/LabGrotesque-Medium.woff2
etag: "d3df48def87009f6ce71e9e5b82a1281"
vary: Verso, bypass-verso-payment
strict-transport-security: max-age=31536000; preload
x-amz-request-id: 5TB26ZA5AZYJMXP0
access-control-allow-origin: *
expires: Wed, 26 Apr 2023 03:32:20 GMT
cache-control: max-age=31536000, immutable
content-security-policy: default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src https: data: blob:; connect-src https: data: blob: wss://*.hotjar.com wss://*.conde.digital; font-src https: data:; img-src https: blob: data: android-webview-video-poster:; media-src blob: data: https:; object-src https:; script-src https: data: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; block-all-mixed-content; upgrade-insecure-requests;
accept-ranges: bytes
content-type: application/x-font-woff2
apple-news-services-host: www.wired.com
x-cache-hits: 8

GET
H/1.1 200
OK Group%204%402x.png
media.wired.com/photos/62740724cfc157274bd1cc8d/master/pass/ 116 KB
116 KB 28ms
27ms Image
image/png 151.101.0.239
FASTLY

General

Check archive.org

Show headers Download Go to Show image

Full URL: https://media.wired.com/photos/62740724cfc157274bd1cc8d/master/pass/Group%204%402x.png?format=original
Protocol: HTTP/1.1
Security: TLS 1.2, ECDHE_RSA, AES_128_GCM
Server: 151.101.0.239 , United States, ASN54113 (FASTLY, US),
Reverse DNS
Software: /
Resource Hash: fb8378d4d1b47a50b7030eed7eae7a704df9bb3d0532d5de67a82e41e5c78525

Request headers

accept-language: en-GB,en;q=0.9
Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36

Response headers

Date: Fri, 20 May 2022 22:58:44 GMT
Connection: keep-alive
Age: 1316229
X-Cache: HIT, HIT
Fastly-Io-Info: ifsz=122341 idim=402x432 ifmt=png ofsz=118309 odim=402x432 ofmt=png
server-timing: geo;desc="continent=EU;country=GB;pop=LHR"
Fastly-Restarts: 1
X-Served-By: cache-iad-kiad7000046-IAD, cache-lhr7331-LHR
experience: katra
Accept-Ranges: bytes
X-Timer: S1653087524.358244,VS0,VE1
Etag: "taZJuV8/CvQFDejR5b5YAKX1hRjHeNFl4ydkBPldSi0"
vary: accept
Content-Type: image/png
Fastly-Stats: io=1
cache-control: max-age=31536, must-revalidate, public
Content-Length: 118309
timing-allow-origin: *
X-Cache-Hits: 1, 7

POST
H3 200 collect
www.google-analytics.com/ 35 B
55 B 46ms
46ms Ping
image/gif 2a00:1450:4001:813::200e
GOOGLE

General

Check archive.org

Show headers Download Go to

Full URL

https://www.google-analytics.com/collect

Requested by

Host: www.google-analytics.com
URL: https://www.google-analytics.com/analytics.js

Protocol

Security

QUIC, , AES_128_GCM

Server

2a00:1450:4001:813::200e Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),

Reverse DNS

Software

Golfe2 /

Resource Hash

8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015

Security Headers

Name	Value
X-Content-Type-Options	nosniff

Request headers

Referer: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1
accept-language: en-GB,en;q=0.9
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36
Content-Type: text/plain;charset=UTF-8

Response headers

pragma: no-cache
date: Fri, 20 May 2022 22:58:44 GMT
x-content-type-options: nosniff
last-modified: Sun, 17 May 1998 03:00:00 GMT
server: Golfe2
content-type: image/gif
access-control-allow-origin: https://www.wired.com
cache-control: no-cache, no-store, must-revalidate
access-control-allow-credentials: true
cross-origin-resource-policy: cross-origin
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
content-length: 35
expires: Fri, 01 Jan 1990 00:00:00 GMT

Failed requests

These URLs were requested, but there was no response received. You will also see them in the list above.

Domain: assoc-na.associates-amazon.com
URL: https://assoc-na.associates-amazon.com/onetag?src=330&pj=%7B%22tracking_id%22%3A%22w050b-20%22%2C%22assocPayloadId%22%3Anull%2C%22refUrl%22%3A%22https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%22%7D&u=https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1

Domain: pbs.getpublica.com
URL: https://pbs.getpublica.com/v1/s2s-hb?site_id=2564&format=json&app_name=CNEVIDEO&adserver=gam&slot_count=1&site_name=wired&content_episode=null&content_length=74&content_season=&content_id=583dce7afd2e614a9500000d&content_title=Jumping%20Spiders%20Jump%20Around%2C%20Jump%20Up%2C%20Jump%20Up%2C%20and%20Get%20Down&content_series=&content_language=en&content_category=Science%2CScience%20%26%20Tech--Science&content_keywords=absurd%20creature%2Cbiology&site_page=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&cb=8555783

Verdicts & Comments Add Verdict or Comment

143 JavaScript Global Variables

These are the non-standard "global" variables defined on the window object. These can be helpful in identifying possible client-side frameworks and code.

56 Cookies

Cookies are little pieces of information stored in the browser of a user. Whenever a user visits the site again, he will also send his cookie values, thus allowing the website to re-identify him even if he changed locations. This is how permanent logins work.

Domain/Path	Expires	Name / Value
.wired.com/	1970-01-20 11:57:24	Name: sailthru_hid Value: e4710b5f81a8ecc617eb69bf938664a76044883d8f3da371ee6eb3d62878e4173492f2b1e41b8f8dd5104156
.wired.com/	1970-01-20 03:21:32	Name: sailthru_bid Value: 27786158.1493579
.wired.com/	1970-01-20 07:30:39	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.wired.com/	1970-01-20 11:57:03	Name: CN_xid_refresh Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
www.wired.com/	1970-01-20 03:11:27	Name: xid1 Value: 1
www.wired.com/	1970-01-20 11:57:03	Name: verso_bucket Value: 617
.wired.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.wired.com/	1970-01-20 20:42:39	Name: CN_segments Value: co.w2220\|en.nl
.condenastdigital.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
www.wired.com/	1970-01-20 11:57:03	Name: usprivacy Value: 1---
www.wired.com/	1970-01-20 11:57:03	Name: OneTrustWPCCPAGoogleOptOut Value: true
www.wired.com/	1970-01-20 03:27:21	Name: CN_visits_m Value: 1654041600667%26vn%3D1
www.wired.com/	1970-01-20 03:11:29	Name: CN_in_visit_m Value: true
.wired.com/	1970-01-20 11:57:25	Name: pay_ent_smp Value: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsInZlciI6MX0.eyJjbnQiOjEsImlhdCI6MTY1MzA4NzUyMDY5MiwiaXNzIjoiam91cm5leSIsIm1heCI6NCwidWF0IjoxNjUzMDg3NTIwNjkyLCJ1cmxzIjpbIi9zdG9yeS90aGUtb25saW5lLXNwaWRlci1tYXJrZXQtaXMtbWFzc2l2ZS1hbmQtY3Jhd2xpbmctd2l0aC1pc3N1ZXMvIl19.hHvn2iy9HqbK3_Sgo_fuoAHE5VolgvX42Nxo_XhHKLA
.epicurious.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.epicurious.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.gq.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.gq.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.glamour.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.glamour.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.allure.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.allure.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.architecturaldigest.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.architecturaldigest.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.bonappetit.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.bonappetit.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.newyorker.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.newyorker.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.cntraveler.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.cntraveler.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.pitchfork.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.pitchfork.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.wired.com/	1970-01-20 11:57:03	Name: OptanonConsent Value: isGpcEnabled=0&datestamp=Fri+May+20+2022+22%3A58%3A40+GMT%2B0000+(GMT)&version=6.32.0&isIABGlobal=false&hosts=&consentId=1684ca3f-1d90-4300-a64a-c476eb4eb54f&interactionCount=0&landingPath=https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1&groups=C0003%3A0%2CC0004%3A0%2CC0001%3A1%2CC0002%3A1%2CSTACK42%3A0
.vogue.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.vogue.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.them.us/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.them.us/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.teenvogue.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.teenvogue.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.self.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.self.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.vanityfair.com/	1970-01-20 07:13:22	Name: CN_xid Value: 346f4b04-1d97-480c-b8c0-21b9cbba4cd5
.vanityfair.com/	1970-01-20 07:30:39	Name: CN_geo_country_code Value: GB
.wired.com/	1970-01-20 03:11:31	Name: AMP_TOKEN Value: %24NOT_FOUND
.wired.com/	1970-01-20 20:42:39	Name: _ga Value: GA1.2.1914554227.1653087521
.wired.com/	1970-01-20 03:12:53	Name: _gid Value: GA1.2.124107926.1653087521
.wired.com/	1970-01-20 03:11:27	Name: _dc_gtm_UA-8293713-27 Value: 1
.wired.com/	1970-01-20 03:11:29	Name: _parsely_session Value: {%22sid%22:1%2C%22surl%22:%22https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%253DCRMWIR012019%250A%250A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1%22%2C%22sref%22:%22%22%2C%22sts%22:1653087521275%2C%22slts%22:0}
.wired.com/	1970-01-20 12:40:51	Name: _parsely_visitor Value: {%22id%22:%22pid=1b3e2fa0b3149ecb5c2dab237069b24e%22%2C%22session_count%22:1%2C%22last_session_ts%22:1653087521275}
.wired.com/	1970-01-20 11:57:03	Name: _hjSessionUser_1537182 Value: eyJpZCI6ImQzYTM2ZmEzLTY0N2ItNTY3Yy05MDZmLTg2NTRjYWI2MTVmMiIsImNyZWF0ZWQiOjE2NTMwODc1MjEzNDAsImV4aXN0aW5nIjpmYWxzZX0=
.wired.com/	1970-01-20 03:11:29	Name: _hjFirstSeen Value: 1
www.wired.com/	1970-01-20 03:11:27	Name: _hjIncludedInSessionSample Value: 1
.wired.com/	1970-01-20 03:11:29	Name: _hjSession_1537182 Value: eyJpZCI6IjZkMTQ4ZDVkLTY2MGItNDkxMi1hNjg5LTRjNTFhM2JhNjkwMCIsImNyZWF0ZWQiOjE2NTMwODc1MjEzNTksImluU2FtcGxlIjp0cnVlfQ==
www.wired.com/	1970-01-20 03:11:27	Name: _hjIncludedInPageviewSample Value: 1
.wired.com/	1970-01-20 03:11:29	Name: _hjAbsoluteSessionInProgress Value: 0
www.wired.com/	1970-01-20 05:35:27	Name: cneplayercount Value: 1

2 Console Messages

A page may trigger messages to the console to be logged. These are often error messages about being unable to load a resource or execute a piece of JavaScript. Sometimes they also provide insight into the technology behind a website.

Source	Level	URL Text
javascript	error	URL: https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1 Message: Access to XMLHttpRequest at 'https://assoc-na.associates-amazon.com/onetag?src=330&pj=%7B%22tracking_id%22%3A%22w050b-20%22%2C%22assocPayloadId%22%3Anull%2C%22refUrl%22%3A%22https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%22%7D&u=https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1' from origin 'https://www.wired.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
network	error	URL: https://assoc-na.associates-amazon.com/onetag?src=330&pj=%7B%22tracking_id%22%3A%22w050b-20%22%2C%22assocPayloadId%22%3Anull%2C%22refUrl%22%3A%22https%3A%2F%2Fwww.wired.com%2Fstory%2Fthe-online-spider-market-is-massive-and-crawling-with-issues%2F%3Fbxid%3D6044883d8f3da371ee6eb3d6%26cndid%3D64035285%26esrc%3Dgrowl2-regGate-1120%26mbid%3Dmbid%253DCRMWIR012019%250A%250A%26source%3DEDT_WIR_NEWSLETTER_0_DAILY_ZZ%26utm_brand%3Dwired%26utm_campaign%3Daud-dev%26utm_content%3DWIR_Daily_052022%26utm_mailing%3DWIR_Daily_052022%26utm_medium%3Demail%26utm_source%3Dnl%26utm_term%3DP1%22%7D&u=https://www.wired.com/story/the-online-spider-market-is-massive-and-crawling-with-issues/?bxid=6044883d8f3da371ee6eb3d6&cndid=64035285&esrc=growl2-regGate-1120&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_052022&utm_mailing=WIR_Daily_052022&utm_medium=email&utm_source=nl&utm_term=P1 Message: Failed to load resource: net::ERR_FAILED

Security Headers

This page lists any security headers set by the main page. If you want to understand what these mean and how to use them, head on over to this page

Header

Value

Content-Security-Policy

Strict-Transport-Security

max-age=31536000; preload

Indicators

This is a term in the security industry to describe indicators such as IPs, Domains, Hashes, etc. This does not imply that any of these indicate malicious activity.

ampcid.google.com
api.condenast.io
apis.google.com
assoc-na.associates-amazon.com
c.amazon-adsystem.com
capture.condenastdigital.com
cdn.cookielaw.org
cdn.parsely.com
connect.facebook.net
cookies-data.onetrust.io
d1z2jf7jlzjs58.cloudfront.net
d2c8v52ll5s99u.cloudfront.net
dp8hsntg6do36.cloudfront.net
dwgyu36up6iuz.cloudfront.net
geolocation.onetrust.com
imasdk.googleapis.com
in.hotjar.com
infinityid.condenastdigital.com
journey.wired.com
js-sec.indexww.com
link.wired.com
mb.moatads.com
media.wired.com
news.google.com
p.skimresources.com
p1.parsely.com
pagead2.googlesyndication.com
pbs.getpublica.com
pitchfork.com
player.cnevids.com
polyfill.io
px.moatads.com
r.skimresources.com
s.skimresources.com
s0.2mdn.net
script.hotjar.com
securepubads.g.doubleclick.net
static.hotjar.com
stats.g.doubleclick.net
t.skimresources.com
vars.hotjar.com
www.allure.com
www.architecturaldigest.com
www.bonappetit.com
www.cntraveler.com
www.epicurious.com
www.glamour.com
www.google-analytics.com
www.google.co.uk
www.google.com
www.googletagmanager.com
www.gq.com
www.newyorker.com
www.self.com
www.teenvogue.com
www.them.us
www.vanityfair.com
www.vogue.com
www.wired.com
z-na.associates-amazon.com
z.moatads.com
assoc-na.associates-amazon.com
pbs.getpublica.com
108.138.2.156
108.156.255.177
108.157.4.21
108.157.4.38
143.204.209.161
143.204.215.26
151.101.0.239
151.101.128.239
151.101.192.239
151.101.2.194
151.101.64.239
151.139.128.11
172.217.16.130
18.233.44.90
18.65.159.128
2.20.86.213
23.35.236.247
2606:4700:10::6814:b844
2606:4700:4400::ac40:9b40
2606:4700::6810:9440
2a00:1450:4001:800::200a
2a00:1450:4001:801::200e
2a00:1450:4001:802::2008
2a00:1450:4001:80f::2006
2a00:1450:4001:812::200e
2a00:1450:4001:813::2004
2a00:1450:4001:813::200e
2a00:1450:4001:827::2002
2a00:1450:4001:828::200e
2a00:1450:4001:82b::2003
2a00:1450:400c:c08::9d
2a03:2880:f02d:100:face:b00c:0:3
2a04:4e42::282
34.232.146.139
35.190.59.101
35.190.91.160
35.201.67.47
52.51.233.122
54.155.18.159
54.77.221.60
65.9.58.150
65.9.58.194
65.9.61.60
72.21.195.65
03aa6fcac2902227e1b66a01b87824692f708bbf9bfe441784f8ed22d677f6de
066fc3ce045361bba8240f583393178cdedced02f8d4bb917c2d3f0520032564
06b99248a163333e36980a6cfb756f1a7de60fa49517162b87b1a44d5d48f844
155f28c98e64816b163401a271a70611552b8f142643d4151f6bd08310ad08e9
173d7c7e266672df75c4e048a934c55ee24d9a9028a87fd2957e74d1bd6a8d08
1909b2a83fd41494d94862c4323944d9d0aa1f1e653f252ea5a73fc5944308b0
1b4482365e2a58facf50f2495c776b236aef0626f0ae276b8459e1e3cec6c126
1d097e69ff47df9414a0ec07dfc70401084f4599617045a3a3edc7661ff76f3f
1da25cf82f065474aadce40a8c7369f6df48e1847cb95c21bd9083488eff7390
1f90db0a2498c14a33ddf6dc7e65b81adeded4d160e4d60740e03b9e11ba829d
1f94185bf320b088eb3c40b75de95ac8516680f4036bd287131b34f9c058146a
2236efed5f46e344722474597cff28f2dbb8c5cd134c79ab744d87a4f9b6581e
2576bfa8494eebe395580bcae1c994c50b1872f2f5a17b805652ff125d26b935
26ed5f6aa822d65b6ea7df8d13f1a217d7a933376a824b7ef52af0e6f64f1a44
27e55b238d002105e17ff50848818d1a4fa2656f650d0d5ca1cd40aa95b2c6ec
29cf1db1cac70e1421ddad6a60111ad50afea460acc234bff478aa8072a6aad5
2c667cdbe90922576bac69bbd0fa8f61d0c410748bf29b5bccea09b21123f1a0
2cdbd73d559e8a310d85e46c4c0c1341feee9b6d9ff8b8c105d937d68395dc3d
2eaad22c1c05e82825717d920a7e69250da0a9117a1ecfbab0e7e2e4d08dee79
2fa8c45d309615057c3ae6c240e1f9eb9c7f15e16d5ef3b6470467aa8d3b6c0a
394e311f0ce57b37ce616f3891c36dd3ff97aab169aa16ead4d3cad4ac6a409b
3c25b077a6d92cd9d3576660b68c4c0bd135b78b3cd3b66491ff2c7aa0eeaad3
3e2c74520955923ebd7093beb2cf2a2fd6b1dabba81de3948e4712620159e61f
3e8e76a70b5ec0a97f60491364274ab39aebd8f949b6a310a174633b015d4738
4041b947acd8eb0e87e3577c9121ddf40336ce3d2bb5033262ab74bb80032682
480bcf79f3ffb2d1f72bba3877edda3483c37e75e4db201770f9c962b881e744
4de3f44f9af02f0a9ac1366998ed8d04b85caee3bccd4552c04edd8dcd926bee
5475ef880793a875564088fea38154cee107eede5a2af036f3774a3dec5e48e6
58a5f8bddb693b55eb999460bbe2d671132ae787eec709202724aed802b51c76
61a0a971698a6b0901df941b1c31c46a41053e7833ce6ab080962ffaf49e5027
62f586be8571b23584eb4a60a45a3157ff7c8388b1b1e3b4e8890e243b3e47de
6414975cf1d0285b683085f648dc5dc2efbdee90406182ad244c30a7a7fa4446
67f8c7fd7353ad063da1f3115924c458c494cb134f4d87de4407a132842c9bc9
6a4dac260dffc284594d633859fb508b2fcfade38b61c8af9cd55eb23adf9e89
6adc3d4c1056996e4e8b765a62604c78b1f867cceb3b15d0b9bedb7c4857f992
6c26965b3758255462804203b738f7b446de4a66ec96fa8095a0dcc52b334918
6ded4fb8e38deab4abb8dedbe990d41a33177bf001810013926a196ed73487b6
71f7b22f7b615b6a6cb2240ba7516fb2e83d2028607d5983fd64d1b755fd11f5
738d35cb217fbd63b59fb51803ea8e491cd8114db5f292358695afe6a77c1802
7475b0e4ec6aaefe03e26dee8683fe8dbb06a19d4443265e59aae73cb9f1e261
7725538fe2f71147bffeba7452b434c826aef9009666cb4360c605d0b2a91d1d
7ff758ced20d4139eb5f580cfc93d1161ed5e19a4a2d4020728143855d17adeb
8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015
833a86642252016b29f08dd45ffd27f9e00ca237f28d8c5f0147a6e15d009377
8392b864ce606bf8ec20cf749f148dd7134d770200dd81df9b7adbc33a88978e
8575c83bcd649c28701830bc98d0051adba45ba685776092d2417c9240ee0c67
87b2408523892f375c00a9d521c67f6eb516ecac25c479a7b15705bfab08fcd3
88082e2436305c53b9849eab602898e4d5b728b68c2439cbfad581846fd32cdf
8cbeee80c026a82a6a5e3b67ee444bf9f42a36b51e512dd9afc0da3220d54bcf
8ce81789e3ba1b78f99aac5cc82d35c652244c072a5cc8df0fd99225ea18fbd9
8edbd08b9bb87f815ad871e44aae03af609fc44b1961d608e94eff3f4e010375
91afb6158e33ea43973ffd3f04b6caf266c5684a6b180a4acf1b001d2afaae78
92bcf815f419b05328f7002e2c5f304632455fa2b0f3d86a16cb47f877e7ae5a
92fca55833f48b4289ac8f1cedd48752b580fce4ec4b5d81670b8193d6e51b54
9e0843dbf1dc0d65a75182a82b945a9373557932e61934c27679c357f20c33a9
9ff1f6746123f8bb8d719e2f1b804a96f3513586f36d2b7c1eb878433be2d6a0
a1925038db769477ab74b4df34350c35688a795bb718727b0f4292a4a78a6210
a566a49a53a5b9c184eb5094a82c3b35e68a25b08bf76215dbf2e375cfe6d9f9
a7b7120dffd25546c93c1367b9c86a3dc87e71d2c89ebb39163a71eb3b659f01
aaecd144d2b8763b2fa5c91f09778294363cef363c10504205f4203922644d11
abb8f643c4c3c538b113f051044746b5e5bf19273ccf217807546d1c3eee64f8
acbe6770b0fc8b621a9d4f7068b241fb403fe999ea33270931ee59ec4cfdf3f1
ad3b5971ebec97a3b507722b499de667292078966cb9dd3b121d7a9646794179
af29cacfb5125f85da0f1557bb56456abcc1556dbd3094bb56e569890348c984
b29dfe7b6e22fffd693378fa14676d1d27a38b0f9c9a88b9e0793af1f34ddf48
b9daef61d4b711f1d28c3eef6bd8d522b8df518e833767512ad79502cc605dbd
bb05bc3976691db38d1c403ede59d1d2181fb64a6227b6ee1d849beba8f8c35d
bde06a0400c168573473e2de967d842eec383f2f755aef4ec017b2f333e7ff85
be51d69b63e7d47d25cfd9bedf139d053f4bd56157527005c3fdea76a4e9c9ed
befa0a7f28e5b50c7af0e063611c12a999d86005f5ab6501f6c623eba45cbac9
bfa1a0373c6e9fb398ea2d490254d5c33353f5ab88491a63cc94a16463432c7b
c2457ed91b16d9dbf7a21917fd4f8c47da415054f1c93354749ed075ad211fb4
c34a9420de5b381d4f2459193c70e4d84d5aaa0928a67907a57e2a9e9d3a8852
c41dc37fea212372d1f53109304ebae695e644f9ce083dcab08d5978c8c3020f
c462d460eab61de19f36cc384c99666e5bf65eaeba0c12b8f594c5410c01f220
c8c9128b649afff93f89f77eb2aa5a4bbbb1443bebc5156d0f697780c8beaa26
c92ee2460b4063f46ccd0ad0e0a68d212c6b756c4a0ef3a7fdf0afe0989781b1
ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356
cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda
d0bffc7261df1454c5e05475cda7d9e6647318dc6c3936767e1252bfe8849c54
d1aabe1212b2f9cb8f6a547454bd4e5f4773485e3e001b327e501ba3e0e77cc7
d281c9d6bae645f3da6d2f0769a2cf0668709fd28e2021ce74821cdd8c7117b3
dcecab1355b5c2b9ecef281322bf265ac5840b4688748586e9632b473a5fe56b
e0a6819d2a0186738790df0b93af04cb893c3040c74a5ffcde80b0eed215e3ba
e2511b147f3cf95f742758d3e2062eac98f5265a859dc07959eb8a32f0a2f528
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
e4312f2f537cbd6e153d8426ab167e1537c2aa53c8be4f62f05de9e666155d13
e5f4543004148e64beaa3dd965ef70153f0dc50dc4d0adc63c2bbac600754ba0
e690c27c97f5a802d795964cff40f2ee90b90d87ec3e8043489840404d1ce6eb
e853373f24fc82d9ddb21a9c0e592f8eb5771e168ab81418fba7e7bcee617d63
ed98b083c5cb0ed2d72279f5fa70a2c33badb8cd1b51475efad0c97d99ec29ed
ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629
f700de8771b792ff0d89ef4213de13fb344b45ac7e41cf0af724886d8ee1763b
f8bd28fee94c800df636a486d42ed91d2df89db1fd3e223d5e89ce3d9dd107fe
fb8378d4d1b47a50b7030eed7eae7a704df9bb3d0532d5de67a82e41e5c78525
fcc83a5b6aef86420c1ad553167106df96bd0ff4192ffe52b1647599948edbcf
fd829117a8525d07eb88964288d6e9401c14e5db450532eae36ab9185ec8349d

www.wired.com Open in urlscan Pro 151.101.2.194 Public Scan

Summary

urlscan.io Verdict: No classification

Domain & IP information

Screenshot Live screenshot Full Image

Page Title

Page URL History Show full URLs

Detected technologies

Page Statistics

129 Requests

96 %HTTPS

36 %IPv6

40 Domains

61 Subdomains

45 IPs

5 Countries

4797 kBTransfer

12082 kBSize

56 Cookies

33 Outgoing links

Page URL History

Redirected requests

129 HTTP transactions Everything HTML Script AJAX CSS Image Expand all 0 data transactions

Request headers

Response headers

Redirect headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

Request headers

Response headers

www.wired.com Open in urlscan Pro
151.101.2.194 Public Scan

Screenshot
Live screenshot

Full Image

129
Requests

96 %
HTTPS

36 %
IPv6

40
Domains

61
Subdomains

45
IPs

5
Countries

4797 kB
Transfer

12082 kB
Size

56
Cookies

129 HTTP transactions
0 data transactions