togetter.com
Open in
urlscan Pro
54.64.20.247
Public Scan
Submitted URL: http://togetter.com/robots.txt
Effective URL: https://togetter.com/robots.txt
Submission: On October 21 via manual from JP — Scanned from JP
Effective URL: https://togetter.com/robots.txt
Submission: On October 21 via manual from JP — Scanned from JP
Form analysis 0 forms found in the DOM
Text Content
User-agent: Googlebot User-agent: bingbot User-agent: Slurp Disallow: /search? Disallow: /create? User-agent: Megalodon Disallow: / User-agent: ia_archiver Disallow: / User-agent: Twitterbot Allow: / User-agent: bingbot Crawl-delay: 1 User-agent: * Allow: /