如何查找网站是不是使用 HSTS

Posted 2023-02-25

【中文标题】如何查找网站是不是使用 HSTS

【英文标题】：How to find if a web site uses HSTS如何查找网站是否使用 HSTS

【发布时间】：2012-07-27 21:38:52

【问题描述】：

我是 curl 的新手，正在尝试确定网站是否使用 Strict-Transport-Security。

我正在逃避建议。我被告知要检查 Chrome's preloaded list 并运行

curl -D - https://www.example.com | head -n 20

检查 Strict-Transport-Security 标头。

但是'head'命令产生了一个错误并且是未知的。

有什么想法吗？

ATM 我运行的是 Win XP，几天后会有一个 linux 发行版。

谢谢。

【参考方案1】：

[扩展@FauxFaux 的答案]

我想看看我的网站与业内其他网站相比如何。所以，我写了一个 bash for 循环。我发现有些网站不仅对待HEAD 请求与GET 不同，而且它们（亚马逊和微软）对待curl 与真正的浏览器不同。因此，我在请求中添加了一些标头以获得真正的响应。

脚本

# NOTE: You can copy/paste this whole block straight into a bash shell

apex_domains=(
    paypal.com
    amazon.com
    google.com
    microsoft.com
)
curl_command=`
`"curl -svo /dev/null "`
    `"-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) "`
        `"AppleWebKit/537.36 (Khtml, like Gecko) Chrome/86.0.4240.183 Safari/537.36' "`
    `"-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif"`
        `",image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' "`
    `"-H 'accept-language: en-US,en;q=0.9' "`
    `"--compressed"

for domain in "$apex_domains[@]"; do
    for scheme in 'http' 'https'; do
        for subdomain in '' 'www.'; do
            echo -e         "\n""  $scheme://$subdomain$domain"
            echo "  $curl_command  $scheme://$subdomain$domain"
            eval   "$curl_command  $scheme://$subdomain$domain" 2>&1 | \
                tr -d '\r' | grep -i --color=always 'strict-transport-security.*';
        done
    done
done

输出

输出在终端上看起来更好，因为 grepped 标头被突出显示。

  http://paypal.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://paypal.com

  http://www.paypal.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://www.paypal.com

  https://paypal.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://paypal.com
< strict-transport-security: max-age=31536000; includeSubDomains

  https://www.paypal.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://www.paypal.com
< strict-transport-security: max-age=63072000; includeSubDomains; preload

  http://amazon.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://amazon.com

  http://www.amazon.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://www.amazon.com

  https://amazon.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://amazon.com

  https://www.amazon.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://www.amazon.com
< strict-transport-security: max-age=47474747; includeSubDomains; preload

  http://google.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://google.com

  http://www.google.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://www.google.com

  https://google.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://google.com

  https://www.google.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://www.google.com
< strict-transport-security: max-age=31536000

  http://microsoft.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://microsoft.com

  http://www.microsoft.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  http://www.microsoft.com

  https://microsoft.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://microsoft.com

  https://www.microsoft.com
  curl -svo /dev/null -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: en-US,en;q=0.9' --compressed  https://www.microsoft.com
< strict-transport-security: max-age=31536000

【参考方案2】：

Chrome 有一个 HSTS 检查功能chrome://net-internals#hsts

但请注意，Chrome 也喜欢在您通过 https 请求网站时添加条目。

刚刚让 chrome 将我重定向到 https 以获得没有 https 证书的内部站点。甚至没有在 443 上听。不出所料，curl 没有返回 Strict 标头。然后我发现 chrome 有一个内部 HSTS 列表。可以从 chrome://net-internals#hsts 中清除，不包括 Google 维护的全球列表。

【参考方案3】：

这个方法很好。

$ curl -s -D- https://paypal.com/ | grep Strict
Strict-Transport-Security: max-age=14400

如您所见，一些网络服务器只是拒绝接受HEAD 请求。 curl 将打印带有-v 的GET 请求的标头：

$ curl -s -vv https://paypal.com/ 2>&1 | grep Strict
< Strict-Transport-Security: max-age=14400

< 表示标头是服务器返回给您的标头。

实际的example.com（如您的示例）将不起作用，因为它根本不听https://：

$ curl -D- https://www.example.com
curl: (7) couldn't connect to host

由于Strict-Transport-Security 标头仅在通过https:// 传递时才有效，因此可以非常安全地假设任何未在https:// 上响应的站点都没有使用 STS，尤其是因为它会使用 STS没有理由这样做。

【讨论】：

实际上，当您联系使用 wget 并通过 http 联系 HSTS 网站时，您会得到：URL transformed to HTTPS due to an HSTS policy，您最终会使用 https

curl -s -D- https://paypal.com/ | grep -i Strict 更适合不区分大小写和更可靠的结果