三大公有云托管 Kubernetes 服务 (EKSGKEAKS) 评估

Posted 2021-10-01 rtoax

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了三大公有云托管 Kubernetes 服务 (EKSGKEAKS) 评估相关的知识，希望对你有一定的参考价值。

EKS vs GKE vs AKS - Evaluating Kubernetes in the Cloud | StackRoxhttps://www.stackrox.com/post/2021/01/eks-vs-gke-vs-aks-jan2021/三大公有云托管 Kubernetes 服务 (EKS、GKE、AKS) 评估 • Feiskyhttps://feisky.xyz/posts/2021-01-20-eks-gke-aks/

作为发展最快的开源项目，Kubernetes 已经在越来越多的企业落地。而作为全球三大公有云提供商，AWS、Azure 和 GCP 都提供了托管 Kubernetes 集群服务，即 Elastic Kubernetes Service (EKS)、Azure Kubernetes Service (AKS) 和 Google Kubernetes Engine (GKE)。

这些托管 Kubernetes 集群服务在标准的 Kubernetes 开源项目之上，构建了一系列的增强特性，形成了云平台独特的生态。了解它们的最新发展以及独特的生态圈，可以帮助我们更好的了解 Kubernetes 生态现状，并可以作为基于 Kubernetes 构建新服务时的最佳实践参考。

以下数据基于 2021 年 1 月三大平台的最新官方文档。

基本信息

AKS 提供了更新的 Kubernetes 版本支持，通常比 GKE 和 EKS 新两到三个版本。
AKS 和 GKE 都提供了控制平面的自动更新机制（AKS 还在预览版），而 EKS 不支持自动更新。
AKS 和 GKE 都提供了 Node 健康检测和自动恢复机制，而 EKS 需要用户自己负责。
GKE 和 EKS 的控制平面都是收费的，而 AKS 还是继续免费。
在运行时上，三大平台都同时支持 Docker 和 containerd，所以上游社区 Docker 弃用不影响这些平台。除此之外，GKE 还支持独有的 gVisor，适用于安全要求更高的场景。

	Amazon EKS	Microsoft AKS	Google GKE	Kubernetes
Currently supported Kubernetes version(s)	1.18 (default) 1.17 1.16 1.15	1.20 (preview) 1.19 1.18 1.17	1.17 1.16 1.15 (default) 1.14	1.20 1.19 1.18
# of supported minor version releases	≥3 + 1 deprecated	3	4	3
Original GA release date	June 2018	June 2018	August 2015	July 2015 (Kubernetes 1.0)
CNCF Kubernetes Conformance	Yes	Yes	Yes	—
Latest CNCF-certified version	1.18	1.19	1.18	—
Control-plane upgrade process	User initiated User must also manually update the system services that run on nodes (e.g., kube-proxy, coredns, AWS VPC CNI)	User initiated All system components update with cluster upgraded	Automatically upgraded by default; can be user-initiated	—
Node upgrade process	Unmanaged node groups: user-initiated and managed Managed node groups: user-initiated; EKS will drain and replace nodes	Automatically upgraded; or user-initated; AKS will drain and replace nodes	Automatically upgraded (default; can be turned off) during cluster maintenance window; can be user-initiated; GKE drains and replaces nodes	—
Node OS	Linux: Amazon Linux 2 (default); Ubuntu (partner AMI) Bottlerocket Windows: Windows Server 2019	Linux: Ubuntu Windows: Windows Server 2019	Linux: Container-Optimized OS (COS) (default), Ubuntu Windows: Windows Server 2019 Windows Server version 1909	Linux: any OS supported by a compatible container runtime Windows: Windows Server 2019 (Kubernetes v1.14+)
Container runtime	Docker (default) containerd (through Bottlerocket)	Docker (default) containerd	Docker (default) containerd gVisor	Linux: Docker Containerd Cri-o rktlet any runtime that implements the Kubernetes CRI (Container Runtime Interface) Windows: Docker EE-basic 18.09
Control plane high availability options	Control plane is deployed across multiple Availability Zones (default)	Control plane components will be spread between the number of zones defined by the Admin	Zonal Clusters: Single Control Plane Regional Clusters: Three Kubernetes control planes quorum	Supported
Control plane SLA	99.95% (default)	99.95% (SLA backed) 99.9% (non-SLA backed)	Zonal clusters: 99.95% Regional clusters: 99.95%	—
SLA financially-backed	Yes	Yes	Yes	—
Pricing	$0.10/hour (USD) per cluster + standard costs of EC2 instances and other resources	Pay-as-you-go: Standard costs of node VMs and other resources	$0.10/hour (USD) per cluster + standard costs of GCE machines and other resources	—
GPU support	Yes (NVIDIA); user must install device plugin in cluster	Yes (NVIDIA); user must install device plugin in cluster	Yes (NVIDIA); user must install device plugin in cluster Compute Engine A2 VMs; are also available	Supported with device plugins
Control plane: log collection	Optional Default: Off Logs are sent to AWS CloudWatch	Optional Default: Off Logs are sent to Azure Monitor	Optional Default: Off Logs are sent to Stackdriver	—
Container performance metrics	Optional Default: Off Metrics are sent to AWS CloudWatch Container Insights	Optional Default: Off Metrics are sent to Azure Monitor	Optional Default: Off Metrics are sent to Stackdriver	—
Node health monitoring	No Kubernetes-aware support; if node instance fails, the AWS autoscaling group of the node pool will replace it	Auto repair is now available. Node status monitoring is available. Use autoscaling rules to shift workloads.	Node auto-repair enabled by default	—

综合这些基本信息，AKS 和 GKE 相对于 EKS 来说，托管服务提供了更多的自动诊断和自动恢复机制，具有明显的优势。而 AKS 的控制平面还是免费的，这对小型客户来说，具有不小的吸引力。

服务限制

在集群节点数量上，GKE 最多支持 15000 个节点，EKS 支持 3000 个节点，而 AKS 暂时只支持 1000 个节点。
在节点池数量上，AKS 支持 100 个节点池，EKS 支持 100，而 GKE 没有明确的文档。
在每个节点池支持的节点数量上，AKS 和 EKS 都支持 100，而 GKE 支持 1000。
在每个节点支持的 Pod 数量上，AKS 支持 250，GKE 支持 110，而 EKS 需要用户根据具体网络配置计算。

	EKS	AKS	GKE	Kubernetes (as of v1.19)
Max clusters	100/region*	1000	50/zone + 50 regional clusters	—
Max nodes per cluster	30 (Managed node groups) * 100 (Max nodes per group) = 3000*	1000 (Virtual Machine Scale Sets) 100 (VM Availability Sets)	15,000 nodes (v1.18 required) 5000 nodes (v1.17 or lower)	5000
Max nodes per node pool/group	Managed node groups: 100*	100	1000	—
Max node pools/groups per cluster	Managed node groups: 30*	100 nodes per node pool	Not documented	—
Max pods per node	Linux: Varies by node instance type: ((# of IPs per Elastic Network Interface - 1) # of ENIs) + 2* Windows: # of IPs per ENI - 1	250 (Azure CNI, max, configured at cluster creation time) 110 (kubenet network) 30 (Azure CNI, default)	110 (default)	100 (recommended value, configurable)

综合这些服务限制，GKE 单集群支持更多的节点数，超过 5000 节点时，GKE 是唯一的选择。

网络和安全

在网络和安全上，三大平台都默认开启了 RBAC，都支持 PodSecurityPolicy。AKS 还支持 Azure Policy，在涵盖 PodSecurityPolicy 基础上，还支持配置 Azure 平台相关的策略。
在 Kubernetes API 的访问限制上，三大平台都提供了白名单机制和私有 API 地址的功能。
在网络策略（Network Policy）的支持上，EKS 需要用户手动去安装 Calico，而 AKS 和 GKE 都支持内置开启。
虽然 EKS 提供了托管节点池的功能，但这个功能要求每个节点都绑定一个公网 IP，这对网络安全来说是个很大的挑战。

	EKS	AKS	GKE	Kubernetes
Network plugin/CNI	Amazon VPC Container Network Interface (CNI)	Azure CNI or kubenet	kubenet (default) Calico (added for Network Policies)	kubenet (default) External CNIs can added
Kubernetes RBAC	Required Immutable after cluster creation	Enabled by default Immutable after cluster creation	Enabled by default Mutable after cluster creation	Supported since 2017
Kubernetes Network Policy	Not enabled by default Calico can be manually installed at any time	Not enabled by default Must be enabled at cluster creation time	Not enabled by default Can be enabled at any time	Not enabled by default CNI implementing Network Policy API can be installed manually
PodSecurityPolicy support	PSP controller installed in all clusters with permissive default policy (v1.13+)	PSP can be installed at any time. Will be deprecated on May 31st 2021 for Azure Policy	PSP can be installed at any time. Currently in Beta	PSP admission controller needs to be enabled as kube-apiserver flag. Set to be deprecated in version 1.21
Private or public IP address for cluster Kubernetes API	Public by default Optional public, hybrid or private setup	Public by default Private-only available where private link is supported	Public by default Optional public, hybrid or private setup	—
Private or Public IP addresses for nodes	Unmanaged node groups: Optional Managed node groups: Optional	Public by default Private can be enabled as well	Public by default Private can be enabled as well	—
Pod-to-pod traffic encryption supported by provider	No by default	No by default	Yes, with Istio implemented	Requires a CNI implementation with functionality
Firewall for cluster Kubernetes API	CIDR allow list option	CIDR allow list option	CIDR allow list option	—
Read-only root filesystem on node	Pod security policy required	Azure policy required	COS: default Alternative: Pod security policy required	Supported

综合这些网络和安全特性，AKS 和 GKE 提供了更完善的安全控制机制，而 EKS 还有很多配置需要用户自己管理。

镜像服务

在镜像服务上，三大平台都提供了镜像仓库服务，支持匿名或私有的镜像托管。ECR 和 ACR 都同时支持 Docker 镜像格式、OCI 镜像格式以及 Helm Chart，而 GKE 已经从大家熟知的 GCR 迁移到了 Artifact Registry （AR），支持 Docker 镜像格式、OCI 镜像格式以及 Maven 和 npm 等。
在镜像安全上，三大平台都提供了镜像安全扫描服务。
在镜像可用性上，三大平台都提供了跨地域冗余的机制，自动把镜像数据复制到其他地域。

	EKS	AKS	GKE
Image repository service	ECR (Elastic Container Registry)	ACR (Azure Container Registry)	AR (Artifact Registry)
Supported formats	Docker Image Manifest V2, Schema 1 Docker Image Manifest V2, Schema 2 Open Container Initiative (OCI) Specifications Helm Charts	Docker Image Manifest V2, Schema 1 Docker Image Manifest V2, Schema 2 Open Container Initiative (OCI) Specifications Helm Charts	Docker Image Manifest V2, Schema 1 Docker Image Manifest V2, Schema 2 Open Container Initiative (OCI) Specifications Maven and Gradle npm
Access security	Permissions managed by AWS IAM Permissions can be applied at repository level Network endpoint is public by default Network endpoint can be limited to specific VPCs	Permissions managed by Azure RBAC Can be applied at repository level (preview) Network endpoint is public by default Network endpoint can be limited to specific VNets (preview)	Permissions managed by GCP IAM Permissions can only be applied at registry level Network endpoint is public by default Network access for GCR registries can be limited to specific VPCs with service perimeters
Supports image signing	No	Yes	Yes, with Binary Authorization and Voucher
Supports immutable image tags	Yes	Yes, and it supports the locking of images and repositories	No
Image scanning service	Yes, free service: OS packages only	Yes, paid service: Uses the Qualys scanner in a sandbox to check for vulnerabilities	Yes, paid Service: OS packages only
Registry SLA	99.9%; financially-backed	99.9%; financially-backed	None
Geo-Redundancy	Yes, configurable	Yes, configurable as part of the premium service	Yes: by default

综合这些镜像服务的特性，可以发现三个平台提供的镜像服务基本类似，AKS 和 EKS 支持托管 Helm Charts，是相对于 GKE 的优势。

参考文档

StackRox 报告：https://www.stackrox.com/post/2021/01/eks-vs-gke-vs-aks-jan2021/
AKS 文档：https://aka.ms/aks/docs
GKE 文档：https://cloud.google.com/kubernetes-engine
EKS 文档：https://aws.amazon.com/eks/

三大公有云托管 Kubernetes 服务 (EKS、GKE、AKS) 评估 • Feiskyhttps://feisky.xyz/posts/2021-01-20-eks-gke-aks/EKS vs GKE vs AKS - Evaluating Kubernetes in the Cloud | StackRoxProviding an update on the core Kubernetes services offered by the big threehttps://www.stackrox.com/post/2021/01/eks-vs-gke-vs-aks-jan2021/Azure Kubernetes Service (AKS) documentation | Microsoft DocsAKS allows you to quickly deploy a production ready Kubernetes cluster in Azure. Learn how to use AKS with these quickstarts, tutorials, and samples.https://docs.microsoft.com/en-us/azure/aks/Amazon EKS | Managed Kubernetes Service | Amazon Web ServicesAmazon EKS makes it easy to run Kubernetes on AWS with fully-managed clusters that are highly available, secure, and integrated with AWS services.https://aws.amazon.com/eks/?whats-new-cards.sort-by=item.additionalFields.postDateTime&whats-new-cards.sort-order=desc&eks-blogs.sort-by=item.additionalFields.createdDate&eks-blogs.sort-order=desc

以上是关于三大公有云托管 Kubernetes 服务 (EKSGKEAKS) 评估的主要内容，如果未能解决你的问题，请参考以下文章

Kubernetes监控手册06-监控APIServer

云服务三大部署模式-私有云公有云混合云谁才是云计算未来的主流

超越Kubernetes：值得关注的5大云原生技术

云原生在京东丨如何在 Kubernetes 上部署有状态的云原生应用？（上）

Multi-Cloud Kubernetes 最佳实践

多种云服务模式下Kubernetes的最佳实践