在 terraform 模块中显式使用提供程序

Posted 2023-03-24

技术标签:

【中文标题】在 terraform 模块中显式使用提供程序【英文标题】：Explicitly use a provider within terraform module 【发布时间】：2021-12-26 11:54:50 【问题描述】：

我试图在我的模块中显式调用提供程序以在 AzureCloud 和 AzureChinaCloud 中创建命名空间。但是，我在这样做时遇到了问题。以下是我的配置：

terraform 
  required_providers 
    azurerm = 
      source  = "hashicorp/azurerm"
      version = "=2.78.0"
    
  
  backend "azurerm" 
    resource_group_name = "Terraform-rg"
    storage_account_name = "terraformstate"
    container_name = "tfstate"
    subscription_id = "00000000-0000-0000-0000-000000000000"
    key = "prod"
  


provider "azurerm" 
  features 



provider "azurerm" 
  features 
  alias           = "sub2"
  subscription_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  client_id       = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  client_secret   = var.client_secret
  tenant_id       = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  environment     = "china"


module "helm_ns_creation" 
  source = "./namespace/"
  providers = 
    azurerm = azurerm
    azurerm.sub2 = azurerm.sub2
   
  applications = var.applications
  geo = var.geo
  ns_values = ["$file("../namespace/values.yaml")"]


-------------------

provider "kubernetes" 
  config_path    = "config"


provider "helm" 
  kubernetes 
    config_path = "config"
  


resource "kubernetes_namespace" "aks_namespace" 
  provider = azurerm.sub2
  for_each = for ns in var.applications : ns.namespace_name => ns
  metadata 
    annotations = 
      name = "$each.value.namespace_name"
    
    labels = 
      name = "$each.value.team_name"
    
    name = "$each.value.namespace_name"
  
  

locals 
# get json
namespace_data = jsondecode(file(var.inputfile))
principal_ids = distinct([for principal in local.namespace_data.applications : principal.principal_id])
principal_ids_cn = distinct([for principal_cn in local.namespace_data.applications : principal_cn.principal_id_cn])
get_principal_ids = (var.geo == "cn" ? local.principal_ids_cn : local.principal_ids)


data "azurerm_subscription" "global" 


resource "azurerm_role_assignment" "custom" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.global.id
# scope = "/subscriptions/$var.subscription_id"
  role_definition_name = var.custom_role
  principal_id = each.key


resource "azurerm_role_assignment" "builtin" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.global.id
  role_definition_name = var.builtin_role
  principal_id = each.key


data "azurerm_subscription" "china" 
  provider = azurerm.sub2


resource "azurerm_role_assignment" "custom_cn" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.china.id
# scope = "/subscriptions/$var.subscription_id"
  role_definition_name = var.custom_role
  principal_id = each.key


resource "azurerm_role_assignment" "builtin_cn" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.china.id
  role_definition_name = var.builtin_role
  principal_id = each.key

当我运行代码在两个不同的云（中国和全球）中创建命名空间时，我仅在中国地区收到以下错误。但是，对于全球也是如此：

│ 错误：无法列出提供者注册状态，可能是由于凭据无效或服务主体没有使用资源管理器API的权限，Azure错误：resources.ProvidersClient#List：响应失败请求：StatusCode=404 -- 原始错误：autorest/azure：服务返回错误。 Status=404 Code="SubscriptionNotFound" Message="找不到订阅'xxxxxxx-xxxxxx-xxxx-xxxx-xxxxxxxxxx'。"

with provider["registry.terraform.io/hashicorp/azurerm"],
│   on main.tf line 18, in provider "azurerm":
│   18: provider "azurerm"

现在中国供应商的订阅失败了。我如何使它适用于两个云（中国和全球）。如果需要任何其他详细信息，请告诉我..

【问题讨论】：

您好@pk_dhruv，我可以知道您在命名空间定义中提到的提供者azurerm.mooncake 块在哪里吗？因为我看到中国云别名为“sub2”.. @AnsumanBal-MT .. 对错误感到抱歉.. 我现在已经编辑了它.. 我也试过在资源部分没有提供程序块，没有区别.. 我试过没有成功..我仍然得到同样的错误.. kubernetes_namespace 不是 azurerm 提供者的一部分，它是kubernetes 提供者的一部分。所以，为了使用 kubernetes namespace ，你应该使用 kubernetes provider 而不是 azure rm 。你可以参考这个link @AnsumanBal-MT .. 我现在在我的查询中更新了代码.. 我需要按照上面的代码使用 azurerm.. 但是，这似乎不起作用并且失败了错误信息.. 【参考方案1】：

为了说明，我将整个代码分为 3 个部分，如下所述：

使用以下提供程序块，您必须已创建 AKS 群集的在公共和中国云中。

terraform 
  required_providers 
    azurerm = 
      source  = "hashicorp/azurerm"
      version = "=2.78.0"
    
  
  backend "azurerm" 
    resource_group_name = "Terraform-rg"
    storage_account_name = "terraformstate"
    container_name = "tfstate"
    subscription_id = "00000000-0000-0000-0000-000000000000"
    key = "prod"
  

provider "azurerm" 
  features 

provider "azurerm" 
  features 
  alias           = "sub2"
  subscription_id = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  client_id       = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  client_secret   = var.client_secret
  tenant_id       = "xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxx"
  environment     = "china"

resource "azurerm_kubernetes_cluster" "aks_cluster_public" 
  provider = azurerm
  name                = "ansuman-aks-001"
  location            = data.azurerm_resource_group.sub1.location
  resource_group_name = data.azurerm_resource_group.sub1.name
  dns_prefix          = "ansuman-aks-cluster"


.....


resource "azurerm_kubernetes_cluster" "aks_cluster_china" 
  provider = azurerm.sub2
  name                = "ansuman-aks-001"
  location            = data.azurerm_resource_group.sub1.location
  resource_group_name = data.azurerm_resource_group.sub1.name
  dns_prefix          = "ansuman-aks-cluster"


.....

创建 AKS 集群后，您可以在公共和中国使用 Kubernetes Providers 和创建 Kubernetes Namespace 云将如下所示：

provider "kubernetes" 
  host                   = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.host"
  username               = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.username"
  password               = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.password"
  client_certificate     = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.client_certificate")
  client_key             = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.client_key")
  cluster_ca_certificate = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.cluster_ca_certificate")

provider "kubernetes" 
  alias = "sub2"
  host                   = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.host"
  username               = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.username"
  password               = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.password"
  client_certificate     = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.client_certificate")
  client_key             = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.client_key")
  cluster_ca_certificate = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.cluster_ca_certificate")


resource "kubernetes_namespace" "app_namespace_public" 
  provider = kubernetes
  metadata 
    name = "my-namespace"
  
  depends_on = [
    azurerm_kubernetes_cluster.aks_cluster_public
  ]

resource "kubernetes_namespace" "app_namespace_china" 
  provider = kubernetes.sub2
  metadata 
    name = "my-namespace"
  
  depends_on = [
    azurerm_kubernetes_cluster.aks_cluster_china
  ]

正如您在 Kubernetes Provider 中看到的，我使用过 aks_cluster_configs 用于公共和中国，因为我也在创建 AKS 集群，如果您不创建 AKS 集群，那么您也可以使用 config paths，但概念将是相同的，即一个提供者用于公共，另一个用于中国和资源块也是如此。

以上完成后，就可以使用azurerm provider作为角色了任务如下：

data "azurerm_subscription" "global" 
  provider = azurerm.sub2


resource "azurerm_role_assignment" "custom" 
  provider = azurerm.sub2
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.global.id
  role_definition_name = var.custom_role
  principal_id = each.key


resource "azurerm_role_assignment" "builtin" 
  provider = azurerm
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.global.id
  role_definition_name = var.builtin_role
  principal_id = each.key


data "azurerm_subscription" "china" 
  provider = azurerm.sub2


resource "azurerm_role_assignment" "custom_cn" 
  provider = azurerm.sub2
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.china.id
  role_definition_name = var.custom_role
  principal_id = each.key


resource "azurerm_role_assignment" "builtin_cn" 
  provider = azurerm.sub2
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.china.id
  role_definition_name = var.builtin_role
  principal_id = each.key

注意：如果您也在使用Helm Provider，那么您必须在此处遵循与 Kubernetes Provider 相同的概念，您可以参考此Terraform Helm Provider Documentation。请确保按照我们配置 azurerm 和 kubernetes 提供程序的方式进行配置，并在模块或资源块中使用相同的方式。

我在具有 AKS 集群、命名空间和内置角色且没有自定义角色的环境中使用以下代码测试了上述内容，输出如下：

我的 Main.tf 文件：

provider "azurerm" 
  features 

provider "azurerm" 
  alias = "sub2"
  subscription_id = "948d4068-xxxx-xxxx-xxxx-e00a844e059b"
  tenant_id = "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
  client_id = "f6a2f33d-xxxx-xxxx-xxxx-d713a1bb37c0"
  client_secret = "inl7Q~Gvddxxxx-xxxx-xxxxaGPF3uSoL"
  features 


data "azurerm_resource_group" "sub2" 
  provider = azurerm.sub2
  name = "ansumantest"

data "azurerm_resource_group" "sub1" 
  provider = azurerm
  name = "xxx-ansbal-xxxx"

resource "azurerm_kubernetes_cluster" "aks_cluster_public" 
  provider = azurerm
  name                = "ansuman-aks-001"
  location            = data.azurerm_resource_group.sub1.location
  resource_group_name = data.azurerm_resource_group.sub1.name
  dns_prefix          = "ansuman-aks-cluster"

  default_node_pool 
    name                  = "default"
    vm_size               = "Standard_D2_v2"
    availability_zones    = [1, 2]
    enable_auto_scaling   = true
    max_count             = 4
    min_count             = 1
    node_count            = 2
    type                  = "VirtualMachineScaleSets"
  

  network_profile 
    network_plugin = "kubenet"
  

  service_principal 
    client_id     = "f6a2f33d-xxxx-xxxx-xxxx-d713a1bb37c0"
    client_secret = "inl7Q~Gvxxxx-xxxx-xxxxiyaGPF3uSoL"
  
  role_based_access_control 
    enabled = true
  



resource "azurerm_kubernetes_cluster" "aks_cluster_china" 
  provider = azurerm.sub2
  name                = "ansuman-aks-001"
  location            = data.azurerm_resource_group.sub2.location
  resource_group_name = data.azurerm_resource_group.sub2.name
  dns_prefix          = "ansuman-aks-cluster"

  default_node_pool 
    name                  = "default"
    vm_size               = "Standard_D2_v2"
    availability_zones    = [1, 2]
    enable_auto_scaling   = true
    max_count             = 4
    min_count             = 1
    node_count            = 2
    type                  = "VirtualMachineScaleSets"
  

  network_profile 
    network_plugin = "kubenet"
  

  service_principal 
    client_id     = "f6a2f33d-xxxx-xxxx-xxxx-d713a1bb37c0"
    client_secret = "inl7Q~Gvddxxxx-xxxx-xxxx6ntiyaGPF3uSoL"
  
  role_based_access_control 
    enabled = true
  



provider "kubernetes" 
  host                   = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.host"
  username               = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.username"
  password               = "$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.password"
  client_certificate     = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.client_certificate")
  client_key             = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.client_key")
  cluster_ca_certificate = base64decode("$azurerm_kubernetes_cluster.aks_cluster_public.kube_config.0.cluster_ca_certificate")

provider "kubernetes" 
  alias = "sub2"
  host                   = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.host"
  username               = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.username"
  password               = "$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.password"
  client_certificate     = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.client_certificate")
  client_key             = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.client_key")
  cluster_ca_certificate = base64decode("$azurerm_kubernetes_cluster.aks_cluster_china.kube_config.0.cluster_ca_certificate")


resource "kubernetes_namespace" "app_namespace_public" 
  provider = kubernetes
  metadata 
    name = "my-namespace"
  
  depends_on = [
    azurerm_kubernetes_cluster.aks_cluster_public
  ]

resource "kubernetes_namespace" "app_namespace_china" 
  provider = kubernetes.sub2
  metadata 
    name = "my-namespace"
  
  depends_on = [
    azurerm_kubernetes_cluster.aks_cluster_china
  ]


data "azurerm_subscription" "global" 
  provider = azurerm

data "azurerm_client_config" "global" 
  provider = azurerm

resource "azurerm_role_assignment" "builtin" 
  provider = azurerm
  scope = data.azurerm_resource_group.sub1.id
  role_definition_name = "Azure Kubernetes Service Cluster Admin Role"
  principal_id = data.azurerm_client_config.global.object_id


data "azurerm_subscription" "china" 
  provider = azurerm.sub2

data "azurerm_client_config" "China" 
  provider = azurerm.sub2

resource "azurerm_role_assignment" "builtin_cn" 
  provider = azurerm.sub2
  scope = data.azurerm_subscription.china.id
  role_definition_name = "Azure Kubernetes Service Cluster Admin Role"
  principal_id = data.azurerm_client_config.China.object_id

输出：

注意：我只在公共云中使用了 2 次订阅，因为我没有中国云订阅，但不同云也一样，请务必添加 environment azurerm 提供程序块中的参数。

【讨论】：

感谢您尝试这个复杂的场景。感谢您的努力。但是，当订阅来自同一个公共云时，我能够让事情正常运行。只有当它在不同的云，比如一个在公共的云和一个在中国的..我在提供程序部分添加了环境，希望它可以工作..但是，它不起作用，而且中国和公共的订阅都没有被阅读..跨度> Hey AnsumanBal.. 我们知道了.. 我会尽快分享答案.. 感谢您的努力.. 确定@pk_dhruv。【参考方案2】：

我通过使用提供程序块中的条件解决了这个问题..

provider "azurerm" 
  subscription_id = var.subscription_id 
  client_id       = var.geo == "cn" ? var.client_id_cn : var.client_id
  client_secret   = var.geo == "cn" ? var.client_secret_cn : var.client_secret
  tenant_id       = var.geo == "cn" ? var.tenant_id_cn : var.tenant_id
  environment     = var.geo == "cn" ? "china" : "public"
  features 


module "helm_ns_creation" 
  source = "./namespace/"
  applications = var.applications
  geo = var.geo
  values = ["$file("../namespace/values.yaml")"]


-------

## data block for reading subscription

data "azurerm_subscription" "current" 


resource "azurerm_role_assignment" "custom" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.current.id
  role_definition_name = var.custom_role
  principal_id = each.key


resource "azurerm_role_assignment" "builtin" 
  for_each = toset(local.get_principal_ids)
  scope = data.azurerm_subscription.current.id
  role_definition_name = var.builtin_role
  principal_id = each.key

这只是多提供者使用的所有别名。因此，当变量 geo 为 china 时，它将使用 provider 中的变量 for china，如果不等于 china，它将使用 provider 中的其他变量。这样，提供者根据条件切换，就像一个魅力..

【讨论】：

以上是关于在 terraform 模块中显式使用提供程序的主要内容，如果未能解决你的问题，请参考以下文章

如何修复您应该只在 IOS 的应用程序中显式呈现一个导航器

sequelize抛出错误方言需要在express js中显式定义

如何在 Django 中显式重置模板片段缓存？

在信号处理程序中显式调用析构函数

如何在使用 IMAPI 创建的 ISO 文件中显式创建目录结构？

如何在 Spring Boot 配置中显式传递数据库名称？