Terraform 学习总结—— 基于 AWS 云平台上的 Terraform 实战
Posted 科技D人生
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Terraform 学习总结—— 基于 AWS 云平台上的 Terraform 实战相关的知识，希望对你有一定的参考价值。
一、AWS 上基础环境介绍
相信大家已经知道 Terraform 的基本使用方式了。在我们以后的场景中，主要讨论一些典型架构的实现。在实际使用中，Terraform 在多云基础架构创建中，是非常方便和简洁的。Terraform 造了足够多的轮子，使用起来非常顺手，用于创建云上的基础架构。在应用发布上，可以使用 Packer 进行应用的集成，与 Terraform 配合做到应用的自动发布。同时在云上的基础构建完成后，想构建一些复杂系统，基于 OS 之上时，就需要交给 Ansible 和 PowerShell 来进行实现了。例如创建 K8S/Openshift 的集群环境，在创建这些复杂架构的方法上，存在多种工具，但是我们需要企业级发布的时候， 在系统层面的配和应用层面的配置就需要更高的控制精细程度。而这个时候，就凸显了 Terraform 和 Ansible 等工具的控制能力。我们可以精确的控制到每个配置项。无论是 Infra 还是 OS。
这里是一个典型的 AWS 多账号设计，左边的是 Operation 运维组用户的人员的核心账号下的运维主机 Terraform，这台主机上同时集成安装了 ansible 和 packer 工具，用于实现后面的实验，但是实际生产环境中可能会依据性能分拆，当前实验环境中，都是单机实现即可。由图可知，我们通过对于右边受管理的账号 Project or Team 的项目账号进行管理。例如：初始化账号，包括创建组网 VPC/Subnet/Role/Tags、创建 IAM、安全组等一系列的动作。完成这个账号的初始化动作。
环境安装：
Terraform下载包位置(建议实际使用中，选择最后版本进行安装)：
Downloads | Terraform by HashiCorp
Downloads | Packer by HashiCorp
2 个账号，左边的账号是运维部门管理的账号，里面创建一台可以访问公网的 VM，建议使用 Amazon Linux 2。 右边账号是一个新账号，用于提供给新项目组或者新部门。具体方法：
选用 amaz2 的 AMI 进行系统准备，使用以下命令，进行环境依赖的安装。
$ sudo amazon-linux-extras install epel -y
$ sudo yum install -y ansible
$ sudo curl -o /tmp/terraform.zip https://releases.hashicorp.com/terraform/0.12.3/terraform_0.12.13_linux_amd64.zip
$ sudo curl -o /tmp/packer.zip https://releases.hashicorp.com/packer/1.4.5/packer_1.4.5_linux_amd64.zip
$ sudo unzip /tmp/terraform.zip -d /usr/local/bin
$ sudo unzip /tmp/packer.zip -d /usr/local/bin权限配置方法：
添加 IAM 权限，我们先去在右侧是受管账号（Project or Team Account）下面去建立Assume Role，具体如图：IAM->Roles->Create Role
这里的 Role 关联了 Policy AdministratorAccess, 仅用于测试。但是实际生产上，要依据实际实际用户来进行权限划分。
这一步创建 Tags, 啥也没有填。你随意。
这里给 Role name 添加了一个名字 ”terraform-assume-role“
请查看这个 role 的 ARN，如下图，请记录好，后面我们在 Operation Account 里面会用到。
下面我们在左侧的运维账号（Operation Account）进行Policy/Role的创建。
-
首先创建Policy, IAM->Policies->Create Policy
"Version": "2012-10-17",
"Statement":
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": [
"arn:aws-cn:iam::123456780001:role/terraform-assume-role"
]
添加 Policy Name: terraform-assume-policy, 后面我们会关联到 terraform ec2 role 里面。
现在建立 Terraform EC2 Role, 请如下操作 IAM-> Roles->Create role， 选择 AWS Service -> EC2
搜索之前创建的 Policy, terraform, 勾选之前创建的 Policy
填写 role 的名字： terraform-ec2-role
选中之前创建的 Terraform EC2, 依次执行如图
更新为我们创建的Role: terraform-ec2-role
如上，我就完成基础环境的准备。
二、与 Packer 配合定制 AMI ，实现 ELB + Auto Scaling Group
围绕 Packer 的集成来实现，自动应用的发布方法。如下图：
如图示架构，我们需要 2 个步骤：
1、使用 Packer 制作 AMI
目录内的文件说明
| 文件 | 描述 |
|---|---|
| amaz2-stress-hk.json和amaz2-stress-nx.json | 这两个文件分别是香港和中国宁夏两个region内的AMI定义的描述文件，用于Packer创建文件 |
| base_install.sh | 这个文件是会在json文件里面定义和被调用的脚本。 |
| burnCPU.sh和index.php | 是压缩包burn.tgz里面的文件内容。 |
| burn.tgz | 是压缩包，里面包含了burnCPU.sh 和index.php |
由于默认 AWS China 不能访问 80、443，需要额外开通，所以我们以 HK 做实验。首先查看 Packer 制作镜像的描述文件：
"builders": [
"type": "amazon-ebs",
"access_key": "Your-AWS-Account-AK",
"secret_key": "Your-AWS-Account-SK",
"region": "ap-east-1",
"source_ami": "ami-570c7726",
"instance_type": "c5.large",
"ssh_username": "ec2-user",
"ami_name": "amaz2-stress timestamp by packer"
],
"provisioners": [
"type": "file",
"source": "burn.tgz",
"destination": "/tmp/burn.tgz"
,
"type": "shell",
"script": "base_install.sh"
]
主要说明如下：builders和provisioners是代表了packer打包ami的最终要的三部分。还有一个Post-Processors，当前环境没有用到。
builders:
| 代码 | 解释 |
|---|---|
| builders | 这部分是看后端接入的类型，比如：AWS/Azure/Google Cloud |
| Type | 这个amazon-ebs是在后端为AWS的其中一种打包方法，也是我们最常见的，启动一个实例，进行修改，然后关机，创建AMI。除此之外，还有另外4种模式：chroot/EBS Surrogate/EBS Volume/Instance 总计5种的修改方法。可以去读文档查看具体的参数: : Amazon EBS - Builders | Packer by HashiCorp |
| AK/SK | AWS AK/SK |
provisioners:
| 代码 | 解释 |
|---|---|
| Type | 主要类型有集成ansible/shell/file/Powershell/Chef.. 等总计10几种的模式。 |
| source | 本地文件存放位置 |
| destination | 文件上传目标路径 |
当前这个语法 file 代表将本地一个压缩文件传入到镜像的 /tmp/ 下面，Shell 这个部分代表，后续执行本地这个 base_install.sh 的脚本。
<!–特别说明：–>
<!–由于创建AMI属于一个临时行为，所以packer这个命令在简化环境下，需要default vpc, 否则会报错。而通常生产环境里面，defalut vpc已经删除。而用生产环境里面的vpc, 需要加入更多参数，如vpc id、subnet id。–>
我们需要进入当前 ami-packer 目录，执行命令打包：
[ec2-user@ip-172-31-22-159 ami-maker]$ packer build amaz2-stree-hk.json
amazon-ebs output will be in this color.
==> amazon-ebs: Prevalidating AMI Name: amaz2-stress 1566530942 by packer
amazon-ebs: Found Image ID: ami-570c7726
==> amazon-ebs: Creating temporary keypair: packer_5d5f5d7e-21ca-5f6c-62c1-14517a092628
==> amazon-ebs: Creating temporary security group for this instance: packer_5d5f5d83-63fc-bffe-8049-b616426fcc26
==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-0eb31ca09c65dfbbb
==> amazon-ebs: Waiting for instance (i-0eb31ca09c65dfbbb) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 18.163.6.72
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Uploading burn.tgz => /tmp/burn.tgz
burn.tgz 890 B / 890 B [===========================================================================================================================================] 100.00% 0s
==> amazon-ebs: Provisioning with shell script: base_install.sh
amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
amazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
amazon-ebs: --> Running transaction check
amazon-ebs: ---> Package httpd.x86_64 0:2.4.39-1.amzn2.0.1 will be installed
amazon-ebs: --> Processing Dependency: httpd-tools = 2.4.39-1.amzn2.0.1 for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: httpd-filesystem = 2.4.39-1.amzn2.0.1 for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: system-logos-httpd for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: mod_http2 for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: httpd-filesystem for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: /etc/mime.types for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.39-1.amzn2.0.1.x86_64
amazon-ebs: ---> Package php.x86_64 0:5.4.16-45.amzn2.0.6 will be installed
amazon-ebs: --> Processing Dependency: php-cli(x86-64) = 5.4.16-45.amzn2.0.6 for package: php-5.4.16-45.amzn2.0.6.x86_64
amazon-ebs: --> Processing Dependency: php-common(x86-64) = 5.4.16-45.amzn2.0.6 for package: php-5.4.16-45.amzn2.0.6.x86_64
amazon-ebs: --> Running transaction check
amazon-ebs: ---> Package apr.x86_64 0:1.6.3-5.amzn2.0.2 will be installed
amazon-ebs: ---> Package apr-util.x86_64 0:1.6.1-5.amzn2.0.2 will be installed
amazon-ebs: --> Processing Dependency: apr-util-bdb(x86-64) = 1.6.1-5.amzn2.0.2 for package: apr-util-1.6.1-5.amzn2.0.2.x86_64
amazon-ebs: ---> Package generic-logos-httpd.noarch 0:18.0.0-4.amzn2 will be installed
amazon-ebs: ---> Package httpd-filesystem.noarch 0:2.4.39-1.amzn2.0.1 will be installed
amazon-ebs: ---> Package httpd-tools.x86_64 0:2.4.39-1.amzn2.0.1 will be installed
amazon-ebs: ---> Package mailcap.noarch 0:2.1.41-2.amzn2 will be installed
amazon-ebs: ---> Package mod_http2.x86_64 0:1.15.1-1.amzn2 will be installed
amazon-ebs: ---> Package php-cli.x86_64 0:5.4.16-45.amzn2.0.6 will be installed
amazon-ebs: ---> Package php-common.x86_64 0:5.4.16-45.amzn2.0.6 will be installed
amazon-ebs: --> Processing Dependency: libzip.so.2()(64bit) for package: php-common-5.4.16-45.amzn2.0.6.x86_64
amazon-ebs: --> Running transaction check
amazon-ebs: ---> Package apr-util-bdb.x86_64 0:1.6.1-5.amzn2.0.2 will be installed
amazon-ebs: ---> Package libzip010-compat.x86_64 0:0.10.1-9.amzn2.0.5 will be installed
amazon-ebs: --> Finished Dependency Resolution
amazon-ebs:
amazon-ebs: Dependencies Resolved
amazon-ebs:
amazon-ebs: ================================================================================
amazon-ebs: Package Arch Version Repository Size
amazon-ebs: ================================================================================
amazon-ebs: Installing:
amazon-ebs: httpd x86_64 2.4.39-1.amzn2.0.1 amzn2-core 1.3 M
amazon-ebs: php x86_64 5.4.16-45.amzn2.0.6 amzn2-core 1.4 M
amazon-ebs: Installing for dependencies:
amazon-ebs: apr x86_64 1.6.3-5.amzn2.0.2 amzn2-core 118 k
amazon-ebs: apr-util x86_64 1.6.1-5.amzn2.0.2 amzn2-core 99 k
amazon-ebs: apr-util-bdb x86_64 1.6.1-5.amzn2.0.2 amzn2-core 19 k
amazon-ebs: generic-logos-httpd noarch 18.0.0-4.amzn2 amzn2-core 19 k
amazon-ebs: httpd-filesystem noarch 2.4.39-1.amzn2.0.1 amzn2-core 23 k
amazon-ebs: httpd-tools x86_64 2.4.39-1.amzn2.0.1 amzn2-core 87 k
amazon-ebs: libzip010-compat x86_64 0.10.1-9.amzn2.0.5 amzn2-core 30 k
amazon-ebs: mailcap noarch 2.1.41-2.amzn2 amzn2-core 31 k
amazon-ebs: mod_http2 x86_64 1.15.1-1.amzn2 amzn2-core 147 k
amazon-ebs: php-cli x86_64 5.4.16-45.amzn2.0.6 amzn2-core 2.9 M
amazon-ebs: php-common x86_64 5.4.16-45.amzn2.0.6 amzn2-core 566 k
amazon-ebs:
amazon-ebs: Transaction Summary
amazon-ebs: ================================================================================
amazon-ebs: Install 2 Packages (+11 Dependent packages)
amazon-ebs:
amazon-ebs: Total download size: 6.7 M
amazon-ebs: Installed size: 22 M
amazon-ebs: Downloading packages:
amazon-ebs: --------------------------------------------------------------------------------
amazon-ebs: Total 18 MB/s | 6.7 MB 00:00
amazon-ebs: Running transaction check
amazon-ebs: Running transaction test
amazon-ebs: Transaction test succeeded
amazon-ebs: Running transaction
==> amazon-ebs: ** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows:
==> amazon-ebs: 32:bind-license-9.9.4-74.amzn2.1.2.noarch is a duplicate with 32:bind-license-9.9.4-73.amzn2.1.2.noarch
==> amazon-ebs: python-libs-2.7.16-2.amzn2.0.1.x86_64 is a duplicate with python-libs-2.7.14-58.amzn2.0.4.x86_64
amazon-ebs: Installing : apr-1.6.3-5.amzn2.0.2.x86_64 1/13
amazon-ebs: Installing : apr-util-bdb-1.6.1-5.amzn2.0.2.x86_64 2/13
amazon-ebs: Installing : apr-util-1.6.1-5.amzn2.0.2.x86_64 3/13
amazon-ebs: Installing : httpd-tools-2.4.39-1.amzn2.0.1.x86_64 4/13
amazon-ebs: Installing : generic-logos-httpd-18.0.0-4.amzn2.noarch 5/13
amazon-ebs: Installing : mailcap-2.1.41-2.amzn2.noarch 6/13
amazon-ebs: Installing : httpd-filesystem-2.4.39-1.amzn2.0.1.noarch 7/13
amazon-ebs: Installing : mod_http2-1.15.1-1.amzn2.x86_64 8/13
amazon-ebs: Installing : httpd-2.4.39-1.amzn2.0.1.x86_64 9/13
amazon-ebs: Installing : libzip010-compat-0.10.1-9.amzn2.0.5.x86_64 10/13
amazon-ebs: Installing : php-common-5.4.16-45.amzn2.0.6.x86_64 11/13
amazon-ebs: Installing : php-cli-5.4.16-45.amzn2.0.6.x86_64 12/13
amazon-ebs: Installing : php-5.4.16-45.amzn2.0.6.x86_64 13/13
amazon-ebs: Verifying : apr-util-1.6.1-5.amzn2.0.2.x86_64 1/13
amazon-ebs: Verifying : libzip010-compat-0.10.1-9.amzn2.0.5.x86_64 2/13
amazon-ebs: Verifying : php-cli-5.4.16-45.amzn2.0.6.x86_64 3/13
amazon-ebs: Verifying : apr-util-bdb-1.6.1-5.amzn2.0.2.x86_64 4/13
amazon-ebs: Verifying : httpd-tools-2.4.39-1.amzn2.0.1.x86_64 5/13
amazon-ebs: Verifying : httpd-2.4.39-1.amzn2.0.1.x86_64 6/13
amazon-ebs: Verifying : httpd-filesystem-2.4.39-1.amzn2.0.1.noarch 7/13
amazon-ebs: Verifying : php-5.4.16-45.amzn2.0.6.x86_64 8/13
amazon-ebs: Verifying : mod_http2-1.15.1-1.amzn2.x86_64 9/13
amazon-ebs: Verifying : apr-1.6.3-5.amzn2.0.2.x86_64 10/13
amazon-ebs: Verifying : mailcap-2.1.41-2.amzn2.noarch 11/13
amazon-ebs: Verifying : generic-logos-httpd-18.0.0-4.amzn2.noarch 12/13
amazon-ebs: Verifying : php-common-5.4.16-45.amzn2.0.6.x86_64 13/13
amazon-ebs:
amazon-ebs: Installed:
amazon-ebs: httpd.x86_64 0:2.4.39-1.amzn2.0.1 php.x86_64 0:5.4.16-45.amzn2.0.6
amazon-ebs:
amazon-ebs: Dependency Installed:
amazon-ebs: apr.x86_64 0:1.6.3-5.amzn2.0.2
amazon-ebs: apr-util.x86_64 0:1.6.1-5.amzn2.0.2
amazon-ebs: apr-util-bdb.x86_64 0:1.6.1-5.amzn2.0.2
amazon-ebs: generic-logos-httpd.noarch 0:18.0.0-4.amzn2
amazon-ebs: httpd-filesystem.noarch 0:2.4.39-1.amzn2.0.1
amazon-ebs: httpd-tools.x86_64 0:2.4.39-1.amzn2.0.1
amazon-ebs: libzip010-compat.x86_64 0:0.10.1-9.amzn2.0.5
amazon-ebs: mailcap.noarch 0:2.1.41-2.amzn2
amazon-ebs: mod_http2.x86_64 0:1.15.1-1.amzn2
amazon-ebs: php-cli.x86_64 0:5.4.16-45.amzn2.0.6
amazon-ebs: php-common.x86_64 0:5.4.16-45.amzn2.0.6
amazon-ebs:
amazon-ebs: Complete!
amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
==> amazon-ebs: Existing lock /var/run/yum.pid: another copy is running as pid 2748.
==> amazon-ebs: Another app is currently holding the yum lock; waiting for it to exit...
==> amazon-ebs: The other application is: yum
==> amazon-ebs: Memory : 42 M RSS (332 MB VSZ)
==> amazon-ebs: Started: Fri Aug 23 03:29:45 2019 - 00:01 ago
==> amazon-ebs: State : Sleeping, pid: 2748
==> amazon-ebs: Another app is currently holding the yum lock; waiting for it to exit...
==> amazon-ebs: The other application is: yum
==> amazon-ebs: Memory : 101 M RSS (392 MB VSZ)
==> amazon-ebs: Started: Fri Aug 23 03:29:45 2019 - 00:03 ago
==> amazon-ebs: State : Running, pid: 2748
amazon-ebs: Cleaning repos: amzn2-core amzn2extra-docker amzn2extra-epel
amazon-ebs: 13 metadata files removed
amazon-ebs: 6 sqlite files removed
amazon-ebs: 0 metadata files removed
amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
amazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
amazon-ebs: --> Running transaction check
amazon-ebs: ---> Package epel-release.noarch 0:7-11 will be installed
amazon-ebs: --> Finished Dependency Resolution
amazon-ebs:
amazon-ebs: Dependencies Resolved
amazon-ebs:
amazon-ebs: ================================================================================
amazon-ebs: Package Arch Version Repository Size
amazon-ebs: ================================================================================
amazon-ebs: Installing:
amazon-ebs: epel-release noarch 7-11 amzn2extra-epel 15 k
amazon-ebs:
amazon-ebs: Transaction Summary
amazon-ebs: ================================================================================
amazon-ebs: Install 1 Package
amazon-ebs:
amazon-ebs: Total download size: 15 k
amazon-ebs: Installed size: 24 k
amazon-ebs: Downloading packages:
amazon-ebs: Running transaction check
amazon-ebs: Running transaction test
amazon-ebs: Transaction test succeeded
amazon-ebs: Running transaction
amazon-ebs: Installing : epel-release-7-11.noarch 1/1
amazon-ebs: Verifying : epel-release-7-11.noarch 1/1
amazon-ebs:
amazon-ebs: Installed:
amazon-ebs: epel-release.noarch 0:7-11
amazon-ebs:
amazon-ebs: Complete!
amazon-ebs: Installing epel-release
amazon-ebs: 0 ansible2 available [ =2.4.2 =2.4.6 ]
amazon-ebs: 2 httpd_modules available [ =1.0 ]
amazon-ebs: 3 memcached1.5 available [ =1.5.1 =1.5.16 ]
amazon-ebs: 4 nginx1.12 available [ =1.12.2 ]
amazon-ebs: 5 postgresql9.6 available [ =9.6.6 =9.6.8 ]
amazon-ebs: 6 postgresql10 available [ =10 ]
amazon-ebs: 8 redis4.0 available [ =4.0.5 =4.0.10 ]
amazon-ebs: 9 R3.4 available [ =3.4.3 ]
amazon-ebs: 10 rust1 available \\
amazon-ebs: [ =1.22.1 =1.26.0 =1.26.1 =1.27.2 =1.31.0 ]
amazon-ebs: 11 vim available [ =8.0 ]
amazon-ebs: 13 ruby2.4 available [ =2.4.2 =2.4.4 ]
amazon-ebs: 15 php7.2 available \\
amazon-ebs: [ =7.2.0 =7.2.4 =7.2.5 =7.2.8 =7.2.11 =7.2.13 =7.2.14
amazon-ebs: =7.2.16 =7.2.17 =7.2.19 ]
amazon-ebs: 16 php7.1 available \\
amazon-ebs: [ =7.1.22 =7.1.25 =7.1.27 =7.1.28 =7.1.30 ]
amazon-ebs: 17 lamp-mariadb10.2-php7.2 available \\
amazon-ebs: [ =10.2.10_7.2.0 =10.2.10_7.2.4 =10.2.10_7.2.5
amazon-ebs: =10.2.10_7.2.8 =10.2.10_7.2.11 =10.2.10_7.2.13
amazon-ebs: =10.2.10_7.2.14 =10.2.10_7.2.16 =10.2.10_7.2.17
amazon-ebs: =10.2.10_7.2.19 ]
amazon-ebs: 18 libreoffice available [ =5.0.6.2_15 =5.3.6.1 ]
amazon-ebs: 19 gimp available [ =2.8.22 ]
amazon-ebs: 20 docker=latest enabled \\
amazon-ebs: [ =17.12.1 =18.03.1 =18.06.1 ]
amazon-ebs: 21 mate-desktop1.x available [ =1.19.0 =1.20.0 ]
amazon-ebs: 22 GraphicsMagick1.3 available [ =1.3.29 =1.3.32 ]
amazon-ebs: 23 tomcat8.5 available \\
amazon-ebs: [ =8.5.31 =8.5.32 =8.5.38 =8.5.40 =8.5.42 ]
amazon-ebs: 24 epel=latest enabled [ =7.11 ]
amazon-ebs: 25 testing available [ =1.0 ]
amazon-ebs: 26 ecs available [ =stable ]
amazon-ebs: 27 corretto8 available \\
amazon-ebs: [ =1.8.0_192 =1.8.0_202 =1.8.0_212 =1.8.0_222 ]
amazon-ebs: 28 firecracker available [ =0.11 ]
amazon-ebs: 29 golang1.11 available [ =1.11.3 =1.11.11 ]
amazon-ebs: 30 squid4 available [ =4 ]
amazon-ebs: 31 php7.3 available \\
amazon-ebs: [ =7.3.2 =7.3.3 =7.3.4 =7.3.6 ]
amazon-ebs: 32 lustre2.10 available [ =2.10.5 ]
amazon-ebs: 33 java-openjdk11 available [ =11 ]
amazon-ebs: 34 lynis available [ =stable ]
amazon-ebs: 35 kernel-ng available [ =stable ]
amazon-ebs: 36 BCC available [ =0.x ]
amazon-ebs: 37 mono available [ =5.x ]
amazon-ebs: Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
==> amazon-ebs: https://mirrors.sonic.net/epel/7/x86_64/repodata/repomd.xml: [Errno -1] repomd.xml does not match metalink for epel
==> amazon-ebs: Trying other mirror.
==> amazon-ebs: https://ewr.edge.kernel.org/fedora-buffet/epel/7/x86_64/repodata/d748a548825eb7ebeca7c8cb8e98387afe904e7bc00dab7c9c35795379cc183d-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found
==> amazon-ebs: Trying other mirror.
amazon-ebs: 191 packages excluded due to repository priority protections
amazon-ebs: Resolving Dependencies
==> amazon-ebs: There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
amazon-ebs: --> Running transaction check
amazon-ebs: ---> Package stress.x86_64 0:1.0.4-16.el7 will be installed
amazon-ebs: --> Finished Dependency Resolution
amazon-ebs:
amazon-ebs: Dependencies Resolved
amazon-ebs:
amazon-ebs: ================================================================================
amazon-ebs: Package Arch Version Repository Size
amazon-ebs: ================================================================================
amazon-ebs: Installing:
amazon-ebs: stress x86_64 1.0.4-16.el7 epel 39 k
amazon-ebs:
amazon-ebs: Transaction Summary
amazon-ebs: ================================================================================
amazon-ebs: Install 1 Package
amazon-ebs:
amazon-ebs: Total download size: 39 k
amazon-ebs: Installed size: 94 k
amazon-ebs: Downloading packages:
==> amazon-ebs: warning: /var/cache/yum/x86_64/2/epel/packages/stress-1.0.4-16.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
amazon-ebs: Public key for stress-1.0.4-16.el7.x86_64.rpm is not installed
amazon-ebs: Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
==> amazon-ebs: Importing GPG key 0x352C64E5:
==> amazon-ebs: Userid : "Fedora EPEL (7) <epel@fedoraproject.org>"
==> amazon-ebs: Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
==> amazon-ebs: Package : epel-release-7-11.noarch (@amzn2extra-epel)
==> amazon-ebs: From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
amazon-ebs: Running transaction check
amazon-ebs: Running transaction test
amazon-ebs: Transaction test succeeded
amazon-ebs: Running transaction
amazon-ebs: Installing : stress-1.0.4-16.el7.x86_64 1/1
amazon-ebs: Verifying : stress-1.0.4-16.el7.x86_64 1/1
amazon-ebs:
amazon-ebs: Installed:
amazon-ebs: stress.x86_64 0:1.0.4-16.el7
amazon-ebs:
amazon-ebs: Complete!
==> amazon-ebs: Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
amazon-ebs: burnCPU.sh
amazon-ebs: index.php
==> amazon-ebs: Stopping the source instance...
amazon-ebs: Stopping instance
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating AMI amaz2-stress 1566530942 by packer from instance i-0eb31ca09c65dfbbb
amazon-ebs: AMI: ami-04262661376a925a7
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
ap-east-1: ami-04262661376a925a7请看最后一行： ami-04262661376a925a7，后面我们将这个作为变量，提供后面的 terraform 执行，LB+ASG 的创建。
2: 创建LB + ASG
查看 main.tf 文件, 修改参数值。
##
provider "aws"
region = "ap-east-1" \\\\ 替换成您实际操作的Region
access_key = "Your-AWS-AK" \\\\ 替换您自己账号的AK
secret_key = "Your-AWS-SK" \\\\ 替换成你自己账号的SK
##
variable "asg_instances_id"
default = "From_Packer_create_AMI_ID" \\\\ 替换成你上面从Packer获取到的AMI ID
###
data "aws_vpc" "default"
default = true
data "aws_subnet_ids" "all"
vpc_id = data.aws_vpc.default.id
data "aws_security_group" "default"
vpc_id = data.aws_vpc.default.id
name = "default"
data "aws_ami" "amazon_linux"
most_recent = true
owners = ["amazon"]
filter
name = "name"
values = [
"amzn-ami-hvm-*-x86_64-gp2",
]
filter
name = "owner-alias"
values = [
"amazon",
]
####
module "elb"
source = "../terraform-modules/terraform-aws-elb/"
name = "elb-example"
subnets = data.aws_subnet_ids.all.ids
security_groups = [data.aws_security_group.default.id]
internal = false
listener = [
instance_port = "80"
instance_protocol = "HTTP"
lb_port = "80"
lb_protocol = "HTTP"
,
]
health_check =
target = "HTTP:80/"
interval = 30
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 5
tags =
Owner = "user"
Environment = "dev"
####
module "example_asg"
source = "../my-modules/terraform-aws-autoscaling/"
name = "example-with-lb-asg"
# Launch configuration
#
# launch_configuration = "my-existing-launch-configuration" # Use the existing launch configuration
# create_lc = false # disables creation of launch configuration
lc_name = "example-lc"
image_id = "$var.asg_instances_id"
instance_type = "c5.large"
security_groups = [data.aws_security_group.default.id]
load_balancers = [module.elb.this_elb_id]
key_name = "hongkong"
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = "true"
cpu_utilization_high_threshold_percent = "70"
cpu_utilization_low_threshold_percent = "20"
root_block_device = [
volume_size = "10"
volume_type = "gp2"
,
]
# Auto scaling group
asg_name = "example-asg"
vpc_zone_identifier = data.aws_subnet_ids.all.ids
health_check_type = "EC2"
min_size = 1
max_size = 3
desired_capacity = 1
wait_for_capacity_timeout = 0
tags = [
key = "Environment"
value = "dev"
propagate_at_launch = true
,
key = "Project"
value = "megasecret"
propagate_at_launch = true
,
]
其他参数按照我们上一个LAB的方法获取参数含义，并自行调整。执行命令：
$ terraform init
$ terrrform plan
$ terraform apply -auto-approve检验结果：
执行成功后，默认只会在 ASG 创建 1 台主机，找到 LB 的 DNS Name，进行访问，例如： https://elb-example-291281982.ap-east-1.elb.amazonaws.com
同时结合 AWS Console 观察 EC2 的 create 和 terminal，也可以通过 ASG 查看：
三、与 Ansible 集成的主要方法，实现中国区自己的 “EFS”
下面介绍与 Ansible 集成的方法，实现一套 Pacemaker + DRBD + NFS 来实现一个跨 AZ 的集群部署方案。所以在这种框架下，我们要做到以下两件事：
1：通过 Terraform 创建目标账号下所需要的 Infra 层面的所有资源.
2：通过 Terraform 调用 Ansible Playbook 实现对所有目标主机的配置。
AWS 上 HA 实现方案：
VIP：在 AWS 由于 Subnet 无法跨 AZ，所以实现的方法就存在两种。Overlay 的假 IP，就是一个存粹的路由表指向，将这个不真实存在于 VPC 内的 IP 指向一个 ENI ; 还有一种方式就是 EIP，也就是拥有固定外网 IP 的 Public IP.
IAM Role 的权限：这个权限定义通常是集群主机内所有主机的开关机权限。然后以 ec2 role 的方式赋予。之所以需要这个权限，是因为系统内的 HA agent 会通过 AWS CLI 进行状态的 monitor 和 Switch 动作。所以需要这样的权限，在发生 Failover 后，能够正确处置后续的资源位置。
整体上HA的架构：
1：新增的 EBS，通过 DRBD 的方式进行底层数据的同步。
2：基于 DRBD 所创建的文件系统 PV-VG-LV, 做到提供给 NFS 作为基本的文件系统，用于文件的分享。
3：最上层的 NFS 服务，构建与文件系统上，提供基础服务。
4：VIP 随着 DRBD 的 Master 绑定进行切换。
而关于我们上面提到的 EIP 的架构图如下：
集成 Ansible
1：与 Ansible 的集成生成 inventory，在生成所需的 ec2 主机后，构建 inventory，请看如下代码：
[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat upload.tf
data "template_file" "inventory"
template = "$file("$path.module/templates/hosts.tpl")"
vars =
dns01_hostname = "nfs01.liujia.com"
dns02_hostname = "nfs02.liujia.com"
dns01_ip = tolist(module.ec2-nfs01.private_ip)[0]
dns02_ip = tolist(module.ec2-nfs02.private_ip)[0]
key_path = "~/.ssh/id_rsa"
resource "local_file" "save_inventory"
content = "$data.template_file.inventory.rendered"
filename = "./ansible-playbook/hosts"
[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat templates/hosts.tpl
[master]
$nfs01_hostname ansible_ssh_host=$nfs01_ip
[slave]
$nfs02_hostname ansible_ssh_host=$nfs02_ip
[all:vars]
ansible_ssh_private_key_file = $key_path
在这里通过对模板 inventory 的定义，在通过 data template_file 的参数传递，最终渲染生成 local_file 到 ansible 的 playbook 里面生成 inventory，用于最终 playbook 的生成。
2：完成 Ansible 的上传和 playbook 的执行
[ec2-user@ip-172-31-22-159 withAnsible-55523423422-nx-centos]$ cat upload.tf
...
resource "null_resource" "utility"
connection
timeout = "5m"
type = "ssh"
user = "ansible"
host = tolist(module.ec2-utility.public_ip)[0]
private_key = "$file("mykey.pem")"
provisioner "local-exec"
command = "tar zcvf ./ansible-playbook.tgz ./ansible-playbook"
provisioner "file"
source = "./ansible-playbook.tgz"
destination = "/tmp/ansible-playbook.tgz"
provisioner "remote-exec"
inline = [
"tar zxvf /tmp/ansible-playbook.tgz -C /tmp",
"cd /tmp/ansible-playbook",
"ansible-playbook -i ./hosts site.yml",
]
depends_on = [local_file.save_inventory]
在这里通过对模板 inventory 的定义，在通过 data template_file 的参数传递，最终渲染生成 local_file 到 ansible 的 playbook 里面生成 inventory，用于最终 playbook 的生成。
四、典型场景的应用和常见问题
场景一： 实现 Openshift 3 在 AWS 上的定制快速部署
以下就是典型 openshift 在 AWS 上部署架构图。其中 ansible config server，就是我们在上面讲到的类似方法，在这台主机上已经操作主机，从而进行整个集群环境的配置和管理。
同理，在 AWS 上部署原生的 Kubernates 有很多工具，例如：kops，可以在 AWS 快速构建起一套环境。但是在插件调整，系统配置和调优方面，基本上完全没有入口可以让你自己精细化的调整整个集群的架构。而通过我们 terraform + ansible 的组合，可以从底层资源到系统 OS 的所有层面可控。
场景二： 通用场景说明
在下面的模式下，我们开通过 Terraform 走绿线创建右侧受管账号下的所有资源，并且实现自动化。当然包括计算资源。然后我们通过推出的一台 Ansible 配置管理主机作为部署服务器，进而推出目标账号下，相应的系统。只要有标准的配置方法，我们都可以实现到自动化的创建。至于究竟是什么监控平台、测试平台、容器平台，都可以。
常见问题总结：
问题 1：为什么我执行 terrafom init 执行这么慢？
回答 1：terraform 基本分拆了所有的调用模块，terraform 默认安装里面只包括主要调用。在 terraform 执行 init 后，依据读取的 tf 文件，再去下载对应的 provider 模块或者其他相关模块。通常这些文件都不小，而且通常都存在工作目录的 .terraform 下，如果你确定是类似工作模块，建议你直接复制这个目录到新的项目目录下。这样比较节省时间。
问题 2：如何 debug Terraform ？
回答 2：在执行命令前加入 TF_LOG=TRACE
[ec2-user@ip-172-31-22-159 log]$ TF_LOG=TRACE terraform init
2019/07/18 01:33:09 [INFO] Terraform version: 0.12.4
2019/07/18 01:33:09 [INFO] Go runtime version: go1.12.4
2019/07/18 01:33:09 [INFO] CLI args: []string"/usr/local/bin/terraform", "init"
2019/07/18 01:33:09 [DEBUG] Attempting to open CLI config file: /home/ec2-user/.terraformrc
2019/07/18 01:33:09 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/07/18 01:33:09 [INFO] CLI command args: []string"init"
Terraform initialized in an empty directory!
The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.以上是关于Terraform 学习总结—— 基于 AWS 云平台上的 Terraform 实战的主要内容，如果未能解决你的问题，请参考以下文章
Terraform 学习总结—— 基于阿里云平台上的 Terraform 实战
Terraform 学习总结（10）—— 阿里云平台 Terraform 代码开发技巧总结
Terraform 学习总结（10）—— 阿里云平台 Terraform 代码开发技巧总结