AWS学习笔记--利用CloudFormation管理AWS资源

Posted 2020-10-25

CloudFormation是管理AWS资源的一项服务。你可以创建一个Template，其中包括所有用到的资源(VPC、EC2、RDS等）。Template支持参数化，可根据条件创建资源，以适应不同的环境。CloudFormation另一个重要概念是Stack，每个Stack包含一个Template，可以设定Template中定义的参数值，增加Stack级tag，设置权限等，通过Stack真正的创建资源。

利用CloudFormation不必再单独的创建和配置AWS资源，解除了繁琐的手工安装配置工作，又可以将代码进行版本管理，利于以后的更新维护，也不用编写安装手册了。

Template

CloudFormation Template支持两种格式：JSON和YAML，可以将模板文件保存为任何扩展名，比如：.json, .yaml, .template, .txt。YAML语法更简捷易读，并且支持注释，但CloudFormation Designer目前只支持JSON。
JSON示例

{  
  "AWSTemplateFormatVersion" : "2010-09-09",  
  "Description" : "A sample template",  
  "Resources" : {  
    "MyEC2Instance" : {  
      "Type" : "AWS::EC2::Instance",  
      "Properties" : {  
        "ImageId" : "ami-2f726546",  
        "InstanceType" : "t1.micro",  
        "KeyName" : "testkey",  
        "BlockDeviceMappings" : [  
          {  
            "DeviceName" : "/dev/sdm",  
            "Ebs" : {  
              "VolumeType" : "io1",  
              "Iops" : "200",  
              "DeleteOnTermination" : "false",  
              "VolumeSize" : "20"  
            }  
          }  
        ]  
      }  
    }  
  }  
}

YAML示例

AWSTemplateFormatVersion: "2010-09-09"  
Description: A sample template  
Resources:  
  MyEC2Instance:  
    Type: "AWS::EC2::Instance"  
    Properties:   
      ImageId: "ami-2f726546"  
      InstanceType: t1.micro  
      KeyName: testkey  
      BlockDeviceMappings:  
        -  
          DeviceName: /dev/sdm  
          Ebs:  
            VolumeType: io1  
            Iops: 200  
            DeleteOnTermination: false  
            VolumeSize: 20

模板基本知识请参考：
Template Anatomy
Learn Template Basics
Parameters
参数类型除String、Number、List<Number>、CommaDelimitedList外，还支持特定的类型，比如AWS::EC2::KeyPair::KeyName、AWS::EC2::AvailabilityZone::Name、AWS::EC2::SecurityGroup::GroupName等，用户必须指定其账户中现有的 AWS 值。

在通过CloudFormation Console创建Stack时，参数默认是按字母排序的，如果参数多看起来会很乱，不易查找，这就需要借助Metadata了，AWS::CloudFormation::Interface用于定义参数在CloudFormation控制台中的分组和排序方式：

"Metadata" : {  
  "AWS::CloudFormation::Interface" : {  
    "ParameterGroups" : [  
      {  
        "Label" : { "default" : "Network Configuration" },  
        "Parameters" : [ "VPCID", "SubnetId", "SecurityGroupID" ]  
      },  
      {  
        "Label" : { "default":"Amazon EC2 Configuration" },  
        "Parameters" : [ "InstanceType", "KeyName" ]  
      }  
    ]  
  }  
}

下面的代码演示了CloudFormation中cfn-init和shell的使用。AMI使用了RHEL 7，其中包括安装cfn-init、awscli、awslogs，配置DNS、awscli、awslog等。

"AWSTemplateFormatVersion": "2010-09-09",  
  "Resources": {  
    "InstanceTest": {  
      "Type": "AWS::EC2::Instance",  
      "Metadata": {  
        "AWS::CloudFormation::Init": {  
          "configSets": {  
            "Install": [  
              "Install"  
            ]  
          },  
          "Install": {  
            "packages": {  
              "yum": {  
                "ntp": [],  
                "nfs-utils": [],  
                "rpcbind": []  
              }  
            },  
            "files": {  
              "/etc/sysconfig/network-scripts/ifcfg-eth0": {  
                "content": { "Fn::Join": ["", [  
                  "DEVICE=\"eth0\"\n",  
                  "BOOTPROTO=\"dhcp\"\n",  
                  "ONBOOT=\"yes\"\n",  
                  "TYPE=\"Ethernet\"\n",  
                  "USERCTL=\"yes\"\n",  
                  "PEERDNS=\"no\"\n",  
                  "IPV6INIT=\"no\"\n",  
                  "DNS1=\"10.184.13.14\""  
                ]]},  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/config": {  
                "content": { "Fn::Join": ["", [
                                  "[default]\n",  
                  "output = json\n",  
                  "region = cn-north-1"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/credentials": {  
                "content": { "Fn::Join": ["", [  
                  "[default]\n",  
                  "aws_access_key_id = AKIAPM6UDU6DGZPYXXXX\n",   
                  "aws_secret_access_key = CrYrR/LeiDXJWbAX3KQwFnaScDU0zGE+XXXXXXXX"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/awslogs.conf": {  
                "content": { "Fn::Join": ["", [  
                  "[general]\n",  
                  "state_file = /var/awslogs/state/agent-state\n",  
                  "[/var/log/messages]\n",  
                  "datetime_format = %Y-%m-%d %H:%M:%S\n",  
                  "file = /var/log/messages\n",  
                  "buffer_duration = 5000\n",  
                  "log_stream_name = server-log\n",  
                  "initial_position = start_of_file\n",  
                  "log_group_name = /var/log/messages"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              }  
            },
                        "commands": {  
              "1-timezone": {  
                "command": "timedatectl set-timezone UTC",  
                "ignoreErrors": "true"  
              },  
              "2-groups": {  
                "command": "groupadd -g 1001 monet",  
                "ignoreErrors": "true"  
              },  
              "3-users": {  
                "command": "useradd -d /monet -g monet -u 1001 monetmanager\nchmod 755 /monet ",  
                "ignoreErrors": "true"  
              }  
            },  
            "services": {  
              "sysvinit": {  
                "ntpd": { "enabled" : "true", "ensureRunning" : "true" },  
                "rpcbind": { "enabled" : "true", "ensureRunning" : "true" },  
                "nfslock": { "enabled" : "true", "ensureRunning" : "true" }  
              }  
            }  
          }  
        }  
      },  
      "Properties": {  
        "AvailabilityZone": "cn-north-1a",  
        "DisableApiTermination": "false",  
        "EbsOptimized": "false",  
        "ImageId": "ami-3ce23651",  
        "InstanceInitiatedShutdownBehavior": "stop",  
        "InstanceType": "t2.small",  
        "KeyName": "Prod Key Pair",  
        "Monitoring": "false",  
        "PrivateIpAddress": "10.184.12.247",
                "SecurityGroupIds": [  
          {  
            "Fn::ImportValue": "prod-network-sg-private-id"  
          }  
        ],  
        "SubnetId": {  
          "Fn::ImportValue": "prod-network-subnet-private-id"  
        },  
        "BlockDeviceMappings": [  
          {  
            "DeviceName": "/dev/sda1",  
            "Ebs": {  
              "VolumeSize": "30"  
            }  
          }  
        ],  
        "Tags": [  
          {  
            "Key": "Name",  
            "Value": "test-ec2"  
          }  
        ],  
        "UserData": {  
          "Fn::Base64": {  
            "Fn::Join" : ["", [  
                "#!/bin/bash\n",  

                "# Install cfn-init\n",  
                "curl -O https://bootstrap.pypa.io/ez_setup.py\n",  
                "python ez_setup.py\n",  
                "easy_install --script-dir /opt/aws/bin https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",  

                "# Install the files and packages from the metadata\n",  
                "/opt/aws/bin/cfn-init -v ",  
                "         --stack ", { "Ref" : "AWS::StackName" },  
                "         --resource InstanceTest ",  
                "         --configsets Install ",  
                "         --region ", { "Ref" : "AWS::Region" }, "\n",  

                "# Install awscli\n",  
                "curl -O https://bootstrap.pypa.io/get-pip.py\n",  
                "python get-pip.py\n",  
                "pip install awscli\n", 

                                "# Install awslogs\n",  
                "curl -O https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py\n",  
                "python ./awslogs-agent-setup.py --region cn-north-1 --non-interactive --configfile=/root/.aws/awslogs.conf\n",  

                "# Delete installation files\n",  
                "rm -f get-pip.py ez_setup.py setuptools*.zip awslogs-agent-setup.py\n"  
              ]  
            ]  
          }  
        }  
      }  
    }  
  }  
}

下面演示了在Windows 2016上通过powershell安装DNS:

{  
  "AWSTemplateFormatVersion": "2010-09-09",  
  "Resources": {  
    "InstanceDNS": {  
      "Type": "AWS::EC2::Instance",  
      "Properties": {  
        "AvailabilityZone": "cn-north-1a",  
        "DisableApiTermination": "false",  
        "EbsOptimized": "false",  
        "ImageId": "ami-2797404a",  
        "InstanceInitiatedShutdownBehavior": "stop",  
        "InstanceType": "t2.small",  
        "KeyName": "Prod Key Pair",  
        "Monitoring": "false",  
        "PrivateIpAddress": "10.184.13.15",  
        "SecurityGroupIds": [  
          "sg-b0525cd5"  
        ],  
        "SubnetId": "subnet-a65208c3",  
        "BlockDeviceMappings": [  
          {  
            "DeviceName": "/dev/sda1",  
            "Ebs": {  
              "VolumeSize": "30"  
            }  
          }  
        ],  
        "UserData": { "Fn::Base64" : { "Fn::Join" : ["", [  
          "<powershell>\n",  
          "Install-WindowsFeature -Name DNS -IncludeManagementTools\n",  
          "Add-DnsServerPrimaryZone -Name \"iata.org\" -ZoneFile \"iata.org.dns\"\n",  
          "Add-DnsServerResourceRecordA -Name \"test\" -ZoneName \"iata.org\" -AllowUpdateAny -IPv4Address \"10.184.12.111\"\n",  
          "</powershell>"  
        ]]}},  
        "Tags": [  
          {  
            "Key": "Name",  
            "Value": "test-dns"  
          }  
        ]  
      }  
    }  
  }  
}

CloudFormation Designer

AWS CloudFormation Console提供图形化的Template设计器，支持拖拽操作，集成JSON编辑器，支持自动完成功能。
使用CloudFormation Designer的拖拽功能，或用其打开已有的Template会增加组件位置信息，这些信息在实际应用中是没有必要的，也增加了Template的复杂度，故本人只利用它的自动完成功能，要编辑已有的Template时，使用Copy/Paste的方式。

快捷键
【Ctrl+Shift+Space】 显示提示列表
【Ctrl+F】 搜索
【Ctrl+\】 格式化Template(会删除所有空行)
【Ctrl+Shift+\】 删除所有空格

Sample Templates

CloudFormation提供了大量的示例模板Sample Templates可供学习，其中China Region的示例较少，推荐US West (Northern California) Region。比如Services中下面的示例，全面演示了从创建VPC到EC2。
Creates a VPC and adds an Amazon EC2 instance with an Elastic IP address and a security group

CloudFormer

CloudFormer是模板创建工具，可从现有AWS资源创建CloudFormation Template，用来作为初始Template，是学习Template语法结构的好工具。

CloudFormer使用很简单，它是一个CloudFormation Stack，可从CloudFormation Console创建，创建后在一个EC2实例上运行，不需其他资源。

创建CloudFormer
登录CloudFormation Console > Create New Stack > Select a sample template > CloudFormer > Next > 输入Stack name、Username、Password > 根据提示信息完成后续步骤

启动CloudFormer
点击CloudFormer Stack选项卡Outputs中的链接，输入用户名，密码即可启动CloudFormer。然后Select the AWS Region > Create Template > 一步步的选择要创建模板的资源。

注：如出现错误NoMethodError in TemplatesController，可能是资源配置有问题，试着减少一些资源。

Best Practices

• 按生命周期和所有权组织堆栈，而不是将所有资源放在一个Stack内，避免资源变化时互相影响。另外，可以根据层次结构和面向的服务（SOA）划分资源。
• 使用跨堆栈引用来导出共享资源,其他堆栈使用 Fn::ImportValue函数调用导出的资源。引用另一个 AWS CloudFormation 堆栈中的资源输出
• 重复使用模板以在多个环境中复制堆栈，要使模板可重用，可使用参数、映射和条件部分，以便能在创建堆栈时对其进行自定义。
• 使用嵌套堆栈来重复使用常见模板，要创建嵌套堆栈，可使用模板中的AWS::CloudFormation::Stack资源来引用其他模板。

AWS CloudFormation CLI

Creating a Stack

You must provide the stack name, the location of a valid template, and any input parameters. If you specify a local template file, AWS CloudFormation uploads it to an Amazon S3 bucket in your AWS account.

$ aws cloudformation create-stack --stack-name myteststack --template-body file:///home/testuser/mytemplate.json --parameters ParameterKey=Parm1,ParameterValue=test1 ParameterKey=Parm2,ParameterValue=test2

Listing Your Stacks

Note The aws cloudformation list-stacks command returns information on deleted stacks for 90 days after they have been deleted.

$ aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE

Describing Your Stacks

$ aws cloudformation describe-stacks --stack-name myteststack

By default, aws cloudformation describe-stacks returns parameter values. To prevent sensitive parameter values such as passwords from being returned, include a NoEcho property set to TRUE in your AWS CloudFormation template.

Viewing Stack Event History

You can track the status of the resources AWS CloudFormation is creating and deleting with the aws cloudformation describe-stack-events command. The amount of time to create or delete a stack depends on the complexity of your stack.

$ aws cloudformation describe-stack-events --stack-name myteststack

Listing Stack Resources

$ aws cloudformation list-stack-resources --stack-name myteststack

Retrieving a Template

AWS CloudFormation stores the template you use to create your stack as part of the stack.

$ aws cloudformation get-template --stack-name myteststack

Validating a Template

You can validate templates locally by using the --template-body parameter, or remotely with the --template-url parameter.

$ aws cloudformation validate-template --template-url https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template
$ aws cloudformation validate-template --template-body file:///home/local/test/sampletemplate.json

Deleting a Stack

$ aws cloudformation delete-stack --stack-name myteststack

Getting Started with AWS CloudFormation
Template Anatomy
Learn Template Basics
AWS CloudFormation Templates
AWS Resource Types Reference
Using CloudFormer to Create AWS CloudFormation Templates from Existing AWS Resources
Using the AWS Command Line Interface
AWS CloudFormation Best Practices
Install roles, role services, and features by using Windows PowerShell cmdlets
Domain Name System (DNS) Server Cmdlets