Cloudformation 模板在 S3 事件上触发 Lambda

Posted

技术标签:

【中文标题】Cloudformation 模板在 S3 事件上触发 Lambda【英文标题】:Cloudformation template to trigger Lambda on S3 event 【发布时间】:2018-04-01 22:23:39 【问题描述】:

我想使用 Cloudformation 创建一个 S3 存储桶,每当发生 S3 事件(例如文件创建、文件删除等)时,它将触发 Lambda 函数。

根据我的研究,我有我的 AWS::Lambda::Function 和 AWS::S3::Bucket 设置,

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  HandleFileCreation: 
    Type: "AWS::Lambda::Function"
    Properties: 
      ...

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/AmazonS3FullAccess
      - arn:aws:iam::aws:policy/AWSLambdaFullAccess
      AssumeRolePolicyDocument:
        ...

  ReportsBucket:
    Type: AWS::S3::Bucket

  BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref ReportsBucket
      PolicyDocument:
        ...

我正在查看AWS::Events::Rule,但该示例仅适用于 EC2,我找不到 S3 的示例

  EventRule: 
    Type: "AWS::Events::Rule"
    Properties: 
      Description: "EventRule"
      EventPattern: 
        source: 
          - "aws.ec2"
        detail-type: 
          - "EC2 Instance State-change Notification"
        detail: 
          state: 
            - "stopping"
      State: "ENABLED"
      Targets: 
        - 
          Arn: 
            Fn::GetAtt: 
              - HandleFileCreation
              - Arn
          Id: TargetFunctionV1
  PermissionForEventsToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: 
        Ref: HandleFileCreation
      Action: "lambda:InvokeFunction"
      Principal: "events.amazonaws.com"
      SourceArn: 
        Fn::GetAtt: 
          - "EventRule"
          - "Arn"

如何编写模板以触发 S3 事件?

【问题讨论】:

【参考方案1】:

这里有一个例子,

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-notificationconfig-lambdaconfig.html

EncryptionServiceBucket:
  Type: "AWS::S3::Bucket"
  Properties:
    BucketName: !Sub $User-encryption-service
    NotificationConfiguration:
      LambdaConfigurations:
        -
          Function: !Ref LambdaDeploymentArn
          Event: "s3:ObjectCreated:*"
          Filter:
            S3Key:
              Rules:
                -
                  Name: suffix
                  Value: zip

我注意到的一个问题是,您需要先创建函数,然后再为其分配触发器。如果您使用 CF,请确保在为它创建触发器之前创建 lambda 函数。

希望对你有帮助。

【讨论】:

我收到一个错误AWS::S3::Bucket ReportsBucket The ARN is not well formed Physical ID:some-prefix-us-west-2-test-43a8dcf,它引用了我的存储桶名称BucketName: !Join ['-', ['some-prefix', !Ref Region, !Ref Stage]]【参考方案2】:

我在使用 Amazon Toolkit 的 Visual Studio 示例项目之一中找到了答案:

"myBucketName": 
    "Type": "AWS::S3::Bucket",
    "Properties":  
,
"csvProcessor" : 
  "Type" : "AWS::Serverless::Function",
  "Properties": 
    "Handler": "appli::appli.csvProcessor::FunctionHandler",
    "Runtime": "dotnetcore2.1",
    "CodeUri": "",
    "Description": "Function processing files when they're dropped in s3 bucket",
    "MemorySize": 256,
    "Timeout": 30,
    "Role": null,
    "Policies": [ "AWSLambdaFullAccess" ],
    "Events": 
        "madeUpEventName" : 
            "Type" : "S3",
            "Properties" : 
                "Bucket" :  "Ref" : "myBucketName" ,
                "Events" : [
                    "s3:ObjectCreated:*"
                ]
            
        
    
  

【讨论】:

【参考方案3】:

我创建了以下代码,它正在使用 CloudFormation ################################################## ######################### 每当有 S3 事件时触发 Lambda 函数

  Type: AWS::S3::Bucket
  Properties:
    BucketName: !Sub '$EnvironmentNameShorthand.product'
    NotificationConfiguration:
      LambdaConfigurations:
      - 
        Function: !Sub 'arn:aws:lambda:$AWS::Region:$AWS::AccountId:function:function name'
        Event: "s3:ObjectCreated:*"
        Filter:
          S3Key:
            Rules:
            - 
              Name: suffix
              Value: .json

LambdaInvokePermission:
  Type: AWS::Lambda::Permission
  Properties:
    FunctionName: !Sub 'arn:aws:lambda:$AWS::Region:$AWS::AccountId:function:function name '
    Action: "lambda:InvokeFunction"
    Principal: "s3.amazonaws.com"
    SourceArn: !Sub 'arn:aws:s3:::$EnvironmentNameShorthand.product'

【讨论】:

是的,会的。但是它会在每个对象创建通知上触发 lambda 函数。如果你想触发基于s3键前缀或后缀过滤器的lambda,你需要按照上面Kanniyan发布的答案【参考方案4】:

这是一个详细的答案(来自https://medium.com/@windix/s3-bucket-notification-to-lambda-in-cloudformation-without-circular-reference-f8f56ec5342c)

AWSTemplateFormatVersion: '2010-09-09'
Description: Example Stack

Parameters:
  BucketName:
    Type: String
    Default: unique-bucket-name

Resources:
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      ...
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: 's3:ObjectCreated:*'
            Filter:
              S3Key:
                Rules:
                  - Name: prefix
                    Value: test/
                  - Name: suffix
                    Value: .txt
            Function: !GetAtt Lambda.Arn
  
  Lambda:
    Type: AWS::Lambda::Function
    ...

  S3InvokeLambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref Lambda
      Principal: s3.amazonaws.com
      SourceArn: !Sub arn:aws:s3:::$BucketName

  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: '/'
      Policies:
      - PolicyName: s3
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
              - s3:Get*
            Resource:
              - !Sub arn:aws:s3:::$BucketName
              - !Sub arn:aws:s3:::$BucketName/*

【讨论】:

虽然此链接可能会回答问题,但最好在此处包含答案的基本部分并提供链接以供参考。如果链接页面发生更改,仅链接答案可能会失效。 - From Review @Terru_theTerror,感谢您的评论,我已经更新了我的答案。

以上是关于Cloudformation 模板在 S3 事件上触发 Lambda的主要内容,如果未能解决你的问题,请参考以下文章

使用 cloudformation 为 S3 存储桶启用 Lambda 函数

CloudFormation 模板设置 S3 存储桶默认加密 [重复]

AWS Cloudformation 模板 - 在 S3 存储桶中设置区域

如何使用 Cloud Formation 模板在 S3 存储桶上设置 SSE-S3 或 SSE-KMS 加密?

无法在 cloudformation 中创建带有事件的 s3 存储桶,以连接到它

Ansible CloudFormation模块无法查看S3对象