python 用于流动和转换流日志的Lambda函数。此示例用于HP ArcSight。可以找到完整的文章

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了python 用于流动和转换流日志的Lambda函数。此示例用于HP ArcSight。可以找到完整的文章相关的知识,希望对你有一定的参考价值。

import boto3
import logging
import json
import gzip
import urllib
import time
from StringIO import StringIO

logger = logging.getLogger()
logger.setLevel(logging.INFO)

s3 = boto3.client('s3')

def lambda_handler(event, context):
    
    #set the name of the S3 bucket
    bucketS3 = 'test-flowlogs'
    folderS3 = 'ArcSight'
    prefixS3 = 'AW1Logs_'
    
    #capture the CloudWatch log data
    outEvent = str(event['awslogs']['data'])
    
    #decode and unzip the log data
    outEvent = gzip.GzipFile(fileobj=StringIO(outEvent.decode('base64','strict'))).read()
    
    #convert the log data from JSON into a dictionary
    cleanEvent = json.loads(outEvent)
    
    #create a temp file
    tempFile = open('/tmp/file', 'w+')
    
    #Create the S3 file key
    key = folderS3 + '/' + prefixS3 + str(int(time.time())) + ".log"
    
    #loop through the events line by line
    for t in cleanEvent['logEvents']:
        
        #Transform the data and store it in the temp file. 
        tempFile.write("CEF:0|AWS CloudWatch|FlowLogs|1.0|src=" + str(t['extractedFields']['srcaddr']) + "|spt=" + str(t['extractedFields']['srcport']) + "|dst=" + str(t['extractedFields']['dstaddr']) + "|dpt=" + str(t['extractedFields']['dstport'])+ "|proto=" + str(t['extractedFields']['protocol'])+ "|start=" + str(t['extractedFields']['start'])+ "|end=" + str(t['extractedFields']['end'])+ "|out=" + str(t['extractedFields']['bytes'])+"\n")

    #close the temp file
    tempFile.close()    
    
    #write the files to s3
    s3Results = s3.upload_file('/tmp/file', bucketS3, key)
    print s3Results

以上是关于python 用于流动和转换流日志的Lambda函数。此示例用于HP ArcSight。可以找到完整的文章的主要内容,如果未能解决你的问题,请参考以下文章

当我使用无服务器运行 lambda 时未创建日志流

访问 AWS Lambda 的 Amazon CloudWatch 日志

Python Lambda函数的几种使用方法

用于错误检查的转换函数算不算好?

Python Lambda 函数解析 DynamoDB 的 JSON 格式

Kappa: Lambda 架构的质疑