AWS Lambda：将可执行文件与 python 一起使用

Posted 2023-02-16

【中文标题】AWS Lambda：将可执行文件与 python 一起使用

【英文标题】：AWS Lambda: Using executable with python

【发布时间】：2017-07-07 04:22:09

【问题描述】：

你好this tutorial 几乎准确地描述了我需要做的事情并很好地解释了它，但它对我不起作用。 我搜索了类似的案例，甚至发现其他人遇到了问题，但他们的解决方案对我没有帮助。

我稍微更改了文章中的更改代码，以便从同一个静态对象中提取元数据，因此我可以轻松地在 lambda 中进行测试，而不必每次都将新文件上传到 s3 来触发该功能。 我还添加了一些输出，表明我应用了其他人的解决方案，例如添加“shell=True”并确保文件确实是可执行的。

import logging
import subprocess
import boto3
import botocore.session

SIGNED_URL_EXPIRATION = 300     # The number of seconds that the Signed URL is valid
DYNAMODB_TABLE_NAME = "metadata_test_db"
DYNAMO = boto3.resource("dynamodb")
TABLE = DYNAMO.Table(DYNAMODB_TABLE_NAME)
logger = logging.getLogger('boto3')
logger.setLevel(logging.INFO)


def lambda_handler(event, context):
    """
    :param event:
    :param context:
    """

    # Loop through records provided by S3 Event trigger
    for s3_record in event['Records']:
        logger.info("Working on new s3_record...")
        # Extract the Key and Bucket names for the asset uploaded to S3
        key = s3_record['s3']['object']['key']
        bucket = s3_record['s3']['bucket']['name']
        logger.info("Bucket:  \t Key: ".format(bucket, key))
        # Generate a signed URL for the uploaded asset
        signed_url = get_signed_url(SIGNED_URL_EXPIRATION, bucket, key)
        logger.info("Signed URL: ".format(signed_url))

        # Launch MediaInfo
        # Pass the signed URL of the uploaded asset to MediaInfo as an input
        # MediaInfo will extract the technical metadata from the asset
        # The extracted metadata will be outputted in XML format and
        # stored in the variable xml_output
        out2 = subprocess.check_output(["ls", "-l", "mediainfo"])
        logger.info(out2)
        xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file"], shell=True)
        logger.info("Output: ".format(xml_output))
        #save_record(key, xml_output)

def save_record(key, xml_output):
    """
    Save record to DynamoDB

    :param key:         S3 Key Name
    :param xml_output:  Technical Metadata in XML Format
    :return:
    """
    logger.info("Saving record to DynamoDB...")
    TABLE.put_item(
       Item=
            'sryKey': key,
            'technicalMetadata': xml_output
        
    )
    logger.info("Saved record to DynamoDB")


def get_signed_url(expires_in, bucket, obj):
    """
    Generate a signed URL
    :param expires_in:  URL Expiration time in seconds
    :param bucket:
    :param obj:         S3 Key name
    :return:            Signed URL
    """
    s3_cli = boto3.client("s3")
    presigned_url = s3_cli.generate_presigned_url('get_object', Params='Bucket': bucket, 'Key': obj,
                                                  ExpiresIn=expires_in)
    return presigned_url

来自 lambda 的 cloudwatch 错误日志返回以下内容

13:27:13
START RequestId: cad3123b-f514-11e6-b8b1-45fa69956450 Version: $LATEST

13:27:13
[INFO]  2017-02-17T13:27:13.562Z    cad3123b-f514-11e6-b8b1-45fa69956450    Working on new s3_record...

13:27:13
[INFO]  2017-02-17T13:27:13.562Z    cad3123b-f514-11e6-b8b1-45fa69956450    Bucket: sourcebucket Key: HappyFace.jpg

13:27:13
[INFO]  2017-02-17T13:27:13.665Z    cad3123b-f514-11e6-b8b1-45fa69956450    Signed URL: https://s3.us-east-2.amazonaws.com/sourcebucket/HappyFace.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=300&X-Amz-Date=20170217T132713Z&X-Amz-SignedHeaders=host&X-Amz-Security-Token=FQoDYXdzEEMaDARR11pSPLcxi%2BkSZyL2AU2NjzOk37%2F2Ruwc33ZY5uN%2Ffg9O1c1awRcJf0qej4b29woEf%2BDhHsehkTr4WaKq19MxLL%2BdqmQBXArWXCYaGIcxdy

13:27:13
[INFO]  2017-02-17T13:27:13.683Z    cad3123b-f514-11e6-b8b1-45fa69956450    -rwxrwxrwx 1 slicer 497 9269401 Feb 17 09:51 mediainfo

13:27:13
Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255: CalledProcessError Traceback (most recent call last): File "/var/task/lambda_function.py", line 38, in lambda_handler xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file
Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255: CalledProcessError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 38, in lambda_handler
xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", "https://public-s3-file"], shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 574, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['./mediainfo', '--full', '--output=XML', 'https://public-s3-file']' returned non-zero exit status 255


13:27:13
END RequestId: cad3123b-f514-11e6-b8b1-45fa69956450

13:27:13
REPORT RequestId: cad3123b-f514-11e6-b8b1-45fa69956450  Duration: 127.38 ms Billed Duration: 200 ms Memory Size: 512 MB Max Memory Used: 32 MB

非常感谢任何提示。

【问题讨论】：

看起来和这个问题很相似***.com/questions/39113953/…

你可以尝试使用stderr=subprocess.STDOUT捕获结果中的标准错误吗？

【参考方案1】：

所以对我来说，解决方案是添加 stderr=subprocess.STDOUT，就像在我的问题的评论中建议的 here 一样，并设置 shell=False。 它不适用于 shell=True。 该提示来自a comment 到this answer 到一个类似的问题，它“神奇地”适用于此设置