错误 s3 put/get ops with localstack:`The Content-MD5 you specified was invalid when execution`

Posted

技术标签:

【中文标题】错误 s3 put/get ops with localstack:`The Content-MD5 you specified was invalid when execution`【英文标题】:Error s3 put/get ops with localstack: `The Content-MD5 you specified was invalid when performing` 【发布时间】:2020-08-31 23:59:48 【问题描述】:

我最近在执行 s3 put 或 get 操作时将 testcontainers localstack 版本更新为1.12.5 后出现错误。 Content-MD5 看起来无效。

所以我发现 s3 完整性验证存在问题:https://github.com/localstack/localstack/issues/869

因此,由于某种原因,校验和数据似乎不匹配。

错误:

com.amazonaws.services.s3.model.AmazonS3Exception: The Content-MD5 you specified was invalid (Service: Amazon S3; Status Code: 400; Error Code: InvalidDigest; Request ID: null; S3 Extended Request ID: null)
, S3 Extended Request ID: null
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1632)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
    at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1755)

【问题讨论】:

【参考方案1】:

这个问题可以通过在你的类的顶部添加禁用两个不同的属性来解决:

对于 GET 操作
    static 
        System.setProperty(SkipMd5CheckStrategy.DISABLE_GET_OBJECT_MD5_VALIDATION_PROPERTY, "true");
    
对于 PUT 操作:
    static 
        System.setProperty(SkipMd5CheckStrategy.DISABLE_PUT_OBJECT_MD5_VALIDATION_PROPERTY, "true");
    

【讨论】:

以上是关于错误 s3 put/get ops with localstack:`The Content-MD5 you specified was invalid when execution`的主要内容,如果未能解决你的问题,请参考以下文章

R中S3类的重载矩阵乘法

VS2017git 提交提示错误 Git failed with a fatal error.

如何在 hive 脚本中将函数作为 s3 位置的输入

使用 `ActiveRecord with_connection do` 和 ActionController::Live 时出现线程错误

(python) Spark .textFile(s3://...) access denied 403 with valid credentials

delete post put get请求参数