Python使用boto3操作AWS S3中踩过的坑

Posted 2020-08-19

 

最近在AWS上开发部署应用。

看了这篇关于AWS中国区填坑的文章，结合自己使用AWS的经历，补充两个我自己填的坑。

http://www.jianshu.com/p/0d0fd39a40c9?utm_source=tuicool&utm_medium=referral

 

1. V4 签名认证

官方文档中给出的例子：

import boto3
s3 = boto3.resource(‘s3‘)
s3.meta.client.upload_file(‘/tmp/hello.txt‘, ‘mybucket‘, ‘hello.txt‘)

运行之后会出现如下错误：

An error occurred (InvalidRequest) when calling the GetBucketLifecycle operation: Missing required header for this request: x-amz-content-sha256

 

在文档中找到如下的解释。概括讲就是以后AWS S3 要使用第四版的签名认证了。尤其是亚太地区新建的服务器。

========================

Protect against reuse of the signed portions of the request – The signed portions (using AWS
Signatures) of requests are valid within 15 minutes of the timestamp in the request. An unauthorized
party who has access to a signed request can modify the unsigned portions of the request without
affecting the request‘s validity in the 15 minute window. Because of this, we recommend that you
maximize protection by signing request headers and body, making HTTPS requests to Amazon S3,
and by using the s3:x-amz-content-sha256 condition key (see Amazon S3 Signature Version 4
Authentication Specific Policy Keys (p. 50)) in AWS policies to require users to sign S3 request bodies.
Note
Amazon S3 supports Signature Version 4, a protocol for authenticating inbound API requests
to AWS services, in all AWS regions. At this time, AWS regions created before January 30, 2014
will continue to support the previous protocol, Signature Version 2. Any new regions after January
30, 2014 will support only Signature Version 4 and therefore all requests to those regions must
be made with Signature Version 4. For more information about AWS Signature Version 2, see
Signing and Authenticating REST Requests in the Amazon Simple Storage Service Developer
Guide.

======================

但是坑爹的是，他没有告诉怎么添加这个header。

好在boto3是Python API，直接去源码中找答案。

import boto3
from botocore.client import Config

s3 = boto3.resource(‘s3‘, config=Config(signature_version=‘s3v4‘))
s3.meta.client.upload_file(‘/tmp/hello.txt‘, ‘mybucket‘, ‘hello.txt‘)

 

 

2. 区域命名

使用过AWS的都应该知道，需要选择服务器所在区域，即region。

但是在boto所提供的文档中，却没有提供区域与region的对照。

例如：你选择亚太地区（首尔）-- Asia Pacific (Seoul)， 但如果你设置region为‘Seoul’，就会有‘’access abort‘’的错误。

如果要设置region，请参考下表

AWS区域设置对照表

us-east-1	美国东部（弗吉尼亚北部）	US East (N. Virginia)
us-west-1	美国西部（加利福尼亚北部）	US West (N. California)
us-west-2	美国西部（俄勒冈）	US West (Oregon)
ap-northeast-1	亚太地区（东京）	Asia Pacific (Tokyo)
ap-southeast-1	亚太地区（新加坡）	Asia Pacific (Singapore)
ap-southeast-2	亚太地区（悉尼）	Asia Pacific (Sydney)
ap-northeast-2	亚太地区（首尔）	Asia Pacific (Seoul)
eu-west-1	欧洲（爱尔兰）	EU (Ireland)
eu-central-1	欧洲（法兰克福）	EU (Frankfurt)
sa-east-1	南美洲（圣保罗）	South America (Sao Paulo)
cn-north-1	中国（北京）	cn-north-1