• Python使用boto3操作AWS S3中踩过的坑


    最近在AWS上开发部署应用。

    看了这篇关于AWS中国区填坑的文章,结合自己使用AWS的经历,补充两个我自己填的坑。

    http://www.jianshu.com/p/0d0fd39a40c9?utm_source=tuicool&utm_medium=referral

    1. V4 签名认证

    官方文档中给出的例子:

    import boto3
    s3 = boto3.resource('s3')
    s3.meta.client.upload_file('/tmp/hello.txt', 'mybucket', 'hello.txt')

    运行之后会出现如下错误:

    An error occurred (InvalidRequest) when calling the GetBucketLifecycle operation: Missing required header for this request: x-amz-content-sha256

    在文档中找到如下的解释。概括讲就是以后AWS S3 要使用第四版的签名认证了。尤其是亚太地区新建的服务器。

    ========================

    Protect against reuse of the signed portions of the request – The signed portions (using AWS
    Signatures) of requests are valid within 15 minutes of the timestamp in the request. An unauthorized
    party who has access to a signed request can modify the unsigned portions of the request without
    affecting the request's validity in the 15 minute window. Because of this, we recommend that you
    maximize protection by signing request headers and body, making HTTPS requests to Amazon S3,
    and by using the s3:x-amz-content-sha256 condition key (see Amazon S3 Signature Version 4
    Authentication Specific Policy Keys (p. 50)) in AWS policies to require users to sign S3 request bodies.
    Note
    Amazon S3 supports Signature Version 4, a protocol for authenticating inbound API requests
    to AWS services, in all AWS regions. At this time, AWS regions created before January 30, 2014
    will continue to support the previous protocol, Signature Version 2. Any new regions after January
    30, 2014 will support only Signature Version 4 and therefore all requests to those regions must
    be made with Signature Version 4. For more information about AWS Signature Version 2, see
    Signing and Authenticating REST Requests in the Amazon Simple Storage Service Developer
    Guide.

    ======================

    但是坑爹的是,他没有告诉怎么添加这个header。

    好在boto3是Python API,直接去源码中找答案。

    import boto3
    from botocore.client import Config
    
    s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
    s3.meta.client.upload_file('/tmp/hello.txt', 'mybucket', 'hello.txt')

    2. 区域命名

    使用过AWS的都应该知道,需要选择服务器所在区域,即region。

    但是在boto所提供的文档中,却没有提供区域与region的对照。

    例如:你选择亚太地区(首尔)-- Asia Pacific (Seoul), 但如果你设置region为‘Seoul’,就会有‘’access abort‘’的错误。

    如果要设置region,请参考下表

    AWS区域设置对照表

    us-east-1 美国东部(弗吉尼亚北部) US East (N. Virginia)
    us-west-1 美国西部(加利福尼亚北部) US West (N. California)
    us-west-2 美国西部(俄勒冈) US West (Oregon)
    ap-northeast-1 亚太地区(东京) Asia Pacific (Tokyo)
    ap-southeast-1 亚太地区(新加坡) Asia Pacific (Singapore)
    ap-southeast-2 亚太地区(悉尼) Asia Pacific (Sydney)
    ap-northeast-2 亚太地区(首尔) Asia Pacific (Seoul)
    eu-west-1 欧洲(爱尔兰) EU (Ireland)
    eu-central-1 欧洲(法兰克福) EU (Frankfurt)
    sa-east-1 南美洲(圣保罗) South America (Sao Paulo)
    cn-north-1 中国(北京) cn-north-1
  • 相关阅读:
    查询计划Hash和查询Hash
    执行计划的重用
    执行计划组件、组件、老化
    执行计划的生成
    查询反模式
    T-SQL 公用表表达式(CTE)
    SQL 操作结果集 -并集、差集、交集、结果集排序
    SQL语句
    POJ 1821 单调队列+dp
    区间gcd问题 HDU 5869 离线+树状数组
  • 原文地址:https://www.cnblogs.com/shizouwei/p/6053809.html
Copyright © 2020-2023  润新知