从谷歌云存储获取状态 503

Posted

技术标签:

【中文标题】从谷歌云存储获取状态 503【英文标题】：getting status 503 from google cloud storage 【发布时间】：2015-07-09 09:27:53 【问题描述】：

我在使用默认存储桶将图像上传到新创建的应用引擎应用中的谷歌云存储时遇到问题。

我使用的上传代码在部署到已经具有活动默认存储桶的旧版应用引擎应用时运行良好。当我在 appengine.google.com 创建一个新的应用引擎项目或激活默认 gcs 存储桶时，我会收到一个奇怪的 500 (503) 错误。

我的上传代码使用了 cloudstorage 客户端，如下所示：

def save_to_gcs(file_obj, obj):
    # pass a file and a datastore entity to associate the file to

    serving_url = ''#just assign it adn reassign later

    time_stamp = int(time.time())
    app_id = app_identity.get_application_id()

    logging.error("got app_id .... %s" % app_id)

    fname = '/%s.appspot.com/post_%s_%s.jpg' % (app_id, obj.key.id(), time_stamp)

    # logging.error(file_obj)
    logging.error("fname .... %s" % fname)

    gcs_file = gcs.open(fname, 'w', content_type="image/jpeg")
    gcs_file.write(file_obj)
    logging.error("written file obj ....")# <-- this is where the logs stop
    gcs_file.close()

    gcs_filename = "/gs%s" % fname
    serving_url = images.get_serving_url(blobstore.create_gs_key(gcs_filename))
    logging.error("got serving url ....")
    media_obj = save_gcs_to_media(gcs_filename, serving_url, obj)
    logging.error("saved media obj ....")

    return media_obj

我认为错误与 file.close() 方法有关，并且可能与默认存储桶有关。

我通过了正确的身份验证，我上传的文件大约 100kb，所以没什么大不了的。

任何帮助都将不胜感激（事实上它可以防止我的头发先变白，然后脱落）。

对不起，这里是日志：

E 11:05:54.412 - - - - COVER - - - -

E 11:05:54.413 True

E 11:05:54.413 got time stamp

E 11:05:54.413 got time file name

E 11:05:54.413 got app_id .... emile-test

E 11:05:54.413 fname .... /emile-test.appspot.com/post_5629499534213120_1436432754.jpg

E 11:05:54.696 written file obj ....

D 11:05:55.025 Tasklet is <bound method _StorageApi.urlfetch_async of <cloudstorage.storage_api._StorageApi object at 0xfbd7bf10>>

D 11:05:55.025 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfbd9a810> from tasklet.

D 11:05:55.025 Retry in 0.1 seconds.

D 11:05:55.477 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfbd9a9b0> from tasklet.

D 11:05:55.478 Retry in 0.2 seconds.

D 11:05:56.152 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfbd9ab30> from tasklet.

D 11:05:56.153 Retry in 0.4 seconds.

D 11:05:57.244 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfbd9acb0> from tasklet.

D 11:05:57.245 Retry in 0.8 seconds.

D 11:05:58.435 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfbd7b690> from tasklet.

D 11:05:58.435 Retry in 1.6 seconds.

D 11:06:00.542 Got result <google.appengine.api.urlfetch._URLFetchResult object at 0xfe4ce630> from tasklet.

D 11:06:00.542 Retry in 3.2 seconds.

D 11:06:04.179 Tasklet failed after 7 attempts and 9.48189997673 seconds in total

**E 11:06:04.180 Expect status [200] from Google Storage. But got status 503.**

  Path: '/emile-test.appspot.com/post_5629499534213120_1436432754.jpg'.
  Request headers: 'content-range': 'bytes 0-108067/108068', 'x-goog-api-version': '2', 'accept-encoding': 'gzip, *'.

  Response headers: 'content-length': '0', 'x-guploader-uploadid': 'AEnB2UqzyZwtoNSy8sTEUq7a8fai-A5aDVlMccCj1UVEo72NAUUw4kyDotehcf8RDusUzm52092rdO0CR9JxIC9ba1J5Kj1xuw'.
  Body: ''.

  Extra info: 'upload_path': '/emile-test.appspot.com/post_5629499534213120_1436432754.jpg?upload_id=AEnB2UqzyZwtoNSy8sTEUq7a8fai-A5aDVlMccCj1UVEo72NAUUw4kyDotehcf8RDusUzm52092rdO0CR9JxIC9ba1J5Kj1xuw'.

  Traceback (most recent call last):
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
      rv = self.handle_exception(request, response, e)

File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
      rv = self.router.dispatch(request, response)

File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
      return route.handler_adapter(request, response)

File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
      return handler.dispatch()

File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
      return self.handle_exception(e, self.app.debug)

File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
      return method(*args, **kwargs)

File "/base/data/home/apps/s~emile-test/1.385589451941224330/main.py", line 757, in post
      media_obj = utils.save_to_gcs(image, post)

File "/base/data/home/apps/s~emile-test/1.385589451941224330/utils.py", line 131, in save_to_gcs
      gcs_file.close()

**File "/base/data/home/apps/s~emile-test/1.385589451941224330/cloudstorage/storage_api.py", line 750, in close**
      self._flush(finish=True)
    File "/base/data/home/apps/s~emile-test/1.385589451941224330/cloudstorage/storage_api.py", line 804, in _flush
      self._send_data(data, self._written, file_len)

File "/base/data/home/apps/s~emile-test/1.385589451941224330/cloudstorage/storage_api.py", line 837, in _send_data
      'upload_path': self._path_with_token)

File "/base/data/home/apps/s~emile-test/1.385589451941224330/cloudstorage/errors.py", line 141, in check_status
      raise ServerError(msg)
  ServerError: Expect status [200] from Google Storage. But got status 503.

  Path: '/emile-test.appspot.com/post_5629499534213120_1436432754.jpg'.
  Request headers: 'content-range': 'bytes 0-108067/108068', 'x-goog-api-version': '2', 'accept-encoding': 'gzip, *'.

  Response headers: 'content-length': '0', 'x-guploader-uploadid': 'AEnB2UqzyZwtoNSy8sTEUq7a8fai-A5aDVlMccCj1UVEo72NAUUw4kyDotehcf8RDusUzm52092rdO0CR9JxIC9ba1J5Kj1xuw'.
  Body: ''.

  Extra info: 'upload_path': '/emile-test.appspot.com/post_5629499534213120_1436432754.jpg?upload_id=AEnB2UqzyZwtoNSy8sTEUq7a8fai-A5aDVlMccCj1UVEo72NAUUw4kyDotehcf8RDusUzm52092rdO0CR9JxIC9ba1J5Kj1xuw'.

【问题讨论】：

...我想没有其他人有这个问题，我真的在做蠢事吗？ 【参考方案1】：

我今天在我自己的 GAE 项目中完全解决了这个问题（相同的堆栈跟踪）（我正在从 blobstore 中的 Files API 迁移到 cloudstorage）。我采取的步骤是：

    前往 Google Developers Console

    https://console.developers.google.com/project/YOUR_APP_ID/storage/browser

    其中 YOUR_APP_ID 是不带 .appspot.com 后缀的应用名称，例如“foo”，而不是“foo.appspot.com”，

    点击我项目右侧的三个垂直点

    选定的存储桶权限

    将我的电子邮件地址添加为 Entity=User、name=myemail@gmail.com、Access=Owner。

我不是 100% 确定为什么这能解决问题，但它成功了。

【讨论】：

你是我的英雄，谢谢一百万！我想知道发生了什么事让这么突然被要求。不管怎样，我很佩服你发现这一点的推理技巧。

以上是关于从谷歌云存储获取状态 503的主要内容，如果未能解决你的问题，请参考以下文章

从谷歌云存储中的 csv 加载数据作为 bigquery 'in' 查询

无法使用 python 将 JSON 文件从谷歌云存储加载到 bigquery

如何在谷歌大查询中从谷歌云存储上传表格

使用node.js中的firebase从谷歌云存储读取时，ESlint问题“每个都应该返回一个值或抛出”

从谷歌云数据存储迁移到谷歌云 sql

使用python从谷歌云存储桶中删除数据时出错