无法从弹性 beanstalk/ec2 服务器上传媒体文件

Posted 2023-03-04

【中文标题】无法从弹性 beanstalk/ec2 服务器上传媒体文件

【英文标题】：Can't upload media files from elastic beanstalk/ec2 server

【发布时间】：2017-02-05 08:55:45

【问题描述】：

嘿，我一直被一个问题困住了一段时间，我可以将媒体文件从本地主机上传到 AWS S3，但是当我部署到 EB/EC2 时，上传媒体文件时出现 504 网关超时。在我的开发服务器上，上传工作正常，并且存储在 s3 中没有任何问题。

我目前正在使用django-storages，但我已经尝试过django-storages-redux 和django-s3-storages，我遇到了同样的问题，它在我的本地主机上工作，但在我部署的服务器上，我得到了 504 网关超时并且文件不是上传到 s3。

我将设置文件分解为 dev 和 prod。在我的产品设置中，我有以下内容：

# S3 FILE SETTINGS
AWS_STORAGE_BUCKET_NAME = 'project-assets'
AWS_S3_CUSTOM_DOMAIN = 'cdn.domain.com'
ASSETS_PREFIX = 'app' # prefix in the bucket
MEDIA_PREFIX = ASSETS_PREFIX + '/uploads/'
DEFAULT_FILE_STORAGE = 'core.storages.MediaStorage'
STATICFILES_STORAGE = 'core.storages.StaticStorage'

为了测试开发，我只需将这些设置移动到我的开发设置的底部。

我有这些自定义前缀将其存储在 s3 上的特定目录中。

# storages.py
from django.conf import settings

from storages.backends.s3boto import S3BotoStorage


class StaticStorage(S3BotoStorage):
    def __init__(self, *args, **kwargs):
        kwargs['location'] = settings.ASSETS_PREFIX
        return super(StaticStorage, self).__init__(*args, **kwargs)


class MediaStorage(S3BotoStorage):
    def __init__(self, *args, **kwargs):
        kwargs['location'] = settings.MEDIA_PREFIX
        return super(MediaStorage, self).__init__(*args, **kwargs)

我之前在其他项目中遇到过此错误，通常与 AWS_REGION 有关。我也尝试过使用该设置，但结果相同。我的 S3 存储在“us-west-2”（默认）中，据我所知，我不需要指定区域。

这是我尝试在已部署的实例上上传时出现在我的日志中的堆栈跟踪。

Traceback (most recent call last):
File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner
[Tue Sep 27 16:01:31.991596 2016] [:error] [pid 13870]     response = get_response(request)
[Tue Sep 27 16:01:31.991598 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response
[Tue Sep 27 16:01:31.991600 2016] [:error] [pid 13870]     response = self.process_exception_by_middleware(e, request)
[Tue Sep 27 16:01:31.991602 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response
[Tue Sep 27 16:01:31.991604 2016] [:error] [pid 13870]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Tue Sep 27 16:01:31.991606 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
[Tue Sep 27 16:01:31.991608 2016] [:error] [pid 13870]     return view_func(*args, **kwargs)
[Tue Sep 27 16:01:31.991609 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/viewsets.py", line 87, in view
[Tue Sep 27 16:01:31.991619 2016] [:error] [pid 13870]     return self.dispatch(request, *args, **kwargs)
[Tue Sep 27 16:01:31.991621 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/views.py", line 474, in dispatch
[Tue Sep 27 16:01:31.991623 2016] [:error] [pid 13870]     response = self.handle_exception(exc)
[Tue Sep 27 16:01:31.991625 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/views.py", line 434, in handle_exception
[Tue Sep 27 16:01:31.991626 2016] [:error] [pid 13870]     self.raise_uncaught_exception(exc)
[Tue Sep 27 16:01:31.991628 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/views.py", line 471, in dispatch
[Tue Sep 27 16:01:31.991630 2016] [:error] [pid 13870]     response = handler(request, *args, **kwargs)
[Tue Sep 27 16:01:31.991631 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/mixins.py", line 21, in create
[Tue Sep 27 16:01:31.991633 2016] [:error] [pid 13870]     self.perform_create(serializer)
[Tue Sep 27 16:01:31.991634 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/mixins.py", line 26, in perform_create
[Tue Sep 27 16:01:31.991636 2016] [:error] [pid 13870]     serializer.save()
[Tue Sep 27 16:01:31.991638 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/serializers.py", line 192, in save
[Tue Sep 27 16:01:31.991639 2016] [:error] [pid 13870]     self.instance = self.create(validated_data)
[Tue Sep 27 16:01:31.991641 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/rest_framework/serializers.py", line 873, in create
[Tue Sep 27 16:01:31.991643 2016] [:error] [pid 13870]     instance = ModelClass.objects.create(**validated_data)
[Tue Sep 27 16:01:31.991644 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/manager.py", line 85, in manager_method
[Tue Sep 27 16:01:31.991646 2016] [:error] [pid 13870]     return getattr(self.get_queryset(), name)(*args, **kwargs)
[Tue Sep 27 16:01:31.991659 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/query.py", line 399, in create
[Tue Sep 27 16:01:31.991661 2016] [:error] [pid 13870]     obj.save(force_insert=True, using=self.db)
[Tue Sep 27 16:01:31.991663 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/base.py", line 796, in save
[Tue Sep 27 16:01:31.991665 2016] [:error] [pid 13870]     force_update=force_update, update_fields=update_fields)
[Tue Sep 27 16:01:31.991666 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/base.py", line 824, in save_base
[Tue Sep 27 16:01:31.991668 2016] [:error] [pid 13870]     updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
[Tue Sep 27 16:01:31.991670 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/base.py", line 908, in _save_table
[Tue Sep 27 16:01:31.991672 2016] [:error] [pid 13870]     result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
[Tue Sep 27 16:01:31.991673 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/base.py", line 947, in _do_insert
[Tue Sep 27 16:01:31.991675 2016] [:error] [pid 13870]     using=using, raw=raw)
[Tue Sep 27 16:01:31.991677 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/manager.py", line 85, in manager_method
[Tue Sep 27 16:01:31.991679 2016] [:error] [pid 13870]     return getattr(self.get_queryset(), name)(*args, **kwargs)
[Tue Sep 27 16:01:31.991680 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/query.py", line 1045, in _insert
[Tue Sep 27 16:01:31.991682 2016] [:error] [pid 13870]     return query.get_compiler(using=using).execute_sql(return_id)
[Tue Sep 27 16:01:31.991686 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1053, in execute_sql
[Tue Sep 27 16:01:31.991688 2016] [:error] [pid 13870]     for sql, params in self.as_sql():
[Tue Sep 27 16:01:31.991690 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1006, in as_sql
[Tue Sep 27 16:01:31.991692 2016] [:error] [pid 13870]     for obj in self.query.objs
[Tue Sep 27 16:01:31.991693 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1006, in <listcomp>
[Tue Sep 27 16:01:31.991695 2016] [:error] [pid 13870]     for obj in self.query.objs
[Tue Sep 27 16:01:31.991697 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1005, in <listcomp>
[Tue Sep 27 16:01:31.991699 2016] [:error] [pid 13870]     [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
[Tue Sep 27 16:01:31.991701 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 955, in pre_save_val
[Tue Sep 27 16:01:31.991702 2016] [:error] [pid 13870]     return field.pre_save(obj, add=True)
[Tue Sep 27 16:01:31.991704 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/fields/files.py", line 292, in pre_save
[Tue Sep 27 16:01:31.991706 2016] [:error] [pid 13870]     file.save(file.name, file, save=False)
[Tue Sep 27 16:01:31.991708 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/db/models/fields/files.py", line 91, in save
[Tue Sep 27 16:01:31.991709 2016] [:error] [pid 13870]     self.name = self.storage.save(name, content, max_length=self.field.max_length)
[Tue Sep 27 16:01:31.991711 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/files/storage.py", line 54, in save
[Tue Sep 27 16:01:31.991713 2016] [:error] [pid 13870]     return self._save(name, content)
[Tue Sep 27 16:01:31.991714 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/storages/backends/s3boto.py", line 413, in _save
[Tue Sep 27 16:01:31.991716 2016] [:error] [pid 13870]     self._save_content(key, content, headers=headers)
[Tue Sep 27 16:01:31.991718 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/storages/backends/s3boto.py", line 424, in _save_content
[Tue Sep 27 16:01:31.991720 2016] [:error] [pid 13870]     rewind=True, **kwargs)
[Tue Sep 27 16:01:31.991721 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/s3/key.py", line 1293, in set_contents_from_file
[Tue Sep 27 16:01:31.991723 2016] [:error] [pid 13870]     chunked_transfer=chunked_transfer, size=size)
[Tue Sep 27 16:01:31.991725 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/s3/key.py", line 750, in send_file
[Tue Sep 27 16:01:31.991727 2016] [:error] [pid 13870]     chunked_transfer=chunked_transfer, size=size)
[Tue Sep 27 16:01:31.991728 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/s3/key.py", line 951, in _send_file_internal
[Tue Sep 27 16:01:31.991730 2016] [:error] [pid 13870]     query_args=query_args
[Tue Sep 27 16:01:31.991732 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/s3/connection.py", line 668, in make_request
[Tue Sep 27 16:01:31.991734 2016] [:error] [pid 13870]     retry_handler=retry_handler
[Tue Sep 27 16:01:31.991735 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/connection.py", line 1071, in make_request
[Tue Sep 27 16:01:31.991737 2016] [:error] [pid 13870]     retry_handler=retry_handler)
[Tue Sep 27 16:01:31.991739 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/connection.py", line 1030, in _mexe
[Tue Sep 27 16:01:31.991743 2016] [:error] [pid 13870]     raise ex
[Tue Sep 27 16:01:31.991744 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/connection.py", line 940, in _mexe
[Tue Sep 27 16:01:31.991746 2016] [:error] [pid 13870]     request.body, request.headers)
[Tue Sep 27 16:01:31.991748 2016] [:error] [pid 13870]   File "/opt/python/run/venv/lib/python3.4/site-packages/boto/s3/key.py", line 844, in sender
[Tue Sep 27 16:01:31.991750 2016] [:error] [pid 13870]     http_conn.send(chunk)
[Tue Sep 27 16:01:31.991751 2016] [:error] [pid 13870]   File "/usr/lib64/python3.4/http/client.py", line 917, in send
[Tue Sep 27 16:01:31.991753 2016] [:error] [pid 13870]     self.sock.sendall(data)
[Tue Sep 27 16:01:31.991755 2016] [:error] [pid 13870]   File "/usr/lib64/python3.4/ssl.py", line 723, in sendall
[Tue Sep 27 16:01:31.991756 2016] [:error] [pid 13870]     v = self.send(data[count:])
[Tue Sep 27 16:01:31.991758 2016] [:error] [pid 13870]   File "/usr/lib64/python3.4/ssl.py", line 684, in send
[Tue Sep 27 16:01:31.991760 2016] [:error] [pid 13870]     v = self._sslobj.write(data)
[Tue Sep 27 16:01:31.991763 2016] [:error] [pid 13870] ConnectionResetError: [Errno 104] Connection reset by peer

我认为 ssl 可能有问题（一切都在 https 上）并尝试设置 AWS_S3_SECURE_URLS = True 设置但结果相同。

我也试过关闭 https 重定向并通过 http:// 上传并再次得到相同的结果。

欢迎任何想法，提前谢谢！

【问题讨论】：

附带说明，您可以让浏览器直接上传到 S3 并减少服务器上的负载。对于用户来说，它也会更快。 ***.com/questions/9421727/…

【参考方案1】：

嗯，正如预期的那样，这很简单。我没有将我的 id/key 加载到环境变量中。