Python - AWS Lambda 和 exchangeelib 的多处理问题

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Python - AWS Lambda 和 exchangeelib 的多处理问题相关的知识,希望对你有一定的参考价值。

当我使用我的代码部署在使用交换elib库的AWS Lambda函数上时,我有一个问题。我的代码在本地工作得很好。

可能是什么原因呢?

从我了解的情况来看,它看起来是对多处理,这不会在AWS Lambda处理。在这种情况下,我应该怎么做才能访问我邮箱中一个文件夹的内容?

谅谅

我的代码样本。

from exchangelib import Account, CalendarItem, Message, Mailbox, FileAttachment, HTMLBody, Credentials, DELEGATE, Configuration, IMPERSONATION, FaultTolerance

class ExchangeIdentification:
    def __init__(self, username: str = settings.exchange_email_login, password: str = settings.exchange_email_password,
                 email_sender_address: str = settings.exchange_email_login):
        credentials = Credentials(username, password)
        config = Configuration(service_endpoint='https://outlook.office365.com/EWS/Exchange.asmx',
                               credentials=credentials, auth_type='basic')
        self.account = Account(primary_smtp_address=username, autodiscover=False, config=config,
                               access_type=DELEGATE)
        self.email_sender_address = email_sender_address

def preprocess_file_from_email():
    exchange = ExchangeIdentification()
    folder = exchange.account.root / 'Top of Information Store' / 'my_folder'
    # code continues but breaks at the line above

以及AWS Lambda的错误输出。

[ERROR] 2020-05-11T18:42:06.619Z    d67670a2-865d-4ce1-909e-a4a441c3fdf0    Exception in ASGI application

Traceback (most recent call last):
  File "/var/task/mangum/protocols/http.py", line 39, in run
    await app(self.scope, self.receive, self.send)
  File "/var/task/fastapi/applications.py", line 149, in __call__
    await super().__call__(scope, receive, send)
  File "/var/task/starlette/applications.py", line 102, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/var/task/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/var/task/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/var/task/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/var/task/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/var/task/starlette/routing.py", line 550, in __call__
    await route.handle(scope, receive, send)
  File "/var/task/starlette/routing.py", line 227, in handle
    await self.app(scope, receive, send)
  File "/var/task/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/var/task/fastapi/routing.py", line 196, in app
    raw_response = await run_endpoint_function(
  File "/var/task/fastapi/routing.py", line 150, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/var/task/starlette/concurrency.py", line 34, in run_in_threadpool
    return await loop.run_in_executor(None, func, *args)
  File "/var/lang/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/var/task/app/sf2py/sf2py.py", line 175, in wrapper
    raise e
  File "/var/task/app/sf2py/sf2py.py", line 170, in wrapper
    result = func(*args, **kwargs)
  File "/var/task/app/quote/creation.py", line 202, in preprocess_file_from_email
    folder = exchange.account.root / 'Top of Information Store' / 'my_folder'
  File "/var/task/cached_property.py", line 73, in __get__
    return obj_dict.setdefault(name, self.func(obj))
  File "/var/task/exchangelib/account.py", line 238, in root
    return Root.get_distinguished(account=self)
  File "/var/task/exchangelib/folders/roots.py", line 106, in get_distinguished
    return cls.resolve(
  File "/var/task/exchangelib/folders/base.py", line 499, in resolve
    folders = list(FolderCollection(account=account, folders=[folder]).resolve())
  File "/var/task/exchangelib/folders/collections.py", line 254, in resolve
    for f in self.__class__(account=self.account, folders=resolveable_folders).get_folders(
  File "/var/task/exchangelib/folders/collections.py", line 316, in get_folders
    for f in GetFolder(account=self.account).call(
  File "/var/task/exchangelib/services/get_folder.py", line 29, in call
    for folder, elem in zip(folders_list, self._pool_requests(
  File "/var/task/exchangelib/services/common.py", line 519, in _pool_requests
    results.append((n, self.protocol.thread_pool.apply_async(
  File "/var/task/cached_property.py", line 73, in __get__
    return obj_dict.setdefault(name, self.func(obj))
  File "/var/task/exchangelib/protocol.py", line 429, in thread_pool
    return ThreadPool(processes=thread_poolsize)
  File "/var/lang/lib/python3.8/multiprocessing/pool.py", line 922, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "/var/lang/lib/python3.8/multiprocessing/pool.py", line 196, in __init__
    self._change_notifier = self._ctx.SimpleQueue()
  File "/var/lang/lib/python3.8/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/var/lang/lib/python3.8/multiprocessing/queues.py", line 336, in __init__
    self._rlock = ctx.Lock()
  File "/var/lang/lib/python3.8/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 38] Function not implemented
答案

今晚发布的 exchangeelib 3.2.0 的更新解决了这个问题(原来是 3.1.1 版本)。

以上是关于Python - AWS Lambda 和 exchangeelib 的多处理问题的主要内容,如果未能解决你的问题,请参考以下文章

Lambda,AWS和Python的自动化管理操作 - 自动创建快照,删除快照

AWS lambda 函数 python/pyspark

Python - AWS Lambda 和 exchangeelib 的多处理问题

AWS Rekognition和s3调用Python Lambda中的子文件夹

通过 Cloudformation、CodeBuild 和 CodePipeline 将 python 包部署到 AWS Lambda

AWS Lambda 自动化和 Python - 自动创建S3 Bucket lifecycle