Linux 上的 MATLAB MEX 文件无法在 libpython.2.7.so 中找到符号

Posted 2023-02-16

技术标签:

【中文标题】Linux 上的 MATLAB MEX 文件无法在 libpython.2.7.so 中找到符号【英文标题】：MATLAB MEX file on Linux fails to find symbols in libpython.2.7.so 【发布时间】：2013-08-30 21:09:29 【问题描述】：

我一直致力于 project 使用 libpython2.7.so 将 Python 解释器作为 MEX 文件嵌入到 MATLAB 中，但每当我尝试导入作为扩展模块实现的 Python 模块时都会遇到问题。例如，尝试导入 itertools 会导致以下错误：

>> py_import itertools
ImportError: /usr/lib64/python2.7/lib-dynload/itertoolsmodule.so: undefined symbol: PyTuple_Type
??? Error using ==> pymex_fns
Python exception inside py_import.

Error in ==> py_import at 24
    py_obj = pymex_fns(py_function_t.IMPORT, name);

无论我是否在调用 Py_Initialize() 之前清除 LD_LIBRARY_PATH 以及从 MATLAB 中调用 ldd（在此示例中为 itertools.so）不会导致任何 @987654332，都会出现此行为@消息。下面，我粘贴了设置LD_DEBUG=libs运行MATLAB的结果，首先是MATLAB的启动设置LD_LIBRARY_PATH，然后在尝试导入之前运行setenv('LD_LIBRARY_PATH', '')。

如何解决这个问题，并允许动态加载扩展模块？

使用 MATLAB-default LD_LIBRARY_PATH:

>> py_import itertools           
      3018: find library=libpython2.7.so.1.0 [0]; searching
      3018:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64      (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64/libpython2.7.so.1.0
      3018:  search path=/usr/local/MATLAB/R2011a/sys/os/glnxa64        (LD_LIBRARY_PATH)
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/os/glnxa64/libpython2.7.so.1.0
      3018:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64       (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libpython2.7.so.1.0
      3018:  search path=/usr/local/MATLAB/R2011a/extern/lib/glnxa64:/usr/local/MATLAB/R2011a/runtime/glnxa64:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64     (LD_LIBRARY_PATH)
      3018:   trying file=/usr/local/MATLAB/R2011a/extern/lib/glnxa64/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/runtime/glnxa64/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server/libpython2.7.so.1.0
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/libpython2.7.so.1.0
      3018:  search cache=/etc/ld.so.cache
      3018:   trying file=/lib64/libpython2.7.so.1.0
      3018: 
      3018: find library=libutil.so.1 [0]; searching
      3018:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64      (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64/libutil.so.1
      3018:  search path=/usr/local/MATLAB/R2011a/sys/os/glnxa64        (LD_LIBRARY_PATH)
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/os/glnxa64/libutil.so.1
      3018:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64       (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3018:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libutil.so.1
      3018:  search path=/usr/local/MATLAB/R2011a/extern/lib/glnxa64:/usr/local/MATLAB/R2011a/runtime/glnxa64:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64     (LD_LIBRARY_PATH)
      3018:   trying file=/usr/local/MATLAB/R2011a/extern/lib/glnxa64/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/runtime/glnxa64/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server/libutil.so.1
      3018:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/libutil.so.1
      3018:  search cache=/etc/ld.so.cache
      3018:   trying file=/lib64/libutil.so.1
      3018: 
      3018: 
      3018: calling init: /lib64/libutil.so.1
      3018: 
      3018: 
      3018: calling init: /lib64/libpython2.7.so.1.0
      3018: 
      3018: 
      3018: calling init: /home/cgranade/academics/software-projects/pymex-embed/src/pymex_fns.mexa64
      3018: 
      3018: /home/cgranade/academics/software-projects/pymex-embed/src/pymex_fns.mexa64: error: symbol lookup error: undefined symbol: mexLibrary (fatal)
      3018: /usr/lib64/python2.7/lib-dynload/itertoolsmodule.so: error: symbol lookup error: undefined symbol: PyTuple_Type (fatal)
??? Error using ==> pymex_fns
Python exception inside py_import.

Error in ==> py_import at 24
    py_obj = pymex_fns(py_function_t.IMPORT, name);

LD_LIBRARY_PATH 被清除：

>> py_import itertools           
      3125: find library=libpython2.7.so.1.0 [0]; searching
      3125:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64      (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64/libpython2.7.so.1.0
      3125:  search path=/usr/local/MATLAB/R2011a/sys/os/glnxa64        (LD_LIBRARY_PATH)
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/os/glnxa64/libpython2.7.so.1.0
      3125:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64       (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libpython2.7.so.1.0
      3125:  search path=/usr/local/MATLAB/R2011a/extern/lib/glnxa64:/usr/local/MATLAB/R2011a/runtime/glnxa64:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64     (LD_LIBRARY_PATH)
      3125:   trying file=/usr/local/MATLAB/R2011a/extern/lib/glnxa64/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/runtime/glnxa64/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server/libpython2.7.so.1.0
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/libpython2.7.so.1.0
      3125:  search cache=/etc/ld.so.cache
      3125:   trying file=/lib64/libpython2.7.so.1.0
      3125: 
      3125: find library=libutil.so.1 [0]; searching
      3125:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64:/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64      (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../bin/glnxa64/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/../../sys/os/glnxa64/libutil.so.1
      3125:  search path=/usr/local/MATLAB/R2011a/sys/os/glnxa64        (LD_LIBRARY_PATH)
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/os/glnxa64/libutil.so.1
      3125:  search path=/usr/local/MATLAB/R2011a/bin/glnxa64       (RPATH from file /usr/local/MATLAB/R2011a/bin/glnxa64/MATLAB)
      3125:   trying file=/usr/local/MATLAB/R2011a/bin/glnxa64/libutil.so.1
      3125:  search path=/usr/local/MATLAB/R2011a/extern/lib/glnxa64:/usr/local/MATLAB/R2011a/runtime/glnxa64:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64     (LD_LIBRARY_PATH)
      3125:   trying file=/usr/local/MATLAB/R2011a/extern/lib/glnxa64/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/runtime/glnxa64/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/server/libutil.so.1
      3125:   trying file=/usr/local/MATLAB/R2011a/sys/java/jre/glnxa64/jre/lib/amd64/libutil.so.1
      3125:  search cache=/etc/ld.so.cache
      3125:   trying file=/lib64/libutil.so.1
      3125: 
      3125: 
      3125: calling init: /lib64/libutil.so.1
      3125: 
      3125: 
      3125: calling init: /lib64/libpython2.7.so.1.0
      3125: 
      3125: 
      3125: calling init: /home/cgranade/academics/software-projects/pymex-embed/src/pymex_fns.mexa64
      3125: 
      3125: /home/cgranade/academics/software-projects/pymex-embed/src/pymex_fns.mexa64: error: symbol lookup error: undefined symbol: mexLibrary (fatal)
      3125: /usr/lib64/python2.7/lib-dynload/itertoolsmodule.so: error: symbol lookup error: undefined symbol: PyTuple_Type (fatal)

【问题讨论】：

有趣的项目。上面的问题我没有仔细研究过，但我只想指出其他类似的将 Python 解释器嵌入 MATLAB 的项目：matpy、pymex、pythoncall、pymex @Amro：我环顾四周时不知何故错过了那些，所以感谢您的链接！我没有注意到任何也支持 Win64 的东西，这对我来说是一个要求，并且 kw/pymex 中缺乏复杂矩阵令人失望，但我将精力集中在分叉该项目上可能是有意义的......绝对值得考虑，所以再次感谢您的链接！ 【参考方案1】：

尝试在 MATLAB 内部和系统 shell 上运行常规 python：

外壳

$ LD_DEBUG_OUTPUT=./ld_debug_out LD_DEBUG=all python -c 'import numpy'

MATLAB

>> !LD_DEBUG_OUTPUT=./ld_debug_out LD_DEBUG=all python -c 'import numpy'

并检查输出文件（也许all 信息有点太多了！）。注意我导入了numpy，因为itertools 在我的安装中没有相应的共享库..

一个想法是在调用Py_Initialize()之前尝试在您的MEX函数中手动加载共享库：

dlopen('libpython2.7.so', RTLD_GLOBAL|RTLD_LAZY);

我在一台 Windows 机器上，但上次我玩这个时我也遇到了一些问题。我记得一旦导入，某些 C 扩展模块无法在同一个会话中卸载和重新加载（在嵌入 Python 的托管进程的生命周期内）。请参阅这些remarks 和相关的PEP。

【讨论】：

添加 dlopen() 调用修复了我在扩展模块导入时遇到的几乎所有问题，其余的都很容易修复。非常感谢！ @ChrisGranade：很高兴我能帮上忙。祝你项目的其余部分好运，我一定会关注它！

以上是关于Linux 上的 MATLAB MEX 文件无法在 libpython.2.7.so 中找到符号的主要内容，如果未能解决你的问题，请参考以下文章

Matlab 缺少依赖项 MEX 文件

如何诊断为啥在 MATLAB 中执行 .mex 文件时无法加载 .so 文件？

从 .mexglx 到 .mex

无法将 matlab.exe 进程附加到 Visual Studio 2013 以调试 mex 文件？

防止 MEX 文件在 MATLAB 中崩溃