即使似乎安装了 graphviz，如何在 HPC 集群上安装 pygraphviz 而不会出错？
Posted
技术标签:
【中文标题】即使似乎安装了 graphviz，如何在 HPC 集群上安装 pygraphviz 而不会出错？【英文标题】：How does one install pygraphviz on a HPC cluster without errors even when graphviz seems to be installed? 【发布时间】：2021-08-03 04:35:06 【问题描述】：我试图 pip 和 conda 安装 pygraphviz。我相信 grahviz 是因为安装了
(metalearningpy1.7.1c10.2) miranda9~/ML4Coq $ conda install -y graphviz
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
但是当我尝试使用 pip 安装 pygraphviz 时，我得到以下信息：
(metalearningpy1.7.1c10.2) miranda9~/ML4Coq $ pip install pygraphviz
Collecting pygraphviz
Using cached pygraphviz-1.7.zip (118 kB)
Building wheels for collected packages: pygraphviz
Building wheel for pygraphviz (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-b61bb2uf
cwd: /tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/
Complete output (55 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/scraper.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/testing.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/graphviz.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/agraph.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/__init__.py -> build/lib.linux-x86_64-3.8/pygraphviz
creating build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_attribute_defaults.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_subgraph.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_readwrite.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_unicode.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_drawing.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_string.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_close.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_scraper.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_edge_attributes.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_graph.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_html.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_clear.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/__init__.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_node_attributes.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_layout.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
running egg_info
writing pygraphviz.egg-info/PKG-INFO
writing dependency_links to pygraphviz.egg-info/dependency_links.txt
writing top-level names to pygraphviz.egg-info/top_level.txt
reading manifest file 'pygraphviz.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.png' under directory 'doc'
warning: no files found matching '*.txt' under directory 'doc'
warning: no files found matching '*.css' under directory 'doc'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.svn' found anywhere in distribution
no previously-included directories found matching 'doc/build'
writing manifest file 'pygraphviz.egg-info/SOURCES.txt'
copying pygraphviz/graphviz.i -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/graphviz_wrap.c -> build/lib.linux-x86_64-3.8/pygraphviz
running build_ext
building 'pygraphviz._graphviz' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/pygraphviz
gcc -pthread -B /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/include/python3.8 -c pygraphviz/graphviz_wrap.c -o build/temp.linux-x86_64-3.8/pygraphviz/graphviz_wrap.o
In file included from /usr/include/graphviz/gvc.h:17:0,
from pygraphviz/graphviz_wrap.c:2712:
/usr/include/graphviz/types.h:49:20: fatal error: cgraph.h: No such file or directory
#include <cgraph.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for pygraphviz
Running setup.py clean for pygraphviz
Failed to build pygraphviz
Installing collected packages: pygraphviz
Running setup.py install for pygraphviz ... error
ERROR: Command errored out with exit status 1:
command: /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-mu112lx1/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/include/python3.8/pygraphviz
cwd: /tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/
Complete output (55 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/scraper.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/testing.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/graphviz.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/agraph.py -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/__init__.py -> build/lib.linux-x86_64-3.8/pygraphviz
creating build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_attribute_defaults.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_subgraph.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_readwrite.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_unicode.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_drawing.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_string.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_close.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_scraper.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_edge_attributes.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_graph.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_html.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_clear.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/__init__.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_node_attributes.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
copying pygraphviz/tests/test_layout.py -> build/lib.linux-x86_64-3.8/pygraphviz/tests
running egg_info
writing pygraphviz.egg-info/PKG-INFO
writing dependency_links to pygraphviz.egg-info/dependency_links.txt
writing top-level names to pygraphviz.egg-info/top_level.txt
reading manifest file 'pygraphviz.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.png' under directory 'doc'
warning: no files found matching '*.txt' under directory 'doc'
warning: no files found matching '*.css' under directory 'doc'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.svn' found anywhere in distribution
no previously-included directories found matching 'doc/build'
writing manifest file 'pygraphviz.egg-info/SOURCES.txt'
copying pygraphviz/graphviz.i -> build/lib.linux-x86_64-3.8/pygraphviz
copying pygraphviz/graphviz_wrap.c -> build/lib.linux-x86_64-3.8/pygraphviz
running build_ext
building 'pygraphviz._graphviz' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/pygraphviz
gcc -pthread -B /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/include/python3.8 -c pygraphviz/graphviz_wrap.c -o build/temp.linux-x86_64-3.8/pygraphviz/graphviz_wrap.o
In file included from /usr/include/graphviz/gvc.h:17:0,
from pygraphviz/graphviz_wrap.c:2712:
/usr/include/graphviz/types.h:49:20: fatal error: cgraph.h: No such file or directory
#include <cgraph.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rc_2rzek/pygraphviz_904720f7fa8b49f4a7006870a0d5e479/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-mu112lx1/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/include/python3.8/pygraphviz Check the logs for full command output.
与康达：
(metalearningpy1.7.1c10.2) miranda9~/ML4Coq $ conda install -y -c anaconda pygraphviz
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages. failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- pygraphviz -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
我的错误的根源是什么？ [注意我在linux机器上，不是mac os，我的mac phygraphviz工作正常]
我看到了以下相关问题：
Pip install pygraphviz fails: Failed building wheel for pygraphviz Couldn't install pygraphviz https://github.com/pygraphviz/pygraphviz/issues/347 https://github.com/pygraphviz/pygraphviz/issues/358但他们没有帮助，因为我无法运行 apt-get
(metalearningpy1.7.1c10.2) miranda9~/ML4Coq $ apt-get install python-dev graphviz libgraphviz-dev pkg-config
-bash: apt-get: command not found
编辑：
请注意，我也尝试使用 pip 安装 grapviz，但这并没有解决我的问题：
(metalearningpy1.7.1c10.2) miranda9~/ML4Coq $ pip install graphviz
Collecting graphviz
Downloading graphviz-0.16-py2.py3-none-any.whl (19 kB)
Installing collected packages: graphviz
Successfully installed graphviz-0.16
编辑 2：
我发现graphviz在那里，但为什么找不到呢？
(metalearningpy1.7.1c10.2) miranda9~ $ rpm -qa |grep graphviz
graphviz-gd-2.30.1-21.el7.x86_64
graphviz-2.30.1-21.el7.x86_64
graphviz-devel-2.30.1-21.el7.x86_64
编辑 3：
我也希望它与 pip install -e . 一起使用，例如查看终极工具的安装 (https://github.com/brando90/ultimate-utils)：
由于对 pygraphviz 的依赖，您可能需要先安装 graphviz。接下来，安装graphviz。在 mac 上，brew 安装 graphviz。在 Ubuntu 上，sudo apt install graphviz。安装graphviz后，以可食用模式安装uutils，并使用pip安装它的所有依赖项：
pip install -e ~/ultimate-utils/ultimate-utils-proj-src
作为一个 gitissue：https://github.com/brando90/ultimate-utils/issues/6
【问题讨论】：
似乎答案将是由系统管理员在 HPC 中的所有节点上下载原始 graphviz[graphviz.org/download/]（不是同名的 pytorch 包......令人困惑！）。 我也希望它与pip install -e .一起工作
有人知道为什么使用conda-forge 有效吗？
【参考方案1】：
我能够在安装了graphviz-devel 的 CentOS 上编译它。最初我得到了与上面相同的错误，缺少cgraph.h，但通过向编译器提供头文件查找路径，我可以构建它：
CPPFLAGS="-I/usr/include/graphviz" pip install pygraphviz
当我尝试使用该软件包时，虽然我遇到了符号查找错误。所以最后我放弃了，选择了 conda。
【讨论】：
【参考方案2】：嗯，看来使用conda-forge 有效！
(metalearningpy1.7.1c10.2) miranda9~ $ conda install -y -c conda-forge pygraphviz
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2
added / updated specs:
- pygraphviz
The following packages will be downloaded:
package | build
---------------------------|-----------------
libgcc-ng-9.3.0 | h2828fa1_19 7.8 MB conda-forge
libgomp-9.3.0 | h2828fa1_19 376 KB conda-forge
pygraphviz-1.7 | py38h0d738da_0 125 KB conda-forge
------------------------------------------------------------
Total: 8.3 MB
The following NEW packages will be INSTALLED:
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
libgomp conda-forge/linux-64::libgomp-9.3.0-h2828fa1_19
pygraphviz conda-forge/linux-64::pygraphviz-1.7-py38h0d738da_0
The following packages will be UPDATED:
certifi pkgs/main::certifi-2020.12.5-py38h06a~ --> conda-forge::certifi-2020.12.5-py38h578d9bd_1
libgcc-ng pkgs/main::libgcc-ng-9.1.0-hdf63c60_0 --> conda-forge::libgcc-ng-9.3.0-h2828fa1_19
The following packages will be SUPERSEDED by a higher-priority channel:
_libgcc_mutex pkgs/main::_libgcc_mutex-0.1-main --> conda-forge::_libgcc_mutex-0.1-conda_forge
ca-certificates pkgs/main::ca-certificates-2021.4.13-~ --> conda-forge::ca-certificates-2020.12.5-ha878542_0
openssl pkgs/main::openssl-1.1.1k-h27cfd23_0 --> conda-forge::openssl-1.1.1k-h7f98852_0
Downloading and Extracting Packages
libgcc-ng-9.3.0 | 7.8 MB | ##################################################################################################################################################################################################################################################################################################################################################################################### | 100%
pygraphviz-1.7 | 125 KB | ##################################################################################################################################################################################################################################################################################################################################################################################### | 100%
libgomp-9.3.0 | 376 KB | ##################################################################################################################################################################################################################################################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
注意：
但是，请注意 pygraphviz 出于某种原因不建议使用 conda 安装其 pkg我们建议避免使用 Anaconda 和 conda-forge 来安装 Graphviz 和 PyGraphviz。
但是由于 pip 对我来说一直失败 - 尽管在集群中安装了 graphviz - 请参阅 rpm 的输出：
[rundblom@vision-12 ~]$ rpm -qa |grep graphviz
graphviz-devel-2.30.1-21.el7.x86_64
graphviz-2.30.1-21.el7.x86_64
graphviz-gd-2.30.1-21.el7.x86_64
所以我知道使用 pip 有什么问题。我本来希望以通常的 HPC 方式加载模块，但现在这不是我的选择......
【讨论】：
有人知道为什么使用conda-forge 有效吗？以上是关于即使似乎安装了 graphviz，如何在 HPC 集群上安装 pygraphviz 而不会出错？的主要内容，如果未能解决你的问题，请参考以下文章