如何让 conda 使用自己的 gcc 版本？

Posted 2023-02-16

【中文标题】如何让 conda 使用自己的 gcc 版本？

【英文标题】：How to make conda use its own gcc version?

【发布时间】：2021-12-11 15:34:08

【问题描述】：

我正在尝试在远程系统上运行 stylegan2-pytorch 的培训。远程系统上安装了 gcc (9.3.0)。我正在使用安装了以下内容的 conda env（cudatoolkit=10.2、torch=1.5.0+ 和 ninja=1.8.2、gcc_linux-64=7.5.0）。 我遇到以下错误：

RuntimeError: Error building extension 'fused': [1/2] 
/home/envs/segmentation_base/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/TH -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/THC -isystem /home/envs/segmentation_base/include -isystem /home/envs/segmentation_base/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_70,code=sm_70 --compiler-options '-fPIC' -std=c++14 -c /home/code/semanticGAN_code/models/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o 
FAILED: fused_bias_act_kernel.cuda.o 
/home/envs/segmentation_base/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/TH -isystem /home/envs/segmentation_base/lib/python3.6/site-packages/torch/include/THC -isystem /home/envs/segmentation_base/include -isystem /home/envs/segmentation_base/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_70,code=sm_70 --compiler-options '-fPIC' -std=c++14 -c /home/code/semanticGAN_code/models/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o 
In file included from /home/envs/segmentation_base/include/cuda_runtime.h:83,
                 from <command-line>:
/home/envs/segmentation_base/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!
  138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!
      |  ^~~~~
ninja: build stopped: subcommand failed.

我想使用我的 conda env (gcc_linux-64=7.5.0) 的 gcc 来构建 cuda。当我在我的 conda 环境中运行 gcc --version 时，我得到了系统的 gcc：

gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

which gcc 当我的 conda env 处于活动状态时返回：

usr/bin/gcc

我希望它返回 gcc 版本 7.5.0（安装在环境中的那个）。我知道 conda 对 gcc 有不同的名称，但环境变量应该指向已安装的 gcc。

运行echo $CC 返回

/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-cc.

按照建议的解决方案here，我在激活我的环境时得到以下信息，但同样的问题存在：

INFO: activate-binutils_linux-64.sh made the following environmental changes:
+ADDR2LINE=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-addr2line
+AR=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-ar
+AS=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-as
+CXXFILT=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-c++filt
+ELFEDIT=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-elfedit
+GPROF=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-gprof
+LD_GOLD=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-ld.gold
+LD=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-ld
+NM=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-nm
+OBJCOPY=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-objcopy
+OBJDUMP=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-objdump
+RANLIB=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-ranlib
+READELF=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-readelf
+SIZE=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-size
+STRINGS=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-strings
+STRIP=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-strip
INFO: activate-gcc_linux-64.sh made the following environmental changes:
+build_alias=x86_64-conda-linux-gnu
+BUILD=x86_64-conda-linux-gnu
+CC_FOR_BUILD=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-cc
+CC=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-cc
+CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix
+CMAKE_ARGS=-DCMAKE_LINKER=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-ld -DCMAKE_STRIP=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-strip -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_FIND_ROOT_PATH=;/x86_64-conda-linux-gnu/sysroot -DCMAKE_INSTALL_PREFIX= -DCMAKE_INSTALL_LIBDIR=lib
+CMAKE_PREFIX_PATH=:/home/envs/segmentation_base/x86_64-conda-linux-gnu/sysroot/usr
+CONDA_BUILD_SYSROOT=/home/envs/segmentation_base/x86_64-conda-linux-gnu/sysroot
+_CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_linux_gnu
+CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /include
+CPP=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-cpp
+DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix
+DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem /include
+GCC_AR=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-gcc-ar
+GCC_NM=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-gcc-nm
+GCC=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-gcc
+GCC_RANLIB=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-gcc-ranlib
+host_alias=x86_64-conda-linux-gnu
+HOST=x86_64-conda-linux-gnu
+LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/lib -Wl,-rpath-link,/lib -L/lib
INFO: activate-gxx_linux-64.sh made the following environmental changes:
+CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix
+CXX_FOR_BUILD=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-c++
+CXX=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-c++
+DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix
+GXX=/home/envs/segmentation_base/bin/x86_64-conda-linux-gnu-g++

如何将 gcc 设置为 conda gcc 而不是系统 gcc？我知道这应该在通过activate.d 中的 bash 脚本激活环境时自动完成。

大多数未解决的问题（关于不受支持的 GNU 版本！）要么需要 sudo 权限才能调整 gcc 版本（我没有），要么在 conda 环境中不被接受。我还没有找到一个明确的解决方案：/

TLDR：如何强制 conda 使用自己安装的 gcc 版本而不是主机系统 gcc？

编辑1：添加conda list输出

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
_sysroot_linux-64_curr_repodata_hack 3                   haa98f57_10  
absl-py                   1.0.0                    pypi_0    pypi
albumentations            0.5.2                    pypi_0    pypi
binutils_impl_linux-64    2.35.1               h27ae35d_9  
binutils_linux-64         2.35.1              h454624a_30  
blas                      1.0                         mkl  
ca-certificates           2021.10.26           h06a4308_2  
cachetools                4.2.4                    pypi_0    pypi
certifi                   2021.5.30        py36h06a4308_0  
charset-normalizer        2.0.9                    pypi_0    pypi
cudatoolkit               10.2.89                       3    hcc
cycler                    0.11.0                   pypi_0    pypi
decorator                 4.4.2                    pypi_0    pypi
freetype                  2.11.0               h70c0345_0  
gcc_impl_linux-64         7.5.0               h7105cf2_17  
gcc_linux-64              7.5.0               h8f34230_30  
google-auth               2.3.3                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
grpcio                    1.42.0                   pypi_0    pypi
gxx_impl_linux-64         7.5.0               h0a5bf11_17  
gxx_linux-64              7.5.0               hffc177d_30  
idna                      3.3                      pypi_0    pypi
imageio                   2.8.0                    pypi_0    pypi
imageio-ffmpeg            0.4.2                    pypi_0    pypi
imgaug                    0.4.0                    pypi_0    pypi
importlib-metadata        4.8.2                    pypi_0    pypi
intel-openmp              2021.4.0          h06a4308_3561  
jpeg                      9d                   h7f8727e_0  
kernel-headers_linux-64   3.10.0              h57e8cba_10  
kiwisolver                1.3.1                    pypi_0    pypi
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-devel_linux-64     7.5.0               hbbeae57_17  
libgcc-ng                 9.3.0               h5101ec6_17  
libgomp                   9.3.0               h5101ec6_17  
libpng                    1.6.37               hbc83047_0  
libstdcxx-devel_linux-64  7.5.0               hf0c5c8d_17  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libtiff                   4.2.0                h85742a9_0  
libwebp-base              1.2.0                h27cfd23_0  
lmdb                      0.98                     pypi_0    pypi
lz4-c                     1.9.3                h295c915_1  
markdown                  3.3.6                    pypi_0    pypi
matplotlib                3.3.4                    pypi_0    pypi
mkl                       2020.2                      256  
mkl-service               2.3.0            py36he8ac12f_0  
mkl_fft                   1.3.0            py36h54f3939_0  
mkl_random                1.1.1            py36h0573a6f_0  
ncurses                   6.3                  h7f8727e_2  
networkx                  2.5.1                    pypi_0    pypi
ninja                     1.8.2                    pypi_0    pypi
numpy                     1.19.5                   pypi_0    pypi
numpy-base                1.19.2           py36hfa32c7d_0  
oauthlib                  3.1.1                    pypi_0    pypi
olefile                   0.46                     py36_0  
opencv-python             4.5.4.60                 pypi_0    pypi
opencv-python-headless    4.5.4.60                 pypi_0    pypi
openjpeg                  2.4.0                h3ad879b_0  
openssl                   1.1.1l               h7f8727e_0  
pillow                    8.4.0                    pypi_0    pypi
pip                       21.2.2           py36h06a4308_0  
protobuf                  3.19.1                   pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pyparsing                 3.0.6                    pypi_0    pypi
python                    3.6.13               h12debd9_1  
python-dateutil           2.8.2                    pypi_0    pypi
pytorch                   1.5.0           py3.6_cuda10.2.89_cudnn7.6.5_0    pytorch
pywavelets                1.1.1                    pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.1                  h27cfd23_0  
requests                  2.26.0                   pypi_0    pypi
requests-oauthlib         1.3.0                    pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
scikit-image              0.17.2                   pypi_0    pypi
scipy                     1.5.0                    pypi_0    pypi
setuptools                58.0.4           py36h06a4308_0  
shapely                   1.8.0                    pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_0  
sqlite                    3.36.0               hc218d9a_0  
sysroot_linux-64          2.17                h57e8cba_10  
tensorboard               2.7.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
tifffile                  2020.9.3                 pypi_0    pypi
tk                        8.6.11               h1ccaba5_0  
torchvision               0.6.0                py36_cu102    pytorch
typing-extensions         4.0.1                    pypi_0    pypi
urllib3                   1.26.7                   pypi_0    pypi
werkzeug                  2.0.2                    pypi_0    pypi
wheel                     0.37.0             pyhd3eb1b0_1  
xz                        5.2.5                h7b6447c_0  
zipp                      3.6.0                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3  
zstd                      1.4.9                haebb681_0  

【问题讨论】：

我看到你只有gcc_impl_linux-64 而不是gcc 包，这与package list from a fresh environment 不同。您是否尝试在您的 conda 环境中安装 gcc 包？

gcc 在 conda 下被称为 gcc_linux-64, 它在我的conda list 中可用。我设法通过将符号链接添加到安装的 conda gcc 来解决它。感谢您的努力:)

【参考方案1】：

除了in this issue 发布的解决方案。我添加了指向 conda 安装的 gcc 的符号链接，但我错过了。

ln -s /home/envs/segmentation_base/bin/x86_64-conda_cos6-linux-gnu-cc gcc
ln -s /home/envs/segmentation_base/bin/x86_64-conda_cos6-linux-gnu-cpp g++

【参考方案2】：

只是分享一下，不知道对你有没有帮助。但是它表明在标准条件下可以使用documentation 中描述的conda gcc 而不是系统gcc。

# system gcc
which gcc && gcc --version
# /usr/bin/gcc
# gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

# creating a conda env with gcc
conda create -n gcc gcc
# activate the environment
conda activating gcc
which gcc && gcc --version
# /opt/conda/envs/gcc/bin/gcc
# gcc (GCC) 11.2.0

这是安装在仅使用gcc 创建的全新环境中的软件包列表。

# packages in environment at /opt/conda/envs/gcc:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
binutils_impl_linux-64    2.36.1               h193b22a_2    conda-forge
gcc                       11.2.0               h702ea55_2    conda-forge
gcc_impl_linux-64         11.2.0              h82a94d6_11    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libgcc-devel_linux-64     11.2.0              h0952999_11    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgomp                   11.2.0              h1d223b6_11    conda-forge
libsanitizer              11.2.0              he4da1e4_11    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge

【讨论】：

这正是我打开问题的原因，在标准条件下我应该能够访问 conda gcc，但我不能。运行您建议的相同行，我的 gcc 仍然指向系统 gcc。

@A95 你能分享一下conda list 在你的环境被激活时的输出吗？