如何在非root的服务器账户下配置python科研环境

Posted 2022-11-22 甘木甘木

Intro

在非root的linux服务器比如ubuntu 18.04上如何配置科研环境，如何在非root（无法sudo apt-get install）安装自己需要的package

• 使用conda在自己的环境下安装，
  • conda是可以非root安装的
  • 另外conda提供安装别的linux package的方案
    • 比如安装screen conda install -c conda-forge screen

Install conda

这里选择miniconda作为演示，因为miniconda比较轻量级
首先从官网下载安装的脚，将脚本传到服务器上
执行bash Miniconda3-py38_4.11.0-Linux-x86_64.sh

conda change source

换源
参考tuna

1. vim ~/.condarc
2. ctrl c+v

channels:
  - defaults
show_channel_urls: true
default_channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud

create new environment

conda 从已有的环境创建environment
下面的配置文件适用于：

• python =3.8
• cudatoolkit=11.3 (verified compatible with device CUDA at 11.4)

安装的pakcage：

• torch=1.9.0
• torchvision=0.10.0
• torchaudio=0.9.0

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
@EXPLICIT
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/_libgcc_mutex-0.1-main.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/blas-1.0-mkl.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/ca-certificates-2021.10.8-ha878542_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/intel-openmp-2021.3.0-h06a4308_3350.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/ld_impl_linux-64-2.35.1-h7274673_9.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libstdcxx-ng-9.3.0-hd4cf53a_17.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libgomp-9.3.0-h5101ec6_17.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/mkl-2021.3.0-h06a4308_520.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/_openmp_mutex-4.5-1_gnu.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/libblas-3.9.0-11_linux64_mkl.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libgcc-ng-9.3.0-h5101ec6_17.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/liblapack-3.9.0-11_linux64_mkl.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/bzip2-1.0.8-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/c-ares-1.17.1-h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/cudatoolkit-11.3.1-h2bc3f7f_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/expat-2.4.1-h2531618_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/gmp-6.2.1-h2531618_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/icu-58.2-he6710b0_3.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/jpeg-9d-h7f8727e_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/lame-3.100-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libev-4.33-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libffi-3.3-he6710b0_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libiconv-1.15-h63c8f33_5.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libsodium-1.0.18-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libtasn1-4.16.0-h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libunistring-0.9.10-h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libuv-1.40.0-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libwebp-base-1.2.0-h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/lz4-c-1.9.3-h295c915_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/ncurses-6.2-he6710b0_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/ninja-1.10.2-hff7bd54_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/openssl-1.1.1k-h7f98852_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/perl-5.26.2-h14c3975_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/sleef-3.5.1-h7f98852_1.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/xz-5.2.5-h7b6447c_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/zlib-1.2.11-h7b6447c_3.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/cudnn-8.2.1.32-h86fa8c9_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libedit-3.1.20210714-h7f8727e_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libidn2-2.3.2-h7f8727e_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libnghttp2-1.41.0-hf8bcb03_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libpng-1.6.37-hbc83047_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/libprotobuf-3.16.0-h780b84a_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libssh2-1.9.0-h1ba5d50_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libtiff-4.2.0-h85742a9_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libxml2-2.9.12-h03d6c58_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/magma-2.5.4-h6103c52_2.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/nccl-2.11.4.1-hdc17891_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/nettle-3.7.3-hbbd107a_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/openh264-2.1.0-hd408876_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/pcre2-10.35-h14c3975_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/readline-8.1-h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/screen-4.8.0-he28a2e2_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/tk-8.6.10-hbc83047_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/zeromq-4.3.4-h2531618_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/zstd-1.4.9-haebb681_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/freetype-2.10.4-h5ab3b9f_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/gettext-0.21.0-hf68c758_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/gnutls-3.6.15-he1e5248_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/krb5-1.19.2-hac12032_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/lcms2-2.12-h3be6417_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/openjpeg-2.4.0-h3ad879b_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/sqlite-3.36.0-hc218d9a_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/linux-64/ffmpeg-4.3-hf484d3e_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/libcurl-7.78.0-h0b77cf5_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/python-3.8.11-h12debd9_0_cpython.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/backcall-0.2.0-pyhd3eb1b0_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/curl-7.78.0-h1ccaba5_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/debugpy-1.4.1-py38h295c915_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/decorator-5.0.9-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/entrypoints-0.3-py38_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/ipython_genutils-0.2.0-pyhd3eb1b0_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/nest-asyncio-1.5.1-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/olefile-0.46-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/parso-0.8.2-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/pickleshare-0.7.5-pyhd3eb1b0_1003.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/ptyprocess-0.7.0-pyhd3eb1b0_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch/pycparser-2.21-pyhd8ed1ab_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/python_abi-3.8-2_cp38.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/pyzmq-22.2.1-py38h295c915_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/six-1.16.0-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/tornado-6.1-py38h27cfd23_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/typing_extensions-3.10.0.0-pyhca03da5_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/wcwidth-0.2.5-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/wheel-0.37.0-pyhd3eb1b0_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/certifi-2021.10.8-py38h578d9bd_2.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/cffi-1.14.6-py38ha65f79e_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/future-0.18.2-py38h578d9bd_5.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/git-2.32.0-pl5262hc120c5b_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/jedi-0.18.0-py38h06a4308_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/mkl-service-2.4.0-py38h7f8727e_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/pexpect-4.8.0-pyhd3eb1b0_3.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/pillow-7.2.0-py38h9776b28_2.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/prompt-toolkit-3.0.17-pyhca03da5_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/python-dateutil-2.8.2-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/traitlets-5.0.5-pyhd3eb1b0_0.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/jupyter_core-4.7.1-py38h06a4308_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/matplotlib-inline-0.1.2-pyhd3eb1b0_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/numpy-base-1.20.3-py38h74d4b33_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/setuptools-52.0.0-py38h06a4308_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/jupyter_client-7.0.1-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch/pygments-2.10.0-pyhd3eb1b0_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/ipython-7.27.0-py38hb070fc8_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/ipykernel-6.2.0-py38h06a4308_1.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/pytorch-1.9.0-cuda112py38h3d13190_1.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/pytorch-gpu-1.9.0-cuda112py38h0bbbad9_1.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64/torchvision-0.10.0-py38cuda112h04b465a_0_cuda.tar.bz2
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/mkl_fft-1.3.0-py38h42c9631_2.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/mkl_random-1.2.2-py38h51133e4_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/numpy-1.20.3-py38hf144106_0.conda
https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/linux-64/torchaudio-0.9.0-py38.tar.bz2

conda install screen

配置好tuna mirror之后直接下载完成
缺点是不同的environment都得下载一遍
conda install -c conda-forge screen

bash auto complete

touch run.sh
将下面的代码加入 run.sh

#!/bin/bash

echo '"\\e[A": history-search-backward' >> ~/.inputrc
echo '"\\e[B": history-search-forward' >> ~/.inputrc
echo 'set show-all-if-ambiguous on' >> ~/.inputrc
echo 'set completion-ignore-case on' >> ~/.inputrc

echo "run ' bind -f  ~/.inputrc ' to enable better history completion."

next

bash run.sh
bind -f  ~/.inputrc