HPC install software on python virtual environment without root right and use X graph (windows) to d
Posted Quinn-Yann
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了HPC install software on python virtual environment without root right and use X graph (windows) to d相关的知识，希望对你有一定的参考价值。
1. Using Python Virtual Environment to install new softwares without root right.
Please try to use Python virtualenv to setup and manage Python environments
For example to use Python 3.6.3 with PyTorch 0.2.0_3, I’m working in the folder /home/wang/pyenv
(1) To create a new folder
mkdir py3.6.3
(2)
module load pytorch/python3.6/0.2.0_3
(3)
virtualenv --system-site-packages py3.6.3
(4)
source py3.6.3/bin/activate
(5)
pip install opencv-python Pillow pywavelets scikit-learn
Now every time after run
module load pytorch/python3.6/0.2.0_3 source /home/wang/pyenv/py3.6.3/bin/activate
you’ll use this environment.
[wang@c17-04 ~]$ module load pytorch/python3.6/0.2.0_3
[wang@c17-04 ~]$ source /home/wang/pyenv/py3.6.3/bin/activate
(py3.6.3) [wang@c17-04 ~]$ python -c "import torch; print(torch.__version__); print(torch.__file__)"
0.2.0_3
/share/apps/pytorch/0.2.0_3/python3.6/lib/python3.6/site-packages/torch/__init__.py
(py3.6.3) [wang@c17-04 ~]$ python -c "import cv2; print(cv2.__version__); print(cv2.__file__)"
3.3.0
/home/wang/pyenv/py3.6.3/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so
(py3.6.3) [wang@c17-04 ~]$
2. Use X graph (windows) to display
Inside NYU campus, please directly login to prince.hpc.nyu.edu without through HPC gateways with command
ssh -Y prince.hpc.nyu.edu
When you are working off campus, please try to use HPC gateways is gw.hpc.nyu.edu, /home on new gateway is mounted as read only, users can not write any data there. HPC gateways are mainly for setup ssh tunneling, after sshtunneling session is running, users can directly login to prince cluster from their local computers.
For Mac, please follow the instructions on HPC wiki to setup ssh tunneling
(1) Open an iTerm, run command
ssh hpcgwtunnel
to setup ssh tunneling through gateway.
(2) Keep iTerm window in (1) open, open a new iTerm run command
ssh prince -Y
you should be able to directly login to prince from your local computer as long as ssh tunneling in step (1) is running, X11 forwarding is enabled.
scp is similar as ssh.