docker 命令实用指南
Posted David_Hernandez
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了docker 命令实用指南相关的知识，希望对你有一定的参考价值。
文章目录
- Docker 简介
- Docker 命令
- 镜像
- 容器
- Docker GPU 使用
- Tutorials
- deep learning dockerfile资源
- Solved Bugs
- 其他教程
- References
Docker 简介
省略。
Docker 命令
- run
- docker images 运行 - images
- docker images 查看 - ps
- docker containers 查看 - rm
- docker containers 删除 - rmi
- docker images 删除 - pull
- dockerhub images 拉取 - push
- dockerhub images 更新 - commit
- dockerhub images 提交 - exec
- docker containers 运行时修改
镜像
创建镜像
方式一：下载
# 下载指定版本容器镜像
docker pull gitlab/gitlab-ce:11.2.3-ce.0
方式二：从已经创建的容器中更新镜像，并且提交这个镜像
docker commit -m="First Docker" -a="wcjiang" a6b0a6cfdacf wcjiang/nginx:v1.2.1
上面命令参数说明：
- m 提交的描述信息
- a 指定镜像作者
- a6b0a6cfdacf 记住这个是容器id，不是镜像id
- wcjiang/nginx:v1.2.1 创建的目标镜像名
方式三：使用 Dockerfile 指令来创建一个新的镜像
参考 https://github.com/jaywcjlove/docker-tutorial
发布镜像
参考 https://github.com/jaywcjlove/docker-tutorial
导出镜像为本地文件
常用于分享。
[Saving Images and Containers as Tar Files for Sharing | dockerlabs (collabnix.com)](https://dockerlabs.collabnix.com/beginners/saving-images-as-tar/#:~:text=The docker export - Export a,a tar archive or STDIN)
容器
容器管理
# 列出包括未运行的容器
docker ps
# 查看所有容器(包括正在运行和已停止的)
docker ps -a
# 列出本机正在运行的容器
docker container ls
docker start [containerID/Names] # 启动容器
docker stop [containerID/Names] # 停止容器
docker rm [containerID/Names] # 删除容器
docker logs [containerID/Names] # 查看日志
容器使用
创建
# 创建一个守护状态的 Docker 容器
docker run -itd my-nginx /bin/bash
进入容器
# 进入容器
docker exec -it [containerID/Names] /bin/bash
一次性使用
# 在docker容器中运行hello world!
docker run centos echo "hello world"
# 在docker容器中，安装wget软件
docker run centos yum install -y wget
文件拷贝
从主机复制到容器 sudo docker cp host_path containerID:container_path
从容器复制到主机 sudo docker cp containerID:container_path host_path
# 从正在运行的 Docker 容器里面，将文件拷贝到本机，注意后面有个【点】拷贝到当前目录
docker container cp [containID]:[/path/to/file] .
Docker GPU 使用
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/user-guide.html
例子
# 参数说明：
# --rm Automatically remove the container when it exits
-
Starting a GPU enabled CUDA container; using
--gpus（常用）docker run --rm --gpus all nvidia/cuda nvidia-smi -
Using
NVIDIA_VISIBLE_DEVICESand specify the nvidia runtime （常用）docker run --rm --runtime=nvidia \\ -e NVIDIA_VISIBLE_DEVICES=all nvidia/cuda nvidia-smi -
Start a GPU enabled container on two GPUs （常用）
docker run --rm --gpus 2 nvidia/cuda nvidia-smi -
Starting a GPU enabled container on specific GPUs
docker run --gpus '"device=1,2"' \\ nvidia/cuda nvidia-smi --query-gpu=uuid --format=csv # uuid # GPU-ad2367dd-a40e-6b86-6fc3-c44a2cc92c7e # GPU-16a23983-e73e-0945-2095-cdeb50696982 -
Alternatively, you can also use
NVIDIA_VISIBLE_DEVICES（常用）docker run --rm --runtime=nvidia \\ -e NVIDIA_VISIBLE_DEVICES=1,2 \\ nvidia/cuda nvidia-smi --query-gpu=uuid --format=csv # uuid # GPU-ad2367dd-a40e-6b86-6fc3-c44a2cc92c7e # GPU-16a23983-e73e-0945-2095-cdeb50696982 -
Query the GPU UUID using
nvidia-smiand then specify that to the containernvidia-smi -i 3 --query-gpu=uuid --format=csv # uuid # GPU-18a3e86f-4c0e-cd9f-59c3-55488c4b0c24 docker run --gpus device=GPU-18a3e86f-4c0e-cd9f-59c3-55488c4b0c24 \\ nvidia/cuda nvidia-smi
Tutorials
Dockerfile
FROM centos:7.2.1511
# yum basic tool
RUN yum clean all \\
&& yum makecache \\
&& yum install -y wget \\
&& mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup \\
&& wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo \\
&& yum makecache \\
&& yum install -y gcc gcc-c++ python-devel bzip2 \\
&& yum install -y make \\
&& yum install -y epel-release \\
&& yum install -y python-devel \\
&& yum install -y python-pip \\
&& yum install -y mysql-devel \\
&& yum clean all \\
&& pip install --upgrade pip -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com \\
&& pip install setuptools==33.1.1 -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com
##守护进程，可配置多进程
#RUN yum install -y python-setuptools
# && pip install supervisor -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com \\
# && mkdir -m 777 /tmp
# && mkdir -m 755 -p /etc/supervisor/conf.d
#COPY supervisor.conf /etc/supervisor/supervisord.conf
#
##镜像启动命令
#CMD ["supervisord","-c","/etc/supervisor/supervisord.conf"]
COPY resource/CRF++-0.58.tar.gz /tmp
# MySQL and CRFPP
RUN pip install MySQL-python -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com \\
&& pip install Cython -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com \\
&& cd /tmp \\
&& tar -xzvf CRF++-0.58.tar.gz \\
&& cd CRF++-0.58 \\
&& ./configure \\
&& make \\
&& make install \\
&& cd python \\
&& python setup.py build \\
&& python setup.py install \\
&& cd .. \\
&& rm -rf CRF++-0.58 \\
&& rm -f CRF++-0.58.tar.gz \\
&& mkdir -p /code/example_project/src \\
&& mkdir -p /code/example_project/conf \\
&& mkdir -p /code/example_project/admin
COPY setup.py /code/example_project
COPY src /code/example_project/src
COPY conf /code/example_project/conf
COPY admin /code/example_project/admin
RUN cd /code/example_project \\
&& python2 setup.py build_ext
CMD cd /code/example_project/admin \\
&& sh start_server.sh
build_docker_image.sh
#!/usr/bin/env bash
#检查是否输入镜像tag的version 信息
version=$1
if [ -z $version ]
then
version="pingan_20180131"
fi
cd ..
# 清理日志，也可以在后面做文件模型拷贝等批处理信息
rm -rf log/*
docker build -t dockerhub.example.com/pingan_contract/seq2seq_offline:$version .
start_docker_server.sh
#!/usr/bin/env bash
name=`docker ps -a|grep pingan_contract_seq2seq_offline|awk 'print $NF'`
if [ -z $name ]
then
docker run -itd -m 256g -v /Users/lotus/PycharmProjects/example_project/admin:/code/example_project/admin -v /Users/lotus/PycharmProjects/example_project/log:/code/example_project/log -v /Users/lotus/PycharmProjects/example_project/data:/code/example_project/data --name example_project_seq2seq_offline dockerhub.example.com/example_project/seq2seq_offline:example_20180131
else
docker start example_project_seq2seq_offline
fi
start_server.sh
#!/usr/bin/env bash
base_home="/code/example_project"
base_source_path="$base_home/src"
base_data_path="$base_home/data"
base_conf_path="$base_home/conf"
cd $base_source_path
python3 main.py
stop_docker_server.sh
#!/usr/bin/env bash
docker stop example_project_seq2seq_offline
stop_server.sh
#!/usr/bin/env bash
ps aux|grep python|grep main|awk 'print $2'|xargs kill -9
docker run 例子2
docker run --runtime=nvidia -it -d --name alex_conda \\
-p <port>:8888 \\
-v /etc/passwd:/etc/passwd:ro \\
-v /etc/group:/etc/group:ro \\
-v $HOME:$HOME \\
-v /mnt:/mnt \\
-v conda_pkgs:/opt/conda/pkgs \\
-v conda_envs:/opt/conda/envs \\
-u `id -u`:`id -g` \\
-w $HOME \\
conda_jupyter
docker run --runtime=nvidia -it -d --name example_container -p 6688:8888 -p 6622:22 -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -v $HOME:$HOME -v /mnt:/mnt -v conda_pkgs:/opt/conda/pkgs -v conda_envs:/opt/conda/envs -u `id -u`:`id -g` -w $HOME conda_jupyter
# 例子：root 身份进入 容器
docker exec -u 0 -it example_container bash
docker exec：使用 root 身份进入容器
docker exec -u 0 -it 容器名 bash
docker 开启 ssh 服务
https://www.jianshu.com/p/94a0a95c9a2c
deep learning dockerfile资源
AWS
https://github.com/aws/deep-learning-containers.git
# 获取 dockerfile
# 位置：https://github.com/aws/deep-learning-containers/blob/master/pytorch/training/docker/1.11/py3/cu113/Dockerfile.gpu
git clone https://github.com/aws/deep-learning-containers.git
# build image
cd deep-learning-containers
cp ./pytorch/training/docker/build_artifacts/changehostname.c ./src
cp ./pytorch/training/docker/build_artifacts/start_with_right_hostname.sh ./src
cd ./src
# build 镜像
# 如果失败，重试几次就可以，需要保证网速
nohup docker build -f /home/xiaqi.xia/workspaces/deep-learning-containers/pytorch/training/docker/1.11/py3/cu113/Dockerfile.gpu -t aws/deep-learning-containers:pytorch1.11-cuda11.3.1-ubuntu20.04 . &
# 创建容器
docker run --runtime=nvidia -it -d --name pytorch_middle_language -p 7788:8888 -p 7722:22 -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro -v $HOME:$HOME -v /mnt:/mnt -v conda_pkgs:/opt/conda/pkgs -v conda_envs:/opt/conda/envs -u `id -u`:`id -g` -w $HOME aws/deep-learning-containers:pytorch1.11-cuda11.3.1-ubuntu20.04
# 进入容器
docker exec -it pytorch_middle_language '/bin/bash'
# root 用户 进入
docker exec -u 0 -it pytorch_middle_language bash
# 开启 ssh 服务
https://www.jianshu.com/p/94a0a95c9a2c
# 创建 conda 虚拟环境
conda create --name kg-bart --clone base
# 服务器上命令方式执行
CUDA_VISIBLE_DEVICES=0 python run_seq2seq.py --data_dir ../dataset/commongen_data/commongen --output_dir ../output/KGBart --log_dir ../log/KGBart --model_recover_path ../output/Pretraining_KG/best_model/model.best.bin --fp16 True --max_seq_length 32 --max_position_embeddings 64 --max_len_a 32 --max_len_b 64 --max_pred 64 --train_batch_size 60 --eval_batch_size 48 --gradient_accumulation_steps 6 --learning_rate 0.00001 --warmup_proportion 0.1 --label_smoothing 0.1 --num_train_epochs 10
anibali/docker-pytorch
https://github.com/anibali/docker-pytorch/blob/master/dockerfiles/1.11.0-cuda11.5-ubuntu20.04/Dockerfile
Solved Bugs
解决：docker build E: Failed to fetch http://security.ubuntu.com
方法一
https://stackoverflow.com/questions/37706635/in-docker-apt-get-install-fails-with-failed-to-fetch-http-archive-ubuntu-com
FROM ubuntu:14.04
RUN sed -i'' 's/archive\\.ubuntu\\.com/us\\.archive\\.ubuntu\\.com/' /etc/apt/sources.list
RUN apt-get -y update
方法二
如果方法一不管用，使用方法二。apt get - Ubuntu apt-get unable to fetch packages - Stack Overflow
# add the following to your resolve.conf file /etc/resolv.conf
# nameserver 8.8.8.8
# nameserver 8.8.4.4
RUN echo "nameserver 8.8.8.8" >> /etc/resolv.conf && echo "nameserver 8.8.4.4" >> /etc/resolv.conf
解决：docker build E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/
https://github.com/deepmind/alphafold/issues/4
# 方法一：（已尝试）
Try this:
Modify the Dockerfile like this if you are in China:
................. # some command
RUN echo deb https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64 / | tee /etc/apt/sources.list.d/cuda.list # HERE! Add this before apt-get update
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \\
................. # some command
And you may also unable to access the github depository. So you can modify the website of https://github.com/soedinglab/hh-suite.git to https://github.com.cnpmjs.org/soedinglab/hh-suite.git
# 方法二：（未尝试）
docker build —add-host “developer.download.nvidia.com:152.195.19.142 -f docker/Dockerfile -t alphafold .
其他教程
References
- https://github.com/jaywcjlove/docker-tutorial
- https://github.com/NVIDIA/nvidia-docker
以上是关于docker 命令实用指南的主要内容，如果未能解决你的问题，请参考以下文章