docker[caffe&&pycaffe]

Posted shouhuxianjian

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了docker[caffe&&pycaffe]相关的知识,希望对你有一定的参考价值。


0 引言

今天花了一天,完成了整个caffe的dockerfile编写,其支持python3.6.6,这里主要的注意点是protobuf的版本,在3.6.0之后,只支持c11.

FROM nvidia/cuda:9.0-cudnn7-devel-centos7

COPY ./Python-3.6.6.tgz /home
COPY ./caffe /caffe
COPY ./Centos-7.repo /etc/yum.repos.d
COPY ./boost_1_67_0.tar.gz /home
COPY ./protobuf-cpp-3.5.1.zip /home
COPY ./protobuf-python-3.5.1.zip /home

ENV LANG=en_US.UTF-8
ARG PYINSTALL=/usr/local/python3
ENV PATH=$PYINSTALL/bin:$PATH
ARG http_proxy=http://xxxxxxxx:xxxx  # 记得修改成你的代理,我们机器需要代理才能上网
ARG https_proxy=https://xxxxxxx:xxxx

RUN rm -f /etc/yum.repos.d/CentOS-Base.repo  /etc/yum.repos.d/CentOS-CR.repo  /etc/yum.repos.d/CentOS-Debuginfo.repo  /etc/yum.repos.d/CentOS-Media.repo  /etc/yum.repos.d/CentOS-Sources.repo  /etc/yum.repos.d/CentOS-Vault.repo  /etc/yum.repos.d/CentOS-fasttrack.repo &&     yum clean all && yum makecache &&     yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm &&     # 安装python numpy
    yum -y install make zlib-devel openssl-devel bzip2-devel expat-devel gdbm-devel readline-devel sqlite-devel &&     yum -y install libSM libXrender libXext wget &&     tar -xvf /home/Python-3.6.6.tgz -C /home &&     cd /home/Python-3.6.6 &&     ./configure --prefix=$PYINSTALL &&     make -j32 && make install &&     ln -s $PYINSTALL/bin/python3 $PYINSTALL/bin/python &&     /usr/local/python3/bin/pip3 install -i http://pypi.douban.com/simple/ --trusted-host=pypi.douban.com numpy  &&     /usr/local/python3/bin/pip3 install -i http://pypi.douban.com/simple/ --trusted-host=pypi.douban.com scikit-image  &&     # 准备caffe依赖
    yum -y install leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel &&     yum -y install gflags-devel glog-devel lmdb-devel &&     yum -y install openblas-devel python36-devel &&     yum -y groupinstall "Development Tools" "Development Libraries" &&     # 编译boost 修复libboost_python3.so找不到的问题
    tar -xvf /home/boost_1_67_0.tar.gz -C /home &&     cd  /home/boost_1_67_0 && ./bootstrap.sh --with-libraries=python --with-toolset=gcc &&     ./b2 cflags=‘-fPIC‘ cxxflags=‘-fPIC‘ --with-python include=/usr/include/python3.6m &&     ./b2 install &&     ln -s /usr/local/lib/libboost_python36.so /usr/lib64/libboost_python3.so &&     echo /usr/local/lib >> /etc/ld.so.conf.d/caffe.conf && ldconfig     # 安装protobuf
    cd /home && unzip protobuf-cpp-3.5.1.zip &&     cd /home/protobuf-3.5.1 &&     ./configure && make -j32 && make install && ldconfig &&     cd /home && rm -rf protobuf-3.5.1 &&     unzip protobuf-python-3.5.1.zip &&     cd /home/protobuf-3.5.1 &&     cd python && /usr/local/python3/bin/python3 setup.py build && /usr/local/python3/bin/python3 setup.py install &&     # 安装caffe
    cd /caffe &&  
    make -j32 && make pycaffe -j32 &&     rm -rf /home/*
     
# docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --runtime=nvidia -tid --name zzc_caffe_demo ImageName /bin/bash

其中caffe中的Makefile.config为:

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1  # 支持cudnn

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1  # 使用opencv
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#   You should not set this flag if you will be reading LMDBs with any
#   possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you‘re using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
# 我基于CUDA 9.0,所以删除前面2个,让其从30开始
CUDA_ARCH :=    -gencode arch=compute_30,code=sm_30         -gencode arch=compute_35,code=sm_35         -gencode arch=compute_50,code=sm_50         -gencode arch=compute_52,code=sm_52         -gencode arch=compute_60,code=sm_60         -gencode arch=compute_61,code=sm_61         -gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/include/openblas
BLAS_LIB := /usr/lib64

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7         /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it‘s in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include         # $(ANACONDA_HOME)/include/python2.7         # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
# 下面要注意你安装python的路径
PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := /usr/include/python3.6m                 /usr/local/python3/lib/python3.6/site-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib64
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c ‘import numpy.core; print(numpy.core.__file__)‘))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that ‘make runtest‘ will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

以上是关于docker[caffe&&pycaffe]的主要内容,如果未能解决你的问题,请参考以下文章

TensorRT&Sample&Python[fc_plugin_caffe_mnist]

caffe的pad的报错

ubuntu14.04&matlab2015b 测试caffe的Matlab接口

caffe.bin caffe的框架

Windows10安装ubuntu & caffe GPU版

TGhack2020 re&misc