通过Dockerfile创建caffe-gpu环境

标签：教程   scipy   his   upgrade   dir   echo   bin   ports   imp   

我这里GPU型号为 tesla T4, 算力为7.5，会有一些特殊处理。首先是我不会在Dockerfile中编译caffe

本次环境安装通过Dockerfile安装，如果按照本教程要先安装docker以及nvidia-docker

1. 下载caffe的代码

git clone https://github.com/BVLC/caffe.git

2. 替换caffe目录下docker/gpu中Dockerfile

内容如下

FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
LABEL maintainer caffe-maint@googlegroups.com

RUN apt-get update && apt-get install -y --no-install-recommends         build-essential         vim         python-opencv         python-tk         cmake         git         wget         libatlas-base-dev         libboost-all-dev         libgflags-dev         libgoogle-glog-dev         libhdf5-serial-dev         libleveldb-dev         liblmdb-dev         libopencv-dev         libprotobuf-dev         libsnappy-dev         protobuf-compiler         python-dev         python-numpy         python-pip         python-setuptools         python-scipy &&     rm -rf /var/lib/apt/lists/*

ENV CAFFE_ROOT=/opt/caffe
WORKDIR $CAFFE_ROOT

# FIXME: use ARG instead of ENV once DockerHub supports this
# https://github.com/docker/hub-feedback/issues/460
ENV CLONE_TAG=1.0

RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git .

RUN pip install --upgrade pip &&     pip install pip -U &&     pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple &&     pip install python-dateutil==2.5.0

# git clone https://github.com/NVIDIA/nccl.git && cd nccl && make -j install && cd .. && rm -rf nccl &&
RUN cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd ..
RUN mkdir build && cd build &&     #cmake -DUSE_CUDNN=1 .. && 
    #make -j"$(nproc)"

ENV PYCAFFE_ROOT $CAFFE_ROOT/python
ENV PYTHONPATH $PYCAFFE_ROOT:$PYTHONPATH
ENV PATH $CAFFE_ROOT/build/tools:$PYCAFFE_ROOT:$PATH
RUN echo "$CAFFE_ROOT/build/lib" >> /etc/ld.so.conf.d/caffe.conf && ldconfig

WORKDIR /workspace

3.开始构建image

nvidia-docker build -t caffe-cuda10:gpu gpu

构建成功后可以通过下面查看

docker images

4.创建container

nvidia-docker run -it caffe-cuda10:gpu /bin/bash

通过

nvidia-docker ps -a

查看刚创建的container_id,然后开始使用

nvidia-docker exec -it container_id /bin/bash

5.编译caffe

因为这张gpu卡的算力是7.5的，所以在Dockerfile中没有编译caffe.

修改/opt/caffe/cmake/的Cuda.cmake在第7行加上75

set(Caffe_known_gpu_archs "20 21(20) 30 35 50 60 61 75")

然后cd到build目录下

cmake -DUSE_CUDNN=1 .. 
make -j"$(nproc)"

完成！

备注

如果算力没有超过6.1可以使用下面这个Dockerfile，修改的地方就是在Dockerfile中编译好环境。
就是不高于这个-gencode arch=compute_61,code=compute_61

备注1

FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04
LABEL maintainer caffe-maint@googlegroups.com

RUN apt-get update && apt-get install -y --no-install-recommends         build-essential         vim         python-opencv         python-tk         cmake         git         wget         libatlas-base-dev         libboost-all-dev         libgflags-dev         libgoogle-glog-dev         libhdf5-serial-dev         libleveldb-dev         liblmdb-dev         libopencv-dev         libprotobuf-dev         libsnappy-dev         protobuf-compiler         python-dev         python-numpy         python-pip         python-setuptools         python-scipy &&     rm -rf /var/lib/apt/lists/*

ENV CAFFE_ROOT=/opt/caffe
WORKDIR $CAFFE_ROOT

# FIXME: use ARG instead of ENV once DockerHub supports this
# https://github.com/docker/hub-feedback/issues/460
ENV CLONE_TAG=1.0

RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git .

RUN pip install --upgrade pip &&     pip install pip -U &&     pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple &&     pip install python-dateutil==2.5.0

# git clone https://github.com/NVIDIA/nccl.git && cd nccl && make -j install && cd .. && rm -rf nccl &&
RUN cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd ..
RUN mkdir build && cd build &&     cmake -DUSE_CUDNN=1 .. &&     make -j"$(nproc)"

ENV PYCAFFE_ROOT $CAFFE_ROOT/python
ENV PYTHONPATH $PYCAFFE_ROOT:$PYTHONPATH
ENV PATH $CAFFE_ROOT/build/tools:$PYCAFFE_ROOT:$PATH
RUN echo "$CAFFE_ROOT/build/lib" >> /etc/ld.so.conf.d/caffe.conf && ldconfig

WORKDIR /workspace

备注2

这些依赖是训练的依赖，没有可以不执行。

apt-get update
apt-get install python-skimage
pip install python-dateutil==2.1 -i https://pypi.tuna.tsinghua.edu.cn/simple some-package

通过Dockerfile创建caffe-gpu环境

标签：教程   scipy   his   upgrade   dir   echo   bin   ports   imp   

原文地址：https://www.cnblogs.com/cyssmile/p/14268909.html