• Ubuntu16.04 +cuda8.0+cudnn+caffe+theano+tensorflow配置明细


     

     

    来自:  http://www.cnblogs.com/xujianqing/p/6142963.html

    欢迎关注我的博客:http://blog.csdn.net/hit2015spring和http://www.cnblogs.com/xujianqing

    本文主要是介绍在ubuntu16.04下,怎么配置当下流行的深度学习框架,cuda8.0+cudnn+caffe+theano+tensorflow

    安装英伟达显卡驱动

    首先去官网上查看适合你GPU的驱动

    http://www.nvidia.com/Download/index.aspx?lang=en-us

    sudo add-apt-repository ppa:graphics-drivers/ppa

    sudo apt-get update

    sudo apt-get install nvidia-375(375是你查到的版本号)

    sudo apt-get install mesa-common-dev

    sudo apt-get install freeglut3-dev

    执行完上述后,重启(reboot)。

    重启后输入

    nvidia-smi

    如果出现了你的GPU列表,则说明驱动安装成功了。另外也可以通过,或者输入

    nvidia-settings

    出现

    1. 配置cuda

    https://developer.nvidia.com/cuda-downloads

    在cuda所在目录打开terminal依次输入以下指令:

    sudo dpkg -i cuda-repo-ubuntu1604-8-0-rc_8.0.27-1_amd64​.deb

    sudo apt-get update

    sudo apt-get install cuda​

    ubuntu的gcc编译器是5.4.0,然而cuda8.0不支持5.0以上的编译器,因此需要降级,把编译器版本降到4.9:

    在terminal中执行:

    sudo apt-get install gcc -4.9 gcc-5 g++-4.9 g++-5

    sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 20

    sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 10

    sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.9 20

    sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 10

    sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30

    sudo update-alternatives --set cc /usr/bin/gcc

    sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30

    sudo update-alternatives --set c++ /usr/bin/g++

    配置cuda8.0之后主要加上的一个环境变量声明,在文件~/.bashrc之后加上

    gedit ~/.bashrc

    export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}

    export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

    然后设置环境变量和动态链接库,在命令行输入

    sudo gedit /etc/profile

    在打开的文件里面加上(注意等号两边不能有空格)

    export PATH=/usr/local/cuda/bin:$PATH

    保存之后,创建链接文件

    sudo gedit /etc/ld.so.conf.d/cuda.conf

    在打开的文件中添加如下语句:

    /usr/local/cuda/lib64

    保存退出执行命令行:

    sudo ldconfig

    使链接立即生效。

    2、测试cuda的Samples

    命令行输入(注意cuda-8.0是要相对应自己的cuda版本)

    cd /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery

    make

    sudo ./deviceQuery

    返回GPU的信息则表示配置成功

    3、使用cudnn

    上官网下载对应的cudnn

    https://developer.nvidia.com/cudnn

    下载完cudnn后,命令行输入文件所在的文件夹 (ubuntu为本机用户名)

    cd home/ubuntu/Downloads/

    tar zxvf cudnn-8.0-linux-x64-v5.1.tgz #解压文件

    cd进入cudnn5.1解压之后的include目录,在命令行进行如下操作:

    sudo cp cudnn.h /usr/local/cuda/include/ #复制头文件

    再cd进入lib64目录下的动态文件进行复制和链接:(5.1.5为对应版本具体可修改)

    sudo cp lib* /usr/local/cuda/lib64/ #复制动态链接库

    cd /usr/local/cuda/lib64/

    sudo rm -rf libcudnn.so libcudnn.so.5 #删除原有动态文件

    sudo ln -s libcudnn.so.5.1.5 libcudnn.so.5 #生成软衔接

    sudo ln -s libcudnn.so.5 libcudnn.so #生成软链接

    4、安装opencv3.1.0

    从官网上下载opencv3.1.0

    http://opencv.org/downloads.html

    并将其解压到你要安装的位置,(下载的位置还是在home/ubuntu、Downloads文件夹下)

    首先安装Ubuntu系统需要的依赖项,虽然我也不知道有些依赖项是干啥的,但是只管装就行,也不会占据很多空间的。

    sudo apt-get install --assume-yes libopencv-dev build-essential cmake git libgtk2.0-dev pkg-config python-dev python-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev libtbb-dev libqt4-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip

    然后安装OpenCV需要的一些依赖项,一些文件编码解码之类的东东。

    sudo apt-get install build-essential cmake git

    sudo apt-get install ffmpeg libopencv-dev libgtk-3-dev python-numpy python3-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev libtbb-dev qtbase5-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip

    在终端中cd到opencv文件夹下(解压的那个文件夹),然后

    mkdir build #新建一个build文件夹,编译的工程都在这个文件夹里

    cd build/

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_V4L=ON -D WITH_QT=ON -D WITH_OPENGL=ON -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES" ..(后面两点不要忘记)

    cmake成功后,会出现如下结果,提示配置和生成成功:

    -- Configuring done

    -- Generating done

    -- Build files have been written to: /home/ise/software/opencv-3.1.0/build

    由于CUDA 8.0不支持OpenCV的 GraphCut 算法,可能出现以下错误:

    /home/usrname/opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp:120:54: error: 'NppiGraphcutState' has not been declared

    typedef NppStatus (*init_func_t)(NppiSize oSize, NppiGraphcutState** ppStat

    ^

    /home/usrname/opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp:135:18: error: 'NppiGraphcutState' does not name a type

    operator NppiGraphcutState*()

    ^

    /home/usrname/opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp:141:9: error: 'NppiGraphcutState' does not name a type

    NppiGraphcutState* pState;

    .......

    进入opencv-3.1.0/modules/cudalegacy/src/目录,修改graphcuts.cpp文件,将:

    #include "precomp.hpp"

    #if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)

    改为

    #include "precomp.hpp"

    #if !defined (HAVE_CUDA) || defined (CUDA_DISABLER) || (CUDART_VERSION >= 8000)

    然后make编译就可以了

    make -j8

    上面是将opencv编译成功,但是并没有安装到我们的系统中,有很多的设置都没有写入到系统中,因此还要进行install。

    sudo make install

    sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'

    sudo ldconfig

    重启系统,重启系统后cd到build文件夹下:

    sudo apt-get install checkinstall

    sudo checkinstall

    然后按照提示安装就可以了。

    使用checkinstall的目的是为了更好的管理我安装的opencv,因为opencv的安装很麻烦,卸载更麻烦,其安装的时候修改了一大堆的文件,当我想使用别的版本的opencv时,将当前版本的opencv卸载就是一件头疼的事情,因此需要使用checkinstall来管理我的安装。

    执行了checkinstall后,会在build文件下生成一个以backup开头的.tgz的备份文件和一个以build开头的.deb安装文件,当你想卸载当前的opencv时,直接执行dpkg -r build即可。

    5、配置caffe环境

    切换编译器

    选择g++ 5.0以上的对应编号

    sudo update-alternatives --config g++

    sudo update-alternatives --config gcc

    安装依赖库

    sudo add-apt-repository universe

    sudo apt-get update -y

    sudo apt-get install cmake -y

    # General Dependencies

    sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev

    libhdf5-serial-dev protobuf-compiler -y

    sudo apt-get install --no-install-recommends libboost-all-dev -y

    # BLAS

    sudo apt-get install libatlas-base-dev -y

    # Remaining Dependencies

    sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev -y

    sudo apt-get install python-dev python-numpy –y

    sudo apt-get install -y python-pip

    sudo apt-get install -y python-dev

    sudo apt-get install -y python-numpy python-scipy

    编译 Caffe,cd到要安装caffe的位置

    git clone https://github.com/BVLC/caffe.git

    cd caffe

    cp Makefile.config.example Makefile.config

    修改Makefile.config:

    gedit Makefile.config

    对打开的文件编辑

    # cuDNN acceleration switch (uncomment to build with cuDNN).

    USE_CUDNN := 1

     

    # Uncomment if you're using OpenCV 3 如果用的是opencv3版本

    OPENCV_VERSION := 3

     

    # Uncomment to support layers written in Python (will link against Python libs)

    WITH_PYTHON_LAYER := 1

    在问件里面添加文本由于hdf5库目录更改,所以需要单独添加:

    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/aarch64-linux-gnu/hdf5/serial/

     

    打开makefile文件

    gedit Makefile

    NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)

    替换

    NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

    编辑/usr/local/cuda/include/host_config.h,将其中的第115行注释掉:

    sudo gedit /usr/local/cuda/include/host_config.h

    #error-- unsupported GNU version! gcc versions later than 4.9 are not supported!

     

    改为

    //#error-- unsupported GNU version! gcc versions later than 4.9 are not supported!

    之后编辑即可

    make -j4 all

    make -j4 runtest

    为了更好地使用pycaffe ,建议安装:

    sudo apt-get install python-numpy python-setuptools python-pip cython python-skimage python-protobuf

    make pycaffe

    cd python

    python

    import caffe #测试安装成功

    到这里Caffe开发环境就配置好了!

    可以测试一下,输出AlexNet的时间测试结果:

    cd ~/caffe

    ./build/tools/caffe time --gpu 0 --model ./models/bvlc_alexnet/deploy.prototxt

    6、theano安装

    1、直接输入命令:

    sudo pip install theano

    2、配置参数文件:.theanorc

    sudo gedit ~/.theanorc

    对打开的文件进行编辑

    [global]

    floatX=float32

    device=gpu

    base_compiledir=~/external/.theano/

    allow_gc=False

    warn_float64=warn

    [mode]=FAST_RUN

     

    [nvcc]

    fastmath=True

     

    [cuda]

    root=/usr/local/cuda

     

    3、运行测试例子:

    sudo Vim test.py

    from theano import function, config, shared, sandbox

    import theano.tensor as T

    import numpy

    import time

     

    vlen = 10 * 30 * 768 # 10 x #cores x # threads per core

    iters = 1000

     

    rng = numpy.random.RandomState(22)

    x = shared(numpy.asarray(rng.rand(vlen), config.floatX))

    f = function([], T.exp(x))

    print(f.maker.fgraph.toposort())

    t0 = time.time()

    for i in range(iters):

    r = f()

    t1 = time.time()

    print("Looping %d times took %f seconds" % (iters, t1 - t0))

    print("Result is %s" % (r,))

    if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):

    print('Used the cpu')

    else:

    print('Used the gpu')

     

    可以看到结果:

    /usr/bin/python2.7 /home/hjimce/PycharmProjects/untitled/.idea/temp.py

    Using gpu device 0: GeForce GTX 960 (CNMeM is disabled, cuDNN not available)

    [GpuElemwise{exp,no_inplace}(<CudaNdarrayType(float32, vector)>), HostFromGpu(GpuElemwise{exp,no_inplace}.0)]

    Looping 1000 times took 0.302778 seconds

    Result is [ 1.23178029 1.61879349 1.52278066 ..., 2.20771813 2.29967761

    1.62323296]

    Used the gpu

    说明安装成功

    7、tensorflow 安装

    https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md

    先安装anaconda

    https://repo.continuum.io/archive/Anaconda2-4.2.0-Windows-x86_64.exe

    上面的地址下载 该包默认在downloads里面

    cd /home/username/Downloads

    sudo bash Anaconda2-4.2.0-Linux-x86_64.sh

    配置环境变量

    gedit /etc/profile

    末尾添上,我是一路yes下来,所以安在了root下,你可以自己选路径,这时候的环境变量要改

    export PATH=/root/anaconda2/bin:$PATH

    重启

    打开终端

    python

    安装成功

    2、创建conda环境 名字叫tensorflow

    conda create -n tensorflow python=2.7

    source activate tensorflow #使能该环境

    #下面这句话只能下载给CPU用的tensorflow

    conda install -c conda-forge tensorflow

    利用pip来下载给GPU用的tensorflow

    export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl

    下载安装

    pip install --ignore-installed --upgrade $TF_BINARY_URL

    安装IPython

    conda install ipython

    关掉该环境

    source deactivate

    测试安装是否正确

    source activate tensorflow

    python

    输入

    import tensorflow as tf

    import numpy as np

    # Create 100 phony x, y data points in NumPy, y = x * 0.1 + 0.3

    x_data = np.random.rand(100).astype(np.float32)

    y_data = x_data * 0.1 + 0.3

    # Try to find values for W and b that compute y_data = W * x_data + b

    # (We know that W should be 0.1 and b 0.3, but TensorFlow will

    # figure that out for us.)

    W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))

    b = tf.Variable(tf.zeros([1]))

    y = W * x_data + b

    # Minimize the mean squared errors.

    loss = tf.reduce_mean(tf.square(y - y_data))

    optimizer = tf.train.GradientDescentOptimizer(0.5)

    train = optimizer.minimize(loss)

    # Before starting, initialize the variables. We will 'run' this first.

    init = tf.initialize_all_variables()

    # Launch the graph.

    sess = tf.Session()

    sess.run(init)

    # Fit the line.

    for step in range(201):

    sess.run(train)

    if step % 20 == 0:

    print(step, sess.run(W), sess.run(b))

    # Learns best fit is W: [0.1], b: [0.3]

    OK

    8、Caffe配置错误

    问题:找不到Python.h

    解决:给anaconda添加环境变量

    gedit ~/.banshrc

    添加

    export PATH=/root/anaconda2/bin:$PATH

    export PYTHONPATH=/path/to/caffe/python:$PATH

    修改Makefile.config

    在终端输入

    locate Python.h

    gedit Makefile.config

    在INCLUDE_DIRS 和LIBRARY_DIRS后面添上

    /root/anaconda2/include/python2.7

    启用

    ANACONDA_HOME := $(HOME)/anaconda2

    PYTHON_ INCLUDE =$(ANACONDA_HOME)/include

    ,把前面的#去掉,那三行都去掉#,并在注释上面,

    注释这两句PYTHON_INCLUDE := /usr/include/python2.7

    /usr/lib/python2.7…………..

    如果编译的时候发现有错,回来改完之后又得重新编译一遍pycaffe,于是出现如下错误

    make: Nothing to be done for 'pycaffe'

    则在终端输入:

    sudo make clean

    修改完后再

    sudo make pycaffe

    这里要从make –j8 all那一步开始编译

    编译完后,显示

    然后 cd python进入该目录

    python

    import caffe

    若此时提示错误:

    Traceback (most recent call last)

    File

    ImportError: /home/../anaconda2/lib/python2.7/site-packages/zmq/backend/cython/../../../../.././libstdc++.so.6: versionGLIBCXX_3.4.21' not found

    解决:

    https://github.com/BVLC/caffe/issues/4953

    https://gitter.im/BVLC/caffe/archives/2015/08/20

    cd ..

    pip install protobuf

    sudo apt-get install python-protobuf

    coda install libgcc

     
    分类: 学习笔记
     
    好文要顶 关注我 收藏该文  
    0
    0
     
     
     
    « 上一篇:Caffe学习笔记3
    » 下一篇:Caffe学习笔记3
    posted @ 2016-12-07 21:58 晨凫追风 阅读(2721) 评论(4) 编辑 收藏

     

     
    #1楼 2016-12-08 20:28 | 龙将  
    博主,你好。我按照你的博客安装tensorflow,在import tensorflow as tf 
    验证时出现
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    ImportError: No module named tensorflow
    前面的都一样的结果,能告知一下是怎么回事,万分感谢
    #2楼[楼主] 2016-12-08 20:35 | 晨凫追风  
    @ 龙将
    那是一整段的Python代码,全部粘贴进去,试试
    #3楼[楼主] 2016-12-08 20:37 | 晨凫追风  
    @ 龙将
    source activate tensorflow
    这句话用来激活该环境
    #4楼 2017-01-12 11:30 | tzx0  
    十分感谢!外,opencv 3.2 不用修改源码就可以正确编译了。
     
     
    发表评论
  • 相关阅读:
    寒假Day31:CSU1508地图的四着色bfs+dfs
    寒假Day32:链式前向星
    寒假Day30:HDU4507吉哥系列故事恨7不成妻数位dp
    寒假Day35:HTML表格+图像+超链接
    寒假Day35:2018蓝桥杯 B组
    寒假Day33:HTML入门
    寒假Day30:二叉树的遍历相关题型(求先序或后序、搜索树的判断)
    POJ 1177 Picture (线段树+离散化+扫描线) 详解
    MFC对话框中文出现乱码的解决方法
    如何枚举系统COM串口
  • 原文地址:https://www.cnblogs.com/leoking01/p/6892737.html
Copyright © 2020-2023  润新知