• [转]caffe的配置过程


    caffe的配置过程

     
    这篇文章可参考:http://www.cnblogs.com/cj695/p/4498270.html

    下载caffe
    拷贝Make.config.example Make.config
    修改Make.config 取消注释cpu-only,注释掉cuda相关的


    -----------------安装依赖的库------------------------

    参考docs/install_yum.md,执行:
    yum install protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel

    实际上,hdf5-devel这个安装会找不到。

    但是:yum install gflags-devel glog-devel lmdb-devel这个不成功,找不到。只能手动安装:
    # glog
        wget https://google-glog.googlecode.com/files/glog-0.3.3.tar.gz
        tar zxvf glog-0.3.3.tar.gz
        cd glog-0.3.3
        ./configure
        make && make install
    连不上 google-glog.googlecode.com
    从这里下载:
    http://download.csdn.net/download/chenguangxing3/6661667
    安装glog。
    ./configure --preifx=mypath
    make
    make install


        # gflags
        wget https://github.com/schuhschuh/gflags/archive/master.zip
        unzip master.zip
        cd gflags-master
        mkdir build && cd build
        export CXXFLAGS="-fPIC" && cmake .. && make VERBOSE=1
        make && make install

    cmake的时候,提示版本不够,需要2.8.12或以上,我的是2.8.11
    升级cmake:https://cmake.org/files/v3.4/cmake-3.4.3.tar.gz
     ./bootstrap
    gmake
    gmake install

    再次安装gflags成功。


        # lmdb
        git clone https://github.com/LMDB/lmdb
        cd lmdb/libraries/liblmdb
        make && make install
    安装成功。


    安装atlas:
    yum install atlas


    安装python 开发包:
     yum install python-devel

    安装cython: http://cython.org/  下载0.24版的
    http://cython.org/release/Cython-0.24.zip
    进入解压路径,python setup.py install
    耗时3分钟左右。


    安装numpy
    http://nbtelecom.dl.sourceforge.net/project/numpy/NumPy/1.8.0/numpy-1.8.0.zip
    耗时3分钟左右。



    ------------------------编译安装caffe--------------------------
    参考:http://caffe.berkeleyvision.org/installation.html中的Compilation with Make部分。


    cp Makefile.config.example Makefile.config
    # Adjust Makefile.config (for example, if using Anaconda Python, or if cuDNN is desired)
    make all
    make test
    make runtest

    修改:
    # NOTE: this is required only if you will compile the python interface.
    # We need to be able to find Python.h and numpy/arrayobject.h.
    PYTHON_INCLUDE := /usr/include/python2.7
                   /usr/lib/python2.7/dist-packages/numpy/core/include

    为:(因为我的arrayobject.h在另外lib64的路径下)
    PYTHON_INCLUDE := /usr/include/python2.7
                    /usr/lib64/python2.7/site-packages/numpy/core/include


    修改:打开CPU_ONLY := 1,并注释掉:USE_CUDNN := 1。因为我的电脑没有gpu相关的东西。

    其他的没改动。

    报错:
    CXX src/caffe/blob.cpp
    In file included from ./include/caffe/blob.hpp:8:0,
                     from src/caffe/blob.cpp:4:
    ./include/caffe/common.hpp:6:26: fatal error: glog/logging.h: No such file or directory
     #include <glog/logging.h>
    因为没有装glog,所以再找办法装上先。

    安装后继续编译,报错:
    CXX src/caffe/blob.cpp
    In file included from ./include/caffe/util/math_functions.hpp:11:0,
                     from src/caffe/blob.cpp:7:
    ./include/caffe/util/mkl_alternate.hpp:11:19: fatal error: cblas.h: No such file or directory
     #include <cblas.h>
    没有装cblas,参考:http://blog.csdn.net/cleverysm/article/details/1925549,http://www.linuxidc.com/Linux/2015-02/113169.htm
    http://www.netlib.org/blas/blast-forum/cblas.tgz
    具体:
    1. 编译blas,进入BLAS目录执行下面的命令

        gfortran -c  -O3    *.f                # 编译所有的 .f 文件,生成 .o文件 
        ar rv libblas.a      *.o                # 链接所有的 .o文件,生成 .a 文件 

    2. 编译cblas,进入CBLAS目录,首先根据自己的操作系统平台,将某个Makefiel.XXX复制为Makefile.in,XXX表示操作系统。如果是Linux,那么就将Makefile.LINUX 复制为 Makefile.in。

        cp ../BLAS/libblas.a  testing  # 将上一步编译成功的 libblas.a 复制到 CBLAS目录下的testing子目录 
        make                                            # 编译所有的目录 

    此时会在CBLAS安装目录下的lib目录中产生一个静态链接库文件cblas_LINUX.a,这个库文件和上面得到的libblas.a文件就是我们所需要的。另外还需要的就是CBLAS/include中的cblas.h头文件。将三个文件全部拷贝到,你需调用的应用程序源码目录中。

    拷贝到路径下:
    cp include/cblas.h /usr/include/
    cp lib/cblas_LINUX.a /usr/local/libcblas.a
    cp testing/libblas.a /usr/local/lib


    再次编译,报错:
    src/caffe/layers/hdf5_data_layer.cpp:13:18: fatal error: hdf5.h: No such file or directory
     #include "hdf5.h"
    记得上面已经安装了hdf5-devel,怎么会出这个错?查看一下,发现yum这个没找到hdf5-devel,所以也就没有安装。
    在docs/installion.md中,建议:but we suggest first installing the [Anaconda](https://store.continuum.io/cshop/anaconda/) Python distribution, which provides most of the necessary packages, as well as the `hdf5` library dependency.

    准备安装hdf5:
    http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.16.tar
    解压后:
    ./configure --prefix=/usr/local/hdf5-1.8.3   ##指定安装路径,否则会默认安装到当前路径下。
    make ##4分钟左右
    make install

    此时,在Makefile.config中增加HDF5_DIRS的路径到搜索路径下:
    HDF5_DIRS :=/usr/local/hdf5-1.8.3/
    # Whatever else you find you need goes here.
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include $(HDF5_DIRS)/include
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib $(HDF5_DIRS)/lib



    再次编译,报错:
    In file included from src/caffe/util/db.cpp:2:0:
    ./include/caffe/util/db_leveldb.hpp:7:24: fatal error: leveldb/db.h: No such file or directory
     #include "leveldb/db.h"
                            ^
    compilation terminated.
    缺少了google的kv数据库:leveldb,这个在installation.md中的Optional dependencies也提到了。
    从这里下载:https://github.com/google/leveldb
    参考:https://techoverflow.net/blog/2012/12/14/compiling-installing-leveldb-on-linux/

    解压后,make
    因为没有不支持install,所以,要手工将这些include和生成的lib拷贝到需要的地方。
    cp --preserve=links out-shared/libleveldb.so* /usr/local/lib  #只拷贝了动态库。
    cp -r include/leveldb/ /usr/local/include/


    再次编译,报错:
    CXX src/caffe/util/upgrade_proto.cpp
    AR -o .build_release/lib/libcaffe.a
    LD -o .build_release/lib/libcaffe.so.1.0.0-rc3
    /usr/bin/ld: cannot find -lcblas
    /usr/bin/ld: cannot find -latlas
    collect2: error: ld returned 1 exit status
    make: *** [.build_release/lib/libcaffe.so.1.0.0-rc3] Error 1

    静态库的链接已经完成了,这个动态库缺少了东西。上面安装cblas的时候,发现确实是只产生了静态库,需要增加动态库。
    进入到CBLAS目录下,将静态库转为动态库:
    gcc -shared lib/cblas_LINUX.a testing/libblas.a -o libcblas.so
    cp libcblas.so /usr/local/


    再次编译,报错:
    CXX src/caffe/util/upgrade_proto.cpp
    AR -o .build_release/lib/libcaffe.a
    LD -o .build_release/lib/libcaffe.so.1.0.0-rc3
    /usr/bin/ld: cannot find -lcblas
    /usr/bin/ld: cannot find -latlas
    collect2: error: ld returned 1 exit status

    mv /usr/local/libcblas.* /usr/local/lib/  #就是把它们转移到 /usr/local/lib下,否则还会报错


    再次编译,报错:
    LD -o .build_release/lib/libcaffe.so.1.0.0-rc3
    /usr/bin/ld: cannot find -latlas
    collect2: error: ld returned 1 exit status
    make: *** [.build_release/lib/libcaffe.so.1.0.0-rc3] Error 1

    之前有安装 yum install atlas
    查看:
    [root@localhost caffe-master]# ll /usr/lib64/atlas/
    total 21304
    lrwxrwxrwx. 1 root root       17 Apr  8 15:54 libsatlas.so.3 -> libsatlas.so.3.10
    -rwxr-xr-x. 1 root root 10852104 Nov 20 13:43 libsatlas.so.3.10
    lrwxrwxrwx. 1 root root       17 Apr  8 15:54 libtatlas.so.3 -> libtatlas.so.3.10
    -rwxr-xr-x. 1 root root 10959464 Nov 20 13:43 libtatlas.so.3.10


    再安装yum install atlas-devel,查看:
    [root@localhost caffe-master]# ll /usr/lib64/atlas/
    total 21304
    lrwxrwxrwx. 1 root root       17 Apr  9 14:03 libsatlas.so -> libsatlas.so.3.10
    lrwxrwxrwx. 1 root root       17 Apr  8 15:54 libsatlas.so.3 -> libsatlas.so.3.10
    -rwxr-xr-x. 1 root root 10852104 Nov 20 13:43 libsatlas.so.3.10
    lrwxrwxrwx. 1 root root       17 Apr  9 14:03 libtatlas.so -> libtatlas.so.3.10
    lrwxrwxrwx. 1 root root       17 Apr  8 15:54 libtatlas.so.3 -> libtatlas.so.3.10
    -rwxr-xr-x. 1 root root 10959464 Nov 20 13:43 libtatlas.so.3.10

    参考:http://blog.sina.com.cn/s/blog_4c4668bb01013gsv.html
    发现里面有说到:
    在 3.10.0 的 ATLAS 中没有了 --with-netlib-lapack指定lapack_LINUX.a 只需要 --with-netlib-lapack-tarfile,给出下载的包就行,不需要自己在去编译 LAPACK了。但是最后生成的 库文件也给整合了,不是上面的6个,而是两个 
    libsatals.so libtatlas.so, 去 BUILD/lib 里面查看 Makefile, 可以看到确实被整合了的。为了依其上的软件能更通用,又不想自己去改Makefile的内容,就用老版本的组合了。

    于是,我建立一个软链接:
    ln -s /usr/lib64/atlas/libsatlas.so /usr/lib64/libatlas.so
    (如果我建立在atlas下,即ln -s /usr/lib64/atlas/libsatlas.so /usr/lib64/atlas/libatlas.so  还是找不到,就把它提到外面来了)


    再次编译,这次ok了。
    [root@localhost caffe-master]# make all
    LD -o .build_release/lib/libcaffe.so.1.0.0-rc3
    CXX tools/caffe.cpp
    CXX/LD -o .build_release/tools/caffe.bin
    CXX tools/compute_image_mean.cpp
    CXX/LD -o .build_release/tools/compute_image_mean.bin
    CXX tools/convert_imageset.cpp
    CXX/LD -o .build_release/tools/convert_imageset.bin
    CXX tools/device_query.cpp
    CXX/LD -o .build_release/tools/device_query.bin
    CXX tools/extract_features.cpp
    CXX/LD -o .build_release/tools/extract_features.bin
    CXX tools/finetune_net.cpp
    CXX/LD -o .build_release/tools/finetune_net.bin
    CXX tools/net_speed_benchmark.cpp
    CXX/LD -o .build_release/tools/net_speed_benchmark.bin
    CXX tools/test_net.cpp
    CXX/LD -o .build_release/tools/test_net.bin
    CXX tools/train_net.cpp
    CXX/LD -o .build_release/tools/train_net.bin
    CXX tools/upgrade_net_proto_binary.cpp
    CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
    CXX tools/upgrade_net_proto_text.cpp
    CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
    CXX tools/upgrade_solver_proto_text.cpp
    CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
    CXX examples/cifar10/convert_cifar_data.cpp
    CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
    CXX examples/cpp_classification/classification.cpp
    CXX/LD -o .build_release/examples/cpp_classification/classification.bin
    CXX examples/mnist/convert_mnist_data.cpp
    CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
    CXX examples/siamese/convert_mnist_siamese_data.cpp
    CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin
    [root@localhost caffe-master]# 


    接着是make test,没有问题,花费5分钟左右。
    接着:make runtest,报错:
    [root@localhost caffe-master]# make runtest
    .build_release/tools/caffe
    .build_release/tools/caffe: error while loading shared libraries: libglog.so.0: cannot open shared object file: No such file or directory
    make: *** [runtest] Error 127

    但是在/usr/local/lib下有阿?
    [root@localhost glog-0.3.3]# ll /usr/local/lib
    total 5532
    drwxr-xr-x. 3 root root     19 Apr  8 16:32 cmake
    -rwxr-xr-x. 1 root root 130336 Mar 25 10:59 default.sfx
    -rw-r--r--. 1 root root 792216 Apr  8 17:17 libblas.a
    -rw-r--r--. 1 root root 387232 Apr  8 17:16 libcblas.a
    -rwxr-xr-x. 1 root root   7691 Apr  9 11:39 libcblas.so
    -rw-r--r--. 1 root root 628106 Apr  8 16:31 libgflags.a
    -rw-r--r--. 1 root root 627712 Apr  8 16:31 libgflags_nothreads.a
    -rw-r--r--. 1 root root 352318 Apr  8 16:59 libglog.a
    -rwxr-xr-x. 1 root root    958 Apr  8 16:59 libglog.la
    lrwxrwxrwx. 1 root root     16 Apr  8 16:59 libglog.so -> libglog.so.0.0.0
    lrwxrwxrwx. 1 root root     16 Apr  8 16:59 libglog.so.0 -> libglog.so.0.0.0
    -rwxr-xr-x. 1 root root 524192 Apr  8 16:59 libglog.so.0.0.0
    -rwxr-xr-x. 3 root root 413462 Apr  9 11:26 libleveldb.so
    -rwxr-xr-x. 3 root root 413462 Apr  9 11:26 libleveldb.so.1
    -rwxr-xr-x. 3 root root 413462 Apr  9 11:26 libleveldb.so.1.18
    -rw-r--r--. 1 root root 644748 Apr  8 16:33 liblmdb.a
    -rwxr-xr-x. 1 root root 304783 Apr  8 16:33 liblmdb.so
    drwxr-xr-x. 2 root root     23 Apr  8 16:59 pkgconfig

    临时解决方法:
    export LD_LIBRARY_PATH=/usr/local/lib/
    继续make runtest,报错:
    [root@localhost caffe-master]# make runtest
    .build_release/tools/caffe
    .build_release/tools/caffe: error while loading shared libraries: libhdf5_hl.so.10: cannot open shared object file: No such file or directory
    make: *** [runtest] Error 127

    再来:export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/hdf5-1.8.3/lib
    这次ok了:
    [ RUN      ] NeuronLayerTest/0.TestExpLayer
    [       OK ] NeuronLayerTest/0.TestExpLayer (0 ms)
    [ RUN      ] NeuronLayerTest/0.TestExpGradientBase2Shift1
    [       OK ] NeuronLayerTest/0.TestExpGradientBase2Shift1 (1 ms)
    [ RUN      ] NeuronLayerTest/0.TestLogLayerBase2
    [       OK ] NeuronLayerTest/0.TestLogLayerBase2 (0 ms)
    [ RUN      ] NeuronLayerTest/0.TestLogGradientBase2Shift1Scale3
    [       OK ] NeuronLayerTest/0.TestLogGradientBase2Shift1Scale3 (2 ms)
    [----------] 48 tests from NeuronLayerTest/0 (175 ms total)

    [----------] Global test environment tear-down
    [==========] 1050 tests from 146 test cases ran. (34229 ms total)
    [  PASSED  ] 1050 tests.
    但这个不是好的解决方法,我得永久修改这个链接路径才行。
    方法:
    查看 /etc/ld.so.conf
    发现引用/etc/ld.so.conf.d/下的所有conf结尾的文件,进入里面
    可以看到那个atlas的内容就一句话,指明了lib的路径:/usr/lib64/atlas
    新建一个conf文件:/etc/ld.so.conf.d/glog.conf,里面写:/usr/local/lib
    再新建一个conf文件:/etc/ld.so.conf.d/hdf5.conf,里面写:/usr/local/hdf5-1.8.3/lib
    最后执行:
    /sbin/ldconfig -v

    这样,就不用再去搞那个LD_LIBRARY_PATH了。


    重新执行一次完整流程:
    make clean
    make all  #3分钟
    make test #4分钟
    make runtest #1分钟
    make pycaffe #40秒

    编译python的包:
    [root@localhost caffe-master]# make pycaffe
    CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
    touch python/caffe/proto/__init__.py
    PROTOC (python) src/caffe/proto/caffe.proto
    [root@localhost caffe-master]# 

    编译好了python包后,要把这个模块加入到python的路径下:(参考安装说明:http://caffe.berkeleyvision.org/installation.html python部分)
    export PYTHONPATH=/home/zzz/OpenSource/caffe-master/python:$PYTHONPATH
    编辑/etc/profile,增加:
    export PYTHONPATH=$PYTHONPATH:/home/zzz/OpenSource/caffe-master/python/


    然后打开python,导入caffe:
    [root@localhost caffe-master]# python
    Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import caffe
    /home/zzz/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Net<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    /home/zzz/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Blob<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    /home/zzz/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Solver<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/zzz/OpenSource/caffe-master/python/caffe/__init__.py", line 1, in <module>
        from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver
      File "/home/zzz/OpenSource/caffe-master/python/caffe/pycaffe.py", line 15, in <module>
        import caffe.io
      File "/home/zzz/OpenSource/caffe-master/python/caffe/io.py", line 2, in <module>
        import skimage.io
    ImportError: No module named skimage.io
    >>> 


    报错缺少 skimage.io
    下载:https://pypi.python.org/packages/source/s/scikit-image/scikit-image-0.12.3.tar.gz#md5=04ea833383e0b6ad5f65da21292c25e1
    解压,进入,
    python setup.py install
    不成功,提示:
    te-packages
    Adding scikit-image 0.12.3 to easy-install.pth file
    Installing skivi script to /usr/bin

    Installed /usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg
    Processing dependencies for scikit-image==0.12.3
    Searching for dask[array]>=0.5.0
    Reading https://pypi.python.org/simple/dask/
    Best match: dask 0.8.1.macosx-10.5-x86-64
    Downloading https://pypi.python.org/packages/any/d/dask/dask-0.8.1.macosx-10.5-x86_64.tar.gz#md5=6271cc3687493136a6b743dc5271ab80
    Processing dask-0.8.1.macosx-10.5-x86_64.tar.gz
    error: Couldn't find a setup script in /tmp/easy_install-h26472/dask-0.8.1.macosx-10.5-x86_64.tar.gz
    [root@localhost scikit-image-0.12.3]# python

    在python里,import skimage.io会报:
    >>> import skimage.io
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/io/__init__.py", line 7, in <module>
        from .manage_plugins import *
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/io/manage_plugins.py", line 28, in <module>
        from .collection import imread_collection_wrapper
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/io/collection.py", line 12, in <module>
        from PIL import Image
    ImportError: No module named PIL



    原来用easy_install 这么方便!这个easy_install 我的系统有。
    easy_install Image
    会自动搜索依赖。
      File "/usr/lib64/python2.7/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "setup.py", line 512, in build_extensions
    ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting

    没有安装成功。



    下载:
    http://effbot.org/media/downloads/Imaging-1.1.7.tar.gz
    解压后,进入目录,执行:
    python setup.py install
    成功:
    running install_scripts
    copying build/scripts-2.7/pilconvert.py -> /usr/bin
    copying build/scripts-2.7/pildriver.py -> /usr/bin
    copying build/scripts-2.7/pilfile.py -> /usr/bin
    copying build/scripts-2.7/pilfont.py -> /usr/bin
    copying build/scripts-2.7/pilprint.py -> /usr/bin
    changing mode of /usr/bin/pilconvert.py to 755
    changing mode of /usr/bin/pildriver.py to 755
    changing mode of /usr/bin/pilfile.py to 755
    changing mode of /usr/bin/pilfont.py to 755
    changing mode of /usr/bin/pilprint.py to 755
    running install_egg_info
    Writing /usr/lib64/python2.7/site-packages/PIL/PIL-1.1.7-py2.7.egg-info
    creating /usr/lib64/python2.7/site-packages/PIL.pth
    [root@localhost Imaging-1.1.7]# 

    再次
    >>> import skimage.io
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/io/__init__.py", line 11, in <module>
        from ._io import *
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/io/_io.py", line 7, in <module>
        from ..color import rgb2grey
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/color/__init__.py", line 1, in <module>
        from .colorconv import (convert_colorspace,
      File "/usr/lib64/python2.7/site-packages/scikit_image-0.12.3-py2.7-linux-x86_64.egg/skimage/color/colorconv.py", line 58, in <module>
        from scipy import linalg
    ImportError: No module named scipy

    安装:
    [root@localhost Downloads]# easy_install scipy
    Searching for scipy
    Reading https://pypi.python.org/simple/scipy/
    Best match: scipy 0.17.0
    Downloading https://pypi.python.org/packages/source/s/scipy/scipy-0.17.0.zip#md5=28a4fe29e980804db162524f10873211
    Processing scipy-0.17.0.zip
    Writing /tmp/easy_install-YgsnC3/scipy-0.17.0/setup.cfg
    Running scipy-0.17.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-YgsnC3/scipy-0.17.0/egg-dist-tmp-wyWVj4
    /usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:1428: UserWarning: 
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      warnings.warn(AtlasNotFoundError.__doc__)
    /usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:1439: UserWarning: 
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      warnings.warn(LapackNotFoundError.__doc__)
    /usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:1442: UserWarning: 
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      warnings.warn(LapackSrcNotFoundError.__doc__)
    Running from scipy source directory.
    error: no lapack/blas resources found
    [root@localhost Downloads]# 
    失败!
    从网上下载源码:http://ufpr.dl.sourceforge.net/project/scipy/scipy/0.16.1/scipy-0.16.1.tar.gz
    python setup.py install
    同样报错:
    File "/usr/lib64/python2.7/site-packages/numpy/distutils/misc_util.py", line 935, in get_subpackage
        caller_level = caller_level + 1)
      File "/usr/lib64/python2.7/site-packages/numpy/distutils/misc_util.py", line 872, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "scipy/linalg/setup.py", line 20, in configuration
        raise NotFoundError('no lapack/blas resources found')
    numpy.distutils.system_info.NotFoundError: no lapack/blas resources found
    [root@localhost scipy-0.16.1]# 
    看来要先解决这两个问题。


    反复折腾都没有搞定。。。。
    参考:http://blog.sina.com.cn/s/blog_62dfdc740101aoo6.html
    重新编译atlas。 16:33~16:39



    在等待过程,重新看numpy:
    cp site.conf.example site.conf
    将site.conf编辑为:
    [DEFAULT]
    library_dirs = /usr/local/lib
    include_dirs = /usr/local/include
    src_dirs = /home/gumh/Downloads/BLAS-3.6.0:/home/gumh/Downloads/lapack-3.1.1

    [blas_opt]
    libraries =f77blas,cblas,atlas

    [lapack_opt]
    libraries=lapack,f77blas,cblas,atlas


     [atlas]
     library_dirs = /usr/lib64/atlas/
     include_dirs = /usr/include/atlas

    [amd]
    amd_libs = amd
    #
    [umfpack]
    umfpack_libs = umfpack

    [fftw]
    libraries = fftw3



    然后执行:python setup.py build --fcompiler=gnu95
    执行下去了,可以看到中间有在对上面设定的src路径下的代码进行编译。
    然后python setup.py install

    按照这个site.conf来去build scipy-0.16.1,失败。
    重新下载scipy-0.11.0 。https://pypi.python.org/packages/source/s/scipy/scipy-0.11.0.tar.gz#md5=842c81d35fd63579c41a8ca21a2419b9
    解压,啥都没改u,阅读了那个INSTALL.txt:
    python setup.py install
    直接就运行了,中间看到有在编译那个lapack-3.1.1的文件,我在这里并没有指定,应该是上面numpy中指定的被保存起来了。
    后面还是挂了:
    /usr/bin/gfortran -Wall -Wall -shared build/temp.linux-x86_64-2.7/scipy/integrate/_odepackmodule.o -L/usr/local/lib -L/usr/lib64 -Lbuild/temp.linux-x86_64-2.7 -lodepack -llinpack_lite -lmach -lblas -lpython2.7 -lgfortran -o build/lib.linux-x86_64-2.7/scipy/integrate/_odepack.so
    /usr/bin/ld: /usr/local/lib/libblas.a(dcopy.o): relocation R_X86_64_PC32 against undefined symbol `memcpy@@GLIBC_2.14' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: final link failed: Bad value
    collect2: error: ld returned 1 exit status
    /usr/bin/ld: /usr/local/lib/libblas.a(dcopy.o): relocation R_X86_64_PC32 against undefined symbol `memcpy@@GLIBC_2.14' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: final link failed: Bad value
    collect2: error: ld returned 1 exit status
    error: Command "/usr/bin/gfortran -Wall -Wall -shared build/temp.linux-x86_64-2.7/scipy/integrate/_odepackmodule.o -L/usr/local/lib -L/usr/lib64 -Lbuild/temp.linux-x86_64-2.7 -lodepack -llinpack_lite -lmach -lblas -lpython2.7 -lgfortran -o build/lib.linux-x86_64-2.7/scipy/integrate/_odepack.so" failed with exit status 1
    [root@localhost scipy-0.11.0]# 

    意思是/usr/local/lib/libblas.a这个是静态库,不能用于pic的动态链接。
    那我就要编译一个动态库出来。
    于是重新进入

        gfortran -c  -O3  -fPIC  *.f                #加上了-fPIC
        gcc -shared *.o -fPIC -o  libblas.so
     cp libblas.so /usr/local/lib/

    再次执行 python setup.py install:
    这次完成了:
    /fftpack/tests/
    copying scipy/fftpack/tests/fftw_single_ref.npz -> /usr/lib64/python2.7/site-packages/scipy/fftpack/tests/
    copying scipy/fftpack/tests/gen_fftw_ref.py -> /usr/lib64/python2.7/site-packages/scipy/fftpack/tests/
    creating /usr/lib64/python2.7/site-packages/scipy/weave/doc
    copying scipy/weave/doc/tutorial.txt -> /usr/lib64/python2.7/site-packages/scipy/weave/doc/
    copying scipy/weave/doc/tutorial_original.html -> /usr/lib64/python2.7/site-packages/scipy/weave/doc/
    running install_egg_info
    Writing /usr/lib64/python2.7/site-packages/scipy-0.11.0-py2.7.egg-info
    running install_clib
    [root@localhost scipy-0.11.0]# 
    [root@localhost Downloads]# python
    Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import scipy
    >>> 

    ok,安装完成!


    回到我们的上一层问题:import skimage.io出错,现在再来:
    >>> import skimage.io
    >>> 

    ok!

    继续回到上一层问题:import caffe:
    >>> import caffe
    4.egg/skimage/_shared/_geometry.py", line 4, in <module>
        from matplotlib import _path, path, transforms
    ImportError: No module named matplotlib

    还有问题,缺少matplotlib:
    用easy_install matplotlib的方式安装。
    太慢。
    直接下载:https://pypi.python.org/packages/source/m/matplotlib/matplotlib-1.5.1.tar.gz#md5=f51847d8692cb63df64cd0bd0304fd20
    解压,进入,执行:
    python setup.py build
    提示缺少必须的依赖:
                            * The following required packages can not be built:
                            * freetype, png

    解决:
    yum install freetype-devel

    下载libpng源码:http://tenet.dl.sourceforge.net/project/libpng/libpng14/older-releases/1.4.14/libpng-1.4.14.tar.gz
    解压,进入,执行:
    ./configure 
    make
    make check
    make install
    可以看到,安装信息:
    make[2]: Leaving directory `/home/zzz/Downloads/libpng-1.4.14'
     /usr/bin/mkdir -p '/usr/local/share/man/man3'
     /usr/bin/install -c -m 644 libpng.3 libpngpf.3 '/usr/local/share/man/man3'
     /usr/bin/mkdir -p '/usr/local/share/man/man5'
     /usr/bin/install -c -m 644 png.5 '/usr/local/share/man/man5'
     /usr/bin/mkdir -p '/usr/local/lib/pkgconfig'
     /usr/bin/install -c -m 644 libpng14.pc '/usr/local/lib/pkgconfig'
     /usr/bin/mkdir -p '/usr/local/include/libpng14'
     /usr/bin/install -c -m 644 png.h pngconf.h '/usr/local/include/libpng14'
    make  install-data-hook
    make[2]: Entering directory `/home/zzz/Downloads/libpng-1.4.14'
    cd /usr/local/include; rm -f png.h pngconf.h
    cd /usr/local/include; ln -s libpng14/png.h png.h
    cd /usr/local/include; ln -s libpng14/pngconf.h
        pngconf.h
    cd /usr/local/lib/pkgconfig; rm -f libpng.pc
    cd /usr/local/lib/pkgconfig; ln -s libpng14.pc libpng.pc
    make[2]: Leaving directory `/home/zzz/Downloads/libpng-1.4.14'
    make[1]: Leaving directory `/home/zzz/Downloads/libpng-1.4.14'
    [root@localhost libpng-1.4.14]# 

    再次执行matplotlib的编译:
    python setup.py build 成功。
    python setup.py install 成功。


    继续import caffe,出错:
    ImportError: No module named google.protobuf.internal
    参考:http://blog.csdn.net/littlestream9527/article/details/38734871

    从网盘下载 :http://pan.baidu.com/s/1pJlZubT
    解压,进入,执行:
    ./configure
    make
    make install
    ldconfig 

    然后进入源码目录的python子目录:
    cd python
    执行:
    python setup.py build 
    python setup.py install

    执行完后,再次的导入caffe模块:
    [root@localhost Downloads]# python
    Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import caffe
    /home/gumh/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Net<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    /home/gumh/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Blob<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    /home/gumh/OpenSource/caffe-master/python/caffe/pycaffe.py:13: RuntimeWarning: to-Python converter for boost::shared_ptr<caffe::Solver<float> > already registered; second conversion method ignored.
      from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
    >>> 
    成功了。
  • 相关阅读:
    OS-TEP: Fetch-And-Add
    ULK --- Chap 4: Tasklets (Note)
    ULK --- Chap 4: ksoftirqd kernel threads (Note)
    ULK --- Chap 4: Softirqs and Tasklets (Note)
    ULK --- Chap 4: Nested Execution of Exception and Interrupt Handlers
    ULK --- Chap 4: Interrupt Descriptor Table
    己所欲,勿施于人 --- 周国平
    善良 丰富 高贵 --- 周国平
    第一重要的是做人 --- 周国平
    Operating System: Three Easy Pieces --- Load-Linked and Store-Conditional (Note)
  • 原文地址:https://www.cnblogs.com/Crysaty/p/6248030.html
Copyright © 2020-2023  润新知