• Mac OSX (EI Capitan)搭建Caffe环境并配置python接口

    Caffe是一个清晰而高效的深度学习框架，其作者是博士毕业于UC Berkeley的贾扬清。Caffe是纯粹的C++/CUDA架构，支持命令行、Python和MATLAB接口；可以在CPU和GPU直接无缝切换。我在MacbookPro（无NVIDIA显卡）上大费周章地配置了Caffe的环境，并花了许多时间配置其python接口。

    一、下载Caffe

    github上的下载地址：https://github.com/BVLC/caffe
    进入到下载后的路径，并复制     Makefile.config.example 重命名为 Makefile.config (我电脑的用户名是cuiqi，注意修改)
    git clone https://github.com/BVLC/caffe.git
cd /Users/cuiqi/Downloads/caffe-master && cp Makefile.config.example Makefile.config

    二、安装相关依赖

       对于需要python接口的情况，需要以下依赖

    1. CUDA 
    由于我的Mac没有NVIDIA的GPU，所以只能使用CPU_ONLY模式，需要在 Makefile.config 中修改 
CPU_ONLY := 1
    
2. BLAS via ATLAS, MKL, or OpenBLAS.
# Basic Linear Algebra Subprograms，基础线性代数程序集

3. Boost >= 1.55
# Deepdream是用Python接Caffe，因此还需要 boost.python 支持
    brew install boost --with-python
brew install boost-python

    4. OpenCV >= 2.4 including 3.0

5. protobuf, glog, gflags
    brew install protobuf
brew install glog
brew install gflags
    6. IO libraries hdf5, leveldb, snappy, lmdb
    brew install leveldb
brew install lmdb
    brew tap homebrew/science
brew install homebrew/science/hdf5
    # python driver for hdf5
    pip install h5py
    7. numpy for python
    brew install numpy

    三、修改Makefile.config中相应的路径

       如果要使用Anaconda的python环境可以在Makefile.config中取消相应的注释，我试过这样做，可是在CMAKE的时候并不奏效：

    Python:

Interpreter : /usr/bin/python2.7 (ver. 2.7.10)

Libraries : /usr/lib/libpython2.7.dylib (ver 2.7.10)

NumPy : /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)

    　所以我放弃了使用Anaconda，改使用系统自带的python。

    PYTHON_INCLUDE := /usr/include/python2.7  

      由于我使用了Homebrew安装了numpy，所以我在makefile.config中修改相应的numpy路径

    PYTHON_INCLUDE := /usr/include/python2.7 
/usr/local/Cellar/numpy/1.11.2/lib/python2.7/site-packages/numpy/core/include/numpy/core/include

      我的Makefile.config文件：

    ## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
 CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#    You should not set this flag if you will be reading LMDBs with any
#    possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 
        -gencode arch=compute_20,code=sm_21 
        -gencode arch=compute_30,code=sm_30 
        -gencode arch=compute_35,code=sm_35 
        -gencode arch=compute_50,code=sm_50 
        -gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 
        /usr/local/Cellar/numpy/1.11.2/lib/python2.7/site-packages/numpy/core/include/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include 
        # $(ANACONDA_HOME)/include/python2.7 
        # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include 

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m 
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @
    View Code

    四、编译

      使用cmake工具，将cmake的sourcecode设为caffe的路径，在caffe目录下建立build文件夹，并将cmake的build路径设为该build的路径，configure，generate。

      此时打开终端工具进入build目录下，make。

    make all
make test
make runtest
make pycaffe

      如果没有出现错误，并全部通过后，此时make告一段落，但是并不能确定可以在python中import。

      在终端上：

    vi etc/profile

      或

    sudo vi /etc/profile

      进入修改系统的环境变量，将 caffe/python/添加到Python系统路径：

    export PYTHONPATH=path to caffe/python:$PYTHONPATH
    例如：
    export PYTHONPATH=/Users/cuiqi/Downloads/caffe-master/python:$PYTHONPATH

      终端输入python，确认这个python环境是刚刚在makefile.config中设定的那个，可以 which python 确认。由于我没有使用Anaconda，我的是

      /usr/local/bin/python ，确认为刚刚在makefile.config中设定的路径。

      >>> import caffe

      # 可能的错误

    　　ImportError: No module named skimage.io

      # 解决

       pip install scikit-image

    参考：https://github.com/rainyear/lolita/issues/10?utm_source=tuicool&utm_medium=referral

    16.12.22更新：

      还可能出现的错误：segment fault：11

      在 ITNOSE 上一作者认为此错误很有可能是系统里也有多个版本的Python，并且编译和运行的时候不是一个版本。

      解决办法：

      当然就是要统一编译与运行的python版本。终端键入 which python ，反馈给我/usr/local/bin/python，而CMAKE工具反馈给我的是系统自带的python路径。

      1.对于我而言，我准备使用系统自带的python。使用Homebrew工具brew install python后，python环境路径为/usr/local/bin/python，而caffe编译时使用了系统自带的python路径     /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python。这个路径可以使用otool反编译工具检出，例如：otool -L python/caffe/_caffe.so （_caffe.so在下载好的caffe的python/caffe下）。

      2.修改bash_profile配置文件以在启动系统时使用自带python，打开终端，键入 vi ~/.bash_profile ，输入：

    # Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
 PATH="/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
 export PATH

      完成后输入：wq（保存退出），终端键入 source ~/.bash_profile 或者注销及重启使修改生效。

      3.再次终端键入 which python，返回 /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python，说明修改生效。键入python，

    >>> import caffe

      无报错，成功。

    参考：

    http://www.itnose.net/detail/6522348.html

    https://www.zhihu.com/question/30941329

     16.12.29更新：

    可能出现的错误：TypeError: __init__() got an unexpected keyword argument 'syntax'；

    参考dupuleng的解决方法，这是由于protobuf版本的问题造成的。

    查看protobuf版本：

    pip show protobuf

    protobuf信息：

    Name: protobuf
Version: 2.6.1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: New BSD License
Location: /Users/cuiqi/anaconda/lib/python2.7/site-packages
Requires: setuptools

    此版本（2.6.1）需要变更：

    pip uninstall protobuf
pip install 'protobuf>=3.0.0a3'
    pip show numpy protobuf

    protobuf信息：

    Name: protobuf
Version: 3.1.0.post1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: New BSD License
Location: /Users/cuiqi/anaconda/lib/python2.7/site-packages
Requires: six, setuptools

    完成

    关于cmake：

    经过我的测试发现修改makefile.config后，cmake的 configure之前还要手动修改python的相关路径，否则cmake会使用mac的系统python环境，造成上述segment fault 11的情况。关键在于要勾选Advanced选项：

    不勾选Advanced：

    更改好之后再configure，generate，cmake就会选择相应的环境路径了。

    学而不思则惘
  • 相关阅读:
    使用BigQuery分析GitHub上的C#代码
    ASP.NET Core 处理 404 Not Found
    C# 7 局部函数剖析
    调试 ASP.NET Core 2.0 源代码
    Entity Framework Core Like 查询揭秘
    ASP.NET Core Razor 视图组件
    Thread 1 cannot allocate new log引起的宕机事故（转载）
    docker默认网段和主机网段冲突解决
    docker安装异常以及网络问题总结
    max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
  • 原文地址：https://www.cnblogs.com/keyky/p/6209456.html
Copyright © 2020-2023  润新知