• tensorflow源码安装


    主要参考:https://www.tensorflow.org/install/install_sources#ConfigureInstallation


    卸载tensorflow
        sudo pip uninstall tensorflow

    安装git

        安装git时记得先安装,后更新系统
        sudo apt install git



    安装jdk8:
        my@ubuntu:~$ java

        my@ubuntu:~$ sudo add-apt-repository ppa:openjdk/ppa

        my@ubuntu:~$ sudo add-apt-repository ppa:openjdk-r/ppa
     
        my@ubuntu:~$ sudo apt-get update

        my@ubuntu:~$ sudo apt-get install openjdk-8-jdk

        my@ubuntu:~$ sudo apt-get install openjdk-8-jre openjdk-8-jdk-headlesss
        
        my@ubuntu:~$ java -version
    openjdk version "1.8.0_131"
    OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
    OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

    Install Bazel:
        sudo apt-get update && sudo apt-get install bazel
        sudo apt-get upgrade bazel
        bazel version

    一、安装virtualenv:
        my@ubuntu:~$ sudo apt-get install python-virtualenv

        在virtualenv中创建tensorfloe环境

        my@ubuntu:~$ virtualenv --system-site-packages ~/tensorflow

        激活tensorflow的virtualenv环境

        my@ubuntu:~$ source ~/tensorflow/bin/activate

        为虚拟环境创建快捷方式:
        sudo printf ' alias tensorflow="source ~/tensorflow/bin/activate"'>>~/.bashrc
        让其生效:
        source ~/.bashrc

    二、在env环境中
        
        git clone https://github.com/tensorflow/tensorflow

        sudo apt-get install python-numpy python-dev python-pip python-wheel

    因为我的vm中没有gpu,省去gpu的配置环节:

        sudo apt-get install libcupti-dev

         sudo pip install six numpy wheel


    下面这段摘自tensorflow的官方文档:
        $cd tensorflow  # cd to the top-level directory created
        $ ./configure
    Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7
    Found possible Python library paths:
      /usr/local/lib/python2.7/dist-packages
      /usr/lib/python2.7/dist-packages
    Please input the desired Python library path to use.  Default is [/usr/lib/python2.7/dist-packages]

    Using python library path: /usr/local/lib/python2.7/dist-packages
    Do you wish to build TensorFlow with MKL support? [y/N]
    No MKL support will be enabled for TensorFlow
    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
    Do you wish to use jemalloc as the malloc implementation? [Y/n]
    jemalloc enabled
    Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
    No Google Cloud Platform support will be enabled for TensorFlow
    Do you wish to build TensorFlow with Hadoop File System support? [y/N]
    No Hadoop File System support will be enabled for TensorFlow
    Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
    Do you wish to build TensorFlow with VERBS support? [y/N]
    No VERBS support will be enabled for TensorFlow
    Do you wish to build TensorFlow with OpenCL support? [y/N]
    No OpenCL support will be enabled for TensorFlow
    Do you wish to build TensorFlow with CUDA support? [y/N] Y
    CUDA support will be enabled for TensorFlow
    Do you want to use clang as CUDA compiler? [y/N]
    nvcc will be used as CUDA compiler
    Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]: 8.0
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 6
    Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
    Please note that each additional compute capability significantly increases your build time and binary size.
    [Default is: "3.5,5.2"]: 3.0
    Do you wish to build TensorFlow with MPI support? [y/N]
    MPI support will not be enabled for TensorFlow
    Configuration finished
    这里是非常重要的编译配置:可以参考http://blog.csdn.net/nicholas_wong/article/details/70215127,但还要结合自己的机器报的警告来配置,
    我的机器报如下警告:
    /cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.  
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.  
    W tensorflow/core/platform,那我安装的时候就配置sse4.1和sse4.2的选项,刚开始直接把人家的复制过来用,安装完后报错,后来自己配置了一下,安装成功。而且上面的警告有时候是不会出现的,
    我在安装SyntaxNet wrapper 地址及安装:https://github.com/short-edition/syntaxnet-wrapper时报错,导致测试走不下去,只能消除警告,
        sudo bazel build -c opt  --copt=-msse4.1 --copt=-msse4.2  //tensorflow/tools/pip_package:build_pip_package
    这个编译时间超长,要有耐心。

    上述命令会生成一个叫做build_pip_package的脚本,按照如下命令运行这个脚本,在/tmp/tensorflow_pkg文件夹中创建pip的安装包:

        bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
        cd  /tmp/tensorflow_pkg
        ls
    会发现一个whl的文件:tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl,这是编译生成的。
    下面进行安装:

        sudo  pip install /tmp/tensorflow_pkg/tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl
    安装完成后进行测试:注意测试的时候不要到安装目录下测试。
    $ python

    Enter the following short program inside the python interactive shell:

    # Python
    import tensorflow as tf
    hello = tf.constant('Hello, TensorFlow!')
    sess = tf.Session()
    print(sess.run(hello))

    If the system outputs the following, then you are ready to begin writing TensorFlow programs:

    Hello, TensorFlow!

    如果不报sse4.1和sse4.2错误,有可能安装成功。

    安装syntaxnet:
        前提是tensorflow已经安装好,
        git clone --recursive https://github.com/tensorflow/models.git

        cd models/syntaxnet/tensorflow
    配置选项:
        $ ./configure

    (tensorflow) my@ubuntu:~/models/syntaxnet/tensorflow$ cd ..

        (tensorflow) my@ubuntu:~/models/syntaxnet$ bazel test syntaxnet/... util/utf8/...

        (tensorflow) my@ubuntu:~/models/syntaxnet$ chmod +x  syntaxnet/demo.sh

    7、pip安装 asciitree、mock

        (tensorflow) my@ubuntu:~/models/syntaxnet$  pip install asciitree

        (tensorflow) my@ubuntu:~/models/syntaxnet$ pip install mock


        测试一个句子:

        (tensorflow) my@ubuntu:~/models/syntaxnet$ sudo  echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh
        

    测试中文:    

    MODEL_DIRECTORY为解压的中文模型的位置,一种方式是从http://download.tensorflow.org/models/parsey_universal/Chinese.zip下载
    然后解压到/home/py/models/syntaxnet/syntaxnet/models/parsey_universal,
    另外一种方式是安装 A lightweight SyntaxNet wrapper 地址及安装:https://github.com/short-edition/syntaxnet-wrapper,它会自动安装语言模型

    测试:
    py@ubuntu:~/models/syntaxnet$ MODEL_DIRECTORY=/home/py/models/syntaxnet/syntaxnet/models/parsey_universal/Chinese

    py@ubuntu:~/models/syntaxnet$ echo '然而,中国经历了30多年的改革开放' | syntaxnet/models/parsey_universal/tokenize_zh.sh $MODEL_DIRECTORY | syntaxnet/models/parsey_universal/parse.sh $MODEL_DIRECTORY


    安装SyntaxNet Wrapper

    直接复制文档了:

    A lightweight SyntaxNet wrapper

    The wrapper allows generic use of SyntaxNet in python. It provides interfaces for morphological analyse, pos tagging and dependency resolution along with optional formatting tool.

    The wrapper does not intend to make any assumptions on the use of SyntaxNet, that's why it provides a simple interface and the raw output as default.

    Disclaimer : Has been inspired from another wrapper but we did not want a server based wrapper.
    Installation

    We assume here that you have SyntaxNet installed and working properly on your workstation. If not, please refer to SyntaxNet official page

    The wrapper has been tester on Ubuntu 14.04 Trusty and 16.04 Xenial.

    (virtualenv)$ git clone https://github.com/short-edition/syntaxnet-wrapper.git
    (virtualenv)$ cd syntaxnet-wrapper
    (virtualenv)/syntaxnet-wrapper$ pip install -r requirements.txt
    (virtualenv)/syntaxnet-wrapper$ vim syntaxnet_wrapper/config.yml
    syntaxnet:
      ROOT_DIR: /home/user/workspace/syntactic_parser/tensorflow_models/syntaxnet
      PARSER_EVAL: bazel-bin/syntaxnet/parser_eval
      CONTEXT: syntaxnet/models/parsey_universal/context.pbtxt
      MODEL: syntaxnet/models/parsey_universal

    (virtualenv)/syntaxnet-wrapper$ python -m unittest discover syntaxnet_wrapper
    (virtualenv)/syntaxnet-wrapper$ pip install .

    这一句比较重要:创建 /usr/share/syntaxnet_wrapper_config.yml,内容就是
    syntaxnet:
      ROOT_DIR: /home/py/models/syntaxnet
      PARSER_EVAL: bazel-bin/syntaxnet/parser_eval
      CONTEXT: syntaxnet/models/parsey_universal/context.pbtxt
      MODEL: syntaxnet/models/parsey_universal


    You should be able to use the wrapper from now. Instead of creating config.yml in the current folder, you can also expose it though the environment variable SYNTAXNET_WRAPPER_CONFIG or copying it in /usr/share/syntaxnet_wrapper_config.yml

    You can also use the Dockerfile provided that will install SyntaxNet itself and the wrapper
    How to use this wrapper

    Two mode

    You can use the wrapper in two modes, embodied in two different classes with the same interface

        SyntaxNetWrapperSubprocess, a python implementation of demo.sh shell script provided in SyntaxNet. It starts new subprocesses at each call.
        SyntaxNetWrapper, using wrapper's syntaxnet python implementation. Have the advantage to be faster and more memory efficient than the version with subprocesses. However, we are experience some trouble with it. See Well-known issues

    The interface

    The wrapper is expecting unicode text compatible with utf-8 format. The interface is the same for both classes :

        morpho_sentence, make morphological analyse for a single sentence

        morpho_sentences, make morphological analyse for a sentences list

        tag_sentence, make pos tagging for a single sentence

        tag_sentences, make pos tagging for a sentences list

        parse_sentence, make dependency parsing for a single sentence

        parse_sentences, make dependency parsing for a sentences list

        transform_morpho, transform_tag and transform_dependency format the outputs in a more readable form. Deleting unfilled field.

    例子:

    >>> from syntaxnet_wrapper.wrapper import SyntaxNetWrapper
    >>> sn_wrapper = SyntaxNetWrapper()
    >>> dependency_output = sn_wrapper.parse_sentence(u"Bob brought a pizza to Alice")
    >>> print dependency_output
    u'1 Bob _ PROPN NNP Number=Sing|fPOS=PROPN++NNP 2 nsubj _ _ 2 brought _ VERB VBD Mood=Ind|Tense=Past|VerbForm=Fin|fPOS=VERB++VBD 0 ROOT _ _ 3 a _ DET DT Definite=Ind|PronType=Art|fPOS=DET++DT 4 det _ _ 4 pizza _ NOUN NN Number=Sing|fPOS=NOUN++NN 2 dobj _ _ 5 to _ ADP IN fPOS=ADP++IN 6 case _ _ 6 Alice _ PROPN NNP Number=Sing|fPOS=PROPN++NNP 4 nmod _ _ '

    Use of different language

    The wrapper use by default english model but you can use every "Parsey Universal" released by Google. You just need to pass the name of the model as a constructor's argument. The wrapper will then automatically download the model and use it.

    >>> sn_wrapper = SyntaxNetWrapper(language='French')



  • 相关阅读:
    javac不是内部或外部命令在win10上的解决方案
    HDU 6191 Query on A Tree(字典树+离线)
    hihoCoder #1558 : H国的身份证号码I
    HDU 6154 CaoHaha's staff(2017中国大学生程序设计竞赛
    湖南省第十二届省赛:Parenthesis
    POJ 3260 The Fewest Coins(完全背包+多重背包=混合背包)
    HDU 2923 Relocation(状压dp+01背包)
    HDU 2546 饭卡(01背包)
    HDU 1247 Hat’s Words(字典树)
    HDU 1711 Number Sequence(KMP)附带KMP的详解
  • 原文地址:https://www.cnblogs.com/herosoft/p/8134123.html
Copyright © 2020-2023  润新知