安装 go 版 TensorFlow的问题记录

文章记录在centos 7上安装golang环境及go版TensorFlow的过程及中间遇到的问题及解决方法。

1 安装golang环境

环境是宿主机安装virtualbox加载的centos 7环境，采用的yum源安装。此部分参考此文 CentOS7安装golang

yum -y install golang

安装完成后，查看

[root@localhost ~]# go version
go version go1.13.14 linux/amd64
[root@localhost ~]# 

设置环境变量，在home目录创建GOPATH目录

[root@localhost home]# mkdir gopath^C
[root@localhost home]# 
[root@localhost home]# 
[root@localhost home]# ll
总用量 16548
drwxr-xr-x.  4 root root       99 8月  27 15:45 gopath
-rwxr-xr-x.  1 root root     8496 8月  26 18:00 hello_tf
-rw-r--r--.  1 root root      151 8月  26 18:00 hello_tf.c
drwxr-xr-x.  3 root root       22 8月  27 07:52 openblas
drwxr-xr-x. 19  501  501     4096 8月  27 11:31 Python-3.7.0
-rw-r--r--.  1 root root 16922100 8月  27 10:29 Python-3.7.0.tar.xz
[root@localhost home]# 

编辑/root/.bashrc文件，添加如下信息

export GOPATH=/home/gopath
export GOROOT=/usr/lib/golang
export GOBIN=$GOROOT/bin
export PATH=$PATH:$GOBIN

重新加载/root/.bashrc文件

source /root/.bashrc

查看go的相关设置是否生效

[root@localhost home]# go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/usr/lib/golang/bin"
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gopath"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/lib/golang"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build389723181=/tmp/go-build -gno-record-gcc-switches"
[root@localhost home]# 

环境设置已完成，进行测试，在/home/gopath下编写测试程序

[root@localhost gopath]# vim test.go 
package main
import "fmt"
func main() {
fmt.Println("Hello world!")
}

保存后，输入命令

go build test.go

在输入

[root@localhost gopath]# ./test 
Hello world!
[root@localhost gopath]#

测试go已经配置完成。过程很顺利的完成。

2 安装C版TensorFlow

此部分是按照 安装 Go 版 TensorFlow 中的说明，先设置TensorFlow C 库，安装过程参考 安装 C 版 TensorFlow

2.1安装压缩包

下载：https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz

webget https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz

解压缩下载的归档文件，其中包含要添加到 C 程序中的头文件以及要与之关联的共享库。在 Linux 和 macOS 上，您可能需要解压缩到 /usr/local/lib：

sudo tar -C /usr/local -xzf (downloaded file)

在 Linux/macOS 上，如果将 TensorFlow C 库解压缩到系统目录（例如 /usr/local），请使用 ldconfig 配置链接器：

sudo ldconfig

2.2编译

安装 TensorFlow C 库后，使用以下源代码创建一个示例程序 (hello_tf.c)：

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
  printf("Hello from TensorFlow C library version %s
", TF_Version());
  return 0;
}

编译示例程序以创建可执行文件，然后运行以下命令：

gcc hello_tf.c -ltensorflow -o hello_tf

./hello_tf

执行上述命令后报错如下：

[root@localhost local]# ./hello_tf
./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

百度相关后，找到如下解决方案：

$ gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow -o hello_tf
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
 $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
 $ ./hello_tf 
 Hello from TensorFlow C library version 1.14.0
 $ unset LD_LIBRARY_PATH 
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

基于该方案，设置LD_LIBRARY_PATH，如下所示：

[root@localhost local]# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
[root@localhost local]# ./hello_tf
Hello from TensorFlow C library version 1.15.0
[root@localhost local]# 

另一种解决方案（备注该方案为佳），打开/etc/ld.so.conf文件，增加/usr/local/lib，主要操作如下：

[root@localhost lib]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib

而后执行ldconfig

[root@localhost lib]# ldconfig
[root@localhost lib]# 

此方法参考 Suppressed: java.lang.UnsatisfiedLinkError: libxxx.so: 无法打开共享对象文件: 没有那个文件或目录

3 安装 Go 版 TensorFlow

此部分是按照 安装 Go 版 TensorFlow 的说明逐步进行的，中间有很多问题，后基于网上一些建议调整安装成功。

3.1安装过程

在设置完C 版 TensorFlow相关配置后，进行go版TensorFlow的安装，使用命令

go get github.com/tensorflow/tensorflow/tensorflow/go

我在安装的过程中，经常出现如下的问题

[root@localhost gopath]# go get -d -v github.com/tensorflow/tensorflow/tensorflow/go
github.com/tensorflow/tensorflow (download)
# cd .; git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: 过早的文件结束符（EOF）
fatal: index-pack failed
package github.com/tensorflow/tensorflow/tensorflow/go: exit status 128
[root@localhost gopath]#

分析发现是先从GitHub上clone相关的文件到本机，所以在/home/gopath目录尝试了如下命令

git clone -- https://github.com/tensorflow/tensorflow /home/gopath/src/github.com/tensorflow/tensorflow

就是使用git clone到本机，但该命令经常在达到10%左右就卡住，不在下载了。故采取从浏览器下载到宿主机在上传到虚拟机的方式进行，打开GitHub的TensorFlow地址，前期在是哟用rasa时用的是TensorFlow的1.13.1版本，故这里也下载1.13.1的版本，下载得到tensorflow-1.13.1.zip压缩包，修改名称为tensorflow.zip，而后通过rz命令上传到宿主机

[root@localhost gopath]# cd src/github.com/
[root@localhost github.com]# ll
总用量 40544
drwxr-xr-x. 3 root root       22 8月  27 15:30 golang
drwxr-xr-x. 3 root root       18 8月  26 18:22 gonum
drwxr-xr-x. 3 root root       26 8月  27 07:56 mattn
drwxr-xr-x. 3 root root       25 8月  27 07:56 olekukonko
drwxr-xr-x. 3 root root       19 8月  27 07:56 peterh
drwxr-xr-x. 3 root root       20 8月  27 07:55 pkg
drwxr-xr-x. 4 root root       32 8月  27 07:55 spf13
drwxr-xr-x. 3 root root       24 8月  27 15:31 tensorflow
-rw-r--r--. 1 root root 41515134 8月  27 15:22 tensorflow.zip
drwxr-xr-x. 3 root root       18 8月  27 07:53 ynqa
[root@localhost github.com]#

解压tensorflow.zip，得到tensorflow目录，回到/home/gopath目录进行构建，命令如下

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
can't load package: package github.com/tensorflow/tensorflow/tensorflow/go/op: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/op" in any of:
	/usr/lib/golang/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOROOT)
	/home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op (from $GOPATH)
[root@localhost gopath]#

报错，说找不到该目录，下载的源码怎么没有该目录内，直接cd查看

[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/op: 没有那个文件或目录
[root@localhost gopath]# cd /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/
-bash: cd: /home/gopath/src/github.com/tensorflow/tensorflow/tensorflow/go/: 没有那个文件或目录
[root@localhost gopath]#

确实没有该目录，回到解压处查看，发现源码解压后只有两层tensorflow，而go get的有三层，则我们直接增加一层tensorflow，在进行构建

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op

package google.golang.org/protobuf/encoding/prototext: unrecognized import path "google.golang.org/protobuf/encoding/prototext" (https fetch: Get https://google.golang.org/protobuf/encoding/prototext?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/encoding/protowire: unrecognized import path "google.golang.org/protobuf/encoding/protowire" (https fetch: Get https://google.golang.org/protobuf/encoding/protowire?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/proto: unrecognized import path "google.golang.org/protobuf/proto" (https fetch: Get https://google.golang.org/protobuf/proto?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoreflect: unrecognized import path "google.golang.org/protobuf/reflect/protoreflect" (https fetch: Get https://google.golang.org/protobuf/reflect/protoreflect?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/reflect/protoregistry: unrecognized import path "google.golang.org/protobuf/reflect/protoregistry" (https fetch: Get https://google.golang.org/protobuf/reflect/protoregistry?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoiface: unrecognized import path "google.golang.org/protobuf/runtime/protoiface" (https fetch: Get https://google.golang.org/protobuf/runtime/protoiface?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
package google.golang.org/protobuf/runtime/protoimpl: unrecognized import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
../genop/main.go:17: running "bash": exit status 1
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]# 

报错protobuf导入超时

import path "google.golang.org/protobuf/runtime/protoimpl" (https fetch: Get https://google.golang.org/protobuf/runtime/protoimpl?go-get=1: dial tcp 216.239.37.1:443: i/o timeout

解决方法参考 go get 找不到 google.golang.org/protobuf github.com/gin-gonic/gin 解决办法 也就是从GitHub上clone代码下来，如下操作

[root@localhost gopath]# mkdir -p $GOPATH/src/google.golang.org
[root@localhost gopath]# git clone https://e.coding.net/robinqiwei/googleprotobuf.git $GOPATH/src/google.golang.org/protobuf
正克隆到 '/home/gopath/src/google.golang.org/protobuf'...
remote: Enumerating objects: 13821, done.
remote: Counting objects: 100% (13821/13821), done.
remote: Compressing objects: 100% (5172/5172), done.
remote: Total 13821 (delta 8112), reused 13821 (delta 8112), pack-reused 0
接收对象中: 100% (13821/13821), 7.44 MiB | 932.00 KiB/s, done.
处理 delta 中: 100% (8112/8112), done.
[root@localhost gopath]# 

完成后，在重新构建成功，如下

[root@localhost gopath]# go generate github.com/tensorflow/tensorflow/tensorflow/go/op
# github.com/golang/protobuf/protoc-gen-go
../../../../../golang/protobuf/protoc-gen-go/main.go:71:6: gen.SupportedFeatures undefined (type *protogen.Plugin has no field or method SupportedFeatures)
../../../../../golang/protobuf/protoc-gen-go/main.go:71:27: undefined: internal_gengo.SupportedFeatures
../genop/main.go:17: running "bash": exit status 2
src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:17: running "go": exit status 1
[root@localhost gopath]#

这里发现exit status 1，也不知是啥问题，没有去研究了。然后验证安装结果

[root@localhost gopath]# go test github.com/tensorflow/tensorflow/tensorflow/go
ok  	github.com/tensorflow/tensorflow/tensorflow/go	1.848s
[root@localhost gopath]# 

程序验证，GO示例，创建tf-hello.go

[root@localhost gopath]# vim tf-hello.go 

package main

import (
    tf "github.com/tensorflow/tensorflow/tensorflow/go"
    "github.com/tensorflow/tensorflow/tensorflow/go/op"
    "fmt"
)
func main() {
    // Construct a graph with an operation that produces a string constant.
    s := op.NewScope()
    c := op.Const(s, "Hello from TensorFlow version " + tf.Version())
    graph, err := s.Finalize()
    if err != nil {
        panic(err)
    }
    // Execute the graph in a session.
    sess, err := tf.NewSession(graph, nil)
    if err != nil {
        panic(err)
    }
    output, err := sess.Run(nil, []tf.Output{c}, nil)
    if err != nil {
        panic(err)
    }
    fmt.Println(output[0].Value())
}

执行

[root@localhost gopath]# go run tf-hello.go
2020-08-27 15:44:47.868756: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-08-27 15:44:47.882914: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1799995000 Hz
2020-08-27 15:44:47.884161: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x22d3190 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-27 15:44:47.884211: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Hello from TensorFlow version 1.15.0
[root@localhost gopath]#

C示例，C和Go实际上共用了相同的动态链接库和头文件，安装好Go版本后，C的API可以直接使用。创建tf.c

[root@localhost gopath]# vim tf.c 

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
    printf("Hello from TensorFlow C library version %s
", TF_Version());
    return 0;
}

编译，执行

[root@localhost gopath]# gcc -I/usr/local/include -L/usr/local/lib tf.c -ltensorflow
[root@localhost gopath]# ll
总用量 1988
-rwxr-xr-x. 1 root root    8496 8月  27 15:45 a.out
drwxr-xr-x. 3 root root      25 8月  27 15:40 pkg
drwxr-xr-x. 5 root root      65 8月  27 15:40 src
-rwxr-xr-x. 1 root root 2008751 8月  26 17:56 test
-rw-r--r--. 1 root root      74 8月  26 17:56 test.go
-rw-r--r--. 1 root root     157 8月  27 15:45 tf.c
-rw-r--r--. 1 root root     671 8月  27 15:44 tf-hello.go
[root@localhost gopath]# ./a.out 
Hello from TensorFlow C library version 1.15.0
[root@localhost gopath]# 

至此，整个环境已经整合完成，后续准备使用golang调用TensorFlow模型。