1、编译环境
- OS ：Loongnix-Server Linux release 8.3
- CPU指令集 ： loongarch64
- 平台 ： 龙芯
- go版本 : go version go1.15.6 linux/loong64
2、go和docker安装
docker安装：
yum -y install docker-ce
golang安装：
yum -y install golang
默认安装golang版本是 go version go1.15.6 linux/loong64，如果需要升级golang版本的话可以
yum -y update yum search golang
然后根据search结果yum -y install指定golang版本。
3、编译kubernetes源码
kubernetes全部组件编译：
KUBE_BUILD_PLATFORMS=linux/loong64 make all
编译指定kubernetes组件：
KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet GOFLAGS=-v GOGCFLAGS="-N -l"
编译完成后，可以在 _output/local/bin/linux/loong64/中找到需要的二进制文件。
1）下载指定版本kubernetes源码
从github获取指定版本的k8s源码
wget https://github.com/kubernetes/kubernetes/archive/refs/tags/v1.21.5.tar.gz
解压并进入到kubernetes-1.21.5目录
tar -zxvf v1.21.5.tar.gz && cd kubernetes-1.21.5
注意：本文以kubernetes1.21.5版本源码编译为例，其他kubernetes版本编译步骤和kubernetes1.21.5版本源码编译步骤一致。
2）组件编译及错误处理
2.1） 编译kubelet
KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet
编译发现报如下错误：
[root@master1 kubernetes-1.21.5]#KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet -bash: make: command not found
安装一下make即可：
yum -y install make
安装完make后重新编译kubelet:
[root@master1 kubernetes-1.21.5]# KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet !!! [0509 06:14:35] Unsupported host arch. Must be x86_64, 386, arm, arm64, s390x or ppc64le. Detected go version: go version go1.15.6 linux/loong64. Kubernetes requires go1.16.0 or greater. Please install go1.16.0 or later. !!! [0509 06:14:35] Call tree: !!! [0509 06:14:35] 1: hack/run-in-gopath.sh:31 kube::golang::setup_env(...) !!! [0509 06:14:35] Unsupported host arch. Must be x86_64, 386, arm, arm64, s390x or ppc64le. Detected go version: go version go1.15.6 linux/loong64. Kubernetes requires go1.16.0 or greater. Please install go1.16.0 or later. !!! [0509 06:14:35] Call tree: !!! [0509 06:14:35] 1: /root/go/kubernetes-1.21.5/hack/lib/golang.sh:786 kube::golang::setup_env(...) !!! [0509 06:14:35] 2: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) !!! [0509 06:14:35] Call tree: !!! [0509 06:14:35] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) make[1]: *** [Makefile.generated_files:209: _output/bin/prerelease-lifecycle-gen] Error 1 make: *** [Makefile:551: generated_files] Error 2
首先会发现golang版本太低，这个直接yum search golang然后通过yum安装>=1.16.0以上版本的即可，详细步骤不再赘余,升级完golang版本后重新编译kubeadm:
[root@master1 kubernetes-1.21.5]# go version go version go1.18 linux/loong64 [root@master1 kubernetes-1.21.5]# KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet !!! [0509 06:19:40] Unsupported host arch. Must be x86_64, 386, arm, arm64, s390x or ppc64le. !!! [0509 06:19:45] Unsupported host arch. Must be x86_64, 386, arm, arm64, s390x or ppc64le.
会发现kubernetes源码不支持loongarch64指令集问题，下面主要说一下如何解决kubernetes源码不支持loongarch64指令集问题。根据安装完make后重新编译kubelet错误提示，编辑hack/run-in-gopath.sh,发现其引用了hack/lib/init.sh，然后继续编辑hack/lib/init.sh，并递归检查他们依赖的.sh文件，查看这些文件中是否存在CPU指令集相关的逻辑判断，如果存在的话加上龙芯平台loongarch64架构的指令集，这里直接说下哪些文件需要调整：
（1）修改hack/lib/golang.sh文件
在KUBE_SUPPORTED_SERVER_PLATFORMS ， KUBE_SUPPORTED_NODE_PLATFORMS ，KUBE_SUPPORTED_CLIENT_PLATFORMS，KUBE_SUPPORTED_TEST_PLATFORMS分别添加上linux/loong64
（2）修改hack/lib/util.sh文件
在kube::util::host_arch() 里添加
loongarch64*) host_arch=loongarch64 ;;
（3）修改hack/local-up-cluster.sh文件
loongarch64*) host_arch=loongarch64 ;;
（4）修改vendor/github.com/opencontainers/runc/libcontainer/system/syscall_linux_64.go文件
第二行注释后面追加loong64
//+build arm64 amd64 mips mipsle mips64 mips64le ppc ppc64 ppc64le riscv64 s390x loong64
修改完以上4个文件后再次编译kubelet
root@master1 kubernetes-1.21.5]# KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet
+++ [0509 06:58:10] Building go targets for linux/loong64:
./vendor/k8s.io/code-generator/cmd/prerelease-lifecycle-gen
find: 'rsync': No such file or directory
find: 'rsync': No such file or directory
Generating prerelease lifecycle code for 27 targets
./hack/run-in-gopath.sh: line 34: _output/bin/prerelease-lifecycle-gen: Permission denied
make[1]: *** [Makefile.generated_files:160: gen_prerelease_lifecycle] Error 1
make: *** [Makefile:551: generated_files] Error 2
发现找不到rsync，需要安装rsync并赋予prerelease-lifecycle-gen可执行权限
yum -y install rsync chmod +x _output/bin/prerelease-lifecycle-gen
解决完上面错误后再次编译kubelet
[root@master1 kubernetes-1.21.5]# KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet
Generating prerelease lifecycle code for 27 targets
+++ [0509 07:04:53] Building go targets for linux/loong64:
./vendor/k8s.io/code-generator/cmd/deepcopy-gen
Generating deepcopy code for 227 targets
+++ [0509 07:05:21] Building go targets for linux/loong64:
./vendor/k8s.io/code-generator/cmd/defaulter-gen
Generating defaulter code for 89 targets
+++ [0509 07:05:57] Building go targets for linux/loong64:
./vendor/k8s.io/code-generator/cmd/conversion-gen
Generating conversion code for 124 targets
+++ [0509 07:07:12] Building go targets for linux/loong64:
./vendor/k8s.io/kube-openapi/cmd/openapi-gen
Generating openapi code for KUBE
Generating openapi code for AGGREGATOR
Generating openapi code for APIEXTENSIONS
Generating openapi code for CODEGEN
Generating openapi code for SAMPLEAPISERVER
+++ [0509 07:08:08] Building go targets for linux/loong64:
./vendor/github.com/go-bindata/go-bindata/go-bindata
+++ [0509 07:08:14] Building go targets for linux/loong64:
cmd/kubelet
# k8s.io/kubernetes/vendor/golang.org/x/sys/unix
vendor/golang.org/x/sys/unix/affinity_linux.go:14:35: undefined: _NCPUBITS
vendor/golang.org/x/sys/unix/affinity_linux.go:17:25: undefined: cpuMask
vendor/golang.org/x/sys/unix/affinity_linux.go:50:27: undefined: cpuMask
vendor/golang.org/x/sys/unix/sockcmsg_unix.go:34:9: undefined: Cmsghdr
vendor/golang.org/x/sys/unix/ztypes_linux.go:23:11: undefined: Timespec
vendor/golang.org/x/sys/unix/ztypes_linux.go:24:11: undefined: Timespec
vendor/golang.org/x/sys/unix/ztypes_linux.go:1132:12: undefined: SockaddrStorage
vendor/golang.org/x/sys/unix/ztypes_linux.go:2313:8: undefined: Timespec
vendor/golang.org/x/sys/unix/ztypes_linux.go:2806:8: undefined: TIPCSubscr
vendor/golang.org/x/sys/unix/ztypes_linux.go:3214:10: undefined: PPSKInfo
vendor/golang.org/x/sys/unix/affinity_linux.go:50:27: too many errors
发现报golang.org/x/sys/unix包下某些函数没有定义等错误，这是因为它找不到当前平台所对应的硬件信息文件。解决办法是，用GOROOT下的golang.org/x/sys/unix包替换vendor/golang.org/x/sys/路径下面的unix包。
sudo cp -r /usr/lib/golang-1.18/src/cmd/vendor/golang.org/x/sys/unix vendor/golang.org/x/sys/
替换完unix包后再次重新编译kubelet：
[root@master1 kubernetes-1.21.5]# KUBE_BUILD_PLATFORMS=linux/loong64 make all WHAT=cmd/kubelet
+++ [0509 07:35:07] Building go targets for linux/loong64:
cmd/kubelet
[root@master1 kubernetes-1.21.5]#
经过几分钟等待发现kubelet编译成功。
这里可以到_output/bin目录下验证下kubelet是否编译成功。
[root@master1 bin]# ./kubelet --version Kubernetes v1.21.5
可以看到kebelet组件没问题，至此kubernetes kueblet组件编译成功。
2.2） 编译kubernetes其他组件
编译kubernetes其他组件（如kubeadm、kubectl...）步骤和编译kubelet一致，这里不再赘余。