1、概述
在使用controller-runtime框架进行kubernetes二次开发的时候，程序中通常都会使用GetConfigOrDie()方法获取集群配置以便与kubernetes集群进行连接，示例如下：
opts := ctrl.Options{
Scheme: scheme,
MetricsBindAddress: metricsAddr,
LeaderElection: enableLeaderElection,
Port: 9443,
}
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), opts)
本文主要通过剖析GetConfigOrDie()方法来进行讲解controller-runtime框架的获取kubernetes集群认证配置文件的加载顺序。
2、 GetConfigOrDie()源码剖析
源码文件位置：sigs.k8s.io/controller-runtime/pkg/client/config/config.go
注意：本文中不特殊说明源码文件位置的代码片段都是sigs.k8s.io/controller-runtime/pkg/client/config/config.go文件,本文sigs.k8s.io/controller-runtime框架版本为v0.9.6。
// GetConfigOrDie creates a *rest.Config for talking to a Kubernetes apiserver.
// If --kubeconfig is set, will use the kubeconfig file at that location. Otherwise will assume running
// in cluster and use the cluster provided kubeconfig.
//
// Will log an error and exit if there is an error creating the rest.Config.
func GetConfigOrDie() *rest.Config {
config, err := GetConfig()
if err != nil {
log.Error(err, "unable to get kubeconfig")
os.Exit(1)
}
return config
}
可以看到GetConfigOrDie()方法如同它的名字，假设获取*rest.Config失败便会以异常状态退出程序。
func GetConfig() (*rest.Config, error) {
return GetConfigWithContext("")
}
func GetConfigWithContext(context string) (*rest.Config, error) {
cfg, err := loadConfig(context)
if err != nil {
return nil, err
}
if cfg.QPS == 0.0 {
cfg.QPS = 20.0
cfg.Burst = 30.0
}
return cfg, nil
}
以上方法可以配置*rest.Config.QPS，访问kubernetes集群时的认证配置文件的加载顺序核心逻辑方法为loadConfig方法，逻辑如下：
var (
kubeconfig string
log = logf.RuntimeLog.WithName("client").WithName("config")
)
func init() {
// 启动程序时客户端传参kubeconfig
flag.StringVar(&kubeconfig, "kubeconfig", "",
"Paths to a kubeconfig. Only required if out-of-cluster.")
}
// loadConfig loads a REST Config as per the rules specified in GetConfig.
func loadConfig(context string) (*rest.Config, error) {
// If a flag is specified with the config location, use that
// 1、如果flag初始化了kubeconfig，则从kubeconfig中读取集群配置
if len(kubeconfig) > 0 {
return loadConfigWithContext("", &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfig}, context)
}
// If the recommended kubeconfig env variable is not specified,
// try the in-cluster config.
// 2、 否则从环境变量KUBECONFIG读取，若没有则从集群内部读取，这种场景适用于部署到kubernetes中的场景
kubeconfigPath := os.Getenv(clientcmd.RecommendedConfigPathEnvVar)
if len(kubeconfigPath) == 0 {
if c, err := loadInClusterConfig(); err == nil {
return c, nil
}
}
// If the recommended kubeconfig env variable is set, or there
// is no in-cluster config, try the default recommended locations.
//
// NOTE: For default config file locations, upstream only checks
// $HOME for the user's home directory, but we can also try
// os/user.HomeDir when $HOME is unset.
//
// TODO(jlanford): could this be done upstream?
loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
// 是否存在环境变量HOME
if _, ok := os.LookupEnv("HOME"); !ok {
u, err := user.Current()
if err != nil {
return nil, fmt.Errorf("could not get current user: %v", err)
}
// 3、如果通过KUBECONFIG环境变量和集群内部这两个方式都没有找到（包括从集群内部读取集群配置报错），则会读取默认配置$HOME/.kube/config，如果不存在环境变量HOME，则会读取配置文件user.HomeDir/.kube/config
loadingRules.Precedence = append(loadingRules.Precedence, path.Join(u.HomeDir, clientcmd.RecommendedHomeDir, clientcmd.RecommendedFileName))
}
return loadConfigWithContext("", loadingRules, context)
}
func loadConfigWithContext(apiServerURL string, loader clientcmd.ClientConfigLoader, context string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
loader,
&clientcmd.ConfigOverrides{
ClusterInfo: clientcmdapi.Cluster{
Server: apiServerURL,
},
CurrentContext: context,
}).ClientConfig()
}
接下来再看下从集群内部和通过环境变量（包括读取默认配置）获取集群配置的代码逻辑。
1）集群内部获取集群配置：
InClusterConfig方法,逻辑很简单：在Pod容器内，通过serviceaccout提供的token值和容器内部集群的环境变量生成*rest.Config对象。
InClusterConfig方法源码文件位置：k8s.io/client-go/rest/config.go
// InClusterConfig returns a config object which uses the service account
// kubernetes gives to pods. It's intended for clients that expect to be
// running inside a pod running on kubernetes. It will return ErrNotInCluster
// if called from a process not running in a kubernetes environment.
func InClusterConfig() (*Config, error) {
const (
tokenFile = "/var/run/secrets/kubernetes.io/serviceaccount/token"
rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
)
host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
if len(host) == 0 || len(port) == 0 {
return nil, ErrNotInCluster
}
token, err := ioutil.ReadFile(tokenFile)
if err != nil {
return nil, err
}
tlsClientConfig := TLSClientConfig{}
if _, err := certutil.NewPool(rootCAFile); err != nil {
klog.Errorf("Expected to load root CA config from %s, but got err: %v", rootCAFile, err)
} else {
tlsClientConfig.CAFile = rootCAFile
}
return &Config{
// TODO: switch to using cluster DNS.
Host: "https://" + net.JoinHostPort(host, port),
TLSClientConfig: tlsClientConfig,
BearerToken: string(token),
BearerTokenFile: tokenFile,
}, nil
}
2）通过环境变量（包括读取默认配置）获取集群配置：
不管是flag初始化了kubeconfig，还是从环境变量KUBECONFIG读取，或者是读取默认配置（user.HomeDir/.kube/config或user.HomeDir/.kube/config) 都会实例化ClientConfigLoadingRules结构体对象。
本小节源码文件位置：k8s.io/client-go/tools/clientcmd/loader.go
type ClientConfigLoadingRules struct {
ExplicitPath string
Precedence []string
// MigrationRules is a map of destination files to source files. If a destination file is not present, then the source file is checked.
// If the source file is present, then it is copied to the destination file BEFORE any further loading happens.
MigrationRules map[string]string
// DoNotResolvePaths indicates whether or not to resolve paths with respect to the originating files. This is phrased as a negative so
// that a default object that doesn't set this will usually get the behavior it wants.
DoNotResolvePaths bool
// DefaultClientConfig is an optional field indicating what rules to use to calculate a default configuration.
// This should match the overrides passed in to ClientConfig loader.
DefaultClientConfig ClientConfig
// WarnIfAllMissing indicates whether the configuration files pointed by KUBECONFIG environment variable are present or not.
// In case of missing files, it warns the user about the missing files.
WarnIfAllMissing bool
}
其中：
- ExplicitPath: 表示kubeconfig的路径，优先级高于Precedence；
- Precedence: 表示从KUBECONFIG等环境变量中获取到的路径；
- MigrationRules: 表示旧路径到新路径的映射关系；
- DoNotResolvePaths: 表示是否需要把相对路径转换成绝对路径；
- DefaultClientConfig: 表示默认的配置。
创建ClientConfigLoadingRules结构体对象方法NewDefaultClientConfigLoadingRules()
const (
RecommendedConfigPathFlag = "kubeconfig"
RecommendedConfigPathEnvVar = "KUBECONFIG"
RecommendedHomeDir = ".kube"
RecommendedFileName = "config"
RecommendedSchemaName = "schema"
)
var (
RecommendedConfigDir = filepath.Join(homedir.HomeDir(), RecommendedHomeDir)
RecommendedHomeFile = filepath.Join(RecommendedConfigDir, RecommendedFileName)
RecommendedSchemaFile = filepath.Join(RecommendedConfigDir, RecommendedSchemaName)
)
// NewDefaultClientConfigLoadingRules returns a ClientConfigLoadingRules object with default fields filled in. You are not required to
// use this constructor
func NewDefaultClientConfigLoadingRules() *ClientConfigLoadingRules {
chain := []string{}
warnIfAllMissing := false
envVarFiles := os.Getenv(RecommendedConfigPathEnvVar)
// 从环境变量KUBECONFIG获取集群配置
if len(envVarFiles) != 0 {
fileList := filepath.SplitList(envVarFiles)
// prevent the same path load multiple times
chain = append(chain, deduplicate(fileList)...)
warnIfAllMissing = true
// 从读取$HOME/kube/config文件中获取集群配置
} else {
chain = append(chain, RecommendedHomeFile)
}
return &ClientConfigLoadingRules{
Precedence: chain,
MigrationRules: currentMigrationRules(),
WarnIfAllMissing: warnIfAllMissing,
}
}
3、总结
controller-runtime框架通过GetConfigOrDie()方法获取kubernetes集群认证配置文件的加载顺序如下：
- 1、如果flag初始化了kubeconfig，则从kubeconfig中读取集群配置；
- 2、 否则从环境变量KUBECONFIG读取，若没有则从集群内部读取，这种场景适用于部署到kubernetes中的场景，在Pod容器内，通过serviceaccout提供的token值和容器内部集群的环境变量生成*rest.Config对象。；
- 3、如果通过KUBECONFIG环境变量和集群内部这两个方式都没有找到（包括从集群内部读取集群配置报错），则会读取默认配置$HOME/.kube/config（如果不存在环境变量HOME，则会从user.HomeDir/.kube/config中读取集群配置）。