一、基本概念
XDS、CDS、EDS、LDS、SDS、ADS、RDS、HDS
二、安装
三、动态文件配置
还没找到好的控制平面,为了解决动态更新CDS、LDS配置所以就采用了动态文件配置。
admin: access_log_path: /home/logs/envoy/admin.access.log address: socket_address: { address: 0.0.0.0, port_value: 9001} node: cluster: test-cluster id: test-id dynamic_resources: cds_config: path: /home/envoy/cds.yaml lds_config: path: /home/envoy/lds.yaml
上面的配置文件定义动态资源的位置。cds_config 指定 cds配置文件存放位置。lds_config 指定lds配置文件存放位置。
当我们修改了其中某个配置项后可以通过 mv 命令 使envoy进行热更新 。例如: mv cds.yaml cds.yaml1 然后再mv cds.yaml1 cds.yaml。envoy 提供热启动器,但是还是建议使用mv 文件的方式来更新文件,热启动器是完全重新加载配置项,一旦某项配置不小心配置错误会导致整个监听失败。使用mv 热更新的好处是 当envoy检测到有错误配置项时它不会更新配置,这就保证了envoy不会因错误的配置导致整个监听不可用。
四、CDS
cds.yaml 文件中定义一组或多组Cluster。
resources: - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster connect_timeout: 1s name: k8s.proxy type: STATIC http2_protocol_options: {} load_assignment: cluster_name: k8s.proxy endpoints: - lb_endpoints: - endpoint: address: socket_address: address: xxx.xxx.xxx.xxx port_value: 31080 - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster connect_timeout: 1s name: k8shttp1.proxy type: STATIC load_assignment: cluster_name: k8shttp1.proxy endpoints: - lb_endpoints: - endpoint: address: socket_address: address: xxx.xxx.xxx.xx port_value: 31080 - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster connect_timeout: 1s name: ActivityService type: STATIC circuit_breakers: thresholds: - priority: "DEFAULT" max_connections: 100000 max_pending_requests: 100000 max_requests: 100000 - priority: "HIGH" max_connections: 100000 max_pending_requests: 100000 max_requests: 100000 health_checks: timeout: 1s interval: 2s unhealthy_threshold: 1 healthy_threshold: 1 grpc_health_check: {} http2_protocol_options: {} load_assignment: cluster_name: ActivityService endpoints: - lb_endpoints: - endpoint: address: socket_address: address: xxx.xxx.xxx.xxx port_value: 10099 - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster connect_timeout: 1s name: bigdata type: STRICT_DNS circuit_breakers: thresholds: - priority: "DEFAULT" max_connections: 100000 max_pending_requests: 100000 max_requests: 100000 - priority: "HIGH" max_connections: 100000 max_pending_requests: 100000 max_requests: 100000 health_checks: timeout: 1s interval: 2s unhealthy_threshold: 1 healthy_threshold: 1 grpc_health_check: {} http2_protocol_options: {} load_assignment: cluster_name: bigdata endpoints: - lb_endpoints: - endpoint: address: socket_address: address:a.service.consul port_value: 10030
上面的配置中定义了两种不同的cluster,一种是STATIC 表明这种Cluster是指定静态的IP表示上游服务,一种是STRICT_DNS表明这种Cluster通过域名解析出具体的endpoint。
envoy 启动后可以通过curl localhost:9001/clusters 来查看指定的域名有几个Endpoint。
配置中还定义了健康检查、熔断。也可以通过 curl localhost:9001/clusters |grep xxx 来查看EndPoint是否健康,对于不健康的服务envoy不会将流量转发过去。健康检查只有在第一次访问该cluster才开始运行,如果没有流量则不开启健康检查。
五、LDS
lds.yaml 文件中定义了一组或多组监听配置项。
resources: - "@type": type.googleapis.com/envoy.config.listener.v3.Listener name: k8shttps.proxy address: socket_address: address: 0.0.0.0 port_value: 443 filter_chains: - transport_socket: name: "a.com" typed_config: "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext common_tls_context: tls_certificates: certificate_chain: filename: /home/envoy/x.com-crt.pem private_key: filename: /home/envoy/x.com-key.pem filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http access_log: - name: envoy.access_loggers.file typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog path: /home/logs/envoy/k8s.https.access.log http_filters: - name: envoy.filters.http.router route_config: name: local_route virtual_hosts: - name: local_service domains: - a.com - a.com:* routes: - match: prefix: "/com.activity.server" route: cluster: ActivityService - match: prefix: "/AppGL" route: cluster: k8sqahttp1.proxy upgrade_configs: - upgrade_type: "websocket" enabled: true - match: prefix: "/" route: cluster: abc timeout: 1200s - name: local_service domains: ["*"] routes: - match: prefix: "/" headers: - name: ":method" exact_match: "HEAD" direct_response: status: 200 body: inline_string: "heihei" - match: prefix: "/abc." headers: name: abc exact_match: abc1 case_sensitive: false route: cluster: abc1 - match: prefix: "/abc." case_sensitive: false headers: name: abc exact_match: abc2 route: cluster: abc2 - match: prefix: "/hostrewrite" case_sensitive: false route: cluster: pcwang host_rewrite_literal: wangpengchong.com - match: prefix: "/AppGL" route: cluster: k8shttp1.proxy upgrade_configs: - upgrade_type: "websocket" enabled: true - match: prefix: "/" route: cluster: k8s.proxy timeout: 1200s - "@type": type.googleapis.com/envoy.config.listener.v3.Listener name: k8s.proxy address: socket_address: address: 0.0.0.0 port_value: 80 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http access_log: - name: envoy.access_loggers.file typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog path: /home/logs/envoy/k8s.access.log http_filters: - name: envoy.filters.http.router route_config: name: local_route virtual_hosts: - name: local_service domains: - b.com - b.com:* routes: - match: prefix: "/AppGL" route: cluster: k8sqahttp1.proxy upgrade_configs: - upgrade_type: "websocket" enabled: true - match: prefix: "/" route: cluster: k8sqa.proxy timeout: 1200s - name: local_service domains: ["*"] routes: - match: prefix: "/" headers: - name: ":method" exact_match: "HEAD" direct_response: status: 200 body: inline_string: "heihei" - match: prefix: "/" route: cluster: k8s.proxy timeout: 1200s
上面的配置中定义了两组监听分别是443、80 ,443端口有配置https证书,提供https服务。
配置文件中还定义了预发布机制,根据特定域名,将该域名的所有请求都转发到预发布的Cluster中去。该机制还可以用作其他用途,可根据实际情况进行操作
还配置了较为复杂的流量转发,根据请求头中的值将流量转发到不同的Cluster中。
还配置了websocket升级机制,可将http1.1 升级为websocket,这需要客户端发送正确的升级请求。
还配置了host重写机制、超时,特定请求(健康检查)返回固定值,有需要的童鞋可以参考。
六、TCP监听
- "@type": type.googleapis.com/envoy.config.listener.v3.Listener name: AlertService address: socket_address: address: 0.0.0.0 port_value: 20010 filter_chains: - filters: name: envoy.filters.network.tcp typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy stat_prefix: ingress_tcp max_connect_attempts: 5 cluster: AlertService
七、jwk认证鉴权
- "@type": type.googleapis.com/envoy.config.listener.v3.Listener name: k8s.proxy address: socket_address: address: 0.0.0.0 port_value: 10021 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http access_log: - name: envoy.access_loggers.file typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog path: /home/logs/envoy/beta.grpc.access.log http_filters: - name: envoy.filters.http.jwt_authn typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication providers: identityserver: issuer: http://xxxx audiences: - grpc1 remote_jwks: http_uri: uri: http://xxxx/.well-known/openid-configuration/jwks cluster: identityserver timeout: 5s cache_duration: 600s rules: - match: prefix: / requires: provider_name: identityserver
这里展示较为简单的配置,更复杂的请参考官方文档:
https://www.envoyproxy.io/docs/envoy/v1.18.3/api-v3/extensions/filters/http/jwt_authn/v3/config.proto#extension-envoy-filters-http-jwt-authn
八、监控
直接使用 prometheus 收集信息通grafana 展示即可