Golang（十二）TLS 相关知识（三）理解并模拟简单代理

0. 前言

• 前面的介绍我们理解了数字签名等知识，同时学习了 OpenSSL 生成私钥和证书并验证
• 之前提过我们基于 BitTorrent 协议开发了一个 docker 镜像分发加速插件
• 中间涉及到了配置 docker 的代理
• 下面我们简单介绍下 Golang 的 http.transport 配置了网络代理后的网络行为并编写一个简单的代理转发，加深理解代理转发行为

1. http.Transport 配置代理

• http 代理配置代码如下：

func TLSTransport(caFile string) (*http.Transport, error) {
    tr := &http.Transport{TLSClientConfig: &tls.Config{}, Proxy: http.ProxyFromEnvironment}
    if len(caFile) == 0 {
        tr.TLSClientConfig.InsecureSkipVerify = true
        return tr, nil
    }

    ca, err := ioutil.ReadFile(caFile)
    if err != nil {
        return nil, fmt.Errorf("read CA file failed: %v", err)
    }
    pool := x509.NewCertPool()
    pool.AppendCertsFromPEM(ca)

    tr.TLSClientConfig.RootCAs = pool

    return tr, nil
}

• 上述代码制定了 Proxy 为 http.ProxyFromEnvironment
• 我们跟踪一下 http.ProxyFromEnvironment 的代码

func ProxyFromEnvironment(req *Request) (*url.URL, error) {
    return envProxyFunc()(req.URL)
}
/////////////////////////////////////////////////////////////////////////
func envProxyFunc() func(*url.URL) (*url.URL, error) {
    envProxyOnce.Do(func() {
        envProxyFuncValue = httpproxy.FromEnvironment().ProxyFunc()
    })
    return envProxyFuncValue
}
/////////////////////////////////////////////////////////////////////////
func FromEnvironment() *Config {
    return &Config{
        HTTPProxy:  getEnvAny("HTTP_PROXY", "http_proxy"),
        HTTPSProxy: getEnvAny("HTTPS_PROXY", "https_proxy"),
        NoProxy:    getEnvAny("NO_PROXY", "no_proxy"),
        CGI:        os.Getenv("REQUEST_METHOD") != "",
    }
}
/////////////////////////////////////////////////////////////////////////
func (cfg *Config) ProxyFunc() func(reqURL *url.URL) (*url.URL, error) {
    // Preprocess the Config settings for more efficient evaluation.
    cfg1 := &config{
        Config: *cfg,
    }
    cfg1.init()
    return cfg1.proxyForURL
}
/////////////////////////////////////////////////////////////////////////
func (cfg *config) proxyForURL(reqURL *url.URL) (*url.URL, error) {
    var proxy *url.URL
    if reqURL.Scheme == "https" {
        proxy = cfg.httpsProxy
    }
    fmt.Printf("WangAo test: proxy: %+v", proxy)
    if proxy == nil {
        proxy = cfg.httpProxy
        if proxy != nil && cfg.CGI {
            return nil, errors.New("refusing to use HTTP_PROXY value in CGI environment; see golang.org/s/cgihttpproxy")
        }
    }
    if proxy == nil {
        return nil, nil
    }
    if !cfg.useProxy(canonicalAddr(reqURL)) {
        return nil, nil
    }

    return proxy, nil
}

• proxy 指定返回给定请求的代理的函数
• 如果函数返回一个非 nil 错误，请求将因提供的错误而中止
• 代理类型由 URL scheme 决定：支持 http、https 和 socks5
• 如果 scheme 为空，则假定为 http
• 如果 proxy 为 nil 或返回 nil 的 *url.URL 类型，则不使用 proxy
• envProxyFunc 返回一个函数，函数读取环境变量确定代理地址
• FromEnvironment 可以看出代码主要读取 HTTP_PROXY、HTTPS_PROXY、NO_PROXY 和 REQUEST_METHOD
• ProxyFunc 中调用 config.init 方法解析环境变量，并返回实际解析 URL 并返回代理地址的函数
• 在 proxyForURL 中我们发现，对于 https 请求首选是采用 https 代理地址，若 https 代理地址为空或者请求为其他请求则采用 http 地址
• 若配置了 http 代理地址同时配置了 REQUEST_METHOD，返回空代理地址和错误信息
• 如果 http 代理也没有配置则返回空代理地址
• 解析请求信息若为 localhost 或者为回环地址不使用代理地址，否则返回配置的代理地址

2. 测试网络行为

• 上述我们简单读取了 http.ProxyFromEnvironment 读取环境变量确定代理地址的行为
• 下面我们简单介绍下测试代码
• 首先是 Server 端：

package main

import (
    "bufio"
    "context"
    "fmt"
    "git.tencent.com/tke/p2p/pkg/util"
    "github.com/elazarl/goproxy"
    "github.com/gorilla/mux"
    "io"
    "k8s.io/klog"
    "log"
    "net"
    "net/http"
)

func main() {
    go func() {
        log.Println("Starting httpServer")
        router := mux.NewRouter().SkipClean(true)
        proxy := goproxy.NewProxyHttpServer()
        proxy.Verbose = true
        proxy.NonproxyHandler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
            req.URL.Host = req.Host
            req.URL.Scheme = "http"
            proxy.ServeHTTP(w, req)
        })
        proxy.OnRequest(goproxy.ReqHostIs("test.openssl.com:1213")).HijackConnect(func(req *http.Request, client net.Conn, _ *goproxy.ProxyCtx) {
            var err error

            log.Printf("getHijhack: %+v", req.URL)
            defer func() {
                if err != nil {
                    klog.Errorf("Transfer HTTP CONNECT request failed: %+v, %v", req, err)
                    if _, writeErr := client.Write([]byte("HTTP/1.1 500 Cannot reach destination

")); err != nil {
                        klog.Errorf("Write CONNECT failing header failed: %v", writeErr)
                    }
                }
                if closeErr := client.Close(); closeErr != nil {
                    klog.Errorf("Close client connection failed: %v", closeErr)
                }
            }()

            log.Println("before connectDial")
            remote, err := connectDial(proxy, "tcp", "127.0.0.1:1213")
            if remote != nil {
                log.Printf("==============> remote: %+v>%+v
", remote.LocalAddr(), remote.RemoteAddr())
            }
            if err != nil {
                return
            }

            bufferedRemote := bufio.NewReadWriter(bufio.NewReader(remote), bufio.NewWriter(remote))
            bufferedClient := bufio.NewReadWriter(bufio.NewReader(client), bufio.NewWriter(client))

            errCh := make(chan error, 1)
            go func() {
                defer close(errCh)
                if _, reverseErr := io.Copy(bufferedRemote, bufferedClient); reverseErr != nil {
                    klog.Errorf("Transfer remote to client failed: %v", reverseErr)
                    errCh <- reverseErr
                }
            }()

            if _, transferErr := io.Copy(bufferedClient, bufferedRemote); transferErr != nil {
                klog.Errorf("Transfer client to remote failed: %v", transferErr)
                err = transferErr
            }

            if reverseErr := <-errCh; reverseErr != nil {
                err = reverseErr
            }
        })
        router.HandleFunc("/http", func(w http.ResponseWriter, r *http.Request) {
            log.Printf("1--------------------->http: /http >>>>>> req.URL: %+v", r.URL)
            cnt, err := w.Write([]byte(fmt.Sprintf("http: /http return response of req: %+v", r)))
            log.Printf("/http write: cnt: %v, err: %v", cnt, err)
        })
        router.HandleFunc("/https", func(w http.ResponseWriter, r *http.Request) {
            log.Printf("2--------------------->http: /https >>>>>>req.URL: %+v", r.URL)
            cnt, err := w.Write([]byte(fmt.Sprintf("http: /https return response of req: %+v", r)))
            log.Printf("/http write: cnt: %v, err: %v", cnt, err)
            //proxy.ServeHTTP(w, r)
        })
        router.NotFoundHandler = proxy
        if err := http.ListenAndServe(":1212", router); err != nil {
            log.Printf("httpServer err: %+v", err)
        }
    }()
    go func() {
        log.Println("Starting httpsServer")
        router := mux.NewRouter().SkipClean(true)
        proxy := goproxy.NewProxyHttpServer()
        proxy.Verbose = true
        proxy.NonproxyHandler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
            req.URL.Host = req.Host
            req.URL.Scheme = "https"
            proxy.ServeHTTP(w, req)
        })
        if tr, err := util.TLSTransport("/home/ao/Documents/certs/review/server.crt"); err == nil {
            proxy.Tr = tr
        }
        router.HandleFunc("/https", func(w http.ResponseWriter, r *http.Request) {
            log.Printf("3--------------------->https: req: %+v", r)
            cnt, err := w.Write([]byte(fmt.Sprintf("https: /https return response of req: %+v", r)))
            log.Printf("/http write: cnt: %v, err: %v", cnt, err)
        })
        if err := http.ListenAndServeTLS(":1213", "/home/ao/Documents/certs/review/server.crt", "/home/ao/Documents/certs/review/server.key", router); err != nil {
            log.Printf("httsServer err: %+v", err)
        }
    }()
    select {}
}

func dial(proxy *goproxy.ProxyHttpServer, network, addr string) (c net.Conn, err error) {
    if proxy.Tr.DialContext != nil {
        return proxy.Tr.DialContext(context.Background(), network, addr)
    }
    return net.Dial(network, addr)
}

func connectDial(proxy *goproxy.ProxyHttpServer, network, addr string) (c net.Conn, err error) {
    if proxy.ConnectDial == nil {
        return dial(proxy, network, addr)
    }
    return proxy.ConnectDial(network, addr)
}

• 服务端启动了两个 goroutine，分别监听 http 和 https 请求
• http 监听地址为配置的代理地址
• https 为请求实际请求的地址，同时我们设置了拦截 CONNECT 方法的目标域名
• 在拦截 CONNECT 方法之后的回调函数我们看到此时会和 https 监听地址交换数据转发给 https 地址
• 然后我们看一下 Client 端：

package main

import (
    "fmt"
    "git.tencent.com/tke/p2p/pkg/util"
    "io/ioutil"
    "net/http"
)

func main() {
    tr, _ := util.TLSTransport("/home/ao/Documents/certs/review/server.crt")
    client := &http.Client{Transport: tr}

    req, _ := http.NewRequest("GET", "https://test.openssl.com:1213/https", nil)
    resp, err := client.Do(req)
    if err != nil {
        fmt.Printf("err: %+v", err)
    } else {
        body, _ := ioutil.ReadAll(resp.Body)
        fmt.Printf("resp: %+v=>%+v", resp.StatusCode, string(body))
    }
}

• Client 端很简单，我们只是制定了证书发送一个 https 请求
• 分别启动 Server 端和 Client 端我们看一下结果：

$ go run server.go
2019/10/10 14:51:08 Starting httpsServer
2019/10/10 14:51:08 Starting httpServer
2019/10/10 14:51:33 [001] INFO: Running 1 CONNECT handlers
2019/10/10 14:51:33 [001] INFO: on 0th handler: &{3 0x69b280 <nil>} test.openssl.com:1213
2019/10/10 14:51:33 [001] INFO: Hijacking CONNECT to test.openssl.com:1213
2019/10/10 14:51:33 getHijhack: //test.openssl.com:1213
2019/10/10 14:51:33 before connectDial
2019/10/10 14:51:33 ==============> remote: 127.0.0.1:55062>127.0.0.1:1213
2019/10/10 14:51:33 3--------------------->https: req: &{Method:GET URL:/https Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Encoding:[gzip] User-Agent:[Go-http-client/1.1]] Body:{} GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:test.openssl.com:1213 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:55062 RequestURI:/https TLS:0xc0000ae6e0 Cancel:<nil> Response:<nil> ctx:0xc000142510}
2019/10/10 14:51:33 /http write: cnt: 434, err: <nil>
2019/10/10 14:52:54 [002] INFO: Running 1 CONNECT handlers
2019/10/10 14:52:54 [002] INFO: on 0th handler: &{3 0x69b280 <nil>} test.openssl.com:1213
2019/10/10 14:52:54 [002] INFO: Hijacking CONNECT to test.openssl.com:1213
2019/10/10 14:52:54 getHijhack: //test.openssl.com:1213
2019/10/10 14:52:54 before connectDial
2019/10/10 14:52:54 ==============> remote: 127.0.0.1:55066>127.0.0.1:1213
2019/10/10 14:52:54 3--------------------->https: req: &{Method:GET URL:/https Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Encoding:[gzip] User-Agent:[Go-http-client/1.1]] Body:{} GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:test.openssl.com:1213 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:55066 RequestURI:/https TLS:0xc000160160 Cancel:<nil> Response:<nil> ctx:0xc000154510}
2019/10/10 14:52:54 /http write: cnt: 434, err: <nil>

$ HTTP_PROXY=http://127.0.0.1:1212 go run connect.go 
resp: 200=>return response of req: &{Method:GET URL:http://test.openssl.com:1213/https Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Encoding:[gzip] User-Agent:[Go-http-client/1.1]] Body:{} GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:test.openssl.com:1213 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:34024 RequestURI:http://test.openssl.com:1213/https TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc00011cba0}% 

$ HTTPS_PROXY=http://127.0.0.1:1212 go run connect.go 
resp: 200=>return response of req: &{Method:GET URL:http://test.openssl.com:1213/https Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Encoding:[gzip] User-Agent:[Go-http-client/1.1]] Body:{} GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:test.openssl.com:1213 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:34024 RequestURI:http://test.openssl.com:1213/https TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc00011cba0}% 

• 设定 HTTPS_PROXY 和 HTTP_PROXY 启动得到的结果都是一样的
• 当我们把 Client 端请求的结果设为 http 时，我们再看一下 Server 和 Client 的输出：

$ go run server.go
2019/10/10 14:55:21 Starting httpsServer
2019/10/10 14:55:21 Starting httpServer
2019/10/10 14:55:23 2--------------------->http: /https >>>>>>req.URL: http://test.openssl.com:1213/https
2019/10/10 14:55:23 /http write: cnt: 482, err: <nil>

$ HTTP_PROXY=http://127.0.0.1:1212 go run client.go
resp: 200=>http: /https return response of req: &{Method:GET URL:http://test.openssl.com:1213/https Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept-Encoding:[gzip] User-Agent:[Go-http-client/1.1]] Body:{} GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:test.openssl.com:1213 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:34630 RequestURI:http://test.openssl.com:1213/https TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc00007bdd0}%

$ HTTPS_PROXY=http://127.0.0.1:1212 go run client.go
resp: 400=>Client sent an HTTP request to an HTTPS server.

• 同样分别设定 HTTP_PROXY 和 HTTPS_PROXY 发送 http 请求得到结果是不同的
• 配置了 HTTP_PROXY 的请求被代理收到但是没有发出 CONNECT 方法，而是以 http 方式直接请求的
• 配置了 HTTPS_PROXY 的请求没有被代理收到，但是由于 https 服务端同样的 IP 地址，被 https 服务端直接收到。但由于是 http 请求直接返回 400 了

3. 小结

• 本文主要介绍了 http.ProxyFromEnvironment 配置下的代理行为和相关测试代码
• 欢迎各位给出意见