• gopacket 抓包 过滤器


    //定义过滤器
    func getFilter(port uint16) string {
    // filter := fmt.Sprintf("udp and ((src port %v) or (dst port %v))", port, port)
    filter := fmt.Sprintf("((src port %v) or (dst port %v))", port, port)
    return filter
    }

    【酷Go推荐】网络流量抓包库 gopacket - 知乎 https://zhuanlan.zhihu.com/p/361737169

    网络流量抓包库 gopacket · GoCN社区 https://gocn.vip/topics/11829

    一、gopacket 简介

    1、gopacket 是什么?

    gopacket 是 google 出品的 golang 三方库,质量还是靠的住,项目地址为:github.com/google/gopacket

    gopacket 到底是什么呢?是个抓取网络数据包的库,这么说可能还有点抽象,但是抓包工具大家可能都使用过。

    Windows 平台下有 Wireshark 抓包工具,其底层抓包库是 npcap(以前是 winpcap);

    Linux 平台下有 Tcpdump,其抓包库是 libpcap;

    而 gopacket 库可以说是 libpcap 和 npcap 的 go 封装,提供了更方便的 go 语言操作接口。

    对于抓包库来说,常规功能就是抓包,而网络抓包有以下几个步骤:

    1、枚举主机上网络设备的接口

    2、针对某一网口进行抓包

    3、解析数据包的 mac 层、ip 层、tcp/udp 层字段等

    4、ip 分片重组,或 tcp 分段重组成上层协议如 http 协议的数据

    5、对上层协议进行头部解析和负载部分解析

    2、应用场景有哪些?

    场景 1:网络流量分析

    对网络设备流量进行实时采集以及数据包分析。

    场景 2:伪造数据包

    不少网络安全工具,需要伪造网络数据包,填充上必要的协议字段后发送给对端设备,从而达到一些目的。

    场景 3:离线 pcap 文件的读取和写入

    二、安装部署

    2、1 安装 libpcap 或 npcap 三方库

    在使用 gopacket 包时,首先要确保在 windows 平台下安装了 npcap 或 winpcap,或者是在 linux 平台下安装了 libpcap 库。

    npcap 下载地址:https://nmap.org/npcap/

    libpcap 下载地址:https://www.tcpdump.org/

    下载自己电脑对应的操作系统版本的库

    如果不想从官网下载 libpcap 库的话,也可以采用 centos 的 yum 命令或 ubuntu 的 apt get 命令来进行安装。

    2、2 安装 gopacket 库

    go get github.com/google/gopacket

    三、使用方法

    3、1 枚举网络设备

    package main
    import (
        "fmt"
        "log"
        "github.com/google/gopacket/pcap"
    )
    func main() {
        // 得到所有的(网络)设备
        devices, err := pcap.FindAllDevs()
        if err != nil {
            log.Fatal(err)
        }
        // 打印设备信息
        fmt.Println("Devices found:")
        for _, device := range devices {
            fmt.Println("
    Name: ", device.Name)
            fmt.Println("Description: ", device.Description)
            fmt.Println("Devices addresses: ", device.Description)
            for _, address := range device.Addresses {
                fmt.Println("- IP address: ", address.IP)
                fmt.Println("- Subnet mask: ", address.Netmask)
            }
        }
    }
    

    先调用 pcap.FindAllDevs() 获取当前主机所有的网络设备,网络设备有哪些属性呢?

    // Interface describes a single network interface on a machine.
    type Interface struct {
        Name        string //设备名称
        Description string //设备描述信息
        Flags       uint32 
        Addresses   []InterfaceAddress //网口的地址信息列表
    }
    // InterfaceAddress describes an address associated with an Interface.
    // Currently, it's IPv4/6 specific.
    type InterfaceAddress struct {
        IP        net.IP
        Netmask   net.IPMask // Netmask may be nil if we were unable to retrieve it.
        Broadaddr net.IP     // Broadcast address for this IP may be nil
        P2P       net.IP     // P2P destination address for this IP may be nil
    }
    

    3、2 打开一个设备进行抓包

    package main
    import (
        "fmt"
        "github.com/google/gopacket"
        "github.com/google/gopacket/pcap"
        "log"
        "time"
    )
    var (
        device       string = "eth0"
        snapshot_len int32  = 1024
        promiscuous  bool   = false
        err          error
        timeout      time.Duration = 30 * time.Second
        handle       *pcap.Handle
    )
    func main() {
        // 打开某一网络设备
        handle, err = pcap.OpenLive(device, snapshot_len, promiscuous, timeout)
        if err != nil {log.Fatal(err) }
        defer handle.Close()
        // Use the handle as a packet source to process all packets
        packetSource := gopacket.NewPacketSource(handle, handle.LinkType())
        for packet := range packetSource.Packets() {
            // Process packet here
            fmt.Println(packet)
        }
    }
    

    1)实时捕获

    2、1 节中我们枚举了当前主机的所有网络设备,现在需要打开网络设备并进行实时捕获数据包,需调用 pcap.OpenLive 来打开网络设备,其函数原型如下:

    func OpenLive(device string, snaplen int32, promisc bool, timeout time.Duration) (handle *Handle, _ error)
    

    device:网络设备的名称,如 eth0,也可以填充 pcap.FindAllDevs() 返回的设备的 Name

    snaplen: 每个数据包读取的最大长度 the maximum size to read for each packet

    promisc:是否将网口设置为混杂模式,即是否接收目的地址不为本机的包

    timeout:设置抓到包返回的超时。如果设置成 30s,那么每 30s 才会刷新一次数据包;设置成负数,会立刻刷新数据包,即不做等待

    函数返回值:是一个 *Handle 类型的返回值,可能作为 gopacket 其他函数调用时作为函数参数来传递。

    注意事项:

    一定要记得释放掉 handle,如文中的 defer handle.Close()。

    2)创建数据包源

    packetSource := gopacket.NewPacketSource(handle, handle.LinkType())

    第一个参数为 OpenLive 的返回值,指向 Handle 类型的指针变量 handle。

    第二个参数为 handle.LinkType() 此参数默认是以太网链路,一般我们抓包,也是从 2 层以太网链路上抓取。

    3)读取数据包

    //packetSource.Packets()是个channel类型,此处是从channel类型的数据通道中持续的读取网络数据包
    for packet := range packetSource.Packets() {
            // Process packet here
            fmt.Println(packet)
        }
    

    3、3 解码数据包的各层

    我们可以获取原始数据包,并尝试将其强制转换为已知格式。如 ethernet、IP 和 TCP 层。

    Layers 包是 gopacket 的 Go 库中的新功能,在底层 libpcap 库中不存在。它是 gopacket 库的非常有用的一部分。它允许我们轻松地识别数据包是否包含特定类型的层。这个代码示例将演示如何使用 layers 包来查看包是否是 ethernet、IP 和 TCP,以及如何轻松访问这些头中的字段。

    package main
    import (
        "fmt"
        "github.com/google/gopacket"
        "github.com/google/gopacket/layers"
        "github.com/google/gopacket/pcap"
        "log"
        "strings"
        "time"
    )
    var (
        device      string = "eth0"
        snapshotLen int32  = 1024
        promiscuous bool   = false
        err         error
        timeout     time.Duration = 30 * time.Second
        handle      *pcap.Handle
    )
    func main() {
        // Open device
        handle, err = pcap.OpenLive(device, snapshotLen, promiscuous, timeout)
        if err != nil {log.Fatal(err) }
        defer handle.Close()
        packetSource := gopacket.NewPacketSource(handle, handle.LinkType())
        for packet := range packetSource.Packets() {
            printPacketInfo(packet)
        }
    }
    func printPacketInfo(packet gopacket.Packet) {
        // Let's see if the packet is an ethernet packet
        // 判断数据包是否为以太网数据包,可解析出源mac地址、目的mac地址、以太网类型(如ip类型)等
        ethernetLayer := packet.Layer(layers.LayerTypeEthernet)
        if ethernetLayer != nil {
            fmt.Println("Ethernet layer detected.")
            ethernetPacket, _ := ethernetLayer.(*layers.Ethernet)
            fmt.Println("Source MAC: ", ethernetPacket.SrcMAC)
            fmt.Println("Destination MAC: ", ethernetPacket.DstMAC)
            // Ethernet type is typically IPv4 but could be ARP or other
            fmt.Println("Ethernet type: ", ethernetPacket.EthernetType)
            fmt.Println()
        }
        // Let's see if the packet is IP (even though the ether type told us)
        // 判断数据包是否为IP数据包,可解析出源ip、目的ip、协议号等
        ipLayer := packet.Layer(layers.LayerTypeIPv4)
        if ipLayer != nil {
            fmt.Println("IPv4 layer detected.")
            ip, _ := ipLayer.(*layers.IPv4)
            // IP layer variables:
            // Version (Either 4 or 6)
            // IHL (IP Header Length in 32-bit words)
            // TOS, Length, Id, Flags, FragOffset, TTL, Protocol (TCP?),
            // Checksum, SrcIP, DstIP
            fmt.Printf("From %s to %s
    ", ip.SrcIP, ip.DstIP)
            fmt.Println("Protocol: ", ip.Protocol)
            fmt.Println()
        }
        // Let's see if the packet is TCP
        // 判断数据包是否为TCP数据包,可解析源端口、目的端口、seq序列号、tcp标志位等
        tcpLayer := packet.Layer(layers.LayerTypeTCP)
        if tcpLayer != nil {
            fmt.Println("TCP layer detected.")
            tcp, _ := tcpLayer.(*layers.TCP)
            // TCP layer variables:
            // SrcPort, DstPort, Seq, Ack, DataOffset, Window, Checksum, Urgent
            // Bool flags: FIN, SYN, RST, PSH, ACK, URG, ECE, CWR, NS
            fmt.Printf("From port %d to %d
    ", tcp.SrcPort, tcp.DstPort)
            fmt.Println("Sequence number: ", tcp.Seq)
            fmt.Println()
        }
        // Iterate over all layers, printing out each layer type
        fmt.Println("All packet layers:")
        for _, layer := range packet.Layers() {
            fmt.Println("- ", layer.LayerType())
        }
        ///.......................................................
        // Check for errors
        // 判断layer是否存在错误
        if err := packet.ErrorLayer(); err != nil {
            fmt.Println("Error decoding some part of the packet:", err)
        }
    }
    

    仅仅以此处 tcp 部分的代码详细解析下

    // 判断数据包是否为TCP数据包,可解析源端口、目的端口、seq序列号、tcp标志位等
        tcpLayer := packet.Layer(layers.LayerTypeTCP)
        if tcpLayer != nil {
            fmt.Println("TCP layer detected.")
            tcp, _ := tcpLayer.(*layers.TCP)
            fmt.Printf("From port %d to %d
    ", tcp.SrcPort, tcp.DstPort)
        }
    

    此处需要研究下源代码中数据结构,以防理解错误

    type Packet interface {
        // Layer returns the first layer in this packet of the given type, or nil
        Layer(LayerType) Layer   //根据给定的类型,在数据包中寻找其第一个层
    }
    //看看Layer的结构
    type Layer interface {
        // LayerType is the gopacket type for this layer.
        LayerType() LayerType
        // LayerContents returns the set of bytes that make up this layer.
        LayerContents() []byte
        // LayerPayload returns the set of bytes contained within this layer, not
        // including the layer itself.
        LayerPayload() []byte
    }
    //tcp数据包格式
    type TCP struct {
        BaseLayer
        SrcPort, DstPort                           TCPPort
        Seq                                        uint32
        Ack                                        uint32
        DataOffset                                 uint8
        FIN, SYN, RST, PSH, ACK, URG, ECE, CWR, NS bool
        Window                                     uint16
        Checksum                                   uint16
        Urgent                                     uint16
        sPort, dPort                               []byte
        Options                                    []TCPOption
        Padding                                    []byte
        opts                                       [4]TCPOption
        tcpipchecksum
    }
    

    TCP 结构体是实现了 Layer 接口的,其实 Ethernet,IPV4,UDP 等结构体也实现了 Layer 接口

    在上述代码中,我们调用函数时,传入的 LayerType 协议层的类型为 layers.LayerTypeTCP,函数返回值为 interface 类型,必须转换成 TCP 结构体

    tcp, _ := tcpLayer.(*layers.TCP)

    tcp 是 layers.TCP 这个具体类型的指针,通过 tcp 则可以获取数据包中 tcp 协议的相关字段。

    3、4 自定义层

    自定义层有助于实现当前不包含在 gopacket layers 包中的协议。

    import (
        "fmt"
        "github.com/google/gopacket"
    )
    // 创建自定义层数据结构,并实现Layer接口中的函数LayerType()、LayerContents()、LayerPayload()
    type CustomLayer struct {
        // This layer just has two bytes at the front
        SomeByte    byte
        AnotherByte byte
        restOfData  []byte
    }
    // 注册自定义层类型,然后我们才可以使用它
    // 第一个参数是ID. 自定义层使用大于2000的数字,它必须是唯一的
    var CustomLayerType = gopacket.RegisterLayerType(
        2001,
        gopacket.LayerTypeMetadata{
            "CustomLayerType",
            gopacket.DecodeFunc(decodeCustomLayer),
        },
    )
    
    //自定义层实现LayerType
    func (l CustomLayer) LayerType() gopacket.LayerType {
        return CustomLayerType
    }
    
    //自定义层实现LayerContents
    func (l CustomLayer) LayerContents() []byte {
        return []byte{l.SomeByte, l.AnotherByte}
    }
    
    //自定义层实现LayerPayload
    func (l CustomLayer) LayerPayload() []byte {
        return l.restOfData
    }
    
    //实现自定义的解码函数
    func decodeCustomLayer(data []byte, p gopacket.PacketBuilder) error {
        p.AddLayer(&CustomLayer{data[0], data[1], data[2:]})
        return p.NextDecoder(gopacket.LayerTypePayload)
    }
    func main() {
        rawBytes := []byte{0xF0, 0x0F, 65, 65, 66, 67, 68}
        packet := gopacket.NewPacket(
            rawBytes,
            CustomLayerType,
            gopacket.Default,
        )
        fmt.Println("Created packet out of raw bytes.")
        fmt.Println(packet)
        // Decode the packet as our custom layer
        customLayer := packet.Layer(CustomLayerType)
        if customLayer != nil {
            fmt.Println("Packet was successfully decoded with custom layer decoder.")
            customLayerContent, _ := customLayer.(*CustomLayer)
            // Now we can access the elements of the custom struct
            fmt.Println("Payload: ", customLayerContent.LayerPayload())
            fmt.Println("SomeByte element:", customLayerContent.SomeByte)
            fmt.Println("AnotherByte element:", customLayerContent.AnotherByte)
        }
    }
    

    结合上述代码可知,实现自定义的层需要 3 步:

    1、创建自定义层的结构体,并实现 Layer 接口中的函数 LayerType()、LayerContents()、LayerPayload()

    2、按照解码函数签名来实现自定义解码函数,名称可自行命名。

    解码函数签名如下:

    type DecodeFunc func([] byte, PacketBuilder) error

    3、使用 gopacket.RegisterLayerType 函数来注册自定义层

    3、5 TCP 流重组

    为什么需要 tcp 流重组?

    package main
    
    import (
        "bufio"
        "flag"
        "io"
        "log"
        "net/http"
        "time"
    
        "github.com/google/gopacket"
        "github.com/google/gopacket/examples/util"
        "github.com/google/gopacket/layers"
        "github.com/google/gopacket/pcap"
        "github.com/google/gopacket/tcpassembly"
        "github.com/google/gopacket/tcpassembly/tcpreader"
    )
    
    var iface = flag.String("i", "eth0", "Interface to get packets from")
    var snaplen = flag.Int("s", 1600, "SnapLen for pcap packet capture")
    
    // Build a simple HTTP request parser using tcpassembly.StreamFactory and tcpassembly.Stream interfaces
    
    // httpStreamFactory implements tcpassembly.StreamFactory
    type httpStreamFactory struct{}
    
    // httpStream will handle the actual decoding of http requests.
    type httpStream struct {
        net, transport gopacket.Flow
        r              tcpreader.ReaderStream
    }
    
    func (h *httpStreamFactory) New(net, transport gopacket.Flow) tcpassembly.Stream {
        hstream := &httpStream{
            net:       net,
            transport: transport,
            r:         tcpreader.NewReaderStream(),
        }
        go hstream.run() // Important... we must guarantee that data from the reader stream is read.
    
        // ReaderStream implements tcpassembly.Stream, so we can return a pointer to it.
        return &hstream.r
    }
    
    func (h *httpStream) run() {
        buf := bufio.NewReader(&h.r)
        for {
            req, err := http.ReadRequest(buf)
            if err == io.EOF {
                // We must read until we see an EOF... very important!
                return
            } else if err != nil {
                log.Println("Error reading stream", h.net, h.transport, ":", err)
            } else {
                bodyBytes := tcpreader.DiscardBytesToEOF(req.Body)
                req.Body.Close()
                log.Println("Received request from stream", h.net, h.transport, ":", req, "with", bodyBytes, "bytes in request body")
            }
        }
    }
    
    func main() {
        defer util.Run()()
        var handle *pcap.Handle
        var err error
    
        // Set up pcap packet capture
        handle, err = pcap.OpenLive(*iface, int32(*snaplen), true, pcap.BlockForever)
        if err != nil {
            log.Fatal(err)
        }
    
        // Set up assembly
        streamFactory := &httpStreamFactory{}
        streamPool := tcpassembly.NewStreamPool(streamFactory)
        assembler := tcpassembly.NewAssembler(streamPool)
    
        // Read in packets, pass to assembler.
        packetSource := gopacket.NewPacketSource(handle, handle.LinkType())
        packets := packetSource.Packets()
        ticker := time.Tick(time.Minute)
        for {
            select {
            case packet := <-packets:
                if packet.NetworkLayer() == nil || packet.TransportLayer() == nil || packet.TransportLayer().LayerType() != layers.LayerTypeTCP {
                    log.Println("Unusable packet")
                    continue
                }
                tcp := packet.TransportLayer().(*layers.TCP)
                //将数据包进行重组
                assembler.AssembleWithTimestamp(packet.NetworkLayer().NetworkFlow(), tcp, packet.Metadata().Timestamp)
    
            case <-ticker:
                //每隔一分钟,刷新之前两分钟内不活动的连接
                assembler.FlushOlderThan(time.Now().Add(time.Minute * -2))
            }
        }
    }
    

    基本步骤如下:

    1、创建 httpStreamFactory 结构体,实现 tcpassembly.StreamFactory 接口

    2、创建连接池

    streamPool := tcpassembly.NewStreamPool(streamFactory)

    3、创建重组器

    assembler := tcpassembly.NewAssembler(streamPool)

    4、将数据包添加到重组器中

    assembler.AssembleWithTimestamp(packet.NetworkLayer().NetworkFlow(), tcp, packet.Metadata().Timestamp)

    三、总结

    首先,gopacket 库是 google 大厂背书,从使用文档、质量、社区活跃度来说都很不错

    其次,使用方式简单,扩展性好。gopacket 提供了自定义的接口,可根据自身需要进行定制化开发

    最后,gopacket 定义的 layers 齐全,如果是实时捕获数据后进行协议解析,采用其内置的 layer 即可,无需自己手动去解析繁杂的协议了。

    golang gopacket网络抓包和分析 - 翔云123456 - 博客园 https://www.cnblogs.com/lanyangsh/p/9821106.html

    Demo

    代码中,抓取与端口3306相关的数据,也就是mysql通信数据。

    package main
    
    import(
    	"fmt"
    	"net"
    	"strings"
    
    	"github.com/google/gopacket"
    	"github.com/google/gopacket/layers"
    	"github.com/google/gopacket/pcap"
    )
    
    func main() {
    
    	fmt.Println("packet start...")
    
    	deviceName := "eth0" 
    	snapLen := int32(65535)
    	port := uint16(3306)
    	filter := getFilter(port) 
    	fmt.Printf("device:%v, snapLen:%v, port:%v
    ", deviceName, snapLen, port)
    	fmt.Println("filter:", filter)
    
    	//打开网络接口,抓取在线数据
    	handle, err := pcap.OpenLive(deviceName, snapLen, true, pcap.BlockForever)
    	if err != nil {
    		fmt.Printf("pcap open live failed: %v", err)
    		return
    	}
    
    	// 设置过滤器
    	if err := handle.SetBPFFilter(filter); err != nil {
    		fmt.Printf("set bpf filter failed: %v", err)
    		return
    	}
    	defer handle.Close()
    
    	// 抓包
    	packetSource := gopacket.NewPacketSource(handle, handle.LinkType())
    	packetSource.NoCopy = true
    	for packet := range packetSource.Packets() {
    		if packet.NetworkLayer() == nil || packet.TransportLayer() == nil || packet.TransportLayer().LayerType() != layers.LayerTypeTCP {
    			fmt.Println("unexpected packet")
    			continue
    		}
    		
    		fmt.Printf("packet:%v
    ",packet)
    
    		// tcp 层
    		tcp := packet.TransportLayer().(*layers.TCP)
    		fmt.Printf("tcp:%v
    ", tcp)		
    		// tcp payload,也即是tcp传输的数据
    		fmt.Printf("tcp payload:%v
    ", tcp.Payload)
    	}
    }
    
    //定义过滤器
    func getFilter(port uint16) string {
     	filter := fmt.Sprintf("tcp and ((src port %v) or (dst port %v))",  port, port)
    	return filter
    }
    

    抓取到的数据包

    packet start...
    device:lo0, snapLen:65535, port:3306
    filter: tcp and ((src port 3306) or (dst port 3306))
    
    packet:PACKET: 75 bytes, wire length 75 cap length 75 @ 2018-10-20 11:13:00.106452 +0800 CST
    - Layer 1 (04 bytes) = Loopback	{Contents=[2, 0, 0, 0] Payload=[..71..] Family=IPv4}
    - Layer 2 (20 bytes) = IPv4	{Contents=[..20..] Payload=[..51..] Version=4 IHL=5 TOS=0 Length=71 Id=0 Flags=DF FragOffset=0 TTL=64 Protocol=TCP Checksum=0 SrcIP=172.16.1.103 DstIP=172.16.1.103 Options=[] Padding=[]}
    - Layer 3 (32 bytes) = TCP	{Contents=[..32..] Payload=[..19..] SrcPort=50351 DstPort=3306(mysql) Seq=110592366 Ack=3116315438 DataOffset=8 FIN=false SYN=false RST=false PSH=true ACK=true URG=false ECE=false CWR=false NS=false Window=12753 Checksum=23336 Urgent=0 Options=[TCPOption(NOP:), TCPOption(NOP:), TCPOption(Timestamps:1064185591/1064170040 0x3f6e2ef73f6df238)] Padding=[]}
    - Layer 4 (19 bytes) = Payload	19 byte(s)
    
    tcp:&{{[196 175 12 234 6 151 129 110 185 191 51 46 128 24 49 209 91 40 0 0 1 1 8 10 63 110 46 247 63 109 242 56] [15 0 0 0 3 115 104 111 119 32 100 97 116 97 98 97 115 101 115]} 50351 3306(mysql) 110592366 3116315438 8 false false false true true false false false false 12753 23336 0 [196 175] [12 234] [TCPOption(NOP:) TCPOption(NOP:) TCPOption(Timestamps:1064185591/1064170040 0x3f6e2ef73f6df238)] [] [{1 1 []} {1 1 []} {8 10 [63 110 46 247 63 109 242 56]} {0 0 []}] {<nil>}}
    tcp payload:[15 0 0 0 3 115 104 111 119 32 100 97 116 97 98 97 115 101 115]
    。。。
    
    

    对抓取到的mysql数据感兴趣的同学,可以参考Mysql 通信协议抓包分析

  • 相关阅读:
    视频实例分割 | Target-Aware Adaptive Tracking for Unsupervised Video Object Segmentation
    目标检测算法:Selective Search(选择性搜索)简述
    [2020BUAA软工助教]期末总结
    WPF,数据验证方案,验证通过才能继续下一步
    WPF:解决数据验证ValidationRule与按钮Canexcute联动的问题
    解决mininet运行报错“ImportError: No module named mininet.log”
    交易所对接ERC20钱包端口要多长时间?
    尺子的刻度
    学习java的第四周
    学习java的第三周
  • 原文地址:https://www.cnblogs.com/rsapaper/p/15493262.html
Copyright © 2020-2023  润新知