• 一个linux下简单的纯C++实现Http请求类(GET,POST,上传,下载)


    一个linux下简单的纯C++实现Http请求类(GET,POST,上传,下载)


    最近写了点关于Http上传下载文件相关的,于是今天整理下代码。

    Http协议简述
    HttpRequest类设计
    使用示例

    Http协议简述

    协议:网络协议的简称,网络协议是通信计算机双方必须共同遵从的一组约定。如怎么样建立连接、怎么样互相识别等。只有遵守这个约定,计算机之间才能相互通信交流,就好比两台计算机交互的语言.

    HTTP协议:超文本传输协议(HyperText Transfer Protocol)是互联网上应用最为广泛的一种网络协议。主要被用于在Web浏览器和网站服务器之间传递信息。 HTTP 是基于 TCP/IP 协议的应用层协议。默认使用80端口。最新版本是HTTP 2.0,目前是用最广泛的是HTTP 1.1。

    HTTP协议工作于客户端-服务端架构为上。浏览器作为HTTP客户端通过URL向HTTP服务端即WEB服务器发送所有请求。Web服务器根据接收到的请求后,向客户端发送响应信息。

    请求方法:
    HTTP/1.1协议中共定义了八种方法(有时也叫“动作”)来表明Request-URI指定的资源的不同操作方式:
    OPTIONS - 返回服务器针对特定资源所支持的HTTP请求方法。也可以利用向Web服务器发送'*'的请求来测试服务器的功能性。
    HEAD- 向服务器索要与GET请求相一致的响应,只不过响应体将不会被返回。这一方法可以在不必传输整个响应内容的情况下,就可以获取包含在响应消息头中的元信息。该方法常用于测试超链接的有效性,是否可以访问,以及最近是否更新。
    GET - 向特定的资源发出请求。注意:GET方法不应当被用于产生“副作用”的操作中,例如在web app.中。其中一个原因是GET可能会被网络蜘蛛等随意访问。
    POST - 向指定资源提交数据进行处理请求(例如提交表单或者上传文件)。数据被包含在请求体中。POST请求可能会导致新的资源的建立和/或已有资源的修改。
    PUT - 向指定资源位置上传其最新内容。
    DELETE - 请求服务器删除Request-URI所标识的资源。
    TRACE- 回显服务器收到的请求,主要用于测试或诊断。
    CONNECT - HTTP/1.1协议中预留给能够将连接改为管道方式的代理服务器。
    PATCH - 用来将局部修改应用于某一资源,添加于规范RFC5789。
    方法名称是区分大小写的。当某个请求所针对的资源不支持对应的请求方法的时候,服务器应当返回状态码405(Method Not Allowed);当服务器不认识或者不支持对应的请求方法的时候,应当返回状态码501(Not Implemented)。
    HTTP服务器至少应该实现GET和HEAD方法,其他方法都是可选的。此外,除了上述方法,特定的HTTP服务器还能够扩展自定义的方法。

    GET和POST请求的区别
    GET请求
    GET /books/?sex=man&name=Professional HTTP/1.1
    Host: www.wrox.com
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6)
    Gecko/20050225 Firefox/1.0.1
    Connection: Keep-Alive
    注意最后有一行空行

    POST请求
    POST / HTTP/1.1
    Host: www.wrox.com
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6)
    Gecko/20050225 Firefox/1.0.1
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 40
    Connection: Keep-Alive

    name=Professional%20Ajax&publisher=Wiley

    1、GET提交:请求的数据会附在URL之后(就是把数据放置在HTTP协议头中),以?分割URL和传输数据,多个参数用&连接;例 如:login.action?name=hyddd&password=idontknow&verify=%E4%BD%A0 %E5%A5%BD。如果数据是英文字母/数字,原样发送,如果是空格,转换为+,如果是中文/其他字符,则直接把字符串用BASE64加密,得出如: %E4%BD%A0%E5%A5%BD,其中%XX中的XX为该符号以16进制表示的ASCII。

    2、POST提交:把提交的数据放置在是HTTP包的包体中。上文示例中红色字体标明的就是实际的传输数据

    HttpRequest类设计

    请求部分

    HttpRequest::HttpRequest() 通过传入Url构造HttpUrl类分离url域名及uri.
    HttpRequest::connect() 通过gethostbyname()获取域名ip,与80端口组成远端地址建立链接.
    HttpRequest::setRequestMethod() 设置请求方法,目前只添加了Get和Post请求.
    HttpRequest::setRequestProperty() 设置属性.
    HttpRequest::setRequestBody() 设置content.
    HttpRequest::send() 将设置的请求流发送出去.

    接收部分

    HttpRequest::handRead() 处理服务器应答头.
    HttpRequest::getResponseCode() handRead()后可以获取到应答code
    HttpRequest::getResponseProperty() handRead()后可以获取到对应的应答属性
    HttpRequest::getResponseContent() handRead()后可以获取到应答content

    #ifndef _HTTP_REQUEST_HH
    #define _HTTP_REQUEST_HH
    
    #include <netdb.h>
    #include <arpa/inet.h>
    #include <algorithm>
    #include <vector>
    #include <string>
    #include <assert.h>
    #include <string.h>
    #include <sstream>
    #include <map>
    #include "Logger.hh"
    
    const size_t kBufferSize = 4096;
    
    /// A buffer class modeled after org.jboss.netty.buffer.ChannelBuffer
    ///
    /// @code
    /// +-------------------+------------------+------------------+
    /// | prependable bytes |  readable bytes  |  writable bytes  |
    /// |                   |     (CONTENT)    |                  |
    /// +-------------------+------------------+------------------+
    /// |                   |                  |                  |
    /// 0      <=      readerIndex   <=   writerIndex    <=     size
    /// @endcode
    class Buffer
    {
    public:
    
      static const size_t kCheapPrepend = 8;
      static const size_t kInitialSize = 4096;
    //public:
    
    	explicit Buffer(size_t initialSize = kInitialSize)
    	: m_buffer(kCheapPrepend + initialSize),
    	  m_readerIndex(kCheapPrepend),
    	  m_writerIndex(kCheapPrepend)
    	  {
    	    assert(readableBytes() == 0);
            assert(writableBytes() == initialSize);
            assert(prependableBytes() == kCheapPrepend);
    	  }
    
      size_t readableBytes() const
      { return m_writerIndex - m_readerIndex; }
    
      size_t writableBytes() const
      { return m_buffer.size() - m_writerIndex; }
    
      size_t prependableBytes() const
      { return m_readerIndex; }
    
      const char* peek() const
      { return begin() + m_readerIndex; }
    
      char* beginWrite()
      { return begin() + m_writerIndex; }
    
      void hasWritten(size_t len)
      {
        assert(len <= writableBytes());
        m_writerIndex += len;
      }
    
      void unwrite(size_t len)
      {
        assert(len <= readableBytes());
        m_writerIndex -= len;
      }
    
      // retrieve returns void, to prevent
      // string str(retrieve(readableBytes()), readableBytes());
      // the evaluation of two functions are unspecified
      void retrieve(size_t len)
      {
        assert(len <= readableBytes());
        if (len < readableBytes())
        {
          m_readerIndex += len;
        }
        else
        {
          retrieveAll();
        }
      }
    
      void retrieveAll()
      {
      	m_readerIndex = kCheapPrepend;
      	m_writerIndex = kCheapPrepend;
      }
    
    private:
      char* begin()
      {return &*m_buffer.begin(); }
    
      const char* begin() const
      {return &*m_buffer.begin(); }
    
    private:
    	std::vector<char> m_buffer;
    	size_t m_readerIndex;
    	size_t m_writerIndex;
    };
    
    
    namespace sockets
    {
    
    /// Creates a non-blocking socket file descriptor,
    /// abort if any error.
    int createSocket(sa_family_t family);
    
    int  connect(int sockfd, const struct sockaddr* addr);
    ssize_t read(int sockfd, void *buf, size_t count);
    ssize_t readv(int sockfd, const struct iovec *iov, int iovcnt);
    ssize_t write(int sockfd, const void *buf, size_t count);
    void close(int sockfd);
    
    void fromIpPort(const char* ip, uint16_t port,
                    struct sockaddr_in* addr);
    
    int getSocketError(int sockfd);
    void delaySecond(int sec);
    //const struct sockaddr* sockaddr_cast(const struct sockaddr_in* addr)
    //const struct sockaddr_in* sockaddr_in_cast(const struct sockaddr* addr);
    
    }
    
    class InetAddress
    {
     public:
      /// Constructs an endpoint with given ip and port.
      /// @c ip should be "1.2.3.4"
      InetAddress(std::string ip, uint16_t port);
    
      /// Constructs an endpoint with given struct @c sockaddr_in
      /// Mostly used when accepting new connections
      explicit InetAddress(const struct sockaddr_in& addr)
        : m_addr(addr)
      { }
    
      sa_family_t family() const { return m_addr.sin_family; }
      //std::string toIp() const;
      //std::string toIpPort() const;
    
      const struct sockaddr* getSockAddr() const { return (struct sockaddr*)(&m_addr); }
    
      uint32_t ipNetEndian() const;
    
      // resolve hostname to IP address, not changing port or sin_family
      // return true on success.
      // thread safe
      // static bool resolve(StringArg hostname, StringArg* ip);
      // static std::vector<InetAddress> resolveAll(const char* hostname, uint16_t port = 0);
    
     private:
        struct sockaddr_in m_addr;
    
    };
    
    
    class HttpUrl
    {
    public:
    	HttpUrl(std::string& httpUrl)
    	:m_httpUrl(httpUrl),
    	 m_smatch(detachHttpUrl())
    	{
    		LOG_DEBUG << "URL : " << m_httpUrl;
    	}
    	~HttpUrl(){};
    
    	enum HttpUrlMatch
    	{
    		URL = 0,
    		HOST = 1,
    		URI = 2
    	};
    
    	std::vector<std::string> detachHttpUrl() const
    	{
    	  std::vector<std::string> v;
    	  std::string::size_type pos1, pos2;
    	  pos2 = m_httpUrl.find('/');
    	  assert(std::string::npos != pos2);
    	  pos1 = pos2 + 2;
    	  pos2 = m_httpUrl.find('/', pos1);
    	  assert(std::string::npos != pos2);
    	  v.push_back(m_httpUrl);
    	  v.push_back(m_httpUrl.substr(pos1, pos2 - pos1));
    	  v.push_back(m_httpUrl.substr(pos2 + 1));
    	  LOG_DEBUG << "detachHttpUrl() url :" << v[0];
    	  LOG_DEBUG << "detachHttpUrl() host :" << v[1];
    	  LOG_DEBUG << "detachHttpUrl() uri :" << v[2];
    	  return v;
    	}
    
    	bool HttpUrlToIp(const std::string& host, char* ip) const
    	{
    		struct hostent* phost = NULL;
    
    		phost = gethostbyname(host.c_str());
    	    if (NULL == phost)
    	    {
    	    	LOG_ERROR << "HttpUrlToIp(): gethostbyname error : " << errno << " : "<< strerror(errno);
    	    	return false;
    	    	//LOG_SYSERR << "urlToIp(): gethostbyname error";
    	    }
    
    	    inet_ntop(phost->h_addrtype,  phost->h_addr, ip, 17);
    
    	    return true;
    	}
    
    	std::string domain() const
    	{
    		return getHttpUrlSubSeg(HOST);
    	}
    
    	std::string getHttpUrlSubSeg(HttpUrlMatch sub = HOST) const{ return m_smatch[sub]; }
    
    private:
    	std::string m_httpUrl;
    	std::vector<std::string> m_smatch;
    };
    
    
    class HttpRequest
    {
    public:
    	enum HttpRequestMethod{
    		GET = 0,
    		POST
    	};
    	HttpRequest(std::string httpUrl);
    	~HttpRequest();
    
    	void connect();
    	//void TEST(const std::string path,const std::string content);
    	void setRequestMethod(const std::string &method);
    	void setRequestProperty(const std::string &key, const std::string &value);
    	void setRequestBody(const std::string &content);
    
    	//void clear() { clearStream(); m_buffer.retrieveAll(); }
    	void clearStream() {m_stream.str("");}
    	std::string strStream() const { return m_stream.str(); };
    
    	int getResponseCode() const {
    		assert(m_haveHandleHead);
    		return m_code;
    	}
    
    	std::string getResponseProperty(const std::string& key) const {
    		assert(m_haveHandleHead);
    		return m_ackProperty.at(key);
    	}
    
    	std::string getResponseContent() {
    		assert(m_haveHandleHead);
    		return std::string(m_buffer.peek(), m_buffer.readableBytes());
    	}
    
    	void handleRead();
    	void uploadFile(const std::string& file, const std::string& contentEnd);
    	void downloadFile(const std::string& file);
    
    	void send(){
    		sockets::write(m_sockfd, strStream().c_str(), strStream().size());
    	}
    
    	void close(){ sockets::close(m_sockfd); }
    
    private:
        void SplitString(const std::string& s, std::vector<std::string>& v, const std::string& c);
    
      Buffer m_buffer;
    	HttpUrl m_httpUrl;
    	std::stringstream m_stream;
    	int m_code;
    	int m_sockfd;
    	bool m_haveHandleHead;
    	std::map<std::string, std::string> m_ackProperty;
    };
    
    #endif
    

    关于上传和下载

    HttpRequest::downloadFile(const std::string& file);
    下载直接在 handRead() 处理完应答头后 , 调用downloadFile() 存储在本地.
    HttpRequest::uploadFile(const std::string& file, const std::string& contentEnd)
    上传部分复杂一点,需要根据kBoundary 设置边界,如下.

    	off_t fileSize = FileSize(uploadFile);
    	LOG_DEBUG << "fileSize : " << fileSize;
    	std::stringstream content;
    	content << "--" + kBoundary << "
    ";
    	content << "Content-Disposition: form-data; name="upload"; filename="test.file"
    ";
    	content << "Content-Type: text/plain
    
    ";
    
    	std::string contentEnd = "
    --" + kBoundary + "--
    ";
    
    	HttpClient upload("http://xxxxx/upload");
    	upload.setRequestMethod("POST");
    	upload.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + kBoundary);
    	upload.setRequestProperty("Cache-Control", "no-cache");
    	upload.setRequestProperty("Content-Length", std::to_string(fileSize + content.str().size() + contentEnd.size()));
    	upload.setRequestProperty("Connection", "close
    ");
    	upload.setRequestBody(content.str());
    

    Cpp实现

    #include <string>
    #include <iostream>
    #include <fcntl.h>
    #include <errno.h>
    #include <sys/uio.h>  // readv
    #include <stdint.h>
    #include <endian.h>
    #include <unistd.h>
    #include <map>
    #include <fstream>
    
    #include "HttpRequest.hh"
    
    const std::map<std::string, int>::value_type init_value[] =
    {
    	std::map<std::string, int>::value_type( "GET", HttpRequest::GET),
    
    	std::map<std::string, int>::value_type( "POST", HttpRequest::POST)
    };
    
    const static std::map<std::string, int> kRequestMethodMap(init_value, init_value + (sizeof init_value / sizeof init_value[0]));
    
    static inline uint16_t hostToNetwork16(uint16_t host16)
    {
      return htobe16(host16);
    }
    
    
    int sockets::createSocket(sa_family_t family){
      // Call "socket()" to create a (family) socket of the specified type.
      // But also set it to have the 'close on exec' property (if we can)
    
    	int sock;
    
    	//CLOEXEC,即当调用exec()函数成功后,文件描述符会自动关闭。
    	//在以往的内核版本(2.6.23以前)中,需要调用 fcntl(fd, F_SETFD, FD_CLOEXEC) 来设置这个属性。
    	//而新版本(2.6.23开始)中,可以在调用open函数的时候,通过 flags 参数设置 CLOEXEC 功能,
    #ifdef SOCK_CLOEXEC
      sock = socket(family, SOCK_STREAM|SOCK_CLOEXEC, 0);
      if (sock != -1 || errno != EINVAL) return sock;
      // An "errno" of EINVAL likely means that the system wasn't happy with the SOCK_CLOEXEC; fall through and try again without it:
    #endif
    
      sock = socket(family, SOCK_STREAM, 0);
    
    #ifdef FD_CLOEXEC
      if (sock != -1) fcntl(sock, F_SETFD, FD_CLOEXEC);
    #endif
      return sock;
    }
    
    int sockets::connect(int sockfd, const struct sockaddr* addr)
    {
      return ::connect(sockfd, addr, sizeof(struct sockaddr));
    }
    
    void sockets::fromIpPort(const char* ip, uint16_t port,
                             struct sockaddr_in* addr)
    {
      addr->sin_family = AF_INET;
      addr->sin_port = hostToNetwork16(port);
      if (::inet_pton(AF_INET, ip, &addr->sin_addr) <= 0)
      {
        LOG_SYSERR << "sockets::fromIpPort";
      }
    }
    
    ssize_t sockets::read(int sockfd, void *buf, size_t count)
    {
      return ::read(sockfd, buf, count);
    }
    
    ssize_t sockets::readv(int sockfd, const struct iovec *iov, int iovcnt)
    {
      return ::readv(sockfd, iov, iovcnt);
    }
    
    ssize_t sockets::write(int sockfd, const void *buf, size_t count)
    {
      return ::write(sockfd, buf, count);
    }
    
    void sockets::close(int sockfd)
    {
      if (::close(sockfd) < 0)
      {
        LOG_SYSERR << "sockets::close";
      }
    }
    
    void sockets::delaySecond(int sec)
    {
      struct  timeval tv;
      tv.tv_sec = sec;
      tv.tv_usec = 0;
      select(0, NULL, NULL, NULL, &tv);
    }
    
    
    
    InetAddress::InetAddress(std::string ip, uint16_t port)
    {
      ::bzero(&m_addr, sizeof m_addr);
      sockets::fromIpPort(ip.c_str(), port, &m_addr);
    }
    
    uint32_t InetAddress::ipNetEndian() const
    {
      assert(family() == AF_INET);
      return m_addr.sin_addr.s_addr;
    }
    
    
    HttpRequest::HttpRequest(std::string httpUrl)
      :m_httpUrl(httpUrl)
    {
    
    }
    
    HttpRequest::~HttpRequest()
    {
    
    }
    
    void HttpRequest::connect()
    {
      char ip[32] = {0};
      while(true)
      {
        struct hostent* phost = NULL;
    
        phost = gethostbyname(m_httpUrl.domain().c_str());
        if (NULL == phost)
        {
          LOG_ERROR << "HttpUrlToIp(): gethostbyname error : " << errno << " : "<< strerror(errno) << " continue.";
          sockets::delaySecond(1);
          continue;
        }
    
        inet_ntop(phost->h_addrtype,  phost->h_addr, ip, sizeof ip);
    
        LOG_DEBUG << "HttpRequest::Connector() gethostbyname Successful";
    
        InetAddress serverAddr = InetAddress(ip, 80);
    
        m_sockfd = sockets::createSocket(serverAddr.family());
        if(m_sockfd < 0) LOG_SYSERR << "HttpRequest::connect() : createSocket error";
        int ret = sockets::connect(m_sockfd, serverAddr.getSockAddr());
        LOG_DEBUG << "sockfd : " << m_sockfd << "sockets::connect ret : " << ret ;
        int savedErrno = (ret == 0) ? 0 : errno;
    
        switch (savedErrno)
        {
          case 0:
          case EINPROGRESS:
          case EINTR:
          case EISCONN:
            LOG_INFO << "HttpRequest::connect() sockfd : " << m_sockfd << " Successful";
            break;
          default :
            LOG_ERROR << "Connect Error ";
            sockets::delaySecond(1);
            continue;
        }
    
        break;
      }
    
      LOG_DEBUG << "HttpRequest::Connector() end";
    }
    
    void HttpRequest::setRequestMethod(const std::string &method)
    {
    	switch(kRequestMethodMap.at(method))
    	{
    		case HttpRequest::GET :
    			m_stream << "GET " << "/" << m_httpUrl.getHttpUrlSubSeg(HttpUrl::URI) << " HTTP/1.1
    ";
    			LOG_DEBUG << m_stream.str().c_str();
    			break;
    		case HttpRequest::POST :
    			m_stream << "POST "  << "/" << m_httpUrl.getHttpUrlSubSeg(HttpUrl::URI) << " HTTP/1.1
    ";
    			LOG_DEBUG << m_stream.str().c_str();
    			break;
    		default :
    			LOG_ERROR << "No such Method : " << method.c_str();
    			break;
    	}
    
    	m_stream << "Host: " << m_httpUrl.getHttpUrlSubSeg(HttpUrl::HOST) << "
    ";
    }
    
    
    void HttpRequest::setRequestProperty(const std::string &key, const std::string &value)
    {
    	m_stream << key << ": " << value << "
    ";
    }
    
    void HttpRequest::setRequestBody(const std::string &content)
    {
    	m_stream << content;
    }
    
    void HttpRequest::handleRead()
    {
    	assert(!m_haveHandleHead);
    	ssize_t nread = 0;
    	ssize_t writtenBytes = 0;
    
    	nread = sockets::read(m_sockfd, m_buffer.beginWrite(), kBufferSize);
    	if(nread < 0) LOG_SYSFATAL << "sockets::read";
    	m_buffer.hasWritten(nread);
    	LOG_TRACE << "sockets::read(): nread: " << nread << " remain: " << m_buffer.writableBytes();
    	size_t remain = kBufferSize - nread;
    	while(remain > 0)
    	{
    		size_t n = sockets::read(m_sockfd, m_buffer.beginWrite(), remain);
    		if(n < 0) LOG_SYSFATAL << "sockets::read";
    		m_buffer.hasWritten(n);
    		if(0 == n)
    		{
    			LOG_DEBUG << "sockets::read finish";
    			break;
    		}
    		remain = remain - n;
    	}
    	//std::cout << m_buffer.peek();
    
    	//for(int i = 0; i < nread; i++) printf("%02x%c",(unsigned char)buffer[i],i==nread - 1 ?'
    ':' ');
    	//LOG_DEBUG << "handleRead Recv Response : 
    " << m_buffer.peek();
    	int headsize = 0;
    	std::string line;
    	std::stringstream ss(m_buffer.peek());
    	std::vector<std::string> v;
    	getline(ss, line);
    	//LOG_DEBUG << line;
    	headsize += line.size() + 1;
    	SplitString(line, v, " ");
    	//for(int i = 0; i < v.size(); i++) std::cout << v[i] << std::endl;
    	m_code = std::stoi(v[1]);
    	if(v[1] != "200")
    	{
    	  LOG_ERROR << "Error Http Server Response : " << v[1].c_str();
    	}
    
    	do{
    		getline(ss, line);
    		headsize += line.size() + 1;  // + 1('
    ')
    		if(!line.empty()) line.erase(line.end()-1); // remove '/r'
    		//LOG_DEBUG << line;
    		v.clear();
    		SplitString(line, v, ":");
    		if(v.size() == 2){
    			m_ackProperty[v[0]] = v[1].erase(0,v[1].find_first_not_of(" "));
    		}
    	}while(!line.empty());
    
    	LOG_DEBUG << "Http Head Size is " << headsize;
    	std::string res(m_buffer.peek(), headsize);
    	LOG_DEBUG << "Http Response :
    " << res;
    	m_buffer.retrieve(headsize);
    
    	m_haveHandleHead = true;
    
    }
    
    void HttpRequest::uploadFile(const std::string& file, const std::string& contentEnd)
    {
    
    	FILE* fp = fopen(file.c_str(), "rb");
    	if(fp == NULL)
    	{
    		LOG_SYSFATAL << "fopen() File :" << file.c_str() << " Errno";
    	}
    
    	bool isEnd = false;
    	ssize_t writtenBytes = 0;
    
    	assert(m_buffer.writableBytes() == Buffer::kInitialSize);
    
    	while(!isEnd)
    	{
    		ssize_t nread = fread(m_buffer.beginWrite(), 1, kBufferSize, fp);
    		m_buffer.hasWritten(nread);
    		while(m_buffer.writableBytes() > 0)
    		{
    			LOG_TRACE << "file read(): nread: " << nread << " remain: " << m_buffer.writableBytes();
    			size_t n = fread(m_buffer.beginWrite(), 1, m_buffer.writableBytes(), fp);
    			m_buffer.hasWritten(n);
    			if(0 == n)
    			{	int err = ferror(fp);
    				if(err)
    				{
    					fprintf(stderr, "fread failed : %s
    ", strerror(err));
    				}
    				LOG_DEBUG << "sockets::read finish";
    				isEnd = true;
    				break;
    			}
    		}
    
    		ssize_t nwrite = sockets::write(m_sockfd, m_buffer.peek(), m_buffer.readableBytes());
    		if(nwrite < 0) LOG_SYSFATAL << "sockets::write";
    		writtenBytes += nwrite;
    		LOG_TRACE << "sockets::write nread " << m_buffer.readableBytes() << " nwrite " << nwrite << " writtenBytes " << writtenBytes;
    		m_buffer.retrieve(nwrite);
    	}
    
    	fclose(fp);
    
    	m_buffer.retrieveAll();
    
    	ssize_t n = sockets::write(m_sockfd, contentEnd.c_str(), contentEnd.size());
    	if(n < 0) LOG_SYSFATAL << "sockets::write";
    }
    
    void HttpRequest::downloadFile(const std::string& file)
    {
    	assert(m_haveHandleHead);
    
    	bool isEnd = false;
    	ssize_t nread = 0;
    	ssize_t writtenBytes = 0;
    	bool haveHandleHead = false;
    	bool isDownFile = false;
    
    	std::ofstream output(file, std::ios::binary);
    	if (!output.is_open()){ // 检查文件是否成功打开
    		LOG_SYSFATAL << "open file error" << file;
    	}
    
    	output.write(m_buffer.peek(), m_buffer.readableBytes());
    	writtenBytes += m_buffer.readableBytes();
    	m_buffer.retrieve(m_buffer.readableBytes());
    
    	LOG_DEBUG << "Content-Length : " << getResponseProperty("Content-Length");
    
    	while(!isEnd)
    	{
    		nread = sockets::read(m_sockfd, m_buffer.beginWrite(), kBufferSize);
    		if(nread < 0) LOG_SYSFATAL << "sockets::read";
    		m_buffer.hasWritten(nread);
    		LOG_TRACE << "sockets::read(): nread: " << nread << " remain: " << m_buffer.writableBytes() << " writtenBytes: " << writtenBytes;
    		size_t remain = kBufferSize - nread;
    		while(remain > 0)
    		{
    			size_t n = sockets::read(m_sockfd, m_buffer.beginWrite(), remain);
    			if(n < 0) LOG_SYSFATAL << "sockets::read";
    			m_buffer.hasWritten(n);
    			if(0 == n)
    			{
    				LOG_DEBUG << "sockets::read finish";
    				isEnd = true;
    				break;
    			}
    			remain = remain - n;
    		}
    
    		output.write(m_buffer.peek(), m_buffer.readableBytes());
    		writtenBytes += m_buffer.readableBytes();
    		m_buffer.retrieve(m_buffer.readableBytes());
    	}
    	LOG_DEBUG << " writtenBytes " << writtenBytes;
    
    	output.close();
    	sockets::close(m_sockfd);
    }
    
    void HttpRequest::SplitString(const std::string& s, std::vector<std::string>& v, const std::string& c)
    {
      std::string::size_type pos1, pos2;
      pos2 = s.find(c);
      pos1 = 0;
      while(std::string::npos != pos2)
      {
        v.push_back(s.substr(pos1, pos2-pos1));
    
        pos1 = pos2 + c.size();
        pos2 = s.find(c, pos1);
      }
      if(pos1 != s.length())
        v.push_back(s.substr(pos1));
    }
    
    

    关于源码中的Logger

    Logger部分使用的前面文章的Logger类,可通过编辑器批量更改,注释掉,或复制第一篇异步日志的代码直接使用.

    使用示例

    下载图片

    #include "HttpRequest.hh"
    
    int main()
    {
    
      HttpRequest ImageReq("http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg");
      ImageReq.setRequestMethod("GET");
      ImageReq.setRequestProperty("Cache-Control", "no-cache");
      ImageReq.setRequestProperty("Content-Type", "application/octet-stream");
      ImageReq.setRequestProperty("Connection", "close
    ");
    
      ImageReq.connect();
      ImageReq.send();
      ImageReq.handleRead();
      ImageReq.downloadFile("./test.jpg");
    
      return 0;
    }
    
    
    2018-10-23 20:24:07.257167 [DEBUG] [HttpRequest.hh:194] [detachHttpUrl] detachHttpUrl() url :http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
    2018-10-23 20:24:07.257411 [DEBUG] [HttpRequest.hh:195] [detachHttpUrl] detachHttpUrl() host :img.zcool.cn
    2018-10-23 20:24:07.257561 [DEBUG] [HttpRequest.hh:196] [detachHttpUrl] detachHttpUrl() uri :community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
    2018-10-23 20:24:07.257661 [DEBUG] [HttpRequest.hh:171] [HttpUrl] URL : http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
    2018-10-23 20:24:07.257734 [DEBUG] [HttpRequest.cpp:178] [setRequestMethod] GET /community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg HTTP/1.1
    
    2018-10-23 20:24:08.192206 [DEBUG] [HttpRequest.cpp:142] [connect] HttpRequest::Connector() gethostbyname Successful
    2018-10-23 20:24:08.232474 [DEBUG] [HttpRequest.cpp:149] [connect] sockfd : 3sockets::connect ret : 0
    2018-10-23 20:24:08.233236 [INFO ] [HttpRequest.cpp:158] HttpRequest::connect() sockfd : 3 Successful
    2018-10-23 20:24:08.233453 [DEBUG] [HttpRequest.cpp:169] [connect] HttpRequest::Connector() end
    2018-10-23 20:24:08.257666 [TRACE] [HttpRequest.cpp:212] [handleRead] sockets::read(): nread: 4096 remain: 0
    2018-10-23 20:24:08.258490 [DEBUG] [HttpRequest.cpp:257] [handleRead] Http Head Size is 798
    2018-10-23 20:24:08.259139 [DEBUG] [HttpRequest.cpp:259] [handleRead] Http Response :
    HTTP/1.1 200 OK
    Server: Tengine
    Content-Type: image/jpeg
    Content-Length: 155984
    Connection: close
    Date: Tue, 23 Oct 2018 10:05:56 GMT
    x-oss-request-id: 5BCEF284EA872628A4BA7B32
    ETag: "3D2EABCF4B0809B38A154C5087E5ECD4"
    Last-Modified: Fri, 18 Mar 2016 03:09:12 GMT
    x-oss-object-type: Normal
    x-oss-hash-crc64ecma: 17069884749112719525
    x-oss-storage-class: Standard
    Expires: Thu, 18 Mar 2021 03:09:12 GMT
    Cache-Control: max-age=86400
    Via: cache10.l2cm9[0,304-0,H], cache35.l2cm9[37,0], cache1.cn683[0,200-0,H], cache1.cn683[2,0]
    Age: 8479
    Ali-Swift-Global-Savetime: 1538030855
    X-Cache: HIT TCP_HIT dirn:6:15817583
    X-Swift-SaveTime: Tue, 23 Oct 2018 10:18:38 GMT
    X-Swift-CacheTime: 85638
    Access-Control-Allow-Origin: *
    Timing-Allow-Origin: *
    EagleId: 71db884115402976359511426e
    
    
    2018-10-23 20:24:08.263882 [DEBUG] [HttpRequest.cpp:335] [downloadFile] Content-Length : 155984
    2018-10-23 20:24:08.264447 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 3298
    2018-10-23 20:24:08.264569 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 7394
    2018-10-23 20:24:08.264978 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 11490
    2018-10-23 20:24:08.265032 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 15586
    2018-10-23 20:24:08.265073 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 19682
    2018-10-23 20:24:08.265112 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 23778
    2018-10-23 20:24:08.265151 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 288 remain: 3808 writtenBytes: 27874
    2018-10-23 20:24:08.277661 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3432 remain: 664 writtenBytes: 31970
    2018-10-23 20:24:08.278022 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 784 remain: 3312 writtenBytes: 36066
    2018-10-23 20:24:08.278264 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2480 remain: 1616 writtenBytes: 40162
    2018-10-23 20:24:08.278580 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1280 remain: 2816 writtenBytes: 44258
    2018-10-23 20:24:08.278913 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1528 remain: 2568 writtenBytes: 48354
    2018-10-23 20:24:08.279254 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1776 remain: 2320 writtenBytes: 52450
    2018-10-23 20:24:08.279402 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 576 remain: 3520 writtenBytes: 56546
    2018-10-23 20:24:08.279920 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 824 remain: 3272 writtenBytes: 60642
    2018-10-23 20:24:08.280905 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 64738
    2018-10-23 20:24:08.280948 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2768 remain: 1328 writtenBytes: 68834
    2018-10-23 20:24:08.281217 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3016 remain: 1080 writtenBytes: 72930
    2018-10-23 20:24:08.281506 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3264 remain: 832 writtenBytes: 77026
    2018-10-23 20:24:08.281948 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2064 remain: 2032 writtenBytes: 81122
    2018-10-23 20:24:08.297685 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3760 remain: 336 writtenBytes: 85218
    2018-10-23 20:24:08.298170 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4008 remain: 88 writtenBytes: 89314
    2018-10-23 20:24:08.298362 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2808 remain: 1288 writtenBytes: 93410
    2018-10-23 20:24:08.298919 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3056 remain: 1040 writtenBytes: 97506
    2018-10-23 20:24:08.299576 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 101602
    2018-10-23 20:24:08.299641 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 656 remain: 3440 writtenBytes: 105698
    2018-10-23 20:24:08.300041 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2352 remain: 1744 writtenBytes: 109794
    2018-10-23 20:24:08.300479 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2600 remain: 1496 writtenBytes: 113890
    2018-10-23 20:24:08.301340 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2848 remain: 1248 writtenBytes: 117986
    2018-10-23 20:24:08.303384 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 122082
    2018-10-23 20:24:08.303646 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1920 remain: 2176 writtenBytes: 126178
    2018-10-23 20:24:08.303926 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 130274
    2018-10-23 20:24:08.304004 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 134370
    2018-10-23 20:24:08.304376 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 138466
    2018-10-23 20:24:08.304540 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 142562
    2018-10-23 20:24:08.304737 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 768 remain: 3328 writtenBytes: 146658
    2018-10-23 20:24:08.318544 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2464 remain: 1632 writtenBytes: 150754
    2018-10-23 20:24:08.320273 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1134 remain: 2962 writtenBytes: 154850
    2018-10-23 20:24:08.320374 [DEBUG] [HttpRequest.cpp:351] [downloadFile] sockets::read finish
    2018-10-23 20:24:08.320460 [DEBUG] [HttpRequest.cpp:362] [downloadFile]  writtenBytes 155984
    
    


  • 相关阅读:
    Pyinstaller(python打包为exe文件)
    matplotlib 填充颜色
    Visual Studio 2015 Enterprise
    latex中pdflatex与xelatex的区别
    latex插图续
    dva+umi+antd项目从搭建到使用(没有剖验证,不知道在说i什么)
    umi+dva+antd新建项目(亲测可用)
    HTTP缓存机制
    企业网站常见需求整理
    立足于运维与监控的前端框架 NoahV
  • 原文地址:https://www.cnblogs.com/ailumiyana/p/9839170.html
Copyright © 2020-2023  润新知