-
C#网页数据采集(三)HttpWebRequest
- <span style="font-family: Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255);">截取到网页数据是js加载完以后的</span>
- <span style="white-space:pre"> </span> HtmlWeb webClient = new HtmlWeb();
- string _url = "http://news.baidu.com/";
- HtmlAgilityPack.HtmlDocument html1 = webClient.Load(_url);
- var end3 = html1.Encoding.BodyName;
- string _htmlSource = GetHtmlSource(_url, System.Text.Encoding.GetEncoding(end3));
- public static string GetHtmlSource(string url, Encoding charset)
- {
- string _html = string.Empty;
- try
- {
- HttpWebRequest _request = (HttpWebRequest)WebRequest.Create(url);
- HttpWebResponse _response = (HttpWebResponse)_request.GetResponse();
- using (Stream _stream = _response.GetResponseStream())
- {
- using (StreamReader _reader = new StreamReader(_stream, charset))
- {
- _html = _reader.ReadToEnd();
- }
- }
- }
- catch (WebException ex)
- {
- using (StreamReader sr = new StreamReader(ex.Response.GetResponseStream()))
- {
- _html = sr.ReadToEnd();
- }
- }
- catch (Exception ex)
- {
- _html = ex.Message;
- }
- return _html;
- }
-
相关阅读:
Maven配置--《maven实战》读书笔记
设置定时任务
C#中的==
C# lock的应用
JDK和JRE
末尾不以.OK文件结尾的正则表达式匹配
ftp访问空目录的返回
正则表达式的结尾匹配
匿名对象和匿名类
匿名内部类的调用
-
原文地址:https://www.cnblogs.com/telwanggs/p/6477670.html
Copyright © 2020-2023
润新知