并发编程：异步+回调机制

一：进程

1、导入模块

2.定义访问下载函数

3.解析下载的内容

4.在进程池里放任务，并把任务提交给get函数

完整代码：

 1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor
 2 import requests
 3 import os
 4 import time
 5 import random
 6 
 7 def get(url):
 8     print('%s GET %s' %(os.getpid(),url))   
 9     response=requests.get(url)               
10     if response.status_code == 200:          
11         return response.text                
12 
13 def pasrse(obj):     
14     res=obj.result()  
15     print('%s 解析结果为：%s' %(os.getpid(),len(res)))
16 
17 if __name__ == '__main__':
18     urls=[
19         'https://www.baidu.com',
20         'https://www.baidu.com',
21         'https://www.baidu.com',
22         'https://www.baidu.com',
23         'https://www.baidu.com',
24         'https://www.baidu.com',
25         'https://www.baidu.com',
26         'https://www.baidu.com',
27         'https://www.python.org',
28     ]
29 
30     pool=ProcessPoolExecutor(4)     
31     for url in urls:                
32         obj=pool.submit(get,url)
33         obj.add_done_callback(pasrse)  
34     print('主进程',os.getpid())

View Code

运行结果：

主进程 5588
6976 GET https://www.baidu.com
17844 GET https://www.baidu.com
16832 GET https://www.baidu.com
8956 GET https://www.baidu.com
17844 GET https://www.baidu.com
5588 解析结果为：2443
8956 GET https://www.baidu.com
5588 解析结果为：2443
16832 GET https://www.baidu.com
5588 解析结果为：2443
6976 GET https://www.baidu.com
5588 解析结果为：2443
17844 GET https://www.python.org
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：48887

Process finished with exit code 0

二：线程

过程与进程相似，但线程没有主次之分，哪个线程闲着了就由哪个线程干回调的活

from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor
from threading import current_thread
import requests
import os
import time
import random

def get(url):
    print('%s GET %s' %(current_thread().name,url))
    response=requests.get(url)
    time.sleep(random.randint(1,3))

    if response.status_code == 200:
        return response.text

def pasrse(obj):
    res=obj.result()
    print('%s 解析结果为：%s' %(current_thread().name,len(res)))

if __name__ == '__main__':
    urls=[
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.python.org',
    ]

    pool=ThreadPoolExecutor(4)
    for url in urls:
        obj=pool.submit(get,url)
        obj.add_done_callback(pasrse)

View Code

运行结果：

ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_1 GET https://www.baidu.com
ThreadPoolExecutor-0_2 GET https://www.baidu.com
ThreadPoolExecutor-0_3 GET https://www.baidu.com
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_2 解析结果为：2443
ThreadPoolExecutor-0_2 GET https://www.baidu.com
ThreadPoolExecutor-0_1 解析结果为：2443
ThreadPoolExecutor-0_1 GET https://www.baidu.com
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_3 解析结果为：2443
ThreadPoolExecutor-0_3 GET https://www.python.org
ThreadPoolExecutor-0_2 解析结果为：2443
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_1 解析结果为：2443
ThreadPoolExecutor-0_3 解析结果为：48887

Process finished with exit code 0

相关阅读:
Java 设计模式之单例
 CentOS 下安装配置Hadoop
Linux 修改IP地址和DNS
Hibernate 主配置文件详解
 HTTP 协议细节
 Java 设计模式之工厂
 mysqldump备份与还原
 [小技巧]windows时间同步
 关于反射
 XHTML、DTD和Web Services
原文地址：https://www.cnblogs.com/zh-xiaoyuan/p/11773867.html