协程介绍
线程和进程的操作是由程序触发系统接口,最后的执行者是系统;协程的操作则是程序员。
协程存在的意义:对于多线程应用,CPU通过切片的方式来切换线程间的执行,线程切换时需要耗时(保存状态,下次继续)。协程,则只使用一个线程,在一个线程中规定某个代码块执行顺序。
协程的适用场景:当程序中存在大量不需要CPU的操作时(IO),适用于协程;
greenlet
greenlet封装好的协程,利用.swith对协程操作进行手动切换
#!/usr/bin/env python # coding=utf-8 from greenlet import greenlet def test1(): print(12) gr2.switch() print(34) gr2.switch() def test2(): print(56) gr1.switch() print(78) gr1 = greenlet(test1) gr2 = greenlet(test2) gr1.switch()
gevent
gevent中用到的主要是Greenlet, 它是对greenlet的又一级封装。
import gevent def foo(): print('Running in foo') gevent.sleep(0) print('Explicit context switch to foo again') def bar(): print('Explicit context to bar') gevent.sleep(0) print('Implicit context switch back to bar') gevent.joinall([ gevent.spawn(foo), gevent.spawn(bar), ])
使用协程抓取网站内容例子
#!/usr/bin/env python # coding=utf-8 from gevent import monkey; monkey.patch_all() import gevent import requests def f(url): print('GET: %s' % url) resp = requests.get(url) data = resp.text print('%d bytes received from %s.' %(len(data),url)) gevent.joinall([ gevent.spawn(f, 'https://www.baidu.com'), gevent.spawn(f, 'http://www.mingxiao.info'), gevent.spawn(f, 'https://www.github.com'), ])
输出:
GET: https://www.baidu.com
GET: http://www.mingxiao.info
GET: https://www.github.com
2443 bytes received from https://www.baidu.com.
10915 bytes received from http://www.mingxiao.info.
25319 bytes received from https://www.github.com.