C语言基础学习PYTHON——基础学习D09
20180903内容纲要:
线程、进程
1、paramiko
2、线程、进程初识
3、多线程
(1)线程的调用方式
(2)join
(3)线程锁、递归锁、信号量
(4)Timer
(5)Event
(6)Queue队列
4、多进程
(1)多进程
(2)获取进程PID
(3)进程锁
(4)进程间的通讯
(5)进程池
5、小结
6、练习:简单主机批量管理工具
1、paramiko
paramiko模块提供了ssh及sft进行远程登录服务器执行命令和上传下载文件的功能。这是一个第三方的软件包,使用之前需要安装。
我个人觉得,这个在Windows上不太好用。在windows上python3需要安装vs2010,但我装了之后还是不行,可能是我自己的原因,在python2.7上好像可以,没试过。
这有链接:https://blog.csdn.net/songfreeman/article/details/50920767
很详细!
2 线程、进程初识
什么是线程(thread)?
线程是操作系统能够进行运算调度的最小单位。它被包含在进程之中,是进程中的实际运作单位。
一条线程指的是进程中一个单一顺序的控制流,一个进程中可以并发多个线程,每条线程并行执行不同的任务
A thread is an execution context, which is all the information a CPU needs to execute a stream of instructions.
Suppose you're reading a book, and you want to take a break right now, but you want to be able to come back and resume reading from the exact point where you stopped. One way to achieve that is by jotting down the page number, line number, and word number. So your execution context for reading a book is these 3 numbers. If you have a roommate, and she's using the same technique, she can take the book while you're not using it, and resume reading from where she stopped. Then you can take it back, and resume it from where you were. Threads work in the same way. A CPU is giving you the illusion that it's doing multiple computations at the same time. It does that by spending a bit of time on each computation. It can do that because it has an execution context for each computation. Just like you can share a book with your friend, many tasks can share a CPU. On a more technical level, an execution context (therefore a thread) consists of the values of the CPU's registers. Last: threads are different from processes. A thread is a context of execution, while a process is a bunch of resources associated with a computation. A process can have one or many threads. Clarification: the resources associated with a process include memory pages (all the threads in a process have the same view of the memory), file descriptors (e.g., open sockets), and security credentials (e.g., the ID of the user who started the process).
什么是进程(process)?
程序并不能单独运行,只有将程序装载到内存中,系统为它分配资源才能运行,而这种执行的程序就称之为进程。
程序和进程的区别就在于:程序是指令的集合,它是进程运行的静态描述文本;进程是程序的一次执行活动,属于动态概念。
An executing instance of a program is called a process.
Each process provides the resources needed to execute a program. A process has a virtual address space, executable code, open handles to system objects, a security context, a unique process identifier, environment variables, a priority class, minimum and maximum working set sizes, and at least one thread of execution. Each process is started with a single thread, often called the primary thread, but can create additional threads from any of its threads.
那么他们的区别呢?
进程与线程的区别?
线程:是操作系统最小的调度单位。 是一串指令的集合。
进程:要操作cpu必须先创建一个线程
所有在同一个进程里的线程是共享一块内存空间
1 1、Threads share the address space of the process that created it; processes have their own address space. 2 1、线程共享内存空间,进程的内存是独立的 3 4 2、Threads have direct access to the data segment of its process; processes have their own copy of the data segment of the parent process. 5 6 3、Threads can directly communicate with other threads of its process; processes must use interprocess communication to communicate with sibling processes. 7 3、两个进程之间可以直接交流,两个进程进行通讯,必须通过一个中间代理来实现。 8 9 4、New threads are easily created; new processes require duplication of the parent process. 10 4、创建新线程很简单。创建新进程需要对其父进程进行一次克隆 11 12 5、Threads can exercise considerable control over threads of the same process; processes can only exercise control over child processes. 13 5、一个线程可以控制和操作同意进程里的其他线程,但是进程只能操作子进程 14 15 6、Changes to the main thread (cancellation, priority change, etc.) may affect the behavior of the other threads of the process; changes to the parent process does not affect child processes. 16
3 多线程
(1)线程的调用方式
直接调用:
1 #Author:ZhangKanghui 2 3 import threading 4 import time 5 6 def run(n): 7 print("task",n,threading.current_thread()) 8 time.sleep(2) 9 '''#这里是只有两个线程,那么更多线程呢? 10 t1 = threading.Thread(target=run,args=("t1",)) 11 t2 = threading.Thread(target=run,args=("t2",)) 12 t1.start() 13 t2.start() 14 ''' 15 #这是一个多线程,那么我们来看一下他整个程序的执行时间为什么不是2s 16 start_time = time.time() 17 for i in range(50): 18 t = threading.Thread(target=run,args=("t-%s" %i,)) #这个地方args必须有,。。因为这是默认一个元组 19 t.start() 20 #t.join() #这样可以把并行变成串行 21 22 print("mian threading has finished....",threading.current_thread(),threading.active_count()) 23 print("cost:",time.time()-start_time) 24 #这是因为主线程和子线程没关系。主线程不会等子线程执行完毕才计算时间 25 #那接下来如果想要等所有的线程结束然后打印程序执行时间可以这么干 26 ''' 27 28 import threading 29 import time 30 31 def run(n): 32 print("task",n) 33 time.sleep(2) 34 35 36 start_time = time.time() 37 t_obj =[] #存线程实例 38 for i in range(50): 39 t = threading.Thread(target=run,args=("t-%s" %i,)) #这个地方args必须有,。。因为这是默认一个元组 40 #t.setDaemon(True) #把当前线程设置成守护线程,必须在启动strat之前 41 t.start() 42 t_obj.append(t) #为了不阻塞后面线程的启动,不在这里加join,先放到一个列表里 43 for i in t_obj: #循环线程实例列表,等待所有线程执行完毕 44 t.join() 45 46 print("cost:",time.time()-start_time) 47 48 '''
继承式调用
1 #Author:ZhangKanghui 2 3 import threading 4 import time 5 6 class MyThread(threading.Thread): 7 def __init__(self,n,sleep_time): 8 super(MyThread,self).__init__() 9 self.n = n 10 self.sleep_time = sleep_time 11 12 def run(self): 13 print("running task",self.n) 14 time.sleep(self.sleep_time) 15 print("task done",self.n) 16 17 t1 =MyThread("t1",2) 18 t2 =MyThread("t2",4) 19 20 t1.start() 21 t2.start() 22 23 #t1.join() 24 #t2.join() 25 26 print("mian threading....")
(2)join&Daemon
其实join就是wait。Daemon就是守护线程。
Some threads do background tasks, like sending keepalive packets, or performing periodic garbage collection, or whatever. These are only useful when the main program is running, and it's okay to kill them off once the other, non-daemon, threads have exited.
Without daemon threads, you'd have to keep track of them, and tell them to exit, before your program can completely quit. By setting them as daemon threads, you can let them run and forget about them, and when your program quits, any daemon threads are killed automatically.
1 #Author:ZhangKanghui 2 import time 3 import threading 4 5 6 def run(n): 7 print('[%s]------running---- ' % n) 8 time.sleep(2) 9 print('--done--') 10 11 12 def main(): 13 for i in range(5): 14 t = threading.Thread(target=run, args=[i, ]) 15 t.start() 16 t.join(1) 17 print('starting thread', t.getName()) 18 19 20 m = threading.Thread(target=main, args=[]) 21 m.setDaemon(True) # 将main线程设置为Daemon线程,它做为程序主线程的守护线程,当主线程退出时,m线程也会退出,由m启动的其它子线程会同时退出,不管是否执行完任务 22 m.start() 23 m.join() 24 # m.join(timeout=2) 25 print("---main thread done----")
1 #Author:ZhangKanghui 2 3 import threading 4 import time 5 6 class MyThread(threading.Thread): 7 def __init__(self,n,sleep_time): 8 super(MyThread,self).__init__() 9 self.n = n 10 self.sleep_time = sleep_time 11 12 def run(self): 13 print("running task",self.n) 14 time.sleep(self.sleep_time) 15 print("task done",self.n) 16 17 t1 =MyThread("t1",2) 18 t2 =MyThread("t2",4) 19 20 t1.start() 21 t2.start() 22 23 t1.join() 24 t2.join() 25 26 print("mian threading....")
1 import threading 2 import time 3 4 def run(n): 5 print("task",n) 6 time.sleep(2) 7 8 start_time = time.time() 9 t_obj =[] #存线程实例 10 for i in range(50): 11 t = threading.Thread(target=run,args=("t-%s" %i,)) #这个地方args必须有,。。因为这是默认一个元组 12 #t.setDaemon(True) #把当前线程设置成守护线程,必须在启动strat之前 13 t.start() 14 t_obj.append(t) #为了不阻塞后面线程的启动,不在这里加join,先放到一个列表里 15 for i in t_obj: #循环线程实例列表,等待所有线程执行完毕 16 t.join() 17 18 print("cost:",time.time()-start_time)
(3)线程锁/递归锁/信号量
线程锁
一个进程下可以启动多个线程,多个线程共享父进程的内存空间,也就意味着每个线程可以访问同一份数据,此时,如果2个线程同时要修改同一份数据,会出现什么状况?
1 import time 2 import threading 3 4 def addNum(): 5 global num #在每个线程中都获取这个全局变量 6 print('--get num:',num ) 7 time.sleep(1) 8 lock.acquire() #修改数据前加锁 9 num -=1 #对此公共变量进行-1操作 10 lock.release() #修改后释放 11 12 num = 100 #设定一个共享变量 13 thread_list = [] 14 lock = threading.Lock() #生成全局锁 15 for i in range(100): 16 t = threading.Thread(target=addNum) 17 t.start() 18 thread_list.append(t) 19 20 for t in thread_list: #等待所有线程执行完毕 21 t.join() 22 23 print('final num:', num )
递归锁
就是在一个大锁中还要再包含子锁
1 import threading,time 2 3 def run1(): 4 print("grab the first part data") 5 lock.acquire() 6 global num 7 num +=1 8 lock.release() 9 return num 10 def run2(): 11 print("grab the second part data") 12 lock.acquire() 13 global num2 14 num2+=1 15 lock.release() 16 return num2 17 def run3(): 18 lock.acquire() 19 res = run1() 20 print('--------between run1 and run2-----') 21 res2 = run2() 22 lock.release() 23 print(res,res2) 24 25 26 if __name__ == '__main__': 27 28 num,num2 = 0,0 29 lock = threading.RLock() 30 for i in range(10): 31 t = threading.Thread(target=run3) 32 t.start() 33 34 while threading.active_count() != 1: 35 print(threading.active_count()) 36 else: 37 print('----all threads done---') 38 print(num,num2)
信号量(Semaphore)
互斥锁 同时只允许一个线程更改数据,而Semaphore是同时允许一定数量的线程更改数据 。
比如厕所有3个坑,那最多只允许3个人上厕所,后面的人只能等里面有人出来了才能再进去。
1 import threading,time 2 3 def run(n): 4 semaphore.acquire() 5 time.sleep(1) 6 print("run the thread: %s " %n) 7 semaphore.release() 8 9 if __name__ == '__main__': 10 11 num= 0 12 semaphore = threading.BoundedSemaphore(5) #最多允许5个线程同时运行 13 for i in range(20): 14 t = threading.Thread(target=run,args=(i,)) 15 t.start() 16 17 while threading.active_count() != 1: 18 pass #print threading.active_count() 19 else: 20 print('----all threads done---') 21 print(num)
(4)Timer
This class represents an action that should be run only after a certain amount of time has passed。
1 #Author:ZhangKanghui 2 import threading 3 def hello(): 4 print("hello, world") 5 6 t = threading.Thread(target=hello) 7 t = threading.Timer(30.0, hello) 8 t.start() # after 30 seconds, "hello, world" will be printed
(5)Event
An event is a simple synchronization object;通过Event来实现两个或多个线程间的交互。
the event represents an internal flag, and threads can wait for the flag to be set, or set or clear the flag themselves.
event.wait()
event.set()
event.clear()
下面是一个红绿灯的例子,即起动一个线程做交通指挥灯,生成几个线程做车辆,车辆行驶按红灯停,绿灯行的规则。
1 import threading,time 2 import random 3 def light(): 4 if not event.isSet(): 5 event.set() #wait就不阻塞 #绿灯状态 6 count = 0 7 while True: 8 if count < 10: 9 print('