• 多进程


    1、启动一个新的进程

    其实启动一个新的进程和启动一个线程的格式是一样的,只不过启动进程是需要从multiprocessing中导入Process(注意此处的Process首字母是大写的)

    格式如下:

    from multiprocessing import Process
    
    def f(name):
        print('The f is running !')
    
    if __name__ == '__main__':
        p = Process(target=f,args=('balabala',))
        p.start()

    2、进程间的通信

    由于进程是资源的集合,所以进程之间是不能相互访问的,是独立的。

    2.1 Queue 其实用法和线程中的队列是一样的,需要从multiprocessing 导入Queue

    from multiprocessing import Queue,Process
    import os
    
    def f(t):
        t.put(['孙悟空','美猴王',500])
        print('The current PID f:',os.getppid(),os.getpid())
    
    if __name__ == '__main__':
        q = Queue()
        print('The current PID:', os.getpid())
        p = Process(target=f,args=(q,))
        p.start()
        print('The data is :',q.get())

    运行结果为:

    The current PID: 8748
    The current PID f: 8748 15712
    The data is : ['孙悟空', '美猴王', 500]
    可以看出,这是两个进程,进程之间的通信和线程中的队列是一个道理,一个进程把数据放到队列中,另一个程序取队列里边取数据,相当于通过中间的池子作为联系的纽带!!

    2.2通过Pipes管道通讯

    from multiprocessing import Pipe,Process
    
    def f(t):
        t.send(['孙悟空','美猴王',1000])
        t.close()
    
    if __name__ == '__main__':
        one_hand,other_hand = Pipe()
        p = Process(target=f,args=(other_hand,))
        p.start()
        print('The one_hand receive data:',one_hand.recv())
        p.join()

    这个模型等价于生成一个管子,两个进程一端一个,一段发送,另一端接受,至于哪个程序在那一段无所谓,都可以!!当然子线程发送给,父进程接受一样也是可以的!!

    2.3Manager代理者

    from multiprocessing import Manager,Process
    import os
    def f(f_dict,f_list):
        f_dict[1] = 'a'
        f_dict[2] = 'b'
        f_dict[3] = 'c'
        f_dict[os.getpid()] = os.getpid()
        print('f_list:',f_list)
        print('f_dict:',f_dict)
    
    if __name__ == '__main__':
        manager = Manager()
        m_dict = manager.dict()
        m_list = manager.list(range(5))
        p_list = []
        for i in range(10):
            p = Process(target=f,args=(m_dict,m_list))
            p.start()
            p_list.append(p)
        for p in p_list:
            p.join()

    其实Manager和Queue效果是一样的,只不过Queue只是生成一种队列,也就是一种类型的池子,而Manager可以生成多种不同类型的池子,可以在池子里放的类型更多了。道理是一样的!!

    3、进程同步

    from multiprocessing import Process,Lock,Pipe
    import os,time
    def f(lock,an,n):
        lock.acquire()
        print('The recv data {n}:'.format(n = n),an.recv())
        lock.release()
    if __name__ == '__main__':
        lock = Lock()
        one, another = Pipe()
    
        for i in range(10):
            one.send('伟大的祖国!!')
            p = Process(target=f,args=(lock,another,i))
            p.start()

    虽然不同进程之间的数据是相互独立的,但是比如不同进程在想屏幕上输出数据是,他们是公用的一块屏幕啊,这就有可能第一个进程在向屏幕输出数据还没输出完毕时,第二个进程就开始向屏幕输出数据了,这时就有可能是进程在向屏幕输出数据是出现乱了。

    4、进程池

    进程池需要导入multiprocessing中的Pool

    进程池有两种方法一种是apply也就是串行,另一种是apply_async,这种是并行,例如运行100个进程,但是pool的数据为5,那意思就是5个进程并行运行,运行结束后和下边的5个并行进程串行运行。

    from multiprocessing import Process,Pool
    import os,time
    def f(a):
        time.sleep(2)
        print('The current data:',a)
        return a + 100
    
    def back(ar):
        print('The main func has runned,the back func is running!!',ar)
    
    if __name__ == '__main__':
        pool = Pool(3)
        for i in range(10):
            # pool.apply(func = f,args = (i,))
            pool.apply_async(func=f, args=(i,), callback=back)  //callback回调函数的参数为func调用函数的返回值,如果没有返回值,则程序出错
    
        print('End!!')
        pool.close()   //这里要注意pool先关闭,然后再等待进程运行,否则进程直接关闭。
        pool.join()

    运行结果如下:

    End!!
    The current data: 0
    The main func has runned,the back func is running!! 100         //callback函数的参数为func函数的返回值
    The current data: 1
    The current data: 2
    The main func has runned,the back func is running!! 101
    The main func has runned,the back func is running!! 102
    The current data: 3
    The main func has runned,the back func is running!! 103
    The current data: 5
    The current data: 4
    The main func has runned,the back func is running!! 105
    The main func has runned,the back func is running!! 104
    The current data: 6
    The main func has runned,the back func is running!! 106
    The current data: 7
    The current data: 8
    The main func has runned,the back func is running!! 108
    The main func has runned,the back func is running!! 107
    The current data: 9
    The main func has runned,the back func is running!! 109

    Process finished with exit code 0

  • 相关阅读:
    Java内存模型(JMM)
    线程安全问题的本质详解: 原子性、有序性、可见性
    Quartz实现分布式可动态配置的定时任务
    Java引用详解-StrongReference SoftReference WeakReference PhantomReference
    流行的报表生成工具-JXLS
    Java线程监控及中断
    IntelliJ IDEA 内存优化最佳实践
    Dapeng框架-开源高性能分布式微服务框架
    Scala实现Try with resources自动关闭IO
    Jvm启动,关闭及对应钩子
  • 原文地址:https://www.cnblogs.com/zhouzhe-blog/p/9435548.html
Copyright © 2020-2023  润新知