• Python的Multiprocessing多进程实例


    最近在拜读RBG大神的faster-rcnn源码时发现他用了多进程去分阶段处理神经网络,原因如下:

        # --------------------------------------------------------------------------
        # Pycaffe doesn't reliably free GPU memory when instantiated nets are
        # discarded (e.g. "del net" in Python code). To work around this issue, each
        # training stage is executed in a separate process using
        # multiprocessing.Process.
        # --------------------------------------------------------------------------

    大致意思是pycaffe在安装网络后当不再需要使用该部分网络时,不能靠谱的释放GPU显存资源。为解决这个问题,每一个训练阶段都做一个独立的进程去执行,也就用到了多进程。

    python的多进程示例如下:

    import multiprocessing as mp
    
    def function_name():
        do sth.
        queue.put({'xxx': yyy})
    
    mp_queue = mp.Queue()
    
    p = mp.Process(target=function_name, kwargs=dict_ur_mp_kwargs)
    p.start()
    output = mp_queue.get()
    p.join()
    
    #其他进程
    p = mp.Process(target=function_name, kwargs=dict_ur_mp_kwargs)
    p.start()
    output = mp_queue.get()
    p.join()
    #.......
  • 相关阅读:
    Tinkoff Challenge
    Tinkoff Challenge
    Tinkoff Challenge
    Tinkoff Challenge
    整体二分
    树链剖分+LCT
    上下界网络流
    莫队
    可并堆
    bzoj_1033: [ZJOI2008]杀蚂蚁antbuster
  • 原文地址:https://www.cnblogs.com/ocean1100/p/7754091.html
Copyright © 2020-2023  润新知