代码之家 › 专栏 › 技术社区 › Matt

如何在multiprocess.dummy池中单独命名线程?

multiprocessing multithreading python

Matt · 技术社区 · 7 年前

multiprocessing.dummy.Pool 这样我打电话时就能看到他们的名字 threading.enumerate() 从主线程开始。当我打电话时,有没有一个关键字可以应用 pool.apply_async 能够命名线程吗?我更愿意在创造时命名它们,而不是从内部命名 tester 功能,只是为了清洁。

例如,如果我有下面的示例代码:

import multiprocessing.dummy
from time import sleep
import threading

def tester():
    sleep(2)
    print("running \n")

def run_conc(number_of_threads, fxn):
    pool = multiprocessing.dummy.Pool(processes=number_of_threads)
    for thread in range(number_of_threads):
        pool.apply_async(tester)
    print(threading.enumerate(), "\n")
    pool.close()
    pool.join()

run_conc(3, tester)

当我运行它时,我收到输出:

[<_MainThread(MainThread, started 140735632434048)>, <Thread(SockThread, started daemon 123145521917952)>, <DummyProcess(Thread-1, started daemon 123145527246848)>, <DummyProcess(Thread-2, started daemon 123145532502016)>, <DummyProcess(Thread-3, started daemon 123145537757184)>, <Thread(Thread-4, started daemon 123145543012352)>, <Thread(Thread-5, started daemon 123145548267520)>, <Thread(Thread-6, started daemon 123145553522688)>] 

running 
running 
running

我希望能够命名线程列表中的3个虚拟线程,以便识别哪个是哪个。或许有一种方法可以从内部做到这一点 concurrent.futures

3 回复 | 直到 7 年前

Darkonaut 7 年前

池中的工作线程是在您呼叫时创建 pool.apply_async

在初始化时,如果不操纵源,就无法命名线程。你的选择是:

重命名工作线程之后池已准备好实例化

第一个选项很容易实现,只需在 ._pool -属性,并更改 .name

from multiprocessing.pool import ThreadPool as Pool


if __name__ == '__main__':

    pool = Pool(4)
    print([w.name for w in pool._pool])
    # ['Thread-1', 'Thread-2', 'Thread-3', 'Thread-4']

    for w in pool._pool:
        w.name = w.name.replace('Thread', 'ThreadPoolWorker')

    print([w.name for w in pool._pool])
    # ['ThreadPoolWorker-1', 'ThreadPoolWorker-2', 'ThreadPoolWorker-3', 'ThreadPoolWorker-4']

    pool.close()
    pool.join()

注意,我正在使用 multiprocessing.pool.ThreadPool 在这里,只需遵循下面第二个选项中的示例 multiprocessing.dummy.Pool 只是一个包装 ThreadPool .

对于第二个选项,可以为辅助线程修补工厂函数 ThreadPool.Process 使用包装器,将默认名称“Thread-%d”(%d由计数器值填充)扩展为更有意义的名称,如“ThreadPoolWorker”。

# threadpool.py
# Module patching the name of worker-threads within ThreadPool

__all__ = ['ThreadPool']

from functools import wraps
from multiprocessing.pool import ThreadPool


def rename_worker(fn):
    @wraps(fn)
    def wrapper(*args, **kwargs):
        w = fn(*args, **kwargs)
        w.name = w.name.replace('Thread', 'ThreadPoolWorker')
        return w
    return wrapper


ThreadPool.Process = staticmethod(rename_worker(ThreadPool.Process))

用法:

from threadpool import ThreadPool as Pool


if __name__ == '__main__':

    pool = Pool(4)
    print([w.name for w in pool._pool])
    # ['ThreadPoolWorker-1', 'ThreadPoolWorker-2', 'ThreadPoolWorker-3', 'ThreadPoolWorker-4']
    pool.close()
    pool.join()

Matt 7 年前

好的,我发现我可以在 tester threading.current_thread().name . 但是,如果有人知道如何在从 pool.apply_async

Jayendra Parmar 7 年前

如果您可以编辑线程名称,那么下面的代码片段将有所帮助

from multiprocessing import Pool,Queue
import threading

thread_names = Queue()
num_process = 4
for e in ['A','B','C','D']:
    thread_names.put('Thread-{}'.format(e))

def initializer(q):
    thread_name = q.get()
    threading.current_thread().name = thread_name

if __name__ == '__main__':
    pool = Pool(num_process=4,initializer=initializer,initargs=(threadnames,))