为什么要为共享锁抛出"multiprocessing.managers.RemoteError"?

人气:291 发布:2022-10-16 标签: python-3.x python-multiprocessing shared-objects locks

问题描述

我使用的是带有Ubuntu 18.04的Python 3.6.7

运行以下脚本后,其中每个进程都有自己的共享锁:

from multiprocessing import Process, Manager


def foo(l1):
    with l1:
        print('lol')


if __name__ == '__main__':
    processes = []
    with Manager() as manager:
        for cluster in range(10):
            lock1 = manager.Lock()
            calc_args = (lock1, )
            processes.append(Process(target=foo,
                                     args=calc_args))
        for p in processes:
            p.start()

        for p in processes:
            p.join()

我有奇怪的异常:

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "temp.py", line 5, in foo
    with l1:
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 991, in __enter__
    return self._callmethod('acquire')
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 235, in serve_client
    self.id_to_local_proxy_obj[ident]
KeyError: '7f49974624e0'

有什么想法或建议如何解决此问题吗?

谢谢

推荐答案

出于某种原因,您必须保留原始对您从SyncManager获得的任何内容的引用,请参见以下内容:

from multiprocessing import Manager, Process, current_process
from multiprocessing.managers import AcquirerProxy, SyncManager

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager: SyncManager = Manager()

    locks = []
    for i in range(3):
        # Always keep the reference in some variable
        locks.append(manager.Lock())

    processes = []
    for i in range(3):
        p = Process(target=foo, args=[locks[i]])
        processes.append(p)

    # If you clear the list, which you lose the reference, it won't work
    # locks.clear()

    for p in processes:
        p.start()
    for p in processes:
        p.join()

很抱歉聚会迟到了,但我希望这会有帮助!

-原始响应如下:-

嘿,这个问题在Python3.8.2中仍然存在

我设法重现了错误:

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    process1 = Process(target=foo, args=[manager.Lock()])

    process1.start()
    process1.join()

但如果我取出manager.Lock(),它就能正常工作!

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    lock1 = manager.Lock()           # Here
    process1 = Process(target=foo, args=[lock1])

    process1.start()
    process1.join()

我真搞不懂为什么把锁拿掉会有什么不同

393