python – 为什么我的多进程队列看起来不是线程安全的？
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了python – 为什么我的多进程队列看起来不是线程安全的？，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含3635字，纯文字阅读大概需要6分钟。
内容图文
我正在构建一个运行另一个Python程序的监视程序计时器,如果它无法从任何线程中找到签入,则关闭整个程序.这样,它最终将能够控制所需的通信端口.计时器的代码如下：
from multiprocessing import Process, Queue
from time import sleep
from copy import deepcopy
PATH_TO_FILE = r'.\test_program.py'
WATCHDOG_TIMEOUT = 2
class Watchdog:
def __init__(self, filepath, timeout):
self.filepath = filepath
self.timeout = timeout
self.threadIdQ = Queue()
self.knownThreads = {}
def start(self):
threadIdQ = self.threadIdQ
process = Process(target = self._executeFile)
process.start()
try:
while True:
unaccountedThreads = deepcopy(self.knownThreads)
# Empty queue since last wake. Add new thread IDs to knownThreads, and account for all known thread IDs
# in queue
while not threadIdQ.empty():
threadId = threadIdQ.get()
if threadId in self.knownThreads:
unaccountedThreads.pop(threadId, None)
else:
print('New threadId < {} > discovered'.format(threadId))
self.knownThreads[threadId] = False
# If there is a known thread that is unaccounted for, then it has either hung or crashed.
# Shut everything down.
if len(unaccountedThreads) > 0:
print('The following threads are unaccounted for:\n')
for threadId in unaccountedThreads:
print(threadId)
print('\nShutting down!!!')
break
else:
print('No unaccounted threads...')
sleep(self.timeout)
# Account for any exceptions thrown in the watchdog timer itself
except:
process.terminate()
raise
process.terminate()
def _executeFile(self):
with open(self.filepath, 'r') as f:
exec(f.read(), {'wdQueue' : self.threadIdQ})
if __name__ == '__main__':
wd = Watchdog(PATH_TO_FILE, WATCHDOG_TIMEOUT)
wd.start()
我还有一个小程序来测试看门狗功能
from time import sleep
from threading import Thread
from queue import SimpleQueue
Q_TO_Q_DELAY = 0.013
class QToQ:
def __init__(self, processQueue, threadQueue):
self.processQueue = processQueue
self.threadQueue = threadQueue
Thread(name='queueToQueue', target=self._run).start()
def _run(self):
pQ = self.processQueue
tQ = self.threadQueue
while True:
while not tQ.empty():
sleep(Q_TO_Q_DELAY)
pQ.put(tQ.get())
def fastThread(q):
while True:
print('Fast thread, checking in!')
q.put('fastID')
sleep(0.5)
def slowThread(q):
while True:
print('Slow thread, checking in...')
q.put('slowID')
sleep(1.5)
def hangThread(q):
print('Hanging thread, checked in')
q.put('hangID')
while True:
pass
print('Hello! I am a program that spawns threads!\n\n')
threadQ = SimpleQueue()
Thread(name='fastThread', target=fastThread, args=(threadQ,)).start()
Thread(name='slowThread', target=slowThread, args=(threadQ,)).start()
Thread(name='hangThread', target=hangThread, args=(threadQ,)).start()
QToQ(wdQueue, threadQ)
正如您所看到的,我需要将线程放入queue.Queue,而单独的对象会慢慢将queue.Queue的输出提供给多处理队列.相反,如果我将线程直接放入多处理队列中,或者在put之间没有QToQ对象休眠,则多处理队列将锁定,并且在看门狗端看起来总是为空.
现在,由于多处理队列应该是线程和进程安全的,我只能假设我在实现中搞砸了一些东西.我的解决方案似乎有效,但也觉得hacky足够我觉得我应该解决它.
我正在使用Python 3.7.2,如果重要的话.
解决方法:
我怀疑test_program.py退出.
我把最后几行更改为：
tq = threadQ
# tq = wdQueue # option to send messages direct to WD
t1 = Thread(name='fastThread', target=fastThread, args=(tq,))
t2 = Thread(name='slowThread', target=slowThread, args=(tq,))
t3 = Thread(name='hangThread', target=hangThread, args=(tq,))
t1.start()
t2.start()
t3.start()
QToQ(wdQueue, threadQ)
print('Joining with threads...')
t1.join()
t2.join()
t3.join()
print('test_program exit')
对join()的调用意味着测试程序永远不会自行退出,因为没有任何线程退出.
因此,t3挂起,看门狗程序检测到这一点并检测到未计入的线程并停止测试程序.
如果从上面的程序中删除了t3,则其他两个线程表现良好,看门狗程序允许测试程序无限期地继续.
内容总结
以上是互联网集市为您收集整理的python – 为什么我的多进程队列看起来不是线程安全的？全部内容，希望文章能够帮你解决python – 为什么我的多进程队列看起来不是线程安全的？所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。