python – ZMQ DEALER ROUTER高频丢失消息？

内容导读

互联网集市收集整理的这篇技术教程文章主要介绍了python – ZMQ DEALER ROUTER高频丢失消息？，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含2390字，纯文字阅读大概需要4分钟。

内容图文

我正在使用pyzmq从经销商向ROUTER发送20000条消息.

当我在每条消息之间暂停0.0001秒时,它们都会到达,但如果我通过暂停每条消息0.00001将它们发送速度提高10倍,则只有大约一半的消息到达.

是什么导致了这个问题？

解决方法:

是什么导致了这个问题？

ZMQ IO线程的默认设置 – 负责操作模式.

如果你投入时间并深入了解优秀的ZMQ概念和架构,我敢于称之为问题.

从早期版本的ZMQ库开始,就有一些重要的参数可以帮助中心杰作(IO线程)保持稳定和可扩展的理由,从而为您提供这个强大的框架.

零共享/零复制/(几乎)零延迟是不能实现零成本的格言.

ZMQ.Context实例具有相当丰富的内部参数化,可以通过API方法进行修改.

让我引用一个奇妙而宝贵的资料–Pieter HINTJENS的书,Code Connected,第1卷.

(绝对值得花时间和逐步完成PDF副本.C语言代码片段不会伤害任何人的pythonic心态,因为关键信息在Pieter已经制作成300个令人兴奋的页面的文本和故事中).

High-Water Marks

When you can send messages rapidly from process to process, you soon discover that memory is a precious resource, and one that can be trivially filled up. A few seconds of delay somewhere in a process can turn into a backlog that blows up a server unless you understand the problem and take precautions.

…

ØMQ uses the concept of HWM (high-water mark) to define the capacity of its internal pipes. Each connection out of a socket or into a socket has its own pipe, and HWM for sending, and/or receiving, depending on the socket type. Some sockets (PUB, PUSH) only have send buffers. Some (SUB, PULL, REQ, REP) only have receive buffers. Some (DEALER, ROUTER, PAIR) have both send and receive buffers.

In ØMQ v2.x, the HWM was infinite by default. This was easy but also typically fatal for high-volume publishers. In ØMQ v3.x, it’s set to 1,000 by default, which is more sensible. If you’re still using ØMQ v2.x, you should always set a HWM on your sockets, be it 1,000 to match ØMQ v3.x or another figure that takes into account your message sizes and expected subscriber performance.

When your socket reaches its HWM, it will either block or drop data depending on the socket type. PUB and ROUTER sockets will drop data if they reach their HWM, while other socket types will block. Over the inproc transport, the sender and receiver share the same buffers, so the real HWM is the sum of the HWM set by both sides.

Lastly, the HWM-s are not exact; while you may get up to 1,000 messages by default, the real buffer size may be much lower (as little as half), due to the way libzmq implements its queues.

内容总结

以上是互联网集市为您收集整理的python – ZMQ DEALER ROUTER高频丢失消息？全部内容，希望文章能够帮你解决python – ZMQ DEALER ROUTER高频丢失消息？所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。

内容备注

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。

内容手机端