笔记 | ZeroMQ +Lua In Action

Posted 2021-04-10 5ithink

Linux环境安装配置Lua+ZeroMQ参考：

一：Request-Reply

a.Hello World client in Lua

b.Hello World server in Lua

c.代码结构

d.客服端日志

e.服务端日志

二：Publish-Subscribe模式

The PUB-SUB socket pair is asynchronous. 

• A subscriber can connect to more than one publisher, using one connect call each time. Data will then arrive and be interleaved ("fair-queued") so that no single publisher drowns out the others.

• If a publisher has no connected subscribers, then it will simply drop all messages.

• If you're using TCP and a subscriber is slow, messages will queue up on the publisher. We'll look at how to protect publishers against this using the "high-water mark" later.

• From ZeroMQ v3.x, filtering happens at the publisher side when using a connected protocol (tcp:// or ipc://). Using the epgm:// protocol, filtering happens at the subscriber side.

三：push/pull模式

        pull/push模式：多任务并行处理，一定程度解决pub/sub模式 “消息阻塞问题”。

四：Pub-Sub Message Envelopes

        In the pub-sub pattern, we can split the key into a separate message frame that we call an envelope.

a.publisher in lua

b.subscriber in lua

五：Advanced Pattern

• Request Distribution

• Extended Request-Reply Pattern
  

• Mutithreaded Server

• asynchronous request-reply flows

• Extended Request-Reply

• Load Balancing Broker

The Simple Pirate Pattern

• High-availability Clone Server Pair

• Pub-Sub Network with a Proxy
  

• Multithread Server

• Pub-Sub Forwarder Proxy

• Parallel Pipeline with Kill Signaling

六：小结

            ZeroMQ looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. 

1）The built-in core ZeroMQ patterns

• Request-reply

• Pub-sub

• Pipeline

• Exclusive pair

Pipeline—which connects nodes in a fan-out/fan-in pattern that can have multiple steps and loops. This is a parallel task distribution and collection pattern.

Exclusive pair—which connects two sockets exclusively. This is a pattern for connecting two threads in a process, not to be confused with "normal" pairs of sockets.

2）ZeroMQ sockets patterns

• PUB and SUB

• REQ and REP

• REQ and ROUTER (take care, REQ inserts an extra null frame)

• DEALER and REP (take care, REP assumes a null frame)

• DEALER and ROUTER

• DEALER and DEALER

• ROUTER and ROUTER

• PUSH and PULL

• PAIR and PAIR

(a).DEALER to REP—This gives us an asynchronous client that can talk to multiple REP servers.

(b).REQ to ROUTER—This gives us an asynchronous server that can talk to multiple REQ clients at the same time. 

(c).DEALER to ROUTER—It gives us asynchronous clients talking to asynchronous servers, where both sides have full control over the message formats.

(d).ZeroMQ sockets

• The REQ socket sends, to the network, an empty delimiter frame in front of the message data. REQ sockets are synchronous. REQ sockets always send one request and then wait for one reply. REQ sockets talk to one peer at a time. If you connect a REQ socket to multiple peers, requests are distributed to and replies expected from each peer one turn at a time.

• The REP socket reads and saves all identity frames up to and including the empty delimiter, then passes the following frame or frames to the caller. REP sockets are synchronous and talk to one peer at a time. If you connect a REP socket to multiple peers, requests are read from peers in fair fashion, and replies are always sent to the same peer that made the last request.

• The DEALER socket is oblivious to the reply envelope and handles this like any multipart message. DEALER sockets are asynchronous and like PUSH and PULL combined. They distribute sent messages among all connections, and fair-queue received messages from all connections.

• The ROUTER socket is oblivious to the reply envelope, like DEALER. It creates identities for its connections, and passes these identities to the caller as a first frame in any received message. Conversely, when the caller sends a message, it uses the first message frame as an identity to look up the connection to send to. ROUTERS are asynchronous.

3）ZeroMQ sockets life

        Creating, destroying, and configuring sockets works as you'd expect for any object. But remember that ZeroMQ is an asynchronous, elastic fabric. This has some impact on how we plug sockets into the network topology and how we use the sockets after that.

• Creating and destroying sockets,（zmq_socket(), zmq_close()).

• Configuring sockets by setting options on them and checking them if necessary (see zmq_setsockopt(), zmq_getsockopt()).

• Plugging sockets into the network topology by creating ZeroMQ connections to and from them (zmq_bind(), zmq_connect()).

• Using the sockets to carry data by writing and receiving messages on them (zmq_msg_send(), zmq_msg_recv()).

4）Sending and Receiving Messages

Let's look at the main differences between TCP sockets and ZeroMQ sockets when it comes to working with data:

• ZeroMQ sockets carry messages like UDP rather than a stream of bytes as TCP does. A ZeroMQ message is length-specified binary data. We'll come to messages shortly; their design is optimized for performance and so a little tricky.

• ZeroMQ sockets do their I/O in a background thread. This means that messages arrive in local input queues and are sent from local output queues, no matter what your application is busy doing.

• ZeroMQ sockets have one-to-N routing behavior built-in, according to the socket type.

The zmq_send() method does not actually send the message to the socket connection(s). It queues the message so that the I/O thread can send it asynchronously. It does not block except in some exception cases. So the message is not necessarily sent when zmq_send()returns to your application.

5）I/O Threads

         ZeroMQ does I/O in a background thread. One I/O thread (for all sockets) is sufficient for all but the most extreme applications. When you create a new context, it starts with one I/O thread. The general rule of thumb is to allow one I/O thread per gigabyte of data in or out per second. To raise the number of I/O threads, use the zmq_ctx_set() call before creating any sockets:

6）zero copy

        ZeroMQ's message API lets you send and receive messages directly from and to application buffers without copying data. We call this zero-copy, and it can improve performance in some applications.zero-copy in the specific case where you are sending large blocks of memory (thousands of bytes), at a high frequency.

参考链接

• http://zeromq.org/

• http://zguide.zeromq.org/page:all

• https://github.com/booksbyus/zguide/tree/master/examples/Lua