Kubernetes 中的 Socket.IO 集群，多个节点未发送给所有客户端

Posted 2023-02-19

【中文标题】Kubernetes 中的 Socket.IO 集群，多个节点未发送给所有客户端

【英文标题】：Socket.IO Cluster In Kubernetes With Multiple Nodes Not Emitting To All Clients

【发布时间】：2022-01-15 23:48:32

【问题描述】：

我有一个 kubernetes 环境，我正在尝试将我的 socket.io nodejs 应用程序发布到它以进行扩展。

它只向连接到同一服务器的客户端发送消息，其他客户端根本没有收到任何消息。

使用：

socket.io@4.1.0

sticky@1.0.1

cluster-adapter@0.1.0

我正在关注此文档：

这是我的 clustered-socket.js：

const cluster = require("cluster");
const http = require("http");
const  Server  = require("socket.io");
const numCPUs = require("os").cpus().length;
const  setupMaster, setupWorker  = require("@socket.io/sticky");
const  createAdapter, setupPrimary  = require("@socket.io/cluster-adapter");

if (cluster.isMaster) 
    console.log(`Master $process.pid is running`);

    const httpServer = http.createServer();

    setupMaster(httpServer, 
        loadBalancingMethod: "least-connection",
    );

    setupPrimary();

    cluster.setupMaster(
        serialization: "advanced",
    );

    httpServer.listen(80);

    console.log(`Starting $numCPUs workers...`);

    for (let i = 0; i < numCPUs; i++) 
        var worker = cluster.fork();
        console.log(`Worker $worker.process.pid started.`);
    

    console.log(`Started $numCPUs workers.`);

    cluster.on("exit", (worker) => 
        console.log(`Worker $worker.process.pid died`);
        cluster.fork();
    );
 else 
    console.log(`Worker $process.pid started`);

    const httpServer = http.createServer();
    const io = new Server(httpServer, 
        transports: ["websocket"],
        cors: 
            origin: "*",
            methods: ["GET", "POST"]
        
    );

    io.adapter(createAdapter());
    setupWorker(io);

    io.on('connection', (socket) => 
        socket.emit("MESSAGE", "Welcome to Stream Socket.");

        socket.on('disconnect', () =>  );

        socket.on('SUBSCRIBE', (msg) => 
            try 
                console.log(msg);
                
                var obj = JSON.parse(msg);
                socket.join(obj.requestedStream);

                if (obj.requestedStream.startsWith("OLD_MESSAGES")) 
                    // This here is only emitting to clients connected to same server.
                    io.to("OLD_MESSAGES").emit("___OLD MESSAGES HERE____");
                

             catch (e)  console.log(e); 
        );

        socket.on('UNSUBSCRIBE', (msg) => 
            var obj = JSON.parse(msg);
            socket.leave(obj.requestedStream);
        );
    );

我已经在 Kubernetes 上使用 nginx 启用了粘性会话，入口：

nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/affinity-mode: "persistent"
nginx.ingress.kubernetes.io/affinity-canary-behavior: "sticky"
nginx.ingress.kubernetes.io/session-cookie-name: "route"
nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
nginx.ingress.kubernetes.io/session-cookie-expires: "10800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "10800"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600" 
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-ssl-server-name: "on"

这是我连接到套接字的方式：

    var server = document.getElementById('address').value;

    socket = io(server, 
        transports: ["websocket"],
        'reconnection': false,
        
    );

    socket.on('MESSAGE', (msg) => 
        logResponse(msg);
    );

    socket.on('MESSAGE', (msg) => 
        logResponse(msg);
    );

最后是我的 dockerfile：

FROM node:14
EXPOSE 80
EXPOSE 443

WORKDIR /usr/src/app
COPY package*.json ./
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    rm -rf /var/lib/apt/lists/*
RUN sed -i "/^# deb.*multiverse/ s/^# //" /etc/apt/sources.list
RUN sed -i "/^# deb.*universe/ s/^# //" /etc/apt/sources.list
RUN npm install
RUN npm ci --only=production
COPY . .
CMD ["node", "clustered-socket.js"]

【参考方案1】：

使用一个适配器，该适配器使用一个连接到所有节点的数据库，例如 Redis、Mongodb 或 Postgres 适配器。

【讨论】：

谢谢，redis 适配器解决了这个问题。