通过 helix api 加入 twitch irc，（5000+ 个频道）连接重置错误

Posted 2023-02-23

【中文标题】通过 helix api 加入 twitch irc，（5000+ 个频道）连接重置错误

【英文标题】：Joining twitch irc through helix api, (5000+ channels) connection reset error

【发布时间】：2021-06-26 14:01:13

【问题描述】：

要开始我使用匿名连接加入频道，这意味着没有加入限制，我尝试了不同的睡眠变体，我刚开始只是从文本加入但是有很多问题，因为它连接了所有加入之前的插座，所以我看不出是什么原因造成的。然而，这是迄今为止我创建的最好的版本，它非常磨损，但我只是想了解问题所在。如果有人对完成这样的大任务有任何见解，我将不胜感激！

（oauth 和 helix 标头来自我为测试而创建的一个随机 alt 帐户，它在示例中尝试加入 10k 个频道，但最多停止在 2k-3k 左右）

import requests
import socket 
import time
import threading 
import random 

connections_made = 0
sockets = []


def connect():
    global sockets
    global connections_made
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    print("CONNECTING TO IRC")
    sock.connect(('irc.chat.twitch.tv', 6667))
    sock.send(bytes('PASS oauth:'+ '\r\n', 'utf-8'))
    sock.send(bytes('NICK justinfan' + str(random.randint(10000,99999)) + '\r\n', 'utf-8'))
    sockets.append(sock)
    connections_made += 1
    print(f"socket: len(sockets)")


for i in range(2):
    connect() # initial for .recv reading

helix_headers = 'client-id': 'q6batx0epp608isickayubi39itsckt', 'authorization': 'Bearer rk0ixn6169ar7y5xey9msvk1h8zrs8'
def request(channels_to_join,cursor):
    
    request_amount = int(channels_to_join / 100)   # 100 requests = 10000 channels   

    user_list = []
    sock_numb = 0
    total_chans_joined = 0 
    count_every_request  = 0
    for i in range(request_amount):
        time.sleep(1)
        # 3k channels with time.sleep(1) 1.5k channels with time.sleep(2)  30 seconds then connection reset error (when bulk joining 100 channels and waiting for the next request)
        # waiting 30 seconds  doesnt fix this either stop at about 500  channels so lasted 2.5minutes? 
        # waiting 60 seconds at 500 channels breaks

        if count_every_request == 1: # for every 100 channels 
            connect() 
            count_every_request = 0 


        r = requests.get("https://api.twitch.tv/helix/streams?first=100&after=" + cursor,headers=helix_headers)
        cursor = r.json()['pagination']['cursor']

        count_every_request += 1


        for everything in r.json()['data']:
            user_list.append(everything['user_login'])
            channel = everything['user_login']
            # join channel

            if sock_numb == connections_made: # makes it so when joining sockets it joins up to the amount of sockets that there are and then loops back
                sock_numb = 0 
            print(f"JOINING  #channel with socket: sock_numb total joined: total_chans_joined")
            sockets[sock_numb].send(bytes('JOIN #' + channel + '\r\n', 'utf-8'))
            total_chans_joined += 1
            sock_numb += 1



def loop():    
    print("Looping")
    try:
        while True:
            time.sleep(0.1)
            for i in range(connections_made): 
                data = sockets[i].recv(4096).decode("utf-8",errors='replace').strip()
                if data == "":
                    continue

                print(data)

                if "PING :tmi.twitch.tv" in data:
                    print("PONG")
                    sockets[i].send(bytes('PONG :tmi.twitch.tv' + '\r\n', 'utf-8'))

    except Exception as e:
        print(str(e) + " error in loop ")
        pass 



thread_loop = threading.Thread(target=loop)
thread_loop.start()


request(channels_to_join=10000,cursor = "eyJiIjp7IkN1cnNvciI6ImV5SnpJam80T0RrMU1TNDVNRFkwTWpnd09URTVNU3dpWkNJNlptRnNjMlVzSW5RaU9uUnlkV1Y5In0sImEiOnsiQ3Vyc29yIjoiZXlKeklqbzFNakF6TGpJM056UTFPVEUzT1RReE1Td2laQ0k2Wm1Gc2MyVXNJblFpT25SeWRXVjkifX0")

【问题讨论】：

我为你撤销了泄露的访问令牌we_do_a_lil_trolling

【参考方案1】：

可能的问题是您的机器人无法跟上消息发送缓冲区的速度。

因此您连接到许多频道，但没有及时处理传入的聊天消息。因此，从 Twitch 发送给您的消息“队列”超出了 Twitch 的缓冲区。 DC就是你

或者根据 IRC 速率限制指南，您发送了太多命令并与服务器断开连接。

大型聊天机器人通常会在多个连接上拆分频道组以解决此问题。