python的多线程多进程代码示例

Posted 2020-07-22 2BiTT

多线程有两种方式：thread和threading

这里应用的场景是map数据分多线程、进度写入codis的示例

这是thread的示例:thread的主进程不会等待线程

 

import thread,math,threading,multiprocessing,os,time
def writeToCodis(prefix,key_list,result_map):
    # client = BfdCodis("xxx", )
    begin = int(time.time())
    for key in key_list:
        print key 
        # client.set(prefix + key, result_map[key])
        # client.expire(prefix + key, 1 * 24 * 3600)
    end = int(time.time())
# 分配key
gidToSid={1:2,3:4}
total_key = len(gidToSid)
gap = int(math.ceil(float(total_key)/process_num))
keys = gidToSid.keys()
for i in range(0,process_num):
    thread.start_new_thread(writeToCodis,(G2S_prefix,keys[i*gap:(i+1)*gap],gidToSid))
    # t.start()
time.sleep(10)

 

threading 会等待线程的执行

for i in range(0,process_num):
    t=threading.Thread(target=writeToCodis,args=(G2S_prefix,keys[i*gap:(i+1)*gap],gidToSid))
    t.start()

 

这是多进程的示例

for i in range(0,process_num):
    p = multiprocessing.Process(target=writeToCodis,args=(G2S_prefix,keys[i*gap:(i+1)*gap],gidToSid))
    p.start()