Go协程池设计思路(Task-Job-Worker)

Posted 2022-11-27 沈子恒

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Go协程池设计思路(Task-Job-Worker)相关的知识，希望对你有一定的参考价值。

1. 铺垫：Go 的接收器Receiver

在go语言中，没有类的概念，但是可以给类型（结构体，自定义类型）定义方法。所谓方法就是定义了接受者的函数。接受者定义在func关键字和函数名之间。可以理解成为结构体定义函数方法，类似于C++中的类方法。

type Person struct 
    name string
    age int


func (p Person) say() 
    fmt.Printf("I'm %s,%d years old\\n",p.name,p.age)

接受者类型可以是struct,也可以是指向struc的指针。

接收者的类型为struct

package main

import "fmt"

type Person struct 
name string
age int

func (p Person) say() 
fmt.Printf("I'm %s,%d years old\\n",p.name,p.age)

func (p Person) older()
    p.age = p.age +1

func main() 
    var p1 Person = Person"lineshen",25
    p1.older()
    p1.say()
    var p2 *Person = &Person"leleyue",22
    p2.older()
    p2.say()

output:

I'm lineshen,25 years old
I'm leleyue,22 years old

对于p1的调用，是没有问题的，因为传递的是结构体，在方法中改变结构体属性，并不能改变原来内存中的值。

对于p2的调用会产生疑问，p2明明是个指针，为什么再调用了older方法之后，打印结果还是22 years old?
方法的接受者是Person而调用者是*Person。其实在p2调用时存在一个转换p2.older() -> *p2.older(); p2.say() -> *p2.say()
p2是指向Person实例的指针。因此，方法执行时的接受者实际上还是一个值而非引用，因此值是不会发生改变的。

接收者的类型为指针

package main

import "fmt"

type Person struct 
name string
age int

func (p *Person) say() 
fmt.Printf("I'm %s,%d years old\\n",p.name,p.age)

func (p *Person) older()
    p.age = p.age +1

func main() 
    var p1 Person = Person"lineshen",25
    p1.older()
    p1.say()
    var p2 *Person = &Person"leleyue",22
    p2.older()
    p2.say()

output：

I'm lineshen,26 years old
I'm leleyue,23 years old

此时传递的是引用，并非值。因此，结构体在内存中属性值会跟随函数方法发生相应的改变。

2. 正题：协程池的设计思路

类似于淘宝电商，可能每分钟有百万级别的请求，最简单直观的方法是来一个请求，new goroutine处理这个请求；那么需要的goroutine也是百万级别的。这样做业务处理逻辑是清楚了，但是会导致CPU做很多无用功，因为CPU需要在不同的goroutin之间流转。所以，在实际业务中，应该限制gorotine的个数（一般协程的数量与CPU的数量保持一致，加大CPU的利用率）。一个可行的架构设计如下：

示例代码如下：

package main
import ("fmt"
		"time")

// 任务的属性应该是一个业务函数
type Task struct 
	f func() error // 函数名f, 无参，返回值为error


// 创建Task任务
func NewTask(arg_f func() error) *Task 
	task := Task
		f: arg_f,
	
	return &task


// Task绑定业务方法
func (task *Task) Execute() 
	task.f() // 调用任务中已经绑定好的业务方法


// ------------------------------------------------
type Pool struct 
	EntryChannel chan *Task  // 对外的Task入口
	JobsChannel  chan *Task  // 内部的Task队列
	workerNum    int         // 协程池中最大的woker数量


// 创建Pool
func NewPool(cap int) *Pool 
	pool := Pool 
	EntryChannel : make(chan *Task),
	JobsChannel  : make(chan *Task),
	workerNum : cap,
	
	return &pool


// Pool绑定干活的方法
func (pool *Pool) worker(workID int) 
	// worker工作 ： 永久从JobsChannel取任务 然后执行任务
	for task := range pool.JobsChannel 
		task.Execute()
		fmt.Println("work ID ", workID, " has executed")
	


// Pool绑定协程池工作方法
func (pool *Pool) run() 
	// 定义worker数量
	for i:=0; i<pool.workerNum; i++ 
		go pool.worker(i)
	
	
	// 从EntryChannel去任务，发送给JobsChannel
	for task := range pool.EntryChannel 
		pool.JobsChannel <- task // 添加task优先级排序逻辑
	


// ------------------------------------------------
func main() 
	// 创建一些任务
	task := NewTask( func() error  // 匿名函数
		fmt.Println(time.Now())
		return nil
	)
	
	// 创建协程池
	pool := NewPool(4)
	
	// 创建多任务，抛给协程池 
	go func()  // 开启新的协程，防止阻塞
		for
			pool.EntryChannel <- task
		
	()
	
	// 启动协程池
	pool.run()

output:

work ID  0  has executed
2020-04-13 00:15:28.6042081 +0800 CST m=+0.040858901
work ID  0  has executed
2020-04-13 00:15:28.605204 +0800 CST m=+0.041854801
work ID  0  has executed
2020-04-13 00:15:28.6062025 +0800 CST m=+0.042853301
work ID  0  has executed
2020-04-13 00:15:28.6062025 +0800 CST m=+0.042853301
work ID  0  has executed
2020-04-13 00:15:28.6071987 +0800 CST m=+0.043849501
work ID  0  has executed
2020-04-13 00:15:28.6071987 +0800 CST m=+0.043849501
work ID  0  has executed
2020-04-13 00:15:28.6081959 +0800 CST m=+0.044846701
work ID  0  has executed
2020-04-13 00:15:28.6081959 +0800 CST m=+0.044846701
work ID  0  has executed
2020-04-13 00:15:28.6091933 +0800 CST m=+0.045844101
work ID  0  has executed
2020-04-13 00:15:28.6091933 +0800 CST m=+0.045844101
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6111882 +0800 CST m=+0.047839001
work ID  0  has executed
2020-04-13 00:15:28.5683041 +0800 CST m=+0.004954901
work ID  1  has executed
2020-04-13 00:15:28.5673053 +0800 CST m=+0.003956101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.5673053 +0800 CST m=+0.003956101
work ID  2  has executed
2020-04-13 00:15:28.6151779 +0800 CST m=+0.051828701
work ID  2  has executed
2020-04-13 00:15:28.6151779 +0800 CST m=+0.051828701
work ID  2  has executed
2020-04-13 00:15:28.6161747 +0800 CST m=+0.052825501
work ID  2  has executed
2020-04-13 00:15:28.6161747 +0800 CST m=+0.052825501
work ID  2  has executed
2020-04-13 00:15:28.6171731 +0800 CST m=+0.053823901
work ID  2  has executed
2020-04-13 00:15:28.6171731 +0800 CST m=+0.053823901
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  2  has executed
2020-04-13 00:15:28.6111882 +0800 CST m=+0.047839001
work ID  0  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  0  has executed
2020-04-13 00:15:28.6211612 +0800 CST m=+0.057812001
work ID  0  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  2  has executed
2020-04-13 00:15:28.6221586 +0800 CST m=+0.058809401
work ID  2  has executed
2020-04-13 00:15:28.6221586 +0800 CST m=+0.058809401
work ID  2  has executed
2020-04-13 00:15:28.6251574 +0800 CST m=+0.061808201
work ID  2  has executed
2020-04-13 00:15:28.6261523 +0800 CST m=+0.062803101
work ID  2  has executed
2020-04-13 00:15:28.6261523 +0800 CST m=+0.062803101
work ID  2  has executed
2020-04-13 00:15:28.6271454 +0800 CST m=+0.063796201
work ID  2  has executed
2020-04-13 00:15:28.6271454 +0800 CST m=+0.063796201
work ID  2  has executed
2020-04-13 00:15:28.6281435 +0800 CST m=+0.064794301
work ID  2  has executed
2020-04-13 00:15:28.6281435 +0800 CST m=+0.064794301
work ID  2  has executed
2020-04-13 00:15:28.6291399 +0800 CST m=+0.065790701
work ID  2  has executed
2020-04-13 00:15:28.6291399 +0800 CST m=+0.065790701
work ID  2  has executed
2020-04-13 00:15:28.6301372 +0800 CST m=+0.066788001
work ID  2  has executed
2020-04-13 00:15:28.6301372 +0800 CST m=+0.066788001
work ID  2  has executed
2020-04-13 00:15:28.6142098 +0800 CST m=+0.050860601
work ID  3  has executed
2020-04-13 00:15:28.6311355 +0800 CST m=+0.067786301
work ID  3  has executed
2020-04-13 00:15:28.6321319 +0800 CST m=+0.068782701
work ID  3  has executed
2020-04-13 00:15:28.6321319 +0800 CST m=+0.068782701
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6391157 +0800 CST m=+0.075766501
work ID  3  has executed

学习视频：https://www.bilibili.com/video/BV1cx41197j5?from=search&seid=15155039435359774234

以上是关于Go协程池设计思路(Task-Job-Worker)的主要内容，如果未能解决你的问题，请参考以下文章

深入浅出Golang的协程池设计

白话 Golang 协程池

Go协程与协程池

Go语言 | 协程池的应用（可能是全网最适合小白的教程）