缓存在内存中并使用go-routine进行更新的最佳方法是什么？

Posted 2021-03-29

案例：天气API - 我将假设任务很简单，我只想制作一个基于另一个API返回天气的API

代码

package main

import (
    "encoding/json"
    "io/ioutil"
    "net/http"

    "github.com/gorilla/mux"
)

type ResponseBody struct {
    CurrentObservation struct {
        Weather         string `json:"weather"`
        Temperature     string `json:"temperature_string"`
        DisplayLocation struct {
            City string `json:"city"`
        } `json:"display_location"`
    } `json:"current_observation"`
}

var weather ResponseBody

func main() {
    // start the api
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    http.ListenAndServe(":8080", r)
}

// handler
func HomeHandler(w http.ResponseWriter, r *http.Request) {
    // load the weather first
    weather = getWeather()
    b, _ := json.Marshal(weather)
    w.Write(b)
}

// get wether from wunderground api
func getWeather() ResponseBody {
    url := "http://api.wunderground.com/api/MY_API_KEY/conditions/q/CA/San_Francisco.json"
    req, err := http.NewRequest("GET", url, nil)
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    var rb ResponseBody
    json.Unmarshal([]byte(body), &rb)
    return rb
}

现在，每当有人点击API时，它都会向天气API发送请求，但是当我有并发请求时，这将不会有效，所以我会将其缓存在内存中，并在每个例程中更新数据第二

首先：我将把getWeather调用移动到main函数

func main() {
    // load the weather first
    weather = getWeather()
    // start the api
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    http.ListenAndServe(":8080", r)
}

// handler
func HomeHandler(w http.ResponseWriter, r *http.Request) {
    b, _ := json.Marshal(weather)
    w.Write(b)
}

并将在主函数中启动一个例行程序

func main() {
    // load the weather first
    weather = getWeather()
    // update data every 1 second
    go func() {
        for {
            time.Sleep(time.Second)
            weather = getWeather()
        }
    }()
    // start the api
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    http.ListenAndServe(":8080", r)
}

所以现在应用程序可以在使用siege工具测试后处理最多250个并发的并发请求

Transactions:                250 hits
Availability:             100.00 %
Elapsed time:               0.47 secs
Data transferred:           0.03 MB
Response time:              0.00 secs
Transaction rate:         531.91 trans/sec
Throughput:             0.07 MB/sec
Concurrency:                2.15
Successful transactions:         250
Failed transactions:               0
Longest transaction:            0.04
Shortest transaction:           0.00

那么以这种方式缓存和更新数据是否正确？或者有什么不对劲我应该以更好的方式做到这一点？

答案

基本方法还可以，但weather上有数据竞争。使用mutex来保护变量：

var mu sync.RWMutex
var weather ResponseBody

func main() {
    // load the weather first
    weather = getWeather()
    // update data every 1 second
    go func() {
        for {
            time.Sleep(time.Second)
            mu.Lock()
            weather = getWeather()
            mu.Unlock()
        }
    }()
    // start the api
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    http.ListenAndServe(":8080", r)
}

func HomeHandler(w http.ResponseWriter, r *http.Request) {
    mu.RLock()
    b, _ := json.Marshal(weather)
    mu.RUnlock()
    w.Write(b)
}

没有必要在weather中保护main的第一个任务，因为赋值是guaranteed to happen before更新goroutine和ListenAndServer启动的请求处理程序。

一个改进是缓存响应体字节：

var mu sync.RWMutex
var resp []byte

func main() {
    // load the weather first
    weather := getWeather()
    resp, _ = json.Marshal(weather)
    // update data every 1 second
    go func() {
        for {
            time.Sleep(time.Second)
            mu.Lock()
            weather = getWeather()
            resp, _ = json.Marshal(weather)
            mu.Unlock()
        }
    }()
    // start the api
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    http.ListenAndServe(":8080", r)
}

func HomeHandler(w http.ResponseWriter, r *http.Request) {
    mu.RLock()
    b := resp
    mu.RUnlock()
    w.Write(b)
}