页面静态化缓存应用（OpenResty+nginx_srcache+redis）
Posted 数睿技术
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了页面静态化缓存应用（OpenResty+nginx_srcache+redis）相关的知识，希望对你有一定的参考价值。
页面静态化缓存应用（OpenResty+nginx_srcache+redis）
介绍
OpenResty是一个基于 Nginx 与 Lua 的高性能 Web 平台，其内部集成了大量精良的 Lua 库、第三方模块以及大多数的依赖项。用于方便地搭建能够处理超高并发、扩展性极高的动态 Web 应用、Web 服务和动态网关。
srcache-nginx-module模块为前淘宝工程师agentzh（章亦春）开发，srcache为location增加了透明的基于subrequest的缓存层。配合lua使用将页面缓存写入redis集群,这样可以大大减轻真实后端服务的压力,提高访问性能！
工作流程
SRCache工作原理
一个请求进来首先从srcache_fetch 指令进行缓存查找，查到返回到缓存，否则，进入下个指令srcache_store将缓存数据插入到redis。
压力测试的结果
用压力测试工具Apache JMeter测试了一下。模拟10个用户每个用户1000请求，10*1000=10000个请求。 没加缓存，平均每个请求处理时间2.7秒
加了缓存之后，平均每个请求处理时间413 毫秒，访问速度提高6.6倍。
srcache缓存管理界面
这样，我们就可以通过管理页面来操作那些url需要缓存和缓存时间，要想清除缓存只要在缓存的具体url中增加/clearcache即可。 只是简单实现了其功能，还有许多需要完善的地方，不过这样的应用，对于提高页面访问速度效果应该还是可以的
判断页面是否缓存
代码示例
nginx配置文件
lua_package_path "/usr/local/nginx/conf/lualib/?.lua"; lua_package_cpath "/usr/local/nginx/conf/lualib/?.so;";
lua_shared_dict limit 10m;
init_by_lua_file /usr/local/nginx/conf/lua/config.lua;
server {
listen 80;
server_name www.awfjplh.com;
root /path/to/root;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
location ~ \.php$ {
srcache_ignore_content_encoding on;
srcache_methods GET HEAD POST;
srcache_store_pass_header Set-Cookie;
srcache_response_cache_control off;
set $cache_key "";
set $cache_expire "";
set $cache_fetch_skip 1;
set $cache_store_skip 1;
set $cache_url '';
rewrite_by_lua_file /usr/local/nginx/conf/lua/rewrite.lua;
srcache_fetch_skip $cache_fetch_skip;
srcache_store_skip $cache_store_skip;
srcache_fetch GET /cache/content key=$cache_key;
srcache_store PUT /cache/content key=$cache_key&expire=$cache_expire;
add_header X-SRCache-Fetch-Status $srcache_fetch_status;
add_header X-SRCache-Store-Status $srcache_store_status;
add_header srcache_fetch_skip $cache_fetch_skip;
add_header srcache_store_skip $cache_store_skip;
add_header cache_key $cache_key;
add_header cache_expire $cache_expire;
add_header cache_url $cache_url;
proxy_set_header Host $host;
proxy_set_header X-Real-Ip $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://hhpp33;
}
#写入redis
location /cache/content {
internal;
content_by_lua_file /usr/local/nginx/conf/lua/content.lua;
}
#用于清除缓存
location ~ /clearcache(/.*) {
content_by_lua_file /usr/local/nginx/conf/lua/clear_cache.lua;
}
}
config.lua文件
--redis群集配置参数
redis_cluster_config = {
name = "redis_cluster",
serv_list = {
{ ip = "192.168.8.2", port = 3001 },
{ ip = "192.168.8.3", port = 3004 },
{ ip = "192.168.8.4", port = 3002 },
{ ip = "192.168.8.5", port = 3003 },
{ ip = "192.168.8.6", port = 3005 },
{ ip = "192.168.8.7", port = 3006 },
{ ip = "192.168.8.8", port = 3011 },
{ ip = "192.168.8.9", port = 3012 },
},
keepalive_timeout = 60000, --redis connection pool idle timeout
keepalive_cons = 1000, --redis connection pool size
connection_timout = 1000, --timeout while connecting
max_redirection = 5 --maximum retry attempts for redirection
}
--redis加载群集模块
redis_cluster = require "rediscluster"
--加载cjson模块
cjson = require "cjson"
rewrite.lua文件
local request_uri_without_args = ngx.re.sub(ngx.var.request_uri, "\\?.*", "")
local red = redis_cluster:new(redis_cluster_config)
local res, flags, err = red:get('config:srcache:url')
local version, flags, err = red:get('config:srcache:version')
local suffix = '1';
if(version ~= ngx.null ) then
suffix = version
end
if res ~= ngx.null then
cache_url_list = cjson.decode(res)
if not ngx.var.cookie_w_info then
for i, v in ipairs(cache_url_list) do
ngx.var.cache_url = v["status"]
local s = string.find(request_uri_without_args, v["url"])
if s and v["status"] == 1 then
local key = { ngx.var.request_method, " ", ngx.var.scheme, "://", ngx.var.host, request_uri_without_args, }
key = table.concat(key)
key = 'srcache:' .. ngx.md5(key)..':'..suffix
ngx.var.cache_key = key
ngx.var.cache_expire = v["expire"]
ngx.var.cache_fetch_skip = 0
ngx.var.cache_store_skip = 0
break
end
end
end
end
content.lua文件
local red = redis_cluster:new(redis_cluster_config)
local method = ngx.req.get_method()
if method == "GET" then
local key = ngx.var.arg_key
local res, flags, err = red:get(key)
red:close()
if err then
ngx.say(ngx.ERR, err)
ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE)
end
if res == nil and flags == nil and err == nil then
ngx.exit(ngx.HTTP_NOT_FOUND)
end
ngx.print(res)
elseif method == "PUT" then
local value = ngx.req.get_body_data()
local expire = ngx.var.arg_expire or 60
local key = ngx.var.arg_key
local ok, err = red:set(key, value)
red:close()
if not ok then
ngx.say(ngx.ERR, err)
ngx.say(ngx.ERR, key .. value)
ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE)
end
red:expire(key,expire)
else
ngx.exit(ngx.HTTP_NOT_ALLOWED)
end
clear_cache.lua文件
ngx.header.content_type = 'application/json;charset=utf-8'
ngx.header.Access_Control_allow_Credentials = 'true'
ngx.header.Access_Control_allow_Origin = '*'
local red = redis_cluster:new(redis_cluster_config)
local request_uri_without_args = ngx.re.sub(ngx.var.request_uri, "\\?.*", "")
request_uri_without_args = ngx.re.gsub(request_uri_without_args, "/clearcache", "")
local key = { ngx.var.request_method, " ", ngx.var.scheme, "://", ngx.var.host, request_uri_without_args, }
local args = ngx.req.get_uri_args()
local query = ngx.encode_args(args)
local version, flags, err = red:get('config:srcache:version')
local suffix = '1';
if(version ~= ngx.null ) then
suffix = version
end
if query ~= "" then
key[#key + 1] = "?"
key[#key + 1] = query
end
local arr = { status = 0, msg = "" }
key = table.concat(key)
key = 'srcache:' .. ngx.md5(key)..':'..suffix
ok, err = red:exists(key)
if ok ~= 1 then
red:close()
ngx.say(cjson.encode(arr))
return
end
ok, err = red:del(key)
if not ok then
red:close()
arr['msg'] = '缓存删除失败'
ngx.say(cjson.encode(arr))
return
end
red:close()
arr['status'] = 1
arr['msg'] = '缓存删除成功'
ngx.say(cjson.encode(arr));
以上是关于页面静态化缓存应用（OpenResty+nginx_srcache+redis）的主要内容，如果未能解决你的问题，请参考以下文章