构建基于 OpenResty + Lua 的短地址服务

Posted 2021-04-10 LaiJingwu

前言

曾经，我用 php 原生编写了一个轻量级自定义短链接生成的库（GitHub: laijingwu/url-shortener），依赖 vlucas/phpdotenv 和 catfan/medoo 组件。时隔两年，短信营销推广需要在短信内容中放置短链接，而这次我选择了 OpenResty 和 Lua 来完成。

创建数据库表

这里我们使用了 mysql 作为持久化的存储，因此需要在 MySQL 中创建一个表用于存储短链接和对应长链接等信息。

CREATE TABLE `short_link` ( `id` int unsigned NOT NULL AUTO_INCREMENT, `key` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL, `value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL, `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `deleted_at` timestamp NULL DEFAULT NULL, PRIMARY KEY (`id`), KEY `idx_key` (`key`), KEY `idx_created_at` (`created_at`) USING BTREE) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci ROW_FORMAT=DYNAMIC COMMENT='短链接';

编写 Lua 脚本

废话不多说，直接上示例代码。示例代码中使用了 MySQL 作为持久化的存储，使用了 OpenResty 自带的 lua_shared_dict 作为缓存，这里可以根据自身的实际需求去更换不同的存储。

local host = os.getenv("APP_URL")local db_host = os.getenv("DATABASE_HOST")local db_port = os.getenv("DATABASE_PORT")local db_database = os.getenv("DATABASE")local db_user = os.getenv("DATABASE_USERNAME")local db_password = os.getenv("DATABASE_PASSWORD")local expire = os.getenv("CACHE_EXPIRED_MINUTES") if host == nil then host = "http://127.0.0.1"end if db_host == nil or db_database == nil or db_user == nil or db_password == nil then ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)end if db_port == nil then db_port = 3306end if expire == nil then expire = 15end local uri = ngx.var.request_uri if not uri or uri == "/" then ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY)end local links = ngx.shared.linkslocal redirect, flags, err = links:get(uri) if not redirect or redirect == nil then local mysql = require "resty.mysql" local db, err = mysql:new() if not db then ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY) end  db:set_timeout(1000)  local ok, err, errno, sqlstate = db:connect{ host = db_host, port = db_port, database = db_database, user = db_user, password = db_password, max_packet_size = 1024 * 1024 }  if not ok then db:close() ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY) end  sql = "SELECT value FROM short_link WHERE `key` = " .. "\'" .. uri .. "\'" .. " AND deleted_at IS NULL ORDER BY created_at DESC LIMIT 1"  local res, err, errno, sqlstate = db:query(sql) if not res or next(res) == nil then db:close() ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY) end  local result = res[1]["value"]  if(type(result) ~= "string") then db:close() ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY) end  if string.match(result, "^https-://[%w+%.]+[/%w+]+.*") ~= nil then redirect = result elseif string.match(result, "^%w[%w+%.]+.*") ~= nil then redirect = "http://" .. result else redirect = host .. "/" .. result end  local ok, err = links:set(uri, redirect, 60 * expire) if not ok then db:close() ngx.redirect(host, ngx.HTTP_MOVED_PERMANENTLY) end  db:close()end ngx.redirect(redirect, ngx.HTTP_MOVED_PERMANENTLY)

准备 OpenResty 配置文件

nginx.conf 文件如下，供参考：

user nobody;worker_processes 2; error_log /var/log/nginx/error.log warn;pid /var/run/nginx.pid; env APP_URL;env DATABASE;env DATABASE_HOST;env DATABASE_PORT;env DATABASE_USERNAME;env DATABASE_PASSWORD;env CACHE_EXPIRED_MINUTES; events { worker_connections 1024;} http { include mime.types; default_type application/octet-stream;  log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';  access_log /var/log/nginx/access.log main;  sendfile on; tcp_nopush on;  keepalive_timeout 65; server_names_hash_bucket_size 64;  lua_shared_dict links 128m; # 用于分配缓存大小  gzip on; gzip_disable "msie6";  include /etc/nginx/conf.d/*.conf;}

shortlink.vhost.conf 配置如下：

server { listen 80; listen [::]:80; server_name xxx.com; access_log /var/log/nginx/shortlink.access.log; error_log /var/log/nginx/shortlink.error.log;  resolver local=on ipv6=off; # 读取/etc/resolve.conf中默认dns地址 resolver_timeout 5s;  location / { default_type text/plain; content_by_lua_file /usr/local/openresty/lualib/shortlink.lua; }  location = /robots.txt { return 200 "User-Agent: *\nDisallow: "; }  location = /probe { return 200 ""; }  #error_page 404 /404.html; error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/local/openresty/nginx/html; }  location ~ /\.ht { deny all; }}

搭建本地测试环境

本地的测试环境使用 docker 进行搭建，编写好 docker-compose 和相关配置文件后就可以进行测试。

version: '3'services: mysql: container_name: local-mysql image: mysql:8.0.20 ports: - "3306:3306" volumes: - $PWD/conf/mysql/conf.d:/etc/mysql/conf.d:ro - $PWD/data/mysql:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: root MYSQL_DATABASE: app MYSQL_USERNAME: test MYSQL_PASSWORD: test TZ: Asia/Shanghai networks: - local openresty: container_name: local-openresty image: openresty/openresty:centos ports: - "80:80" volumes: - $PWD/log/openresty:/var/log/nginx - $PWD/scripts/openresty/shortlink.lua:/usr/local/openresty/lualib/shortlink.lua - $PWD/scripts/openresty/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf:ro - $PWD/scripts/openresty/shortlink.conf:/etc/nginx/conf.d/shortlink.conf:ro entrypoint: - /usr/bin/openresty - -g - daemon off; environment: TZ: Asia/Shanghai APP_URL: http://127.0.0.1 DATABASE_HOST: mysql DATABASE_PORT: 3306 DATABASE: fusion DATABASE_USERNAME: root DATABASE_PASSWORD: root CACHE_EXPIRED_MINUTES: 15 depends_on: - mysql networks: - localnetworks: local:

编写部署文件

完成上述工作后，就可以着手开始进行部署工作了，部署文件示例如下。ConfigMap 配置文件如下：

apiVersion: v1kind: ConfigMapmetadata: name: cm-shortlink-proxy annotations: configVersion: CONFIG_VERSION namespace: APP_ENVdata: nginx.conf: | (填入上面准备好的 nginx 配置)  shortlink.conf: | (填入上面准备好的 vhost 配置)  shortlink.lua: | (填入上面编写好的 lua 脚本)

Deployment 配置文件如下：

apiVersion: apps/v1kind: Deploymentmetadata: name: dp-shortlink-proxy labels: app: shortlink-proxy namespace: APP_ENVspec: replicas: 1 selector: matchLabels: app: shortlink-proxy template: metadata: annotations: configVersion: CONFIG_VERSION service.shortlink.version: SHORTLINK_VERSION labels: app: shortlink-proxy spec: imagePullSecrets: - name: registry-key containers: - name: shortlink-proxy image: SHORTLINK_IMAGE_REPOSITORY:SHORTLINK_VERSION imagePullPolicy: IfNotPresent env: - name: APP_FRONTEND_URL value: "https://APP_FRONTEND_SERVER_NAME" - name: DATABASE_HOST valueFrom: secretKeyRef: name: APP_ENV-database-secret key: host - name: DATABASE_PORT value: "DB_PORT" - name: DATABASE value: "DB_DATABASE" - name: DATABASE_USERNAME valueFrom: secretKeyRef: name: APP_ENV-database-secret key: username - name: DATABASE_PASSWORD valueFrom: secretKeyRef: name: APP_ENV-database-secret key: password - name: CACHE_EXPIRED_MINUTES value: "SHORTLINK_CACHE_TIMEOUT" ports: - name: shortlink-svc containerPort: 80 volumeMounts: - name: shortlink-proxy-logs mountPath: /usr/local/nginx/logs - name: shortlink-proxy-conf mountPath: /usr/local/openresty/nginx/conf/nginx.conf subPath: nginx.conf - name: shortlink-proxy-conf mountPath: /etc/nginx/conf.d/shortlink.conf subPath: shortlink.conf - name: shortlink-proxy-conf mountPath: /usr/local/openresty/lualib/shortlink.lua subPath: shortlink.lua resources: requests: cpu: 100m memory: 300Mi limits: cpu: 2000m memory: 2000Mi command: ["/usr/bin/openresty"] args: ["-g", "daemon off;"] readinessProbe: httpGet: path: /probe port: 80 initialDelaySeconds: 10 periodSeconds: 10 volumes: - name: shortlink-proxy-logs emptyDir: {} - name: shortlink-proxy-conf configMap: name: cm-shortlink-proxy

Service 配置文件如下：

apiVersion: v1kind: Servicemetadata: name: svc-shortlink-proxy annotations: service.shortlink.protocol: http labels: app: shortlink-proxy namespace: APP_ENVspec: type: LoadBalancer ports: - name: shortlink-svc port: 80 protocol: TCP targetPort: shortlink-svc selector: app: shortlink-proxy

Secret 配置文件如下：

apiVersion: v1kind: Secretmetadata: name: APP_ENV-database-secrettype: OpaquestringData: host: DB_HOST username: DB_USERNAME password: DB_PASSWORD

上面的配置含有很多环境变量，我采取的方案是结合 env 文件和脚本进行统一替换，编写好配置文件后，依次将上述配置文件应用到部署集群后就可以正式对外提供服务了。完结撒花，如果有什么写的不对的地方，欢迎各位观众老爷指出。

短地址怎么来

这里有个小 PHP Demo，仅供参考。

function generate_unique_short_keys(string $string, int $short_len = 6) : array{ $base_chars = [ 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' ]; $short_arr = []; $md5_arr = str_split(md5($string), 8); foreach ($md5_arr as $md5_str) { $hex = hexdec($md5_str) &amp; 0x3fffffff; $short_str = null; for ($i = 0; $i < $short_len; $i++) { $index = 0x0000003d &amp; $hex; $short_str .= $base_chars[$index]; $hex = $hex >> intval(30 / $short_len); } array_push($short_arr, $short_str); } return $short_arr;}