Tengine 安装

Posted 2023-04-20 evescn

Tengine 编译安装和Docker镜像制作

Tengine 安装

背景信息：公司业务使用中，需要使用到域名通配，调研 nginx 发现无法支持正则表达式，需要使用 lua 语句才支持，选择使用了 Tengine ，测试使用 OpenResty 也可以支持域名通配

• 业务模式

域名	后端地址	域名个数
d1-*.evescn.com	10.0.0.11	60个
t1-*.evescn.com	10.0.0.21	60个
t2-*.evescn.com	10.0.0.22	60个
t3-*.evescn.com	10.0.0.23	60个

传统 nginx 不支持 * 在中间这个域名匹配，而且类似的业务域名每套环境有 50-60 个域名，单独的一个个配置过慢，随即选择了域名通配这种方式

1 ｜ 二进制文件编译安装

• 安装必要的依赖项

sudo yum install -y gcc gcc-c++ automake pcre-devel zlib-devel openssl-devel

• 下载 Tengine 源代码

wget http://tengine.taobao.org/download/tengine-2.4.0.tar.gz

• 解压源代码包

tar -zxvf tengine-2.4.0.tar.gz

• 进入源代码目录

cd tengine-2.4.0

• 创建 nginx 用户

useradd -s /sbin/nologin -M nginx

• 配置编译选项

./configure \\
    --prefix=/etc/nginx \\
    --user=nginx \\
    --group=nginx \\
    --with-http_ssl_module \\
    --with-http_v2_module \\
    --with-http_realip_module \\
    --with-http_stub_status_module \\
    --with-http_gzip_static_module \\
    --with-pcre \\
    --add-module=./modules/ngx_http_upstream_check_module

• 编译并安装

make && sudo make install

• 设置目录属主属组

chown -R nginx:nginx /etc/nginx

• 设置软连接

ln -sv /etc/nginx/sbin/nginx /usr/sbin/nginx

• 启动服务

nginx -t
nginx

2 ｜ 制作 Docker 镜像安装

使用了 GitHub 上项目，自己手动进行了打包镜像安装

DockerHub地址 | GitHub地址

• 克隆仓库到本地

git clone https://github.com/Axizdkr/tengine.git

• 制作 Docker 镜像

cd tengine

docker build -t harbor.evescn.com/devops/tengine:2.4.0 .

docker push harbor.evescn.com/devops/tengine:2.4.0

• 启动服务

# cat docker-compose.yml 
version: \'3\'

services:
  rabbitmq:
    image: harbor.evescn.com/devops/tengine:2.4.0
    hostname: tengine
    container_name: tengine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      # 配置文件目录
      - /data/tengine/conf.d/:/etc/nginx/conf.d/
      # ssl 证书文件
      - /data/tengine/certs/:/etc/nginx/certs/
      # 日志文件
      - /data/tengine/logs/:/var/log/nginx/
    restart: always

# docker-compose up -d

# docker-compose ls
NAME                STATUS              CONFIG FILES
tengine             running(1)          /data/docker-compose/tengine/docker-compose.yml

GitHub 仓库中文件信息

• nginx.conf 文件

user  nginx;
# This number should be, at maximum, the number of CPU cores on your system. 
worker_processes auto;

error_log  /var/log/nginx/error.log error;
pid        /var/run/nginx.pid;


events 
    # The effective method, used on Linux 2.6+, optmized to serve many clients with each thread.
    use epoll;
    # Determines how many clients will be served by each worker process.
    worker_connections 4000;
    # Accept as many connections as possible, after nginx gets notification about a new connection.
    multi_accept on;



http 
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    #long time
    check_shm_size 5M;
    # Allow the server to close the connection after a client stops responding. 
    reset_timedout_connection on;
    client_header_timeout 15;
    # Send the client a "request timed out" if the body is not loaded by this time.
    client_body_timeout 10;
    # If the client stops reading data, free up the stale client connection after this much time.
    send_timeout 15;
    # Timeout for keep-alive connections. Server will close connections after this time.
    keepalive_timeout 30;
    # Number of requests a client can make over the keep-alive connection.
    keepalive_requests 30;

    log_format  main  \'$remote_addr - $remote_user [$time_local] "$request" \'
                      \'$status $body_bytes_sent "$http_referer" \'
                      \'"$http_user_agent" "$http_x_forwarded_for"\';


    client_body_buffer_size 128k;
    client_max_body_size 10m;
    proxy_read_timeout 180s;
 
    # Compression.
    gzip on;
    gzip_min_length 10240;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;
    gzip_disable "msie6";

    # Sendfile copies data between one FD and other from within the kernel.
    sendfile on; 
    # Don\'t buffer data-sends (disable Nagle algorithm).
    tcp_nodelay on; 
    # Causes nginx to attempt to send its HTTP response head in one packet,  instead of using partial frames.
    tcp_nopush on;
	

    # Hide web server information
    server_tokens off;
    server_info off;
    server_tag off;

    # redirect server error pages to the static page
    error_page 404             /404.html;
    error_page 500 502 503 504 /50x.html;

    include /etc/nginx/conf.d/*.conf;

• Dockerfile 文件

FROM alpine:3.17


ENV TENGINE_VERSION 2.4.0

# nginx: https://git.io/vSIyj

RUN rm -rf /var/cache/apk/* && \\
    rm -rf /tmp/*

ENV CONFIG "\\
        --prefix=/etc/nginx \\
        --sbin-path=/usr/sbin/nginx \\
        --modules-path=/usr/lib/nginx/modules \\
        --conf-path=/etc/nginx/nginx.conf \\
        --error-log-path=/var/log/nginx/error.log \\
        --http-log-path=/var/log/nginx/access.log \\
        --pid-path=/var/run/nginx.pid \\
        --lock-path=/var/run/nginx.lock \\
        --http-client-body-temp-path=/var/cache/nginx/client_temp \\
        --http-proxy-temp-path=/var/cache/nginx/proxy_temp \\
        --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \\
        --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \\
        --http-scgi-temp-path=/var/cache/nginx/scgi_temp \\
        --user=nginx \\
        --group=nginx \\
        --with-http_ssl_module \\
        --with-http_realip_module \\
        --with-http_addition_module \\
        --with-http_sub_module \\
        --with-http_dav_module \\
        --with-http_flv_module \\
        --with-http_mp4_module \\
        --with-http_gunzip_module \\
        --with-http_gzip_static_module \\
        --with-http_random_index_module \\
        --with-http_secure_link_module \\
        --with-http_stub_status_module \\
        --with-http_auth_request_module \\
        --with-http_xslt_module=dynamic \\
        --with-http_image_filter_module=dynamic \\
        --with-http_geoip_module=dynamic \\
        --with-threads \\
        --with-stream \\
        --with-stream_ssl_module \\
        --with-stream_ssl_preread_module \\
        --with-stream_realip_module \\
        --with-stream_geoip_module=dynamic \\
        --with-http_slice_module \\
        --with-mail \\
        --with-mail_ssl_module \\
        --with-compat \\
        --with-file-aio \\
        --with-http_v2_module \\
        --add-module=modules/ngx_http_upstream_check_module \\
        --add-module=modules/headers-more-nginx-module-0.33 \\
	--add-module=modules/ngx_http_upstream_session_sticky_module \\
        "
RUN     addgroup -S nginx \\
        && adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx nginx \\
        && adduser -u 82 -D -S -G www-data www-data \\
        && apk add --no-cache --virtual .build-deps \\
                gcc \\
                libc-dev \\
                make \\
                openssl-dev \\
                pcre-dev \\
                zlib-dev \\
                linux-headers \\
                curl \\
                libxslt-dev \\
                gd-dev \\
                geoip-dev \\
        && curl -L "https://github.com/alibaba/tengine/archive/$TENGINE_VERSION.tar.gz" -o tengine.tar.gz \\
        && mkdir -p /usr/src \\
        && tar -zxC /usr/src -f tengine.tar.gz \\
        && rm tengine.tar.gz \\
        && cd /usr/src/tengine-$TENGINE_VERSION \\
        && curl -L "https://github.com/openresty/headers-more-nginx-module/archive/v0.33.tar.gz" -o more.tar.gz \\
        && tar -zxC /usr/src/tengine-$TENGINE_VERSION/modules -f more.tar.gz \\
	&& rm  more.tar.gz \\
	&& ls -l /usr/src/tengine-$TENGINE_VERSION/modules \\
	&& ./configure $CONFIG --with-debug \\
        && make -j$(getconf _NPROCESSORS_ONLN) \\
        && mv objs/nginx objs/nginx-debug \\
        && mv objs/ngx_http_xslt_filter_module.so objs/ngx_http_xslt_filter_module-debug.so \\
        && mv objs/ngx_http_image_filter_module.so objs/ngx_http_image_filter_module-debug.so \\
        && mv objs/ngx_http_geoip_module.so objs/ngx_http_geoip_module-debug.so \\
        && mv objs/ngx_stream_geoip_module.so objs/ngx_stream_geoip_module-debug.so \\
        && ./configure $CONFIG \\
        && make -j$(getconf _NPROCESSORS_ONLN) \\
        && make install \\
        && rm -rf /etc/nginx/html/ \\
        && mkdir /etc/nginx/conf.d/ \\
        && mkdir -p /usr/share/nginx/html/ \\
        && install -m644 html/index.html /usr/share/nginx/html/ \\
        && install -m644 html/50x.html /usr/share/nginx/html/ \\
        && install -m755 objs/nginx-debug /usr/sbin/nginx-debug \\
        && install -m755 objs/ngx_http_xslt_filter_module-debug.so /usr/lib/nginx/modules/ngx_http_xslt_filter_module-debug.so \\
        && install -m755 objs/ngx_http_image_filter_module-debug.so /usr/lib/nginx/modules/ngx_http_image_filter_module-debug.so \\
        && install -m755 objs/ngx_http_geoip_module-debug.so /usr/lib/nginx/modules/ngx_http_geoip_module-debug.so \\
        && install -m755 objs/ngx_stream_geoip_module-debug.so /usr/lib/nginx/modules/ngx_stream_geoip_module-debug.so \\
        && ln -s ../../usr/lib/nginx/modules /etc/nginx/modules \\
        && strip /usr/sbin/nginx* \\
        && strip /usr/lib/nginx/modules/*.so \\
        && rm -rf /usr/src/tengine-$NGINX_VERSION \\
        \\
        # Bring in gettext so we can get `envsubst`, then throw
        # the rest away. To do this, we need to install `gettext`
        # then move `envsubst` out of the way so `gettext` can
        # be deleted completely, then move `envsubst` back.
        && apk add --no-cache --virtual .gettext gettext \\
        && mv /usr/bin/envsubst /tmp/ \\
        \\
        && runDeps="$( \\
                scanelf --needed --nobanner --format \'%n#p\' /usr/sbin/nginx /usr/lib/nginx/modules/*.so /tmp/envsubst \\
                        | tr \',\' \'\\n\' \\
                        | sort -u \\
                        | awk \'system("[ -e /usr/local/lib/" $1 " ]") == 0  next   print "so:" $1 \' \\
        )" \\
        && apk add --no-cache --virtual .nginx-rundeps $runDeps \\
        && apk del .build-deps \\
        && apk del .gettext \\
        && mv /tmp/envsubst /usr/local/bin/ \\
        \\
        # Bring in tzdata so users could set the timezones through the environment
        # variables
        && apk add --no-cache tzdata \\
        # Bring in curl and ca-certificates to make registering on DNS SD easier
        && apk add --no-cache curl ca-certificates \\
        # forward request and error logs to docker log collector
        && ln -sf /dev/stdout /var/log/nginx/access.log \\
        && ln -sf /dev/stderr /var/log/nginx/error.log

COPY nginx.conf /etc/nginx/nginx.conf

EXPOSE 80 443

STOPSIGNAL SIGTERM

CMD ["nginx", "-g", "daemon off;"]

tengine安装问题

官方安装文档：http://tengine.taobao.org/document/install.html

首先安装了依赖的pcre，但是安装tengine的时候要指定pcre的安装包路径，而不是编译安装后的prefix路径，具体如下
./configure --prefix=/usr/local/nginx --with-pcre=/usr/local/pcre/
make
报错：

[[email protected] tengine-2.2.0]# make
make -f objs/Makefile
make[1]: Entering directory `/root/software/tengine-2.2.0‘
cd /usr/local/pcre/     && if [ -f Makefile ]; then make distclean; fi     && CC="cc" CFLAGS="-O2 -fomit-frame-pointer -pipe "     ./configure --disable-shared
/bin/sh: line 2: ./configure: 没有那个文件或目录
make[1]: *** [/usr/local/pcre//Makefile] 错误 127
make[1]: Leaving directory `/root/software/tengine-2.2.0‘
make: *** [build] 错误 2
[[email protected] tengine-2.2.0]#

原因：
不是
./configure --prefix=/usr/local/nginx --with-pcre=/usr/local/pcre/
应该是
./configure --prefix=/usr/local/nginx --with-pcre=/root/software/pcre-8.39