ansible搭建高可用负载均衡的nginx+tomcat

Posted 2021-03-28 运维小分享

ansible搭建高可用、负载均衡的nginx+tomcat

因为硬件资源受限，所以并搭建mysql

.部署架构图

.目录结构

1、hosts文件和site.yml文件

vim hosts

[nginxkpd]

192.168.100.13

192.168.100.14

[nginxkpd:vars]

pcre=pcre-8.35

nginx=nginx-1.6.2

backup=BACKUP

master=MASTER

[tomcat]

192.168.100.11

192.168.100.12

vim site.yml

---

- hosts: all

 gather_facts: no

 remote_user: root

 roles:

   - base

- hosts: tomcat

 remote_user: root

 roles:

   - tomcat

- hosts: nginxkpd

 remote_user: root

 roles:

   - nginx

   - keepalived

2、base角色，替换每台主机的yum源，yum源自行下载

cd roles/base/files

wget http://mirrors.aliyun.com/repo/epel-7.repo

wget http://mirrors.163.com/.help/CentOS6-Base-163.repo

#若不导入如下密钥，可更改yum源的gpgcheck认证为0

wget https://archive.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7

vim main.yml

---

- name: copy 163yum and eployum

 copy: src={{item}} dest=/etc/yum.repos.d/{{item}}

 with_items:

   - CentOS6-Base-163.repo

   - epel.repo

- name: copy RPM-GPG-KEY-EPEL-7 /etc/pki/rpm-gpg

 copy: src=RPM-GPG-KEY-EPEL-7 dest=/etc/pki/rpm-gpg/

3、tomcat角色，访问的目标服务器

cd roles/tomcat/files

自行下载tomcat服务器和jdk放到此目录

cd roles/tomcat/templates

vim profile.j2,此文件与/etc/profile文件一致，末尾添加如下内容

export JAVA_HOME=/usr/local/jdk1.8.0_251

export PATH=${JAVA_HOME}/bin:$PATH

export CLASSPATH=.:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar

vim index.html.j2

{{ansible_default_ipv4.address}}

vim main.yml

---

- name: copy tomcat.tar.gz

 copy: src={{item}} dest=/opt/{{item}}

 with_items:

   - apache-tomcat-9.0.36.tar.gz

   - jdk-8u251-linux-x64.tar.gz

- name: tar xzvf jdk包

 shell: ls /usr/local/jdk1.8.0_251/ || tar xzvf /opt/jdk-8u251-linux-x64.tar.gz -C /usr/local/

- name: tar tomcat

 shell: ls /usr/local/apache-tomcat-9.0.36/ || tar xzvf /opt/apache-tomcat-9.0.36.tar.gz -C /usr/local/

- name: mkdir temp

 shell: cd /usr/local/apache-tomcat-9.0.36/webapps/;mkdir temp

- name: template index.html.j2

 template: src=index.html.j2 dest=/usr/local/apache-tomcat-9.0.36/webapps/temp/index.html

- name: template profile.j2

 template: src=profile.j2 dest=/etc/profile

- name: add port 8080

 shell: firewall-cmd --permanent --add-port=8080/tcp;firewall-cmd --reload

- name: start tomcat service

 shell: source /etc/profile;/usr/local/apache-tomcat-9.0.36/bin/startup.sh

注：profile.j2文件是设置tomcat环境变量，index.html.j2文件是获取本机IP；以上安装的tomcat服务不一定能成功启动，根据自己所装路径手动启动tomcat:/usr/local/apache-tomcat-9.0.36/bin/startup.sh

注：tomcat开机自启动脚本，此环境没有添加这个脚本

前提：在tomcat的bin目录下，vim catalina.sh加入jdk和jre的路径

根据自己tomcat的路径配置

vim catalina.sh

export JAVA_HOME=/usr/local/jdk1.8.0_251

export JAVA_JRE=$JAVA_HOME/jre

vim /etc/init.d/tomcat

#!bin/bash

# chkconfig: 345 80 20

# description: start the tomcat deamon

TOMCAT_HOME=/usr/local/apache-tomcat-9.0.36/bin/

case $1 in

start)

sh $TOMCAT_HOME/startup.sh

;;

stop)

sh $TOMCAT_HOME/shutdown.sh

;;

restart)

sh $TOMCAT_HOME/shutdown.sh

sleep 2

sh $TOMCAT_HOME/startup.sh

;;

*)

echo "please use: tomcat {stat|stop|restart}"

;;

esac

exit 0

#chmod 755 /etc/init.d/tomcat #更改可执行权限

#/etc/init.d/tomcat start #脚本启动tomcat(stop,restart)

#chkconfig --add tomcat #注册tomcat服务

#chkconfig tomcat on #把tomcat服务加入开机自启

#chkconfig --list #查看开机自启的服务

重启linux服务器即可

4、nginx角色，高可用、负载均衡，代理转发服务器

cd roles/nginx/files

nginx和pcre自行网上下载放到此目录

cd roles/nginx/templates

vim nginx.conf.j2，就是nginx.conf文件，在http模块中写入如下内容

#gzip  on;

   upstream myserver{

       server 192.168.100.11:8080;

       server 192.168.100.12:8080;

       }

   server {

       listen       80;

       server_name  {{ ansible_default_ipv4.address }};

       location / {

           proxy_pass http://myserver;

       }

vim main.yml

---

- name: yum install nginx plugins

 yum: name={{item}} state=present

 with_items:

   - make

   - zlib

   - zlib-devel

   - gcc-c++

   - libtool

   - openssl

   - openssl-devel

- name: copy pcre

 copy: src={{pcre}}.tar.gz dest=/opt/

- name: tar xzvf pcre-8.35

 shell: ls /usr/local/{{pcre}} || tar xzvf /opt/{{pcre}}.tar.gz -C /usr/local/

- name: ./configure pcre

 shell: cd /usr/local/{{pcre}};./configure

- name: make &&　make install　pcre

 shell: cd /usr/local/{{pcre}};make;make install

- name: copy nginx

 copy: src={{nginx}}.tar.gz dest=/opt/

- name: tar xzvf nginx-1.6.2

 shell: ls /usr/local/{{nginx}} || tar xzvf /opt/{{nginx}}.tar.gz -C /usr/local/

- name: ./configure nginx

 shell: cd /usr/local/{{nginx}};./configure

- name: make &&　make install nginx　

 shell: cd /usr/local/{{nginx}};make;make install

- name: copy nginx.conf.j2

 template: src=nginx.conf.j2 dest=/usr/local/nginx/conf/nginx.conf

- name: start nginx service

 shell: /usr/local/nginx/sbin/nginx

5、keepalived角色，健康检查nginx服务是否正常运行

vim nginxcheck.sh,监控nginx服务运行的脚本

#!/bin/bash

A=`ps -C nginx --no-header |wc -l`

if [ $A -eq 0 ];then

       /usr/local/nginx/sbin/nginx

       sleep 2

       if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then

               systemctl stop keepalived

       fi

fi

cat roles/keepalived/templates/keepalived.conf.j2

global_defs {

  notification_email {

    [e-mail]

    [e-mail]

    [e-mail]

  }

  notification_email_from [e-mail]

  smtp_server {{ansible_default_ipv4.address}}

  smtp_connect_timeout 30

  router_id LVS_DEVEL

}

vrrp_script chk_nginx {

       script /usr/local/src/nginxcheck.sh #运行脚本文件

       interval 2

       weight -2

}

vrrp_instance VI_1 {

   {% if ansible_default_ipv4.address == '192.168.100.13' %}

   state    BACKUP #13作为nginx备用服务器

   {% else %}

   state    MASTER

   {% endif %}        

   interface ens33         #网卡名称

   virtual_router_id 51     #id主备必须同  

   {% if ansible_default_ipv4.address == '192.168.100.13' %}

   priority    98

   {% else %}

   priority    100 #优先级别，主高备低

   {% endif %}      

   advert_int 1

   authentication {

       auth_type PASS

       auth_pass 1111

   }

  track_script { #此模块调用脚本

       chk_nginx

   }

   virtual_ipaddress {

   }

}

vim main.yml

---

- name: yum keepalied

 yum: name=keepalived state=present

- name: copy nginxcheck.sh

 copy: src=nginxcheck.sh dest=/usr/local/src/ mode=777

- name: template keepalived.conf.j2

 template: src=keepalived.conf.j2 dest=/etc/keepalived/keepalived.conf

- name: add port 80/tcp

 shell: firewall-cmd --permanent --add-port=80/tcp;firewall-cmd --reload

- name: serevice keepalived start

 service: name=keepalived state=restarted enabled=yes

6、group_vars变量文件

cat all

---

ansible_ssh_pass: centos

7、执行：cd项目目录：ansible-playbook -i hosts site.yml

测试访问：http://192.168.100.100

若访问失败，检查tomcat服务有没有启动，需手动启动；检查防火墙是否开启；selinux是否关闭