大型网站架构之百万PV网站架构案例

Posted 2020-12-26

一、案例概述

本案例采用四层模式实现，主要分为前端反向代理、web层、数据库缓存层和数据库层。

• 前端反向代理采用主备模式
• web层采用群集模式
• 数据库缓存层采用主备模式
• 数据库层采用主从模式

由于实验条件限制，本次实验共打开四台虚拟机，此处实验将前端代理层、数据库缓存层、数据库层服务搭建在前两台虚拟服务器上，web层采用群集模式，用于单独放置两台虚拟机。故本次实验实际模型为了模拟实际环境，服务搭建按照如下拓扑搭建。

二、实验环境

主机名	操作系统	IP地址	用途
server1	centosx84_64	192.168.144.112	前端反向代理nginx、Redis缓存主机、mysql主数据库
server2	centosx84_64	192.168.144.111	前端反向代理备Nginx、Redis备缓存主机、MySQL备数据库
web1	centosx84_64	192.168.144.113	Web服务tomcat
web2	centosx84_64	192.168.144.114	Web服务tomcat

三、实验部署

3.1、master和slave设置keepalive与Nginx反向代理

• 配置yum源

rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

• 安装keepalive软件和Nginx

yum install -y keepalived nginx

3.1.1配置keepalive

• 配置主从的keepalive，两台配置基本相同，配置文件不同处已做说明

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
    route_id NGINX_HA    //主从不同
}

vrrp_script nginx {
    script "/opt/shell/nginx.sh"    //配置带动Nginx启动脚本
    interval 2   //每隔2s响应
}

vrrp_instance VI_1 {
    state MASTER    //备为BACKUP
    interface ens33 //注意主机网卡端口名称
    virtual_router_id 51   //从需不同
    priority 100    //从需比主低
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
}

track_script {
    nginx        //调度Nginx启动脚本的函数名
}

virtual_ipaddress {
    192.168.144.188    //设置虚拟IP
    }
}

3.1.2创建keepalive带动Nginx启动脚本

mkdir -p /opt/shell
vim /opt/shell/nginx.sh

#!/bin/bash
k=`ps -ef | grep keepalived | grep -v grep | wc -l`
if [ $k -gt 0 ];then
    /bin/systemctl start nginx.service
else
/bin/systemctl stop nginx.service
fi

chmod +x /opt/shell/nginx.sh

3.1.3配置Nginx前端调度功能

vim /etc/nginx/nginx.conf

• 在include 上面一行新增

upstream tomcat_pool {
                server 192.168.144.113:8080;
                server 192.168.144.114:8080;
                ip_hash;        //会话稳固功能，否则无法通过vip地址登陆
        }
        server {
                listen 80;
                server_name 192.168.144.188;  //虚拟IP
                location / {
                        proxy_pass http://tomcat_pool;
                        proxy_set_header X-Real-IP $remote_addr;
                }
        }

• 配置完成后，检查Nginx配置文件语法

nginx -t -c /etc/nginx/nginx.conf

• 关闭防火墙和SELinux，准备启动keepalive，随后会通过配置文件带动脚本，启动Nginx，此处需要注意，若要停止Nginx，则需要先关闭keepalive，然后才可以。

• 查看Nginx是否启动

netstat -ntap | grep nginx

3.2、部署两台web服务器

3.2.1两台web服务器部署tomcat，部署步骤完全相同，为了试验区分，需要在首页内容做区别

tar xf apache-tomcat-8.5.23.tar.gz
tar xf jdk-8u144-linux-x64.tar.gz
mv jdk1.8.0_144/ /usr/local/java
mv apache-tomcat-8.5.23/ tomcat8

• 解压完成后，配置jdk环境变量,让jdk的各种命令为系统识别

vim /etc/profile

export JAVA_HOME=/usr/local/java
export JRE_HOME=/usr/local/java/jre
export PATH=$PATH:/usr/local/java/bin
export CLASSPATH=./:/usr/local/java/lib:/usr/local/java/jre/lib

source /etc/profile

• 配置tomcat启动与关闭命令为系统识别

  ln -s /usr/local/tomcat8/bin/startup.sh /usr/bin/tomcatup
  ln -s /usr/local/tomcat8/bin/shutdown.sh /usr/bin/tomcatdown

• 启动服务，查看端口状态，使用自身网页测试，观察服务是否能够提供。

tomcatup
netstat -anpt | grep 8080

http://192.168.144.113:8080/ //测试默认测试页是否正常显示
http://192.168.144.114:8080/

• 为了试验区分两台web服务器，修改首页内容

vim /usr/local/tomcat8/webapps/ROOT/index.jsp

<h1>Server 129！！</h1>    //注意，web2需要首页内容不同

• 输入调度器地址，也就是虚拟地址，测试两台节点的调度情况。

http://192.168.175.188/

3.2.2搭建会员商城

• 首先在tomcat配置文件中添加支持商城模块

cd /usr/local/tomcat8/conf/
vim server.xml

• 跳到行尾，在Host name下新增 在148行位置

  <Context path="" docBase="SLSaleSystem" reloadable="true" debug="0"></Context>
  //日志调试信息debug为0表示信息越少，docBase指定访问目录

• 将会员商城软件包解压缩

tar zxvf SLSaleSystem.tar.gz -C /usr/local/tomcat8/webapps/

cd /usr/local/tomcat8/webapps/SLSaleSystem/WEB-INF/classes

vim jdbc.properties //修改数据库IP地址是VRRP的虚拟IP，以及授权的用户名root和密码abc123。

driverClassName=com.mysql.jdbc.Driver
url=jdbc:mysql://192.168.144.188:3306/slsaledb?useUnicode=true&characterEncoding=UTF-8  //该成我们设定的虚拟IP
uname=root
password=123456
minIdle=10
maxIdle=50
initialSize=5
maxActive=100
maxWait=100
removeAbandonedTimeout=180
removeAbandoned=true

• 客户端测试

http://192.168.144.113:8080/ //默认的用户名admin 密码：123456
http://192.168.144.114:8080/

http://192.168.175.188 //输入虚拟地址测试登录，并且关闭主再测试登录

3.3、部署MySQL以及主从

3.3.1安装mysql

• 在实际生产环境中，此处应当搭建MySQL服务，但由于实验条件限制，本实验使用mariadb代替MySQL。

yum install -y mariadb-server mariadb

systemctl start mariadb.service
systemctl enable mariadb.service

netstat -anpt | grep 3306

mysql_secure_installation //

Set root password? [Y/n] y         //设置MySQL管理员账户的密码，我选择密码为abc123
New password: 
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
 ... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] n        //删除匿名用户？
 ... skipping.

Normally, root should only be allowed to connect from ‘localhost‘.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] n   //拒绝root用户远程登录？
 ... skipping.

By default, MariaDB comes with a database named ‘test‘ that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] n  //删除test数据库？
 ... skipping.

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y    //重新加载数据库的所有表？
 ... Success!

Cleaning up...

All done!  If you‘ve completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

mysql -uroot -p //进入数据库

3.3.2导入数据库，授权

mysql -u root -p < slsaledb-2014-4-10.sql
mysql -uroot -p

show databases;

GRANT all ON slsaledb.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘abc123‘;    //授予slsaledb数据库所有表所有权限给root用户在任意网段登录，密码为abc123

flush privileges; 

3.3.3 MySQL主从配置

• mysql主服务器下。

vim /etc/my.cnf

[mysqld]下添加

binlog-ignore-db=mysql,information_schema       //二进制日志格式
character_set_server=utf8
log_bin=mysql_bin  //开启二进制日志
server_id=1  
log_slave_updates=true
sync_binlog=1   //同步日志

systemctl restart mariadb

netstat -anpt | grep 3306

mysql -u root -p

show master status; //记录日志文件名称和 位置值

+------------------+----------+--------------+--------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB         |
+------------------+----------+--------------+--------------------------+
| mysql_bin.000001 |      626 |              | mysql,information_schema |
+------------------+----------+--------------+--------------------------+

grant replication slave on *.* to ‘rep‘@‘192.168.144.%‘ identified by ‘123456‘;  //授予主从状态

flush privileges;

• MySQL从服务器下

vim /etc/my.cnf

[mysqld]下添加

server_id=2

systemctl restart mariadb

netstat -anpt | grep 3306
mysql -u root -p

change master to master_host=‘192.168.144.112‘,master_user=‘rep‘,master_password=‘123456‘,master_log_file=‘mysql_bin.000001‘,master_log_pos=2626;

start slave;

show slave status;

            Slave_IO_Running: Yes
            Slave_SQL_Running: Yes

• 测试主从状态是否可行，方可进行下一步骤。

3.4部署Redis数据库缓存层

3.4.1安装redis

• 很对主从redis数据库缓存，首先需要安装redis软件

yum install -y epel-release //安装扩展源
yum install redis -y

3.4.2设置主从关系

• 配置主Redis

vim /etc/redis.conf

bind 0.0.0.0   //将监听网址修改成任意网段

systemctl start redis.service
netstat -anpt | grep 6379

• 测试本身安装完成能否使用

redis-cli -h 192.168.144.112 -p 6379 //测试连接

192.168.144.112:6379> set name test //设置name 值是test

192.168.144.112:6379> get name //获取name值

• 配置从Redis

vim /etc/redis.conf

bind 0.0.0.0   //61行，修改监听地址
...
slaveof 192.168.114.112  6379 //266行下添加主服务器的IP，不是虚拟IP

• 两台redis服务器服务重启

systemctl restart redis.service

• 从服务器上进入redis，发现已经复制完成

redis-cli -h 192.168.144.111 -p 6379
192.168.144.111:6379> get name
"test"

• 至此，redis主从配置完成。

3.4.3配置商城项目中连接redis的参数

• web节点中配置商城项目，指定Redis虚拟IP

vim /usr/local/tomcat8/webapps/SLSaleSystem/WEB-INF/classes/applicationContext-mybatis.xml

      <!--redis 配置 开始-->

         <constructor-arg value="192.168.144.188"/>    //47行
         <constructor-arg value="6379"/>              //48行

3.4.4测试缓存效果

redis-cli -h 192.168.144.188 -p 6379

192.168.175.188:6379> info

keyspace_hits:1  或者 keyspace_misses:2//关注这个值，命中数和未命中数

登录商城，然后反复点击需要数据库参与的操作页面，再回来检查keyspace_hits或者keyspace_misses: 值变化。