百万PV架构搭建详解
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了百万PV架构搭建详解相关的知识,希望对你有一定的参考价值。
百万PV架构
●先了解一下什么是PV
PV(page view)即页面浏览量,通常是衡量一个网络新闻频道或网站甚至一条网络新闻的主要指标。网页浏览数是评价网站流量最常用的指标之一,简称为PV。监测网站PV的变化趋势和分析其变化原因是很多站长定期要做的工作。 Page Views中的Page一般是指普通的html网页,也包含php、jsp等动态产生的html内容。来自浏览器的一次html内容请求会被看作一个PV,逐渐累计成为PV总数。
●环境及组件介绍
操作系统 | IP地址 | 角色 | web组件 |
---|---|---|---|
Centos7 | 192.168.70 .136 | 主服务器 | keepalived、nginx、mysql、redis |
Centos7 | 192.168.70.137 | 从服务器 | keepalived、nginx、mysql、redis |
Centos7 | 192.168.70 .134 | web后端1 | tomcat、项目 |
Centos7 | 192.168.70 .132 | web后端2 | tomcat、项目 |
●使用的软件包
nginx1.8.1:http://101.96.10.46/nginx.org/download/nginx-1.8.1.tar.gz
web服务包:https://pan.baidu.com/s/143ZRkqfUxJJIBzO_yz7gPg
密码:wsgd
mysql解压版https://pan.baidu.com/s/11b_ccrosT0IPdnXhRrU4yQ
密码:ruh5
一、主从服务器配置keepalived
1、安装
[[email protected] ~]# yum install keepalived -y
2、修改keepalive配置文件
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.70.131 #指向本机
smtp_connect_timeout 30
router_id NGINX_01 #备机为NGINX_02
}
vrrp_instance VI_1 {
state MASTER #备机为BACKUP
interface ens33 #网卡名称
virtual_router_id 51 #备机52,不与主相同
priority 100 #优先级,备机优先级要低于主机
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.70.100 #虚拟Ip地址
}
}
==========以下是配置keepalive的顺带着启动nginx===========
个人看法,不太希望这样做,因为每两秒尝试启动nginx,nginx会两秒打一次error日志,提示端口被占用无法启动,这样反而造成了不必要的磁盘读写。
Configuration File for keepalived
#定义NGINX启动脚本位置,每两秒检查一次
vrrp_script nginx {
script "/opt/shell/nginx.sh"
interval 2
}
#删除原本内容,添加route_id,备机为NGINX_02
global_defs {
route_id NGINX_01
}
vrrp_instance VI_1 {
state MASTER #主机为MASTER
interface ens33 #网卡名称
virtual_router_id 51 #备机为52
priority 100 #优先级,备机为99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
#触发脚本
track_script {
nginx
}
#虚拟IP
virtual_ipaddress {
192.168.70.100
}
}
3、启动keepalived
[[email protected] ~]# systemctl start keepalived.service
4、查看虚拟ip
[[email protected] ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:5b:aa:ce brd ff:ff:ff:ff:ff:ff
inet 192.168.70.131/24 brd 192.168.70.255 scope global dynamic ens33
valid_lft 1403sec preferred_lft 1403sec
inet 192.168.70.100/32 scope global ens33 #虚拟ip
valid_lft forever preferred_lft forever
inet6 fe80::49c4:1329:39cd:4427/64 scope link
valid_lft forever preferred_lft forever
注:备机也要启动keepalived,然后关闭主机keepalived,看虚拟Ip会不会漂移到备机。
二、安装nginx
上传nginx安装包至/opt目录下
1、安装环境包
[[email protected] opt]# yum -y install gcc gcc-c++ autoconf gd-devel automake zlib zlib-devel openssl openssl-devel pcre*
2、解压,编译nginx
[[email protected] opt]# tar zxf nginx-1.8.1.tar.gz
[[email protected] opt]# cd nginx-1.8.1/
[[email protected] nginx-1.8.1]# ./configure
--prefix=/usr/local/nginx
--user=nginx
--group=nginx
--with-http_ssl_module
--with-http_gzip_static_module
--with-http_image_filter_module
--with-http_stub_status_modulev
3、安装
[[email protected] nginx-1.8.1]# make && make install
4、创建nginx用户
[[email protected] nginx-1.8.1]# useradd -M -s /sbin/nologin nginx
5、优化命令路径
[[email protected] nginx-1.8.1]# ln -s /usr/local/nginx/sbin/nginx /usr/sbin/
6、添加代理并简单优化
[[email protected] nginx-1.8.1]# cd /usr/local/nginx/conf/
[[email protected] conf]# vim nginx.conf
#用户nginx,单核
user nginx;
worker_processes 1;
#每个核心连接数2048
events {
worker_connections 2048;
}
http {
include mime.types;
default_type application/octet-stream;
#隐藏版本号
server_tokens off;
#sendfile参数用于开启文件的高效传输模式。
sendfile on;
tcp_nopush on;
tcp_nodelay on;
server_names_hash_bucket_size 128;
server_names_hash_max_size 512;
client_header_timeout 15s;
client_body_timeout 15s;
send_timeout 60s;
keepalive_timeout 65;
#压缩模块
gzip on;
gzip_buffers 4 64k;
gzip_http_version 1.1;
gzip_comp_level 2;
gzip_min_length 1k;
gzip_vary on;
gzip_types text/plain text/javascript application/x-javascript text/css text/xml application/xml applicatin/xml+rss;
#反向代理
upstream tomcat_pool {
server 192.168.70.134:8080;
server 192.168.70.132:8080;
ip_hash; #会话稳固,防止停留页面过久导致需要重新登录。
}
server {
listen 80;
server_name 192.168.70.100; #虚拟Ip
location / {
proxy_pass http://tomcat_pool;
proxy_set_header X-Real-IP $remote_addr;
expires 1d;
}
}
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
7、检查配置文件语法
[[email protected] conf]# nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
8、启动nginx
[[email protected] conf]# nginx
三、安装tomcat
上传jdk和tomcat到/opt目录下
1、解压jdk和tomcat
[[email protected] opt]# tar zxf jdk-8u144-linux-x64.tar.gz
[[email protected] opt]# tar zxf apache-tomcat-8.5.23.tar.gz
2、更名jdk为java
[[email protected] opt]# mv jdk1.8.0_144/ java
3、添加环境变量
[[email protected] opt]# vim ~/.bashrc
#末行添加如下三行
export JAVA_HOME=/opt/java
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
4、刷新环境变量,查看java版本
[[email protected] opt]# source ~/.bashrc
[[email protected] opt]# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
5、更名tomcat为tomcat8
[[email protected] opt]# mv apache-tomcat-8.5.23 tomcat8
6、修改默认首页进行测试,两台tomcat都需要操作
[[email protected] opt]# cd tomcat8/webapps/ROOT/
[[email protected] ROOT]# vim index.jsp
#删除原有的内容,添加如下本机ip,用于测试
#134主机添加如下
<h1> this is 134 server <h1>
#132主机添加如下
<h1> this is 132 server <h1>
7、建立命令软链接,启动tomcat
[[email protected] ROOT]# ln -s /opt/tomcat8/bin/startup.sh /usr/bin/tomcatup
[[email protected] ROOT]# ln -s /opt/tomcat8/bin/shutdown.sh /usr/bin/tomcatdown
[[email protected] ROOT]# tomcatup #启动tomcat
Using CATALINA_BASE: /opt/tomcat8
Using CATALINA_HOME: /opt/tomcat8
Using CATALINA_TMPDIR: /opt/tomcat8/temp
Using JRE_HOME: /opt/java
Using CLASSPATH: /opt/tomcat8/bin/bootstrap.jar:/opt/tomcat8/bin/tomcat-juli.jar
Tomcat started.
8、web网页用虚拟ip访问
192.168.70.100
显示后台我们设置的tomcat页面即可,然后关闭一台tomcat看页面会不会变
四、mysql主从复制
上传mysql压缩包到/opt目录下
1、解压mysql
[[email protected] opt]# tar zxf mysql-5.7.21-linux-glibc2.12-x86_64.tgz
[[email protected] opt]# mv mysql-5.7.21-linux-glibc2.12-x86_64 mysql
2、修改配置文件
[[email protected] opt]# vim /etc/my.cnf
[client]
port = 3306
socket = /tmp/mysql.sock
[mysqld]
character-set-server = utf8mb4
skip_name_resolve = 1
user = mysql
port = 3306
server-id = 1 #注意,备机的mysql,server-id不能与主机相同
socket = /tmp/mysql.sock
basedir = /opt/mysql/
datadir = /opt/mysql/data
pid-file = /opt/mysql/data/mysql.pid
log_bin=/opt/mysql/data/mysql-bin
log-error = /opt/mysql/data/log-error.log
innodb_data_home_dir = /opt/mysql/data
slow-query-log-file=/opt/mysql/data/slow.log
relay-log-index = /opt/mysql/data/relaylog
relay-log-info-file = /opt/mysql/data/relaylog
relay-log = /opt/mysql/data/relaylog
open_files_limit = 10240
table_open_cache = 2048
back_log = 300
max_connections = 10000
max_connect_errors = 20
explicit_defaults_for_timestamp = 1
max_allowed_packet = 64M
thread_cache_size = 300
query_cache_size = 256M
query_cache_limit = 2M
query_cache_min_res_unit = 2k
default-storage-engine = InnoDB
thread_stack = 512K
transaction_isolation = READ-COMMITTED
tmp_table_size = 256M
max_heap_table_size = 256M
key_buffer_size = 2G
sort_buffer_size = 2M
join_buffer_size = 6M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 15G
myisam_repair_threads = 1
interactive_timeout = 1800
wait_timeout = 28800
innodb_data_file_path = ibdata1:120M;ibdata2:200M;ibdata3:200M:autoextend
innodb_buffer_pool_size = 1G
innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M
innodb_log_file_size = 512M
innodb_log_files_in_group = 3
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120
innodb_purge_threads = 0
slow_query_log = 1
long_query_time = 3
replicate-ignore-db = mysql
replicate-ignore-db = test
replicate-ignore-db = information_schema
#slave-skip-errors = 1032,1062,1026,1114,1146,1048,1396
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
[mysqldump]
quick
max_allowed_packet = 64M
3、初始化mysql,设置密码
[[email protected] opt]# ln -s /opt/mysql/bin/* /usr/bin/
[[email protected] opt]# mysql/bin/mysqld --initialize-insecure #初始化
[[email protected] opt]# mysqld_safe & #启动
[[email protected] opt]# mysqladmin -uroot password #设置密码
[[email protected] opt]# mysql -uroot -p #登陆查看
4、设置mysql主从复制
#以下在主库上操作
mysql> grant replication slave on *.* to ‘replication‘@‘192.168.70.%‘ identified by ‘123123‘ with grant option; #授权replication用户权限
mysql> flush privileges; #刷新权限
mysql> show master status; #查看二进制日志位置
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000002 | 885 | | | |
+------------------+----------+--------------+------------------+-------------------+
#以下在从库上操作
mysql> change master to master_host=‘192.168.70.136‘,master_user=‘replication‘,master_password=‘123123‘,,master_log_file=‘mysql-bin.000002‘,master_log_pos=885;
mysql> start slave;
mysql> show slave statusG;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.70.136
..........................
Slave_IO_Running: Yes #要看到这两项为yes,主从复制成
Slave_SQL_Running: Yes
5、主库导入sql语句
[[email protected] opt]# mysql -uroot -p < slsaledb-2014-4-10.sql
6、项目的mysql用户授权
mysql> grant all on . to ‘root‘@‘%‘ identified by ‘123123‘;
mysql> flush privileges;
五、redis缓存服务器配置
1、下载epel源和redis
[[email protected] opt]# yum install -y epel-release
[[email protected] opt]# yum install redis -y
2、修改配置文件
[[email protected] opt]# vim /etc/redis.conf
主从服务器
61行 bind 0.0.0.0
从服务器添加
266行 slaveof 192.169.70.136 6379
3、主从服务器启动redis
[[email protected] opt]# systemctl start redis
4、验证主从功能
[[email protected] opt]# redis-cli -h 192.168.70.136 -p 6379
#主服务器写入
192.168.70.136:6379> set name lisi
OK
192.168.70.136:6379> get name
"lisi"
#从服务器
192.168.70.137:6379> get name #能获取值说明主从同步成功
"lisi"
5、搭建主从切换
注:主服务器上操作
[[email protected] opt]# vim /etc/redis-sentinel.conf
protected-mode no#是否开启保护模式
sentinel monitor mymaster 192.168.70.137 6379 1 #1表示1台从
sentinel down-after-milliseconds mymaster 3000 #切换时间为3000毫秒
#主从都启动群集
[[email protected] opt]# systemctl start redis-sentinel.service
#查看群集状态
[[email protected] opt]# redis-cli -h 192.168.70.136 -p 26379 info Sentinel
# Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 #可以看到下面行有从137 master0:name=mymaster,status=odown,address=192.168.70.137:6379,slaves=0,sentinels=2
六、部署项目
#两台tomcat主机上操作
1、上传项目包,解压缩到指定目录
[[email protected] opt]# tar zxf SLSaleSystem.tar.gz -C /opt/tomcat8/webapps/
2、修改tomcat配置文件,确定项目位置
[[email protected] opt]# cd /opt/tomcat8/conf/
[[email protected] conf]# vim server.xml
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
#接近尾行位置添加如下
<Context path="" docBase="SLSaleSystem" reloadable="true" debug="0"></Context>
3、修改项目文件,指定连接数据库和redis缓存服务
[[email protected] conf]# cd /opt/tomcat8/webapps/SLSaleSystem/WEB-INF/classes/
#修改为虚拟ip
url=jdbc:mysql://192.168.70.100:3306/slsaledb?useUnicode=true&characterEncoding=UTF-8
#数据库授权的用户
uname=root
#密码
password=123123
#修改redis缓存服务器ip地址
[[email protected] classes]# vim applicationContext-mybatis.xml
<!--redis 配置 开始-->
<bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
<property name="maxActive" value="90" />
<property name="maxIdle" value="5" />
<property name="maxWait" value="1000" />
<property name="testOnBorrow" value="true" />
</bean>
<bean id="jedisPool" class="redis.clients.jedis.JedisPool" destroy-method="destroy" >
<constructor-arg ref="jedisPoolConfig"/>
<constructor-arg value="192.168.70.100"/> #虚拟ip
<constructor-arg value="6379"/>
</bean>
<bean id="redisAPI" class="org.slsale.common.RedisAPI">
<property name="jedisPool" ref="jedisPool"/>
</bean>
4、配置完重新启动tomcat
[[email protected] classes]# tomcatdown
[[email protected] classes]# tomcatup
5、网页中访问
192.168.70.100
#默认账号admin
#默认密码123456
6、查看redis缓存服务
[[email protected] opt]# redis-cli -h 192.168.70.136 -p 6379
192.168.70.136:6379> info
keyspace_hits:2 #找到这两行,这个是命中次数
keyspace_misses:0
注:点击进网页内容出现感叹属于正常,缓存服务器需要一段时间缓存内容。
redis缓存服务器主从切换的问题,主宕机,从会顶替上来,但再次启动主的时候,从不会让出当前master的角色,除非手动切换,或从宕机。
这个项目是很久很久的一个项目,拿来学习理解下百万pv架构还行,学习为主,另外想要达到百万pv,硬件性能是必不可少的,软件层面上的优化也是必须的。
本文主要介绍百万pv的架构及其中一些服务之间的关系,谨以介绍学习,实际操作以自己公司环境为主。
以上是关于百万PV架构搭建详解的主要内容,如果未能解决你的问题,请参考以下文章