运维工作中经常用到的一些知识总结

Posted 2020-11-11

接上篇：
http://blog.51cto.com/bobo365/2125159

54、rabbitMQ

http://blog.csdn.net/lishaojun0115/article/details/53152255

用户管理

用户管理包括增加用户，删除用户，查看用户列表，修改用户密码。

(1) 新增一个用户

rabbitmqctl add_user Username Password

(2) 删除一个用户

rabbitmqctl delete_user Username

(3) 修改用户的密码

rabbitmqctl change_password Username Newpassword

(4) 查看当前用户列表

rabbitmqctl list_users

设置用户角色的命令为：

rabbitmqctl set_user_tags User Tag

用户角色可分为五类，超级管理员, 监控者, 策略制定者, 普通管理者以及其他。

(1) 超级管理员(administrator)

可登陆管理控制台(启用management plugin的情况下)，可查看所有的信息，并且可以对用户，策略(policy)进行操作。

(2) 监控者(monitoring)

可登陆管理控制台(启用management plugin的情况下)，同时可以查看rabbitmq节点的相关信息(进程数，内存使用情况，磁盘使用情况等)

(3) 策略制定者(policymaker)

可登陆管理控制台(启用management plugin的情况下), 同时可以对policy进行管理。但无法查看节点的相关信息(上图红框标识的部分)。

(4) 普通管理者(management)

仅可登陆管理控制台(启用management plugin的情况下)，无法看到节点信息，也无法对策略进行管理。

(5) 其他

无法登陆管理控制台，通常就是普通的生产者和消费者。

设置用户权限

rabbitmqctl set_permissions -p VHostPath User ConfP WriteP ReadP 

例如：

(1)设置guest用户对路径 / 的configuration、write、read权限

rabbitmqctl set_permissions -p / guest “” “.” “.*”

(2) 查看(指定hostpath)所有用户的权限信息

rabbitmqctl list_permissions [-p VHostPath]

(3) 查看指定用户的权限信息

rabbitmqctl list_user_permissions User

(4) 清除用户的权限信息

rabbitmqctl clear_permissions [-p VHostPath] User

55、yourls部署：

LNMP环境

（1）创建数据库：

CREATE DATABASE yourls;
GRANT ALL PRIVILEGES ON yourls.* TO ‘yourls‘@‘%‘ IDENTIFIED BY ‘bobo365‘;
FLUSH PRIVILEGES;
\q

（2）主配置文件

Copy user/config-sample.php to user/config.php
define( ‘YOURLS_DB_USER‘, ‘yourls‘ );
define( ‘YOURLS_DB_PASS‘, ‘bobo365‘ );
define( ‘YOURLS_DB_NAME‘, ‘yourls‘ );
define( ‘YOURLS_DB_HOST‘, ‘10.150.27.51‘ );
define( ‘YOURLS_DB_PREFIX‘, ‘yourls_‘ );
define( ‘YOURLS_SITE‘, ‘http://bobo365.com‘ );
define( ‘YOURLS_HOURS_OFFSET‘, 0 ); 
define( ‘YOURLS_LANG‘, ‘‘ ); 
define( ‘YOURLS_UNIQUE_URLS‘, true );
define( ‘YOURLS_PRIVATE‘, true );
define( ‘YOURLS_CObobo365IEKEY‘, ‘qQ4KhL_pu|[email protected]#%:b^{A[vhm‘ );
define( ‘YOURLS_DEBUG‘, false );
define( ‘YOURLS_URL_CONVERT‘, 36 );

（3）nginx配置

[[email protected] conf.d]# more your.conf

server {
        listen       80;
        server_name  bobo365.com www.bobo365.com;

        client_header_buffer_size 32k;
        large_client_header_buffers 4 32k;
        client_max_body_size 500m;
        rewrite ^/([0-9A-Za-z]+)/?$ /yourls-go.php?id=$1 last;
        rewrite ^/([0-9A-Za-z]+)\+/?$ /yourls-infos.php?id=$1 last;
        rewrite ^/([0-9A-Za-z]+)\+all/?$ /yourls-infos.php?id=$1&all=1 last;

    location / {
        root /data/php;
        index index.html;
    }

    location ~ \.php$ {
        root           /data/php;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;
    }

}

（4）安装

http://bobo365.com/admin/install.php

56、Tomcat多实例

/usr/local/apache-tomcat-8.0.26

[[email protected] ~]# cd /usr/local/tomcat_instance/

drwxr-xr-x 8 root root 4096 Feb  8 10:06 tomcat1
drwxr-xr-x 8 root root 4096 Feb  8 10:56 tomcat2
drwxr-xr-x 8 root root 4096 Feb  8 10:56 tomcat3
drwxr-xr-x 8 root root 4096 Feb  8 10:15 tomcat4
drwxr-xr-x 8 root root 4096 Feb  8 10:56 tomcat5

[[email protected] tomcat_instance]# cd tomcat1/

drwxr-xr-x 2 root root 4096 Mar  8 16:41 bin
drwxr-xr-x 3 root root 4096 Feb 11 17:19 conf
drwxr-xr-x 2 root root 4096 Mar  8 11:28 logs
drwxr-xr-x 2 root root 4096 Feb  8 10:06 temp
drwxr-xr-x 3 root root 4096 Mar  8 13:57 webapps
drwxr-xr-x 3 root root 4096 Feb  8 10:06 work

端口：

shutdown 8015 8025 8035 8045 8055
HTTP     8888 8889 8890 8891 8892
AJP      8019 8029 8039 8049 8059

webapps目录：

/usr/local/tomcat_instance/tomcat1/webapps
/usr/local/tomcat_instance/tomcat2/webapps
/usr/local/tomcat_instance/tomcat3/webapps
/usr/local/tomcat_instance/tomcat4/webapps
/usr/local/tomcat_instance/tomcat5/webapps

[[email protected] tomcat1]# tree
.
├── bin
│?? ├── shutdown.sh
│?? ├── startup.sh
│?? └── tomcat.pid
├── conf
│?? ├── Catalina
│?? │?? └── localhost
│?? ├── catalina.policy
│?? ├── catalina.properties
│?? ├── context.xml
│?? ├── logging.properties
│?? ├── server.xml
│?? ├── server.xml_ori
│?? ├── tomcat-users.xml
│?? ├── tomcat-users.xsd
│?? └── web.xml

more conf/server.xml

<Server port="8015" shutdown="SHUTDOWN">

<Connector port="8888" redirectPort="8443"   
   minProcessors="10" maxProcessors="1000"    
   maxThreads="5001" minSpareThreads="1000" maxSpareThreads="2000"    
   acceptCount="31500"  connectionTimeout="60000" disableUploadTimeout="true" debug="0"   URIEncoding="UTF-8"/>

 <Connector port="8019" protocol="AJP/1.3" redirectPort="8443" />

 <Host name="localhost"  appBase="/usr/local/tomcat_instance/tomcat1/webapps"
            unpackWARs="true" autoDeploy="true">

startup.sh
[[email protected] bin]# vim ../../tomcat1/bin/startup.sh

#!/bin/bash
export JRE_HOME=/usr/local/jdk1.8.0_60/jre
export CATALINA_HOME=/usr/local/apache-tomcat-8.0.26
export CATALINA_BASE="/usr/local/tomcat_instance/tomcat1"
export CATALINA_TMPDIR="$CATALINA_BASE/temp"
export CATALINA_PID="$CATALINA_BASE/bin/tomcat.pid"
export JAVA_OPTS="-Xmx4096m -Xms4096m -Xmn2g -Xss256k -Dtomcat.name=tomcat1"

#创建logs目录

if [ ! -d "$CATALINA_BASE/logs" ]; then
  mkdir $CATALINA_BASE/logs
fi

#创建temp目录

if [ ! -d "$CATALINA_BASE/temp" ]; then
  mkdir $CATALINA_BASE/temp
fi

#调用tomcat启动脚本

bash $CATALINA_HOME/bin/startup.sh "[email protected]"         

57、sftp上传文件

[[email protected] script]# more test.sh

#!/bin/bash

USER=bobo365
PASSWORD=bobo365
SRCDIR=/home/bobo365
TODAY=`date +"%Y%m%d"`
DESDIR=./upload/${TODAY}
IP=192.168.2.133
PORT=20002

cd ${SRCDIR}
FILES=`find  ${SRCDIR} -name ‘*.tar.gz‘`

for FILE in ${FILES}
do
        echo ${FILE}

lftp -u ${USER},${PASSWORD} sftp://${IP}:${PORT} <<EOF
#mkdir ${DESDIR}
cd ${DESDIR}/
lcd ${SRCDIR}
put ${FILE}
echo "put ${FILE} finished."
by
EOF

done

lftp -u lzdat,[email protected] sftp://192.168.2.23:52000 <<EOF
cd upload/20180313/
put /upload/in_sheet_20180313.csv
echo "put in_sheet_20180313.csv finished."
by
EOF

58、Nginx转发配置

在Nginx中的默认Proxy是只能对后面Real Server做端口转发的，而不能做域名转发。如果想使用Nginx对后端是同一IP、同一端口 转发不同的域名则需要配置Nginx Proxy。
这个是因为默认情况下：
proxy_set_header Host $proxy_host;
这样就等于前端输入域名后到nginx这里直接转换成IP进行转发了。
于是我们需要修改proxy_set_header的参数。
proxy_set_header Host $http_host;

例：

proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_set_header Host $host;
        proxy_set_header Referer $http_referer;
        proxy_set_header Cobobo365ie $http_cobobo365ie;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Accept-Encoding ‘‘;
        proxy_hide_header Vary;
        proxy_redirect off;

59、压力测试

 yum -y install epel-release
 yum install python-pip

[[email protected] ~]# more load_test.py

from locust import HttpLocust, TaskSet, task

class UserBehavior(TaskSet):

    @task(1)
    def baidu(self):
        self.client.get("/")

class WebsiteUser(HttpLocust):
    task_set = UserBehavior
    min_wait = 3000
    max_wait = 6000

locust -f load_test.py --host=https://www.baidu.com  

60、JVM故障排查

jps获取java进程的pid
jstack -F PID >> java.txt 导出cpu占用高进程的线程栈
top -H -p PID查看对应进程的哪个线程占用cpu过高
echo "obase=16; PID" | bc将线程的PID转化为16进制
在第二步导出的java.txt中查找转化为16进制的线程的pid。找到对应的线程栈。
分析负载高的线程栈都是什么业务操作。优化程序并处理问题。

61、mysql优化

http://blog.51cto.com/leesir/2084306

硬件层优化

目标一： 全面关闭节能模式，让MySQL跑在高性能模式下
关闭CPU节能，OPI Link Speed Select “Max Performance”
关闭内存节能模式，Memory Speed Select “Max Performance”
Power C-States Select “Disable”
C1 Enhanced Mode Select "Disable"
目标二：关闭NUMA，让CPU能始终高效的使用内存
关闭NUMA，Socket Interleave,"Non-NUMA"
目标三：提升IOPS性能
SSD/PCIe-SSD
机械盘搭配阵列卡，cache策略，BBU电池，RAID-10,15KRPM

系统层的优化

目标一：提升IOPS性能
配置合理的I/O调度器
机械盘配deadline，执行命令echo deadline >/sys/block/sda/queue/scheduler
固态盘配noop，执行命令echo noop >/sys/block/sda/queue/scheduler
文件系统尽量使用XFS
mount参数增加noatime，nodiratime，nobarrier
vi /etc/fstab

/dev/sda1 /data xfs defaults,noatime,nodiratime,nobarrier 0 0
/dev/sda2 / xfs defaults,noatime,nodiratime,nobarrier 0 0
/dev/sda3 swap swap defaults,noatime,nodiratime,nobarrier 0 0
mount -o remount /data
mount

目标二：减少SWAP使用倾向甚至禁掉，稳定磁盘I/O和网络减少等待时间
1.vm.swappiness设为5甚至0，假如不关心发生OOM

echo ‘vm.swappiness = 5‘ >>/etc/sysctl.conf
/sbin/sysctl -p
2.vm.dirty_background_ratio设为5，vm.dirty_ratio设为10，让脏页持续刷入磁盘，避免磁盘I/O瞬间写产生TIME_WAIT

echo ‘vm.dirty_background_ration = 5‘ >>/etc/sysctl.conf
echo ‘vm.dirty_ratio = 10‘ >>/etc/sysctl.conf
/sbin/sysctl -p

3.net.ipv4.tcp_tw_recycle和net.ipv4.tcp_tw_reuse设为双1，减少网络等待时间，提高效率

echo ‘net.ipv4.tcp_tw_recycle = 1‘ >>/etc/sysctl.conf
echo ‘net.ipv4.tcp_tw_reuse = 1‘ >>/etc/sysctl.conf
/sbin/sysctl -p

MySQL层的优化

略...

62、vim是自动开启智能缩进导致#号

开启

set paste

关闭

set nopaste
或
set paste!

63、PHP

yum install https://mirrors.tuna.tsinghua.edu.cn/remi/enterprise/remi-release-7.rpm -y
yum --enablerepo remi --enablerepo remi-php56 install php ……
yum --enablerepo remi --enablerepo remi-php70 install php ……
yum --enablerepo remi --enablerepo remi-php71 install php ……

升级：

yum -y install epel-release
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
yum -y remove php* php-common? 
yum -y install php70w? php70w-opcache
yum -y install php70w-mysql php70w-xml php70w-soap php70w-xmlrpc php70w-mbstring php70w-json php70w-gd php70w-mcrypt
yum list | grep php70w*
yum -y install php70w-pecl-redis.x86_64 php70w-pecl-memcached.x86_64
systemctl status php-fpm

64、NGINX

yum install http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm -y
yum install nginx

65、MySQL：

toolkit：

- 分析慢查询：

./pt-query-digest /data/percona-server/logs/mysql-slow.log > query.log

- 碎片整理：

/pt-online-schema-change --user=root --password=bobo365 --host=localhost --alter="ENGINE=Innodb" D=yourls,t=yourls_url --execute

表统计信息：

- 统计每个库的大小：

select table_schema,sum(data_length)/1024/1024/1024 as data_length,sum(index_length)/1024/1024/1024 as index_length,sum(data_length+index_length)/1024/1024/1024 as sum_data_index from information_schema.tables where table_schema!=‘information_schema‘ and table_schema!=‘mysql‘ group by table_schema;
+-------------------------+----------------+----------------+----------------+
| table_schema            | data_length    | index_length   | sum_data_index |
+-------------------------+----------------+----------------+----------------+
| activemq                | 0.000000037253 | 0.000009536743 | 0.000009573996 |
| alarms                  | 0.000045776367 | 0.000061035156 | 0.000106811523 |
| ams                     | 0.000030517578 | 0.000015258789 | 0.000045776367 |
| dashboard               | 0.000045776367 | 0.000061035156 | 0.000106811523 |
| falcon_portal           | 0.000183105469 | 0.000183105469 | 0.000366210938 |
| graph                   | 0.000045776367 | 0.000045776367 | 0.000091552734 |
| performance_schema      | 0.000000000000 | 0.000000000000 | 0.000000000000 |
| replication_wwww.ymq.io | 0.000015258789 | 0.000000000000 | 0.000015258789 |
| sys                     | 0.000015258789 | 0.000000000000 | 0.000015258789 |
| uic                     | 0.000061035156 | 0.000091552734 | 0.000152587891 |
| yourls                  | 0.000045776367 | 0.000061035156 | 0.000106811523 |
+-------------------------+----------------+----------------+----------------+
11 rows in set (0.09 sec)

- 统计库中每个表的大小：

select table_name,data_length,index_length,sum(data_length+index_length) as total_size from information_schema.tables where table_schema=‘yourls‘ group by table_name;
+----------------+-------------+--------------+------------+
| table_name     | data_length | index_length | total_size |
+----------------+-------------+--------------+------------+
| yourls_log     |       16384 |        16384 |      32768 |
| yourls_options |       16384 |        16384 |      32768 |
| yourls_url     |       16384 |        32768 |      49152 |
+----------------+-------------+--------------+------------+
3 rows in set (0.00 sec)

- 统计所有数据库的大小

select sum(data_length+index_length)/1024/124/1024 from information_schema.tables;
+---------------------------------------------+
| sum(data_length+index_length)/1024/124/1024 |
+---------------------------------------------+
|                             18.510122414558 |
+---------------------------------------------+
1 row in set (0.03 sec)

- 统计信息的收集方法：

mysql> select * from information_schema.tables where table_name=‘yourls_url‘\G
*************************** 1. row ***************************
  TABLE_CATALOG: def
   TABLE_SCHEMA: yourls
     TABLE_NAME: yourls_url
     TABLE_TYPE: BASE TABLE
         ENGINE: InnoDB
        VERSION: 10
     ROW_FORMAT: Dynamic
     TABLE_ROWS: 6
 AVG_ROW_LENGTH: 2730
    DATA_LENGTH: 16384
MAX_DATA_LENGTH: 0
   INDEX_LENGTH: 32768
      DATA_FREE: 0
 AUTO_INCREMENT: NULL
    CREATE_TIME: 2018-05-25 14:36:30
    UPDATE_TIME: 2018-05-25 14:36:30
     CHECK_TIME: NULL
TABLE_COLLATION: utf8_unicode_ci
       CHECKSUM: NULL
 CREATE_OPTIONS: 
  TABLE_COMMENT: 
1 row in set (0.00 sec)

66、SLA几个9的计算方式

99%     365*24*（1-99%）=87.6小时
99.9%   365*24*（1-99.9%）=8.76小时
99.99%  365*24*（1-99.99%）*60=52.56分钟
99.999% 365*24*（1-99.999%）*60=5.256分钟```