运维工作中经常用到的一些知识总结

Posted 2020-11-11

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了运维工作中经常用到的一些知识总结相关的知识，希望对你有一定的参考价值。

在日常运维工作中，会有一些知识使用频率较高，以下为个人在工作中常用的一些操作，没有做详细的分类，排版比较凌乱。有需要的同学们可以参考，希望能有所帮助。

1、查看当前系统所以变量

sysctl -a

2、修改Centos7 网卡为eth

install centos 7 按 tab键，增加：

net.ifnames=0 biosdevname=0

3、yum删除软件

yum erase $(rpm -qa|grep java)
yum erase $(rpm -qa|grep zabbix)

4、列行转换

    cat a.txt | xargs           # 列转行
    cat a.txt | xargs           # 列转行
    cat a.txt | xargs -n1       # 行转列

5、du统计排除某文件夹

du -sh * --exclude=gfs

6、gitlab配置备份及定期删除

gitlab-ctl start|stop|status|restart
59 23 * * * gitlab-rake gitlab:backup:create > /dev/null 2>&1
00 00 * * * /bin/bash /home/script/del_git_bak.sh > /dev/null 2>&1

[[email protected] ~]# more /home/script/del_git_bak.sh

#!/bin/bash
git_dir=/var/opt/gitlab/backups/
find $git_dir -type f -mtime +6 -exec rm -f {} \;

7、查找已删除但未释放文件

lsof -n | grep deleted

8、docker容器系统初始化脚本

more /bin/load.sh

#!/bin/bash
ulimit -n 655350
source /data/env.conf

mkdir -p /data/nginx/conf.d
mkdir -p /data/log/nginx
mkdir -p /data/log/glusterfs
mkdir -p /opt/gfs

if [ ${IS_SINGLE} == 1 ]; then
        mount -t glusterfs gfs-master:datastore  /opt/gfs
else
        glusterfs -f /etc/glusterfs/datastore.vol --log-file=/data/log/glusterfs/glusterfs.log /opt/gfs
fi

9、docker容器启动命令

docker run -itd  -v /data/nginx/NGINX:/data --restart=always --net=host --name=NGINX --privileged=true nginx:1.12 /bin/start.sh /bin/bash

10、计划任务debug写入具体文件：

crond debug >result.txt 2>&1

11、glusterFS SLAVE创建示例：

docker run -itd --privileged=true -v /data/gfs/GFS:/data --restart=always --net=host --name=GFS --privileged=true gfs:0.7 /bin/start.sh /bin/bash
gluster volume create datastore replica 2 transport tcp master:/data/export slave:/data/export
gluster volume create datastore replica 2 transport tcp master:/data/export slave:/data/export force
gluster volume create datastore_1 replica 2 transport tcp gfs-master:/data/export_1 gfs-slave:/data/export_1 force
gluster volume start datastore_1

glusterfsd -f /etc/glusterfs/datastore.vol
glusterfs -f /etc/glusterfs/datastore.vol --log-file=/data/log/glusterfs/glusterfs.log /opt/emm
glusterfs -f /etc/glusterfs/datastore_1.vol --log-file=/data/log/glusterfs/glusterfs_1.log /data/gfs/GFS/export_1

[[email protected] glusterfs]# more /etc/glusterfs/datastore.vol

volume remote1
  type protocol/client
  option transport-type tcp
  option remote-host master
  option remote-subvolume /data/export
end-volume

volume remote2
  type protocol/client
  option transport-type tcp
  option remote-host slave
  option remote-subvolume /data/export
end-volume

volume replicate
  type cluster/replicate
  subvolumes remote1 remote2
end-volume

volume writebehind
  type performance/write-behind
  option window-size 1MB
  subvolumes replicate
end-volume

yum install -y glusterfs glusterfs-server glusterfs-fuse

12、文件copy

CP不覆盖现有文件：
cp -n 源目地

cd /opt/tmp/xx

cp -R -n * /opt/IBM/pic/product/
cp -R -n * /opt/IBM/pic/product/  
cp -R -n * /opt/IBM/pic/product/
cp -R -n * /opt/IBM/pic/product/
cp -R -n * /opt/IBM/pic/product/

find ./ -mtime -22 -type d -name "117*"|awk -F/ ‘{ print $2 }‘| uniq > 117.txt
tar -T 117.txt -zcvf /opt/zengliang/117.tar.gz

cp直接覆盖：

\cp -r -f aa ../zhangsan/aa

13、新增虚拟内存：

物理内存交换分区（SWAP）
<= 4G 至少4G
4~16G 至少8G
16G~64G 至少16G
64G~256G 至少32G

SWAP增加：
1、查看 free -m
2、选择合适的分区，创建适合的swap文件
dd if=/dev/zero of=/db/swapfile1 bs=1M count=8192
3、将生成的文件设置为swap分区文件，命令：”mkswap /swap1/swapfile“ mkswap /db/swapfile1 swapon /db/swapfile1
4、激活新创建的swap分区,命令:”swapon /swap1/swapfile“
5、查看生效的swap分区大小，命令:”free -m“
6、设置开机启动新创建的swap分区，编辑”/etc/fstab”配置文件，在文件末尾添加
”/swap1/swapfile swap swap defaults 0 0“

14、排序

ll -rt 按时间排序
ll -Sh 按大小排序

15、查看服务器硬件配置：

cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l
cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
cat /proc/cpuinfo| grep "cpu cores"| uniq
cat /proc/cpuinfo| grep "processor"| wc -l

16、查看nginx日志：

awk ‘{print $1}‘ 26.txt  | sort | uniq -c | sort -n -k 1 -r | head -n 100
awk ‘{print $11}‘ vm.txt  | sort | uniq -c | sort -n -k 1 -r | head -n 100

ps aux |grep -v USER | sort -nk +4 | tail

显示消耗内存最多的10个运行中的进程，以内存使用量排序 cpu+3

netstat -n | awk ‘/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}‘ | sort -n -t 2

SYN_RECV # 正在等待处理的请求
ESTABLISHED # 正常数据传输状态,既当前并发数
TIME_WAIT # 处理完毕，等待超时结束的请求
CLOSE_WAIT # 客户端异常关闭,没有完成4次挥手如大量可能存在×××行为

Active connections: 2 
server accepts handled requests
 6 6 25 
Reading: 0 Writing: 1 Waiting: 1

active connections – 活跃的连接数量
server accepts handled requests — 从nginx启动到现在总共处理了11989个连接 , 成功创建11989次握手, 总共处理了11991个请求
reading — 读取客户端的连接数.
writing — 响应数据到客户端的数量
waiting — 开启 keep-alive 的情况下,这个值等于 active – (reading+writing), 意思就是 Nginx 已经处理完正在等候下一次请求指令的驻留连接.

NGINX安装参数：

--prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt=‘-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic‘

17、CentOS7网卡设置：

[[email protected] ~]# more /etc/sysconfig/network-scripts/ifcfg-ens33

TYPE="Ethernet"
BOOTPROTO="none"
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens33"
UUID="fde08d92-9adf-4eb6-89c6-918445ebdf53"
DEVICE="ens33"
ONBOOT="yes"
IPADDR0=192.168.220.128
PREFIX0=24
GATEWAY0=192.168.220.2
DNS1=192.168.220.2

18、VSFTP相关：

useradd virtual -d /mnt/mfs -s /bin/false
chown virtual:virtual /mnt/mfs -R

sed -i "s/anonymous_enable=YES/anonymous_enable=NO/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#anon_upload_enable=YES/anon_upload_enable=NO/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#anon_mkdir_write_enable=YES/anon_mkdir_write_enable=YES/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#chown_uploads=YES/chown_uploads=NO/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#async_abor_enable=YES/async_abor_enable=YES/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#ascii_upload_enable=YES/ascii_upload_enable=YES/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#ascii_download_enable=YES/ascii_download_enable=YES/g" ‘/etc/vsftpd/vsftpd.conf‘
sed -i "s/#ftpd_banner=Welcome to blah FTP service./ftpd_banner=Welcome to FTP service./g" ‘/etc/vsftpd/vsftpd.conf‘

echo -e "use_localtime=YES\nlisten_port=21\nchroot_local_user=YES\nidle_session_timeout=300
\ndata_connection_timeout=1\nguest_enable=YES\nguest_username=vsftpd
\nuser_config_dir=/etc/vsftpd/vconf\nvirtual_use_local_privs=YES
\npasv_min_port=10060\npasv_max_port=10090
\naccept_timeout=5\nconnect_timeout=1" >> /etc/vsftpd/vsftpd.conf

[[email protected] vsftpd]# grep -Ev "#|^$" vsftpd.conf

anonymous_enable=YES
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
ascii_upload_enable=YES
ascii_download_enable=YES
listen=NO
listen_ipv6=YES
pam_service_name=vsftpd
userlist_enable=NO
tcp_wrappers=YES
guest_enable=YES
guest_username=virtual
user_config_dir=/etc/vsftpd/vsftpd_user_conf
[[email protected] vsftpd]# pwd
/etc/vsftpd

[[email protected] pam.d]# more vsftpd

#%PAM-1.0
auth required pam_userdb.so db=/etc/vsftpd/vsftpd_login
account required pam_userdb.so db=/etc/vsftpd/vsftpd_login

#session    optional     pam_keyinit.so    force revoke
#auth       required    pam_listfile.so item=user sense=deny file=/etc/vsftpd/ftpusers onerr=succeed
#auth       required    pam_shells.so
#auth       include     password-auth
#account    include     password-auth
#session    required     pam_loginuid.so
#session    include     password-auth

[[email protected] pam.d]# pwd
/etc/pam.d

[[email protected] vsftpd]# more vusers.list

ftpuser
123456

[[email protected] vsftpd]# cd vsftpd_user_conf/
[[email protected] vsftpd_user_conf]# ls
ftpuser
[[email protected] vsftpd_user_conf]# more ftpuser

local_root=/usr/share/nginx/html
write_enable=YES
virtual_use_local_privs=YES

db_load -T -t hash -f /etc/vsftpd/vusers.list /etc/vsftpd/vsftpd_login.db

参考：
http://yuanbin.blog.51cto.com/363003/129071/

19、grep：

grep -A 5 可以显示匹配内容以及后面的5行内容
grep -B 5 可以显示匹配内容以及前面的5行内容
grep -C 5 可以显示匹配内容以及前后面的5行内容

find / -size +200M -exec ls -lh {} \;

/etc/my.conf 文件属于哪个包 (package) :

rpm -qf /etc/my.conf

20、SAMBA:

yum -y install samba samba-client
mkdir /home/share
chmod 777 /home/share/
cd /etc/samba/
cp smb.conf smb.conf_bak
more smb.conf

[global]
        workgroup = WORKGROUP
        security = user

        passdb backend = tdbsam

        printing = cups
        printcap name = cups
        load printers = yes
        cups options = raw

[homes]
        comment = Home Directories
        valid users = %S, %D%w%S
        browseable = No
        read only = No
        inherit acls = Yes

[printers]
        comment = All Printers
        path = /var/tmp
        printable = Yes
        create mask = 0600
        browseable = No

[print$]
        comment = Printer Drivers
        path = /var/lib/samba/drivers
        write list = root
        create mask = 0664
        directory mask = 0775

[work]
        path = /home/share
        writable = yes
        validuser = smbuser
        browseable = yes

useradd -s /sbin/nologin smbuser
smbpasswd -a smbuser
systemctl restart smb nmb

21、IPTABLES：

iptables -t nat -A OUTPUT -d 172.16.170.55 -j DNAT --to 172.16.170.168

将对55的访问定向到168

22、安装xenserver tool：

点击XenServer Tools not install，然后然后选择 Install XenServer Tools。
mount /dev/xvdd /mnt
/mnt/Linux/install.sh

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

23、2960交换机开启snmp：

Switch(config)#snmp-server community public ro
Switch(config)#snmp-server enable traps

24、免密：

在A上的命令:
#ssh-keygen -t rsa (连续三次回车,即在本地生成了公钥和私钥,不设置密码)
#ssh [email protected] "mkdir .ssh;chmod 0700 .ssh" (需要输入密码，注:必须将.ssh的权限设为700)
#scp ~/.ssh/id_rsa.pub [email protected]:.ssh/id_rsa.pub (需要输入密码)

在B上的命令:
#touch /root/.ssh/authorized_keys (如果已经存在这个文件, 跳过这条)
#chmod 600 ~/.ssh/authorized_keys (# 注意：必须将~/.ssh/authorized_keys的权限改为600, 该文件用于保存ssh客户端生成的公钥，可以修改服务器的ssh服务端配置文件/etc/ssh/sshd_config来指定其他文件名）
#cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys (将id_rsa.pub的内容追加到 authorized_keys 中, 注意不要用 > ，否则会清空原有的内容，使其他人无法使用原有的密钥登录)

回到A机器:
#ssh [email protected] (不需要密码, 登录成功)

25、私有云服务器扩展磁盘空间：

fdisk /dev/sda
[[email protected] dir]# fdisk /dev/sdc #给新加硬分区

WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to
         switch off the mode (command ‘c‘) and change display units to
         sectors (command ‘u‘).

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1307-2610, default 1307):
Using default value 1307
Last cylinder, +cylinders or +size{K,M,G} (1307-2610, default 2610): +5G

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdc: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe4571f5c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        1306    10490413+  8e  Linux LVM  
/dev/sdc2            1307        1960     5253255   8e  Linux LVM    ##新建的分区/dev/sdc2，将用于LVM扩容

w

partprobe

lvextend -L 190G /dev/centos/root
[[email protected] dir]# pvcreate /dev/sdc2 #将分区/dev/sdc2创建为物理卷
Physical volume "/dev/sdc2" successfully created
[[email protected] dir]# vgextend vg_server1 /dev/sdc2 #将物理卷/dev/sdc2 加入卷组vg_server1
Volume group "vg_server1" successfully extended
[[email protected] dir]# lvextend -L +5G /dev/vg_server1/vg_server1_lvm1 /dev/sdc2 #扩容/dev/vg_server1/vg_server1_lvm1逻辑卷，其中-L指定大小
#+5G 容量增加5G ,如果没有”+“ 表示扩展到多少
Size of logical volume vg_server1/vg_server1_lvm1 changed from 15.00 GiB (3840 extents) to 20.00 GiB (5120 extents).
Logical volume vg_server1_lvm1 successfully resized

/sbin/resize2fs /dev/centos/root
lvextend 

xfs_growfs /dev/centos/root
df -hT

fdisk /dev/sda 
partprobe

pvcreate /dev/sda3
vgextend centos /dev/sda3
lvextend -L +595G /dev/centos/root /dev/sda3
xfs_growfs /dev/centos/root 
df -h

/sbin/resize2fs /dev/centos/root 
lvextend

26、阿里云服务器扩展磁盘空间：

fdisk /dev/vdb 
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p
w
partprobe

pvcreate /dev/vdb1
vgcreate elk /dev/vdb1
lvcreate -l 100%FREE -n lv_data elk
mkdir /data
mkfs -t ext4 /dev/elk/lv_data 
mount /dev/elk/lv_data /data/
df -h

more /etc/fstab
/dev/elk/lv_data /data ext4 defaults 1 1

扩容：
创建pv ,通过pvcreate命令将磁盘/dev/sdb创建为一个系统PV
#pvcreate /dev/sdb
将 PV /dev/sdb添加到卷组vg_openstack中。参考下图中标记的数据可知，磁盘已经添加到vg_openstack，而且卷组的空间增加了50G
#vgextend vg_openstack /dev/sdb
为/soft 添加30G的空间。图中标记的位置给出了/soft的大小，以及/soft的lv名称为 /dev/vg_openstack/LogVol04
#lvextend -L +30G -n /dev/vg_openstack/LogVol04
逻辑卷扩展后并不会马上生效，需要使用“resize2fs”命令重新加载逻辑卷的大小,图中可知，/soft已经添加30G空间
#resize2fs /dev/vg_openstack/LogVol04

yum -y install lvm2 docker

阿里云新增：

fdisk -l
cd /var/lib/
mv docker/ docker_bak
fdisk /dev/vdb
n
t
8e
w
partprobe 
pvcreate /dev/vdb1 
vgcreate vg-docker /dev/vdb1
lvcreate -L 50G -n dockerdata vg-docker
lvcreate -L 5G -n metadata vg-docker
lvcreate -l 100%FREE -n data vg-docker
lvdisplay

mkdir /data
mkfs -t ext4 /dev/vg-docker/data
mount /dev/vg-docker/data  /data
vim /etc/fstab 
/dev/vg-docker/data /data                                         ext4    defaults        1 1

cd /etc/sysconfig/
cp docker docker_bak
systemctl start docker
systemctl enable docker
systemctl status docker
docker info

27、redis：

http://blog.csdn.net/github_26672553/article/details/69568259
dbfilename "dump.rdb"
dir "/data"
requirepass ok

/usr/local/redis/bin/redis-server /usr/local/redis/conf/6379.conf

/usr/local/bin/redis-server /etc/redis.conf &
/usr/local/bin/redis-cli -h localhost -p 6379 -a ok shutdown
/usr/local/bin/redis-cli -p 6380 -a ok

28、JDK：

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
rpm -ivh jdk1.8
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
yum install jenkins

Build Timestamp Plugin
Copy Artifact Plugin
GitHub Organization Folder Plugin
Javadoc Plugin
Maven Integration plugin
promoted builds plugin
Publish Over SSH
Role-based Authorization Strategy

29、confluence:

申请试用[email protected]

30、jira:

yum install http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm -y
yum clean all
yum install Percona-Server-server-56 -y

or:

wget -c https://www.percona.com/downloads/Percona-Server-5.6/Percona-Server-5.6.36-82.0/binary/redhat/7/x86_64/Percona-Server-5.6.36-82.0-r58e846a-el7-x86_64-bundle.tar

CREATE DATABASE jiradb CHARACTER SET utf8 COLLATE utf8_bin;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on <JIRADB>.* TO ‘<USERNAME>‘@‘<JIRA_SERVER_HOSTNAME>‘ IDENTIFIED BY ‘<PASSWORD>‘;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on jiradb.* TO ‘jira‘@‘127.0.0.1‘ IDENTIFIED BY ‘jiradb‘;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on jiradb.* TO ‘jira‘@‘localhost‘ IDENTIFIED BY ‘jiradb‘;

flush privileges;

SHOW GRANTS FOR <USERNAME>@<JIRA_SERVER_HOSTNAME>;

[mysqld]
...
default-storage-engine=INNODB
...

[mysqld]
...
max_allowed_packet=256M
...

// remove this if it exists
sql_mode = NO_AUTO_VALUE_ON_ZERO

https://dev.mysql.com/downloads/connector/j/
mysql-connector-java-5.1.42.jar

/opt/atlassian/jira/atlassian-jira/WEB-INF/lib

以上是关于运维工作中经常用到的一些知识总结的主要内容，如果未能解决你的问题，请参考以下文章

运维工作中经常用到的一些知识总结

总结ASP.NET C#中经常用到的13个JS脚本代码

(转)Android学习路线总结，绝对干货

测试工作中经常用到的几个Linux命令（第一弹）