unison+inotify实现数据实时双向同步

Posted 2020-10-27

unison+inotify实现数据实时双向同步

1、unison简介

Unison是windows和unix平台下都可以使用的文件同步工具，它能使两个文件夹（本地或网络上的）保持内容的一致，也支持经由过程SSH、RSH和Socket同步支持双向同步。Unison有文字界面和图形界面，这里只介绍如何在文字界面下使用. unison拥有其它一些同步工具或文件系统的相同特性，也有自己的特点，可以跨平台使用，对内核和用户权限没有特别要求，unison是双向的，它能自动处理两分拷贝中更新没有冲突的部分，有冲突的部分将会显示出来让用户选择更新策略；只要是能连通的两台主机，就可以运行unison，可以直接使用socket连接或安全的ssh连接方式，对带宽的要求不高，使用类似rsync的压缩传输协议。

unison可以作为两台以上web服务器的双向同步工具，当其中一台web服务器宕机时，另一台web服务器可以正常工作。

 

2、环境准备（本次操作系统为centos7）

（1）准备两台web服务器

web1:192.168.115.120  需要同步的目录为/data/www

web2:192.168.115.150  需要同步的目录为/data/www

 

注意：Unison对版本要求很高，进行同步的两台主机需要相同版本的unison

3、编译安装相关软件（两台机器都需安装）

（1）安装ocaml，版本至少为3.07或更高

[[email protected] ~]# yum -y install gcc gcc-c++

[[email protected] ~]# wget http://caml.inria.fr/pub/distrib/ocaml-4.06/ocaml-4.06.1.tar.gz

[[email protected] ~]# tar -xf ocaml-4.06.1.tar.gz

[[email protected] ~]# cd ocaml-4.06.1

[[email protected] ocaml-4.06.1]# ./configure

[[email protected] ocaml-4.06.1]# make world opt

[[email protected] ocaml-4.06.1]# make install

 

（2）安装unison，两台机器要求版本一样

[[email protected] ~]# wget https://github.com/bcpierce00/unison/archive/v2.51.2.tar.gz

[[email protected] ~]# tar -xf v2.51.2.tar.gz

[[email protected] ~]# cd unison-2.51.2/

[[email protected] unison-2.51.2]# cd src/

[[email protected] src]# make UISTYLE=text THREADS=true

[[email protected] src]# cp unison /usr/local/bin/

[[email protected] src]# unison -version

unison version 2.51.2 (ocaml 4.06.1)

有版本信息出现，则安装成功

特别注意：这个软件不需要执行./configure， make的时候后面的选项也比较特殊，千万不要弄错了。最后也不需要执行make install

（3）安装inotify

[[email protected] ~]# wget https://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

[[email protected] ~]# tar -xf inotify-tools-3.14.tar.gz

[[email protected] ~]# cd inotify-tools-3.14

[[email protected] inotify-tools-3.14]# ./configure && make && make install

4、配置双机ssh信任

web1上生成密钥，不输入私钥密码并上传到web2服务器

[[email protected] ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

32:63:9f:92:7c:fc:7b:9d:8d:70:bc:f0:f1:a3:16:d1 [email protected]

The key's randomart image is:

+--[ RSA 2048]----+

|                 |

|                 |

|              .  |

|             . E |

|      = S    ..  |

|     o B .  o.+  |

|      + =    *.B |

|       o .  ..*.o|

|          oo... .|

+-----------------+

[[email protected] ~]# ssh-copy-id 192.168.115.150

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

[email protected]'s password:

 

Number of key(s) added: 1

 

Now try logging into the machine, with:   "ssh '192.168.115.150'"

and check to make sure that only the key(s) you wanted were added.

 

 

web2上生成密钥，不输入私钥密码并上传到web1服务器

[[email protected] ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

fd:9a:e2:75:39:03:27:d0:01:e3:73:ca:2c:d3:51:f3 [email protected]

The key's randomart image is:

+--[ RSA 2048]----+

|        o.+      |

|       . + +     |

|        = o E    |

|       + B       |

|      o S + .    |

|       o   = .   |

|          . *    |

|        .. + o   |

|       ...o      |

+-----------------+

[[email protected] ~]# ssh-copy-id 192.168.115.120

The authenticity of host '192.168.115.120 (192.168.115.120)' can't be established.

ECDSA key fingerprint is c7:79:b3:87:3f:cf:d9:18:34:e7:38:bd:2d:8c:db:36.

Are you sure you want to continue connecting (yes/no)? yes

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

[email protected]'s password:

 

Number of key(s) added: 1

 

Now try logging into the machine, with:   "ssh '192.168.115.120'"

and check to make sure that only the key(s) you wanted were added.

在2台机器上分别ssh对方IP，能无密码登录则表示配置成功。

5、unison的使用

unison的用法非常灵活和简单，可以通过如下三种方式调用unison。
（1）第一种方式：”unison profile_name [options]”
unison默认会读取~/.unison目录下的配置文件”profile_name.prf”。

注意，在这种方式下，命令行中并没有指出要进行同步的两个地址，所以，此种调用unison的方式必须在配置文件profile_name.prf中通过相关的root指令设置同步的路径和同步的参数，如：

#Unison preferences file

root = /tmp/test

root = ssh://[email protected]//tmp/test/

#force =

#ignore =

batch = true

（2）第二种方式：”unison profile root1 root2 [options]”
root1、root2分别表示要执行同步的两个路径。这两个路径可以是本地目录路径，也可以是远程服务器的路径，如ssh://[email protected]//tmp/test 。由于同步的路径已经在命令行指定了，所以这里无需在profile.prf配置文件中进行root指令的相关设置。

（3）第三种方式：”unison root1 root2 [options]”
这种方式相当于执行”unison default root1 root2”命令，即unison默认读取default.prf的配置。

6、配置双机web目录同步

（1）web1上脚本

[[email protected] ~]# vim unison_inotify.sh

#!/bin/bash

src=/data/www/

host=192.168.115.150

cd ${src}

/usr/local/bin/inotifywait -mrq --format  '%Xe %w%f' -e modify,create,delete,attrib,close_write,move ./ | while read file

   do

/usr/local/bin/unison -batch $src ssh://$host/$src/

echo -n "$file " >> /var/log/inotify.log

echo `date | cut -d " " -f1-4` >> /var/log/inotify.log

done

[[email protected] ~]# chmod +x unison_inotify.sh

[[email protected] ~]# nohup ./unison_inotify.sh &

（2）web2上脚本

[[email protected] ~]# vim unison_inotify.sh

#!/bin/bash

src=/data/www/

host=192.168.115.120

cd ${src}

/usr/local/bin/inotifywait -mrq --format  '%Xe %w%f' -e modify,create,delete,attrib,close_write,move ./ | while read file

   do

/usr/local/bin/unison -batch $src ssh://$host/$src/

echo -n "$file " >> /var/log/inotify.log

echo `date | cut -d " " -f1-4` >> /var/log/inotify.log

done

[[email protected] ~]# chmod +x unison_inotify.sh    

[[email protected] ~]# nohup ./unison_inotify.sh &

 

最后分别在web1和web2上执行上面两个脚本，这样两台服务器的目录会保持相互实时同步了！

7、总结

unison+inotify的双向同步的确给我们带来了极大的方便，但同时也有一个缺点：对于一个文件在两个同步文件夹中都被修改时，unison是不会去同步的，因为unison无法判断以哪个为准，需要人工干预处理，这就需要我们自己权衡利弊。