mydumper 安装

Posted P10ZHUO

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了mydumper 安装相关的知识，希望对你有一定的参考价值。

mydumper 安装


mydumper是社区开源产品，支持并行导出，导入。功能单一，当我们在进行逻辑备份时，可以使用它进行单纯的数据备份。但是它不是mysql自带的工具，需要我们自己进行安装。下面就说下安装过程

编译安装

版本信息：mydumper 0.11.6, built against MySQL 8.0.27
os：RHEL 7.6

下载安装包

github下载地址：https://github.com/mydumper/mydumper

这是最新版，我们看到11天前才更新过的。下载下来即可。

上传软件

[root@postgre ~]# cd /usr/local/
[root@postgre local]# ll mydumper-master.zip 
-rw-r--r-- 1 root root 137471 Dec 31 20:48 mydumper-master.zip

解压

[root@postgre local]# unzip mydumper-master.zip
[root@postgre local]# ll mydumper-master

安装依赖包

[root@postgre local]# yum install pcre-devel.x86_64 glib2-devel zlib-devel cmake

cmake编译

cmake工程一般都不在源码目录直接编译，通常新建目录进行编译，配置失败可以直接删除新目录再重新配置。

比如有个hello工程。

cd hello
mkdir build
cd build
cmake ..
make

[root@postgre local]# cd mydumper-master/
[root@postgre mydumper-master]# mkdir build
[root@postgre mydumper-master]# cd build/
[root@postgre build]# cmake ../
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error: your C compiler: "CMAKE_C_COMPILER-NOTFOUND" was not found.   Please set CMAKE_C_COMPILER to a valid compiler path or name.
CMake Error: your CXX compiler: "CMAKE_CXX_COMPILER-NOTFOUND" was not found.   Please set CMAKE_CXX_COMPILER to a valid compiler path or name.
-- Using mysql-config: /usr/local/mysql/bin/mysql_config
-- MySQL not found.
-- Could NOT find ZLIB (missing:  ZLIB_LIBRARY) (found version "1.2.7")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1") 
-- checking for one of the modules 'glib-2.0'
-- checking for one of the modules 'gthread-2.0'
-- checking for one of the modules 'gio-2.0'
-- checking for one of the modules 'gobject-2.0'
-- checking for module 'libpcre'
--   found libpcre, version 8.32
-- Found PCRE: /usr/include  
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing:  OPENSSL_LIBRARIES OPENSSL_INCLUDE_DIR) 

CMake Warning at docs/CMakeLists.txt:9 (message):
  Unable to find Sphinx documentation generator


-- ------------------------------------------------
-- MYSQL_CONFIG = /usr/local/mysql/bin/mysql_config
-- CMAKE_INSTALL_PREFIX = /usr/local
-- BUILD_DOCS = ON
-- WITH_BINLOG = OFF
-- WITH_ZSTD = OFF
-- OpenSSL_FOUND = 
-- WITH_SSL = OFF
-- RUN_CPPCHECK = OFF
-- Change a values with: cmake -D<Variable>=<Value>
-- ------------------------------------------------
-- 
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
MYSQL_LIBRARIES_dl
    linked by target "mydumper" in directory /usr/local/mydumper-master
    linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_m
    linked by target "mydumper" in directory /usr/local/mydumper-master
    linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_pthread
    linked by target "mydumper" in directory /usr/local/mydumper-master
    linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_resolv
    linked by target "mydumper" in directory /usr/local/mydumper-master
    linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_rt
    linked by target "mydumper" in directory /usr/local/mydumper-master
    linked by target "myloader" in directory /usr/local/mydumper-master

-- Configuring incomplete, errors occurred!
See also "/usr/local/mydumper-master/build/CMakeFiles/CMakeOutput.log".
See also "/usr/local/mydumper-master/build/CMakeFiles/CMakeError.log".

报错：“CMake Error: The following variables are used in this project, but they are set to NOTFOUND.”。找不到环境变量。
查看具体报错日志：

[root@postgre bin]# more /usr/local/mydumper-master/build/CMakeFiles/CMakeError.log
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: 

The output was:
No such file or directory


Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: -c

The output was:
No such file or directory


Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: -Aa

The output was:
No such file or directory


Checking whether the C compiler is IAR using "" did not match "IAR .+ Compiler":
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND 
Build flags: 
Id flags: 

The output was:
No such file or directory


Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND 
Build flags: 
Id flags: -c

The output was:
No such file or directory


Checking whether the CXX compiler is IAR using "" did not match "IAR .+ Compiler":
[root@postgre bin]# more /usr/local/mydumper-master/build/CMakeFiles/CMakeError.log
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: 

The output was:
No such file or directory


Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: -c

The output was:
No such file or directory


Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND 
Build flags: 
Id flags: -Aa

The output was:
No such file or directory


Checking whether the C compiler is IAR using "" did not match "IAR .+ Compiler":
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND 
Build flags: 
Id flags: 

The output was:
No such file or directory


Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND 
Build flags: 
Id flags: -c

The output was:
No such file or directory


Checking whether the CXX compiler is IAR using "" did not match "IAR .+ Compiler":

参考：https://blog.csdn.net/cica0cica/article/details/84470347
linux上使用Cmake出现错误CMake错误CMAKE_CXX_COMPILER-NOTFOUND" was not found
要注意查看每一行错误日志
显示The CXX compiler identification is unknown
sudo yum install gcc-c++安装即可
安装gcc-c++ 包。
其实根据下载页面的readme，也能看出来我们装的包少了。

[root@postgre bin]# yum install gcc-c++

重新编译，还是一样的报错。
删掉之前编译的文件，重新编译。

[root@postgre build]# rm -rf *

[root@postgre build]# cmake ../
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Using mysql-config: /usr/local/mysql/bin/mysql_config
-- Found MySQL: /usr/local/mysql/include, /usr/local/mysql/lib/libmysqlclient.so;/usr/lib64/libpthread.so;/usr/lib64/libdl.so;/usr/local/mysql/lib/private/libcrypto.so;/usr/local/mysql/lib/private/libssl.so;/usr/lib64/libresolv.so;/usr/lib64/libm.so;/usr/lib64/librt.so
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7") 
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1") 
-- checking for one of the modules 'glib-2.0'
-- checking for one of the modules 'gthread-2.0'
-- checking for one of the modules 'gio-2.0'
-- checking for one of the modules 'gobject-2.0'
-- checking for module 'libpcre'
--   found libpcre, version 8.32
-- Found PCRE: /usr/include  
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing:  OPENSSL_LIBRARIES OPENSSL_INCLUDE_DIR) 

CMake Warning at docs/CMakeLists.txt:9 (message):
  Unable to find Sphinx documentation generator


-- ------------------------------------------------
-- MYSQL_CONFIG = /usr/local/mysql/bin/mysql_config
-- CMAKE_INSTALL_PREFIX = /usr/local
-- BUILD_DOCS = ON
-- WITH_BINLOG = OFF
-- WITH_ZSTD = OFF
-- OpenSSL_FOUND = 
-- WITH_SSL = OFF
-- RUN_CPPCHECK = OFF
-- Change a values with: cmake -D<Variable>=<Value>
-- ------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/mydumper-master/build

Configuring done，表示编译成功。

安装

[root@postgre build]# make
Scanning dependencies of target mydumper
[  6%] Building C object CMakeFiles/mydumper.dir/mydumper.c.o
[ 13%] Building C object CMakeFiles/mydumper.dir/server_detect.c.o
[ 20%] Building C object CMakeFiles/mydumper.dir/g_unix_signal.c.o
[ 26%] Building C object CMakeFiles/mydumper.dir/connection.c.o
[ 33%] Building C object CMakeFiles/mydumper.dir/getPassword.c.o
[ 40%] Building C object CMakeFiles/mydumper.dir/logging.c.o
[ 46%] Building C object CMakeFiles/mydumper.dir/set_verbose.c.o
[ 53%] Building C object CMakeFiles/mydumper.dir/common.c.o
Linking C executable mydumper
[ 53%] Built target mydumper
Scanning dependencies of target myloader
[ 60%] Building C object CMakeFiles/myloader.dir/myloader.c.o
[ 66%] Building C object CMakeFiles/myloader.dir/server_detect.c.o
[ 73%] Building C object CMakeFiles/myloader.dir/connection.c.o
[ 80%] Building C object CMakeFiles/myloader.dir/getPassword.c.o
[ 86%] Building C object CMakeFiles/myloader.dir/logging.c.o
[ 93%] Building C object CMakeFiles/myloader.dir/set_verbose.c.o
[100%] Building C object CMakeFiles/myloader.dir/common.c.o
Linking C executable myloader
[100%] Built target myloader
[root@postgre build]# make install
[ 53%] Built target mydumper
[100%] Built target myloader
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/bin/mydumper
-- Removed runtime path from "/usr/local/bin/mydumper"
-- Installing: /usr/local/bin/myloader
-- Removed runtime path from "/usr/local/bin/myloader"

验证缺少库文件

[root@postgre build]# ldd /usr/local/bin/mydumper 
        linux-vdso.so.1 =>  (0x00007fff71fea000)
        libmysqlclient.so.21 => not found
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9862a31000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f986282d000)
        libcrypto.so.1.1 => not found
        libssl.so.1.1 => not found
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f9862614000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f9862312000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f986210a000)
        libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f9861df4000)
        libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007f9861bf2000)
        libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f9861853000)
        libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f9861603000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f98613a1000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f986118b000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f9860e84000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f9860ab7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9862c4d000)
        libffi.so.6 => /lib64/libffi.so.6 (0x00007f98608af000)
        libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f98606ab000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f9860484000)
        libmount.so.1 => /lib64/libmount.so.1 (0x00007f9860241000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f986002b000)
        libblkid.so.1 => /lib64/libblkid.so.1 (0x00007f985fdeb000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f985fbe6000)

做软链接，识别库文件.
我们会发现，mydumper使用的库文件和mysqld进程使用的库文件一致，所以软连接过去即可。
注意路径是 ：/lib64。

[root@postgre bin]# find / -name libmysqlclient.so.21
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21
[root@postgre bin]# find / -name libssl.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1
[root@postgre bin]# find / -name libcrypto.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21 /lib64/libmysqlclient.so.21
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1 /lib64/libssl.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1 /lib64/libcrypto.so.1.1
[root@postgre bin]# ldd /usr/local/bin/mydumper      
        linux-vdso.so.1 =>  (0x00007ffe07aa9000)
        libmysqlclient.so.21 => /lib64/libmysqlclient.so.21 (0x00007ff1ed6f4000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff1ed4d8000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff1ed2d4000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007ff1ece22000)
        libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007ff1ecb92000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ff1ec979000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff1ec677000)
        librt.so.1 => /lib64/librt.so.1 (0x00007ff1ec46f000)
        libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007ff1ec159000)
        libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007ff1ebf57000)
        libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007ff1ebbb8000)
        libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007ff1eb968000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007ff1eb706000)
        libz.so.1 => /lib64/libz.so.1 (0x00007ff1eb4f0000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff1eb1e9000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff1eae1c000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff1eac06000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff1ee037000)
        libffi.so.6 => /lib64/libffi.so.6 (0x00007ff1ea9fe000)
        libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007ff1ea7fa000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007ff1ea5d3000)
        libmount.so.1 => /lib64/libmount.so.1 (0x00007ff1ea390000)
        libblkid.so.1 => /lib64/libblkid.so.1 (0x00007ff1ea150000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007ff1e9f4b000)

测试

[root@postgre test_3306_full]# /usr/local/bin/mydumper --version
mydumper 0.11.6, built against MySQL 8.0.27
[root@postgre build]# cd 
[root@postgre ~]# mydumper -S /tmp/mysql3306.sock -u root -p mysql -o test_3306_full
[root@postgre ~]# ll

总结

所以整个过程：
1、上传文件
[root@postgre ~]# cd /usr/local/
[root@postgre local]# ll mydumper-master.zip
-rw-r–r-- 1 root root 137471 Dec 31 20:48 mydumper-master.zip
2、解压
[root@postgre local]# unzip mydumper-master.zip
[root@postgre local]# ll mydumper-master
3、安装依赖包
[root@postgre local]# yum install zlib-devel pcre-devel.x86_64 glib2-devel zlib-devel cmake gcc-c++
4、cmake编译
[root@postgre local]# cd mydumper-master/
[root@postgre mydumper-master]# mkdir build
[root@postgre mydumper-master]# cd build/
[root@postgre build]# cmake …/
5、安装
[root@postgre build]# make
[root@postgre build]# make install
6、验证缺少库文件：
[root@postgre build]# ldd /usr/local/bin/mydumper
7、链接库文件
[root@postgre bin]# find / -name libmysqlclient.so.21
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21
[root@postgre bin]# find / -name libssl.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1
[root@postgre bin]# find / -name libcrypto.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21 /lib64/libmysqlclient.so.21
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1 /lib64/libssl.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1 /lib64/libcrypto.so.1.1

rpm包安装

按照github的步骤：https://github.com/mydumper/mydumper

下载安装包

github下载地址：https://github.com/mydumper/mydumper/releases

安装

[root@test28 ~]# rpm -ivh mydumper-0.11.5-1.el7.x86_64.rpm

测试

[root@test28 ~]# mydumper -S /tmp/mysql3306.sock -u root -p mysql -o test_3306_full
[root@test28 ~]# ls -ltr
total 2368
-rw-------. 1 root root    1414 Dec 27 19:58 anaconda-ks.cfg
drwxr-xr-x. 3 root root      17 Dec 30 19:30 gauss_om
-rw-r--r--. 1 root root 1219359 Dec 31 12:30 db.sql
-rw-r--r--. 1 root root 1178157 Dec 31 12:53 db3306.sql
drwxr-x---. 2 root root   16384 Dec 31 19:55 test_3306_full

[root@test28 ~]# cd test_3306_full/
[root@test28 test_3306_full]# ll
total 2320
-rw-r--r--. 1 root root     177 Dec 31 19:56 metadata
-rw-r--r--. 1 root root       1 Dec 31 19:56 mysql.columns_priv-metadata
-rw-r--r--. 1 root root     883 Dec 31 19:56 mysql.columns_priv-schema.sql
-rw-r--r--. 1 root root       1 Dec 31 19:56 mysql.component-metadata
-rw-r--r--. 1 root root     407 Dec 31 19:56 mysql.component-schema.sql
-rw-r--r--. 1 root root     363 Dec 31 19:56 mysql.db.00000.sql
。。。。
-rw-r--r--. 1 root root    1646 Dec 31 19:56 sys.waits_global_by_latency-schema-view.sql
-rw-r--r--. 1 root root     154 Dec 31 19:56 test1-schema-create.sql
-rw-r--r--. 1 root root       1 Dec 31 19:56 test1.test1-metadata
-rw-r--r--. 1 root root     245 Dec 31 19:56 test1.test1-schema.sql
-rw-r--r--. 1 root root     150 Dec 31 19:56 test1.zhuo.00000.sql
-rw-r--r--. 1 root root       1 Dec 31 19:56 test1.zhuo-metadata
-rw-r--r--. 1 root root     252 Dec 31 19:56 test1.zhuo-schema.sql
-rw-r--r--. 1 root root     152 Dec 31 19:56 test2.jinrui.00000.sql
-rw-r--r--. 1 root root       1 Dec 31 19:56 test2.jinrui-metadata
-rw-r--r--. 1 root root     254 Dec 31 19:56 test2.jinrui-schema.sql
-rw-r--r--. 1 root root     154 Dec 31 19:56 test2-schema-create.sql
[root@test28 local]# /usr/bin/mydumper --version
mydumper 0.11.5, built against MySQL 5.7.34-37

mydumper导出生成个文件夹，文件夹里面都是数据库和表对应的DDL。
查询rpm安装的版本，是0.11.5，但是怎么是安装在MySQL 5.7.34-37这个版本的。而我们的数据库版本是Mysql 8.0.27.

遗留问题

1、网上说mydumper不支持MySQL 8.0，编译安装的时候会报错。整个类库不识别MySQL 8.0的，只能用MySQL 5.7的类库来代替。但是我们安装并没有出现此情况，不知道为什么？
（可能由于我安装的mydumper版本比较新，后面又正常支持了）
2、如上，rpm包和编译安装均能成功。既然rpm安装很简单，一条命令搞定，而且功能一样。为什么还要用编译安装的方法来安装哪？
（可能为了多使用一种方法，两者应该么区别）
3、rpm安装的mydumper版本正确，我们MySQL版本是8.0.27，但安装后的版本信息是安装在MySQL 5.0.31上面？

以上是关于mydumper 安装的主要内容，如果未能解决你的问题，请参考以下文章

mydumper 安装

mydumper 安装

mydumper安装及使用

my22_mydumper 注意事项

mydumper安装及安装故障汇总

mydumper安装笔记