mydumper 安装
Posted P10ZHUO
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了mydumper 安装相关的知识，希望对你有一定的参考价值。
mydumper 安装
mydumper是社区开源产品，支持并行导出，导入。功能单一，当我们在进行逻辑备份时，可以使用它进行单纯的数据备份。但是它不是mysql自带的工具，需要我们自己进行安装。下面就说下安装过程
编译安装
版本信息：mydumper 0.11.6, built against MySQL 8.0.27
os：RHEL 7.6
下载安装包
github下载地址：https://github.com/mydumper/mydumper
这是最新版，我们看到11天前才更新过的。下载下来即可。
上传软件
[root@postgre ~]# cd /usr/local/
[root@postgre local]# ll mydumper-master.zip
-rw-r--r-- 1 root root 137471 Dec 31 20:48 mydumper-master.zip
解压
[root@postgre local]# unzip mydumper-master.zip
[root@postgre local]# ll mydumper-master
安装依赖包
[root@postgre local]# yum install pcre-devel.x86_64 glib2-devel zlib-devel cmake
cmake编译
cmake工程一般都不在源码目录直接编译，通常新建目录进行编译，配置失败可以直接删除新目录再重新配置。
比如有个hello工程。
cd hello
mkdir build
cd build
cmake ..
make
[root@postgre local]# cd mydumper-master/
[root@postgre mydumper-master]# mkdir build
[root@postgre mydumper-master]# cd build/
[root@postgre build]# cmake ../
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error: your C compiler: "CMAKE_C_COMPILER-NOTFOUND" was not found. Please set CMAKE_C_COMPILER to a valid compiler path or name.
CMake Error: your CXX compiler: "CMAKE_CXX_COMPILER-NOTFOUND" was not found. Please set CMAKE_CXX_COMPILER to a valid compiler path or name.
-- Using mysql-config: /usr/local/mysql/bin/mysql_config
-- MySQL not found.
-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.7")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
-- checking for one of the modules 'glib-2.0'
-- checking for one of the modules 'gthread-2.0'
-- checking for one of the modules 'gio-2.0'
-- checking for one of the modules 'gobject-2.0'
-- checking for module 'libpcre'
-- found libpcre, version 8.32
-- Found PCRE: /usr/include
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_LIBRARIES OPENSSL_INCLUDE_DIR)
CMake Warning at docs/CMakeLists.txt:9 (message):
Unable to find Sphinx documentation generator
-- ------------------------------------------------
-- MYSQL_CONFIG = /usr/local/mysql/bin/mysql_config
-- CMAKE_INSTALL_PREFIX = /usr/local
-- BUILD_DOCS = ON
-- WITH_BINLOG = OFF
-- WITH_ZSTD = OFF
-- OpenSSL_FOUND =
-- WITH_SSL = OFF
-- RUN_CPPCHECK = OFF
-- Change a values with: cmake -D<Variable>=<Value>
-- ------------------------------------------------
--
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
MYSQL_LIBRARIES_dl
linked by target "mydumper" in directory /usr/local/mydumper-master
linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_m
linked by target "mydumper" in directory /usr/local/mydumper-master
linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_pthread
linked by target "mydumper" in directory /usr/local/mydumper-master
linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_resolv
linked by target "mydumper" in directory /usr/local/mydumper-master
linked by target "myloader" in directory /usr/local/mydumper-master
MYSQL_LIBRARIES_rt
linked by target "mydumper" in directory /usr/local/mydumper-master
linked by target "myloader" in directory /usr/local/mydumper-master
-- Configuring incomplete, errors occurred!
See also "/usr/local/mydumper-master/build/CMakeFiles/CMakeOutput.log".
See also "/usr/local/mydumper-master/build/CMakeFiles/CMakeError.log".
报错：“CMake Error: The following variables are used in this project, but they are set to NOTFOUND.”。找不到环境变量。
查看具体报错日志：
[root@postgre bin]# more /usr/local/mydumper-master/build/CMakeFiles/CMakeError.log
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags:
The output was:
No such file or directory
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags: -c
The output was:
No such file or directory
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags: -Aa
The output was:
No such file or directory
Checking whether the C compiler is IAR using "" did not match "IAR .+ Compiler":
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND
Build flags:
Id flags:
The output was:
No such file or directory
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND
Build flags:
Id flags: -c
The output was:
No such file or directory
Checking whether the CXX compiler is IAR using "" did not match "IAR .+ Compiler":
[root@postgre bin]# more /usr/local/mydumper-master/build/CMakeFiles/CMakeError.log
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags:
The output was:
No such file or directory
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags: -c
The output was:
No such file or directory
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: CMAKE_C_COMPILER-NOTFOUND
Build flags:
Id flags: -Aa
The output was:
No such file or directory
Checking whether the C compiler is IAR using "" did not match "IAR .+ Compiler":
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND
Build flags:
Id flags:
The output was:
No such file or directory
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: CMAKE_CXX_COMPILER-NOTFOUND
Build flags:
Id flags: -c
The output was:
No such file or directory
Checking whether the CXX compiler is IAR using "" did not match "IAR .+ Compiler":
参考：https://blog.csdn.net/cica0cica/article/details/84470347
linux上使用Cmake出现错误CMake错误CMAKE_CXX_COMPILER-NOTFOUND" was not found
要注意查看每一行错误日志
显示The CXX compiler identification is unknown
sudo yum install gcc-c++安装即可
安装gcc-c++ 包。
其实根据下载页面的readme，也能看出来我们装的包少了。
[root@postgre bin]# yum install gcc-c++
重新编译，还是一样的报错。
删掉之前编译的文件，重新编译。
[root@postgre build]# rm -rf *
[root@postgre build]# cmake ../
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Using mysql-config: /usr/local/mysql/bin/mysql_config
-- Found MySQL: /usr/local/mysql/include, /usr/local/mysql/lib/libmysqlclient.so;/usr/lib64/libpthread.so;/usr/lib64/libdl.so;/usr/local/mysql/lib/private/libcrypto.so;/usr/local/mysql/lib/private/libssl.so;/usr/lib64/libresolv.so;/usr/lib64/libm.so;/usr/lib64/librt.so
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
-- checking for one of the modules 'glib-2.0'
-- checking for one of the modules 'gthread-2.0'
-- checking for one of the modules 'gio-2.0'
-- checking for one of the modules 'gobject-2.0'
-- checking for module 'libpcre'
-- found libpcre, version 8.32
-- Found PCRE: /usr/include
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_LIBRARIES OPENSSL_INCLUDE_DIR)
CMake Warning at docs/CMakeLists.txt:9 (message):
Unable to find Sphinx documentation generator
-- ------------------------------------------------
-- MYSQL_CONFIG = /usr/local/mysql/bin/mysql_config
-- CMAKE_INSTALL_PREFIX = /usr/local
-- BUILD_DOCS = ON
-- WITH_BINLOG = OFF
-- WITH_ZSTD = OFF
-- OpenSSL_FOUND =
-- WITH_SSL = OFF
-- RUN_CPPCHECK = OFF
-- Change a values with: cmake -D<Variable>=<Value>
-- ------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/mydumper-master/build
Configuring done，表示编译成功。
安装
[root@postgre build]# make
Scanning dependencies of target mydumper
[ 6%] Building C object CMakeFiles/mydumper.dir/mydumper.c.o
[ 13%] Building C object CMakeFiles/mydumper.dir/server_detect.c.o
[ 20%] Building C object CMakeFiles/mydumper.dir/g_unix_signal.c.o
[ 26%] Building C object CMakeFiles/mydumper.dir/connection.c.o
[ 33%] Building C object CMakeFiles/mydumper.dir/getPassword.c.o
[ 40%] Building C object CMakeFiles/mydumper.dir/logging.c.o
[ 46%] Building C object CMakeFiles/mydumper.dir/set_verbose.c.o
[ 53%] Building C object CMakeFiles/mydumper.dir/common.c.o
Linking C executable mydumper
[ 53%] Built target mydumper
Scanning dependencies of target myloader
[ 60%] Building C object CMakeFiles/myloader.dir/myloader.c.o
[ 66%] Building C object CMakeFiles/myloader.dir/server_detect.c.o
[ 73%] Building C object CMakeFiles/myloader.dir/connection.c.o
[ 80%] Building C object CMakeFiles/myloader.dir/getPassword.c.o
[ 86%] Building C object CMakeFiles/myloader.dir/logging.c.o
[ 93%] Building C object CMakeFiles/myloader.dir/set_verbose.c.o
[100%] Building C object CMakeFiles/myloader.dir/common.c.o
Linking C executable myloader
[100%] Built target myloader
[root@postgre build]# make install
[ 53%] Built target mydumper
[100%] Built target myloader
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/bin/mydumper
-- Removed runtime path from "/usr/local/bin/mydumper"
-- Installing: /usr/local/bin/myloader
-- Removed runtime path from "/usr/local/bin/myloader"
验证缺少库文件
[root@postgre build]# ldd /usr/local/bin/mydumper
linux-vdso.so.1 => (0x00007fff71fea000)
libmysqlclient.so.21 => not found
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9862a31000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f986282d000)
libcrypto.so.1.1 => not found
libssl.so.1.1 => not found
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f9862614000)
libm.so.6 => /lib64/libm.so.6 (0x00007f9862312000)
librt.so.1 => /lib64/librt.so.1 (0x00007f986210a000)
libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f9861df4000)
libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007f9861bf2000)
libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f9861853000)
libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f9861603000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f98613a1000)
libz.so.1 => /lib64/libz.so.1 (0x00007f986118b000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f9860e84000)
libc.so.6 => /lib64/libc.so.6 (0x00007f9860ab7000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9862c4d000)
libffi.so.6 => /lib64/libffi.so.6 (0x00007f98608af000)
libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f98606ab000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f9860484000)
libmount.so.1 => /lib64/libmount.so.1 (0x00007f9860241000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f986002b000)
libblkid.so.1 => /lib64/libblkid.so.1 (0x00007f985fdeb000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f985fbe6000)
做软链接，识别库文件.
我们会发现，mydumper使用的库文件和mysqld进程使用的库文件一致，所以软连接过去即可。
注意路径是 ：/lib64。
[root@postgre bin]# find / -name libmysqlclient.so.21
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21
[root@postgre bin]# find / -name libssl.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1
[root@postgre bin]# find / -name libcrypto.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21 /lib64/libmysqlclient.so.21
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1 /lib64/libssl.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1 /lib64/libcrypto.so.1.1
[root@postgre bin]# ldd /usr/local/bin/mydumper
linux-vdso.so.1 => (0x00007ffe07aa9000)
libmysqlclient.so.21 => /lib64/libmysqlclient.so.21 (0x00007ff1ed6f4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff1ed4d8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ff1ed2d4000)
libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007ff1ece22000)
libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007ff1ecb92000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ff1ec979000)
libm.so.6 => /lib64/libm.so.6 (0x00007ff1ec677000)
librt.so.1 => /lib64/librt.so.1 (0x00007ff1ec46f000)
libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007ff1ec159000)
libgthread-2.0.so.0 => /lib64/libgthread-2.0.so.0 (0x00007ff1ebf57000)
libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007ff1ebbb8000)
libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007ff1eb968000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007ff1eb706000)
libz.so.1 => /lib64/libz.so.1 (0x00007ff1eb4f0000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff1eb1e9000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff1eae1c000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff1eac06000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff1ee037000)
libffi.so.6 => /lib64/libffi.so.6 (0x00007ff1ea9fe000)
libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007ff1ea7fa000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007ff1ea5d3000)
libmount.so.1 => /lib64/libmount.so.1 (0x00007ff1ea390000)
libblkid.so.1 => /lib64/libblkid.so.1 (0x00007ff1ea150000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007ff1e9f4b000)
测试
[root@postgre test_3306_full]# /usr/local/bin/mydumper --version
mydumper 0.11.6, built against MySQL 8.0.27
[root@postgre build]# cd
[root@postgre ~]# mydumper -S /tmp/mysql3306.sock -u root -p mysql -o test_3306_full
[root@postgre ~]# ll
总结
所以整个过程：
1、上传文件
[root@postgre ~]# cd /usr/local/
[root@postgre local]# ll mydumper-master.zip
-rw-r–r-- 1 root root 137471 Dec 31 20:48 mydumper-master.zip
2、解压
[root@postgre local]# unzip mydumper-master.zip
[root@postgre local]# ll mydumper-master
3、安装依赖包
[root@postgre local]# yum install zlib-devel pcre-devel.x86_64 glib2-devel zlib-devel cmake gcc-c++
4、cmake编译
[root@postgre local]# cd mydumper-master/
[root@postgre mydumper-master]# mkdir build
[root@postgre mydumper-master]# cd build/
[root@postgre build]# cmake …/
5、安装
[root@postgre build]# make
[root@postgre build]# make install
6、验证缺少库文件：
[root@postgre build]# ldd /usr/local/bin/mydumper
7、链接库文件
[root@postgre bin]# find / -name libmysqlclient.so.21
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21
[root@postgre bin]# find / -name libssl.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1
[root@postgre bin]# find / -name libcrypto.so.1.1
/soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/libmysqlclient.so.21 /lib64/libmysqlclient.so.21
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libssl.so.1.1 /lib64/libssl.so.1.1
[root@postgre bin]# ln -s /soft/mysql-8.0.27-linux-glibc2.12-x86_64/lib/private/libcrypto.so.1.1 /lib64/libcrypto.so.1.1
rpm包安装
按照github的步骤：https://github.com/mydumper/mydumper
下载安装包
github下载地址：https://github.com/mydumper/mydumper/releases
安装
[root@test28 ~]# rpm -ivh mydumper-0.11.5-1.el7.x86_64.rpm
测试
[root@test28 ~]# mydumper -S /tmp/mysql3306.sock -u root -p mysql -o test_3306_full
[root@test28 ~]# ls -ltr
total 2368
-rw-------. 1 root root 1414 Dec 27 19:58 anaconda-ks.cfg
drwxr-xr-x. 3 root root 17 Dec 30 19:30 gauss_om
-rw-r--r--. 1 root root 1219359 Dec 31 12:30 db.sql
-rw-r--r--. 1 root root 1178157 Dec 31 12:53 db3306.sql
drwxr-x---. 2 root root 16384 Dec 31 19:55 test_3306_full
[root@test28 ~]# cd test_3306_full/
[root@test28 test_3306_full]# ll
total 2320
-rw-r--r--. 1 root root 177 Dec 31 19:56 metadata
-rw-r--r--. 1 root root 1 Dec 31 19:56 mysql.columns_priv-metadata
-rw-r--r--. 1 root root 883 Dec 31 19:56 mysql.columns_priv-schema.sql
-rw-r--r--. 1 root root 1 Dec 31 19:56 mysql.component-metadata
-rw-r--r--. 1 root root 407 Dec 31 19:56 mysql.component-schema.sql
-rw-r--r--. 1 root root 363 Dec 31 19:56 mysql.db.00000.sql
。。。。
-rw-r--r--. 1 root root 1646 Dec 31 19:56 sys.waits_global_by_latency-schema-view.sql
-rw-r--r--. 1 root root 154 Dec 31 19:56 test1-schema-create.sql
-rw-r--r--. 1 root root 1 Dec 31 19:56 test1.test1-metadata
-rw-r--r--. 1 root root 245 Dec 31 19:56 test1.test1-schema.sql
-rw-r--r--. 1 root root 150 Dec 31 19:56 test1.zhuo.00000.sql
-rw-r--r--. 1 root root 1 Dec 31 19:56 test1.zhuo-metadata
-rw-r--r--. 1 root root 252 Dec 31 19:56 test1.zhuo-schema.sql
-rw-r--r--. 1 root root 152 Dec 31 19:56 test2.jinrui.00000.sql
-rw-r--r--. 1 root root 1 Dec 31 19:56 test2.jinrui-metadata
-rw-r--r--. 1 root root 254 Dec 31 19:56 test2.jinrui-schema.sql
-rw-r--r--. 1 root root 154 Dec 31 19:56 test2-schema-create.sql
[root@test28 local]# /usr/bin/mydumper --version
mydumper 0.11.5, built against MySQL 5.7.34-37
mydumper导出生成个文件夹，文件夹里面都是数据库和表对应的DDL。
查询rpm安装的版本，是0.11.5，但是怎么是安装在MySQL 5.7.34-37这个版本的。而我们的数据库版本是Mysql 8.0.27.
遗留问题
1、网上说mydumper不支持MySQL 8.0，编译安装的时候会报错。整个类库不识别MySQL 8.0的，只能用MySQL 5.7的类库来代替。但是我们安装并没有出现此情况，不知道为什么？
（可能由于我安装的mydumper版本比较新，后面又正常支持了）
2、如上，rpm包和编译安装均能成功。既然rpm安装很简单，一条命令搞定，而且功能一样。为什么还要用编译安装的方法来安装哪？
（可能为了多使用一种方法，两者应该么区别）
3、rpm安装的mydumper版本正确，我们MySQL版本是8.0.27，但安装后的版本信息是安装在MySQL 5.0.31上面？
以上是关于mydumper 安装的主要内容，如果未能解决你的问题，请参考以下文章