centos7 安装esrally 踩坑过程

Posted bohu83

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了centos7 安装esrally 踩坑过程相关的知识,希望对你有一定的参考价值。

背景:

为了压测下新ES集群性能,使用es官网推荐的Esrally。因为网络差异, 按照官网的步骤来,一步一个坑那基本上就是下载很慢,半天过去爬不出来。

所以那个简洁的命令有很多依赖前置条件。不然就是装不上。

pip安装方式:

需要依赖python3,很多大佬用了Python3.8的版本。我偷个懒,也是给自己挖了坑。

依赖条件:

<1>. 安装Python3.5及以上版本,系统默认可能是2.x版本,

<2>. 安装git1.9及以上版本

<3>. JDK 1.8+ (下载解压、配置环境变量即可)

对于很多后端同学,jdk解压配置不用说了,网上很多。

我基本上安装过程参照了干货:Elasticsearch 压测工具之 esrally的安装与使用(上) - 腾讯云开发者社区-腾讯云

区别是我的python 版本是3.7.14。到了pip3 install esrally就不行了

我是root账户,提示:

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

 所以,采用了虚拟环境来替代

 yum install python-virtualenv

python3 -m venv xp-env

这种最后的xp-env 自己起名字就好

加载虚拟环境

source xp-env/bin/activate

我为啥要单独测试bz2.

进行安装:pip install esrally

这个过程有点长。

不能有warning,执行esrally

之前安装完就报了 

Traceback (most recent call last):
  File "/usr/local/src/venv/bin/esrally", line 5, in <module>
    from esrally.rally import main
  File "/usr/local/src/venv/lib/python3.7/site-packages/esrally/rally.py", line 27, in <module>
    from esrally import PROGRAM_NAME, BANNER, SKULL, check_python_version, doc_link, telemetry
  File "/usr/local/src/venv/lib/python3.7/site-packages/esrally/telemetry.py", line 25, in <module>
    from esrally import metrics, time, exceptions
  File "/usr/local/src/venv/lib/python3.7/site-packages/esrally/metrics.py", line 33, in <module>
    from esrally import time, exceptions, config, version, paths
  File "/usr/local/src/venv/lib/python3.7/site-packages/esrally/config.py", line 27, in <module>
    from esrally.utils import io, git, console, convert
  File "/usr/local/src/venv/lib/python3.7/site-packages/esrally/utils/io.py", line 18, in <module>
    import bz2
  File "/root/training/Python-3.7.14/lib/python3.7/bz2.py", line 19, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'

尝试使用pip3安装不行

  1. pip3 install bz2

改为下载源码进行安装


wget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
tar -zxf  bzip2-1.0.6.tar.gz 
 
cd bzip2-1.0.6  
 
make -f  Makefile-libbz2_so 
 
make && make install

还得再次安装Python,这个很耗时,所以很麻烦。

得验证到上面import bz2 不报错,下面这种不行

Python 3.7.14 (default, Oct  9 2022, 10:52:02) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bz2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/training/Python-3.7.14/lib/python3.7/bz2.py", line 19, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'

解决了bz2的问题后,有报了别的错

Traceback (most recent call last):
  File "/usr/local/xp-env/bin/esrally", line 5, in <module>
    from esrally.rally import main
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/rally.py", line 28, in <module>
    from esrally import version, actor, config, paths, racecontrol, reporter, metrics, track, chart_generator, exceptions, \\
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/racecontrol.py", line 26, in <module>
    from esrally import actor, config, doc_link, driver, exceptions, mechanic, metrics, reporter, track, PROGRAM_NAME
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/driver/__init__.py", line 19, in <module>
    from .driver import DriverActor, PrepareBenchmark, PreparationComplete, StartBenchmark, BenchmarkComplete, TaskFinished
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/driver/driver.py", line 27, in <module>
    from esrally import actor, config, exceptions, metrics, track, client, paths, PROGRAM_NAME, telemetry
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/track/__init__.py", line 18, in <module>
    from .loader import (
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/track/loader.py", line 26, in <module>
    import jinja2
  File "/usr/local/xp-env/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/usr/local/xp-env/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/usr/local/xp-env/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
    from jinja2.utils import Markup
  File "/usr/local/xp-env/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/xp-env/lib/python3.7/site-packages/markupsafe/__init__.py)

这个,看了大佬的办法不一样,我试了pip install --upgrade aws-sam-cli 

还有一些其他错误,可能python版本不同不太一样

Traceback (most recent call last):
  File "/usr/local/xp-env/bin/esrally", line 5, in <module>
    from esrally.rally import main
  File "/usr/local/xp-env/lib/python3.7/site-packages/esrally/__init__.py", line 24, in <module>
    __version__ = pkg_resources.require("esrally")[0].version
  File "/usr/local/xp-env/lib/python3.7/site-packages/pkg_resources/__init__.py", line 909, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/xp-env/lib/python3.7/site-packages/pkg_resources/__init__.py", line 800, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (boto3 1.24.89 (/usr/local/xp-env/lib/python3.7/site-packages), Requirement.parse('boto3==1.10.32'), 'esrally')

这种,就是重新pip装,指定的版本就行。

最后,终于跑起来了

 

以上是关于centos7 安装esrally 踩坑过程的主要内容,如果未能解决你的问题,请参考以下文章

centos7 安装esrally 踩坑过程

推荐一款es轻量级的压测工具

推荐一款es轻量级的压测工具

使用u盘安装centos7.2踩坑过程

CentOS7.4安装MySQL踩坑记录

RabbitMq在CentOs7下的完整安装步骤,带你踩坑