pgbench 安装试用

Posted 2021-02-11 rongfengliang

pgbench 是一个方便的pg 性能测试工具，以下是简单的测试试用

安装

安装pg

yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm
yum install postgresql10
yum install postgresql10-server
yum install -y postgresql10-contrib

配置环境变量

默认安装是/usr/pgsql-10/bin，所以需要配置path 路径

/etc/profile
export PATH=$PATH:/usr/pgsql-10/bin

简单试用

启动参数

以下为支持的参数

pgbench is a benchmarking tool for PostgreSQL.

Usage:
  pgbench [OPTION]... [DBNAME]

Initialization options:
  -i, --initialize invokes initialization mode
  -F, --fillfactor=NUM set fill factor
  -n, --no-vacuum do not run VACUUM after initialization
  -q, --quiet quiet logging (one message each 5 seconds)
  -s, --scale=NUM scaling factor
  --foreign-keys create foreign key constraints between tables
  --index-tablespace=TABLESPACE
                           create indexes in the specified tablespace
  --tablespace=TABLESPACE create tables in the specified tablespace
  --unlogged-tables create tables as unlogged tables

Options to select what to run:
  -b, --builtin=NAME[@W] add builtin script NAME weighted at W (default: 1)
                           (use "-b list" to list available scripts)
  -f, --file=FILENAME[@W] add script FILENAME weighted at W (default: 1)
  -N, --skip-some-updates skip updates of pgbench_tellers and pgbench_branches
                           (same as "-b simple-update")
  -S, --select-only perform SELECT-only transactions
                           (same as "-b select-only")

Benchmarking options:
  -c, --client=NUM number of concurrent database clients (default: 1)
  -C, --connect establish new connection for each transaction
  -D, --define=VARNAME=VALUE
                           define variable for use by custom script
  -j, --jobs=NUM number of threads (default: 1)
  -l, --log write transaction times to log file
  -L, --latency-limit=NUM count transactions lasting more than NUM ms as late
  -M, --protocol=simple|extended|prepared
                           protocol for submitting queries (default: simple)
  -n, --no-vacuum do not run VACUUM before tests
  -P, --progress=NUM show thread progress report every NUM seconds
  -r, --report-latencies report average latency per command
  -R, --rate=NUM target rate in transactions per second
  -s, --scale=NUM report this scale factor in output
  -t, --transactions=NUM number of transactions each client runs (default: 10)
  -T, --time=NUM duration of benchmark test in seconds
  -v, --vacuum-all vacuum all four standard tables before tests
  --aggregate-interval=NUM aggregate data over NUM seconds
  --log-prefix=PREFIX prefix for transaction time log file
                           (default: "pgbench_log")
  --progress-timestamp use Unix epoch timestamps for progress
  --sampling-rate=NUM fraction of transactions to log (e.g., 0.01 for 1%)

Common options:
  -d, --debug print debugging output
  -h, --host=HOSTNAME database server host or socket directory
  -p, --port=PORT database server port number
  -U, --username=USERNAME connect as specified database user
  -V, --version output version information, then exit
  -?, --help show this help, then exit

Report bugs to <[email protected]>.

准备一个数据库

为了方便使用docker-compose 运行,内容有点多，包含了mysql pg 以及一个pgbouncer（连接池工具）

version: "3"
services:
  mysql:
    image: mysql:5.7.16
    volumes:
      - ./gogs/mysql:/var/lib/mysql
    ports:
      - 3306:3306
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    environment:
      MYSQL_ROOT_PASSWORD: dalongrong
      MYSQL_DATABASE: gogs
      MYSQL_USER: gogs
      MYSQL_PASSWORD: dalongrong
      TZ: Asia/Shanghai
  postgres:
    image: postgres:9.6
    ports:
    - "5432:5432"
    environment:
    - "POSTGRES_PASSWORD:dalong"
    volumes:
    - ./db_data2:/var/lib/postgresql/data
  pgbouncer:
    image: brainsam/pgbouncer
    environment:
    - DB_HOST=postgres
    - DB_USER=postgres
    - DB_PASSWORD=dalong
    ports:
    - "6432:6432"

启动数据库

docker-compose up -d

简单测试

• 初始化

pgbench -i --unlogged-tables -s 2 -U postgres -p 5432 -d postgres -h localhost

效果

pgbench -i --unlogged-tables -s 2 -U postgres -p 5432 -d postgres -h localhost
NOTICE: table "pgbench_history" does not exist, skipping
NOTICE: table "pgbench_tellers" does not exist, skipping
NOTICE: table "pgbench_accounts" does not exist, skipping
NOTICE: table "pgbench_branches" does not exist, skipping
creating tables...
100000 of 200000 tuples (50%) done (elapsed 0.03 s, remaining 0.03 s)
200000 of 200000 tuples (100%) done (elapsed 0.06 s, remaining 0.00 s)
vacuum...
set primary keys...
done.

pgbench -i --unlogged-tables -s 16 -U postgres -p 5432 -h localhost -d postgres
creating tables...
100000 of 1600000 tuples (6%) done (elapsed 0.03 s, remaining 0.42 s)
200000 of 1600000 tuples (12%) done (elapsed 0.06 s, remaining 0.40 s)
300000 of 1600000 tuples (18%) done (elapsed 0.09 s, remaining 0.37 s)
400000 of 1600000 tuples (25%) done (elapsed 0.14 s, remaining 0.41 s)
500000 of 1600000 tuples (31%) done (elapsed 0.17 s, remaining 0.37 s)
600000 of 1600000 tuples (37%) done (elapsed 0.20 s, remaining 0.33 s)
700000 of 1600000 tuples (43%) done (elapsed 0.23 s, remaining 0.29 s)
800000 of 1600000 tuples (50%) done (elapsed 0.25 s, remaining 0.25 s)
900000 of 1600000 tuples (56%) done (elapsed 0.28 s, remaining 0.22 s)
1000000 of 1600000 tuples (62%) done (elapsed 0.31 s, remaining 0.19 s)
1100000 of 1600000 tuples (68%) done (elapsed 0.42 s, remaining 0.19 s)
1200000 of 1600000 tuples (75%) done (elapsed 0.52 s, remaining 0.17 s)
1300000 of 1600000 tuples (81%) done (elapsed 0.75 s, remaining 0.17 s)
1400000 of 1600000 tuples (87%) done (elapsed 0.96 s, remaining 0.14 s)
1500000 of 1600000 tuples (93%) done (elapsed 1.15 s, remaining 0.08 s)
1600000 of 1600000 tuples (100%) done (elapsed 1.55 s, remaining 0.00 s)
vacuum...
set primary keys...
done.

• 执行压力测试

pgbench -M prepared -r -c 8 -j 2 -T 10 -U postgres -h localhost -p 5432 -d postgres -l

参数说明
以上参数中，-M prepared表示绑定变量形式的调用SQL，-r表示报告测试文件中每条SQL的平均执行延迟，
-c 8表示模拟8个客户端，-j 2表示pgbench的工作线程是2个，
-T 10表示压力测试的时间是10秒，-l表示把事务统计写入log，其余的是postgres连接相关的参数
以上参数，也可以结合命令帮助看出来
效果

scaling factor: 16
query mode: prepared
number of clients: 8
number of threads: 2
duration: 10 s
number of transactions actually processed: 4115
latency average = 19.924 ms
tps = 401.527925 (including connections establishing)
tps = 401.690314 (excluding connections establishing)
script statistics:
 - statement latencies in milliseconds:
         0.370 set aid random(1, 100000 * :scale)
         0.005 set bid random(1, 1 * :scale)
         0.005 set tid random(1, 10 * :scale)
         0.005 set delta random(-5000, 5000)
         3.051 BEGIN;
         2.684 UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         1.693 SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         2.799 UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         3.317 UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         3.203 INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
         2.740 END;

说明

我们主要比较关注的是tps 参数，结合一些系统调优的技巧进行tps 参数的调整，类似的sysbench 也是可以进行pg 压测的,以下为
sysbench 的简单测试命令

sysbench --db-driver=pgsql --pgsql-host=127.0.0.1 --pgsql-port=5432 --pgsql-user=postgres --pgsql-password=dalong --pgsql-db=postgres --oltp-table-size=10000 --rand-init=on --threads=10 --time=120 --events=0 --report-interval=10 --percentile=99 /usr/share/sysbench/tests/include/oltp_legacy/oltp.lua prepare

sysbench --db-driver=pgsql --pgsql-host=127.0.0.1 --pgsql-port=5432 --pgsql-user=postgres --pgsql-password=dalong --pgsql-db=postgres --oltp-table-size=10000 --threads=10 --time=120 --events=0 --report-interval=10 --percentile=99 /usr/share/sysbench/tests/include/oltp_legacy/oltp.lua run

sysbench --db-driver=pgsql --pgsql-host=127.0.0.1 --pgsql-port=5432 --pgsql-user=postgres --pgsql-password=dalong --pgsql-db=postgres --oltp-table-size=10000 --threads=10 --time=120 --events=0 --report-interval=10 --percentile=99 /usr/share/sysbench/tests/include/oltp_legacy/oltp.lua cleanup

参考资料

https://www.postgresql.org/docs/10/pgbench.html
https://github.com/rongfengliang/mysql-postgres-pgbouncer-docker-compose