So Tired !_! 逆水行舟, 不进则退!

17Jul/14

SysBench系统性能基准测试工具

1 CPU测试
sysbench采用寻找最大素数的方式来测试CPU的性能

[root@xx sysbench-0.4.12]# sysbench --test=cpu --cpu-max-prime=2000 run

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
WARNING: Operation time (0.000000) is less than minimal counted value, counting as 1.000000
WARNING: Percentile statistics will be inaccurate
Done.

Maximum prime number checked in CPU test: 2000

Test execution summary:
total time:                          1.5034s
total number of events:              10000
total time taken by event execution: 1.4998
per-request statistics:
min:                                  0.00ms
avg:                                  0.15ms
max:                                  0.57ms
approx.  95 percentile:               0.31ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   1.4998/0.00

2 线程测试
sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run

[root@xx sysbench-0.4.12]# sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Doing thread subsystem performance test
Thread yields per test: 100 Locks used: 2
Threads started!
WARNING: Operation time (0.000000) is less than minimal counted value, counting as 1.000000
WARNING: Percentile statistics will be inaccurate
Done.
Test execution summary:
total time:                          2.0189s
total number of events:              10000
total time taken by event execution: 128.2852
per-request statistics:
min:                                  0.00ms
avg:                                 12.83ms
max:                                108.17ms
approx.  95 percentile:              42.09ms

Threads fairness:
events (avg/stddev):           156.2500/12.16
execution time (avg/stddev):   2.0045/0.00

 

3 文件IO性能测试
首先生成需要的测试文件,文件总大小1000M,16个并发线程,随机读写模式。执行完后会在当前目录下生成一堆小文件。
3.1 准备测试文件:sysbench --test=fileio --num-threads=16 --file-total-size=1000M --file-test-mode=rndrw prepare
[root@xx sysbench-0.4.12]# sysbench --test=fileio --num-threads=16 --file-total-size=1000M --file-test-mode=rndrw prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128 files, 8000Kb each, 1000Mb total
Creating files for the test...
3.2 执行测试
sysbench --test=fileio --num-threads=16 --file-total-size=1000M --file-test-mode=rndrw run

[root@xx sysbench-0.4.12]# sysbench --test=fileio --num-threads=16 --file-total-size=1000M --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 7.8125Mb each
1000Mb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
WARNING: Operation time (0.000000) is less than minimal counted value, counting as 1.000000
WARNING: Percentile statistics will be inaccurate
Done.

Operations performed:  6007 Read, 4005 Write, 12675 Other = 22687 Total
Read 93.859Mb  Written 62.578Mb  Total transferred 156.44Mb  (214.4Mb/sec)  看到这里发现write很高,我这里是SSD,普通机械磁盘可能就没有这么高了。
13721.48 Requests/sec executed

Test execution summary:
total time:                          0.7297s
total number of events:              10012
total time taken by event execution: 1.3667
per-request statistics:
min:                                  0.00ms
avg:                                  0.14ms
max:                                 20.50ms
approx.  95 percentile:               0.41ms

Threads fairness:
events (avg/stddev):           625.7500/159.20
execution time (avg/stddev):   0.0854/0.02

3.3 清理测试生成的临时文件
sysbench --test=fileio --num-threads=16 --file-total-size=1000M --file-test-mode=rndrw cleanup
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Removing test files...
 4. mutex测试
sysbench --test=mutex --num-threads=16 \--mutex-num=1024 --mutex-locks=10000 --mutex-loops=5000 run
[root@xx sysbench-0.4.12]# sysbench --test=mutex --num-threads=16 \--mutex-num=1024 --mutex-locks=10000 --mutex-loops=5000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Doing mutex performance test
Threads started!
Done.
Test execution summary:
total time:                          0.1167s
total number of events:              16
total time taken by event execution: 1.7472
per-request statistics:
min:                                 96.94ms
avg:                                109.20ms
max:                                114.26ms
approx.  95 percentile:             114.19ms

Threads fairness:
events (avg/stddev):           1.0000/0.00
execution time (avg/stddev):   0.1092/0.01

 

 5,内存测试
 sysbench --test=memory --num-threads=512 --memory-block-size=262144 --memory-total-size=32G run
[root@xx sysbench-0.4.12]#  sysbench --test=memory --num-threads=512 --memory-block-size=262144 --memory-total-size=32G run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 512

Doing memory operations speed test
Memory block size: 256K

Memory transfer size: 32768M

Memory operations type: write
Memory scope type: global
Threads started!
WARNING: Operation time (0.000000) is less than minimal counted value, counting as 1.000000
(last message repeated 1 times)
WARNING: Percentile statistics will be inaccurate
(last message repeated 1 times)
Done.

Operations performed: 131072 (60730.95 ops/sec)

32768.00 MB transferred (15182.74 MB/sec)
Test execution summary:
total time:                          2.1582s
total number of events:              131072
total time taken by event execution: 643.2354
per-request statistics:
min:                                  0.00ms
avg:                                  4.91ms
max:                               1173.07ms
approx.  95 percentile:               0.42ms

Threads fairness:
events (avg/stddev):           256.0000/84.51
execution time (avg/stddev):   1.2563/0.32

 

6 MySQL数据库测试 select
首先需要创建默认的test数据库,或者使用–mysql-db指定一个已经存在的数据库生成测试数据,引擎为innodb,表大小为30000000条记录
6.1 准备数据
--oltp-test-mode
=STRING         test type to use {simple,complex,nontrx,sp} [complex]
--oltp-nontrx-mode=STRING       mode for non-transactional test {select, update_key, update_nokey, insert, delete} [select]
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --mysql-db=test  --oltp-table-size=300000000 --oltp-table-name=t1 --oltp-nontrx-mode=insert --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock prepare

报错如下:

FATAL: no database driver specified
FATAL: failed to initialize database driver!

test库没有建立,去create database
mysql> create database test;
Query OK, 1 row affected (0.01 sec)

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Creating table 't1'...
Creating 30000000 records in table 't1'...

real    13m49.102s
user    0m11.982s
sys     0m0.646s
6.2 执行测试
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --mysql-db=test  --oltp-table-size=300000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
Running the test with following options:
Number of threads: 1

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 10000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            140000
write:                           50000
other:                           20000
total:                           210000
transactions:                        10000  (350.28 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 190000 (6655.38 per sec.)
other operations:                    20000  (700.57 per sec.)

Test execution summary:
total time:                          28.5483s
total number of events:              10000
total time taken by event execution: 28.4897
per-request statistics:
min:                                  1.80ms
avg:                                  2.85ms
max:                                 47.35ms
approx.  95 percentile:               5.95ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   28.4897/0.00
real    0m28.646s
user    0m2.270s
sys     0m1.516s
6.3 clean test data
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --mysql-db=test  --oltp-table-size=30000000 --oltp-table-name=t1   --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock clearup

 

7 MySQL数据库测试 insert 19304W

7.1 执行insert测试,数据准备参考6.1步骤
--oltp-nontrx-mode=insert
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=insert --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run

[root@xx bin]# time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=insert --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 10000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            140000
write:                           50000
other:                           20000
total:                           210000
transactions:                        10000  (222.06 per sec.)
deadlocks:                           0      (0.00 per sec.)
 read/write requests:                 190000 (4219.22 per sec.)
    other operations:                    20000  (444.13 per sec.)

Test execution summary:
total time:                          45.0321s
total number of events:              10000
total time taken by event execution: 44.9398
per-request statistics:
min:                                  1.96ms
avg:                                  4.49ms
max:                                 37.26ms
approx.  95 percentile:               6.96ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   44.9398/0.00
real    0m45.063s
user    0m3.279s
sys     0m2.295s

 

7.2 --oltp-nontrx-mode=update_key 修改测试 带index
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=update_key --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
[root@xx bin]# time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=update_key --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 10000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            140000
write:                           50000
other:                           20000
total:                           210000
transactions:                        10000  (304.16 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 190000 (5778.95 per sec.)
other operations:                    20000  (608.31 per sec.)

Test execution summary:
total time:                          32.8780s
total number of events:              10000
total time taken by event execution: 32.8053
per-request statistics:
min:                                  1.89ms
avg:                                  3.28ms
max:                                 28.03ms
approx.  95 percentile:               6.06ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   32.8053/0.00
real    0m32.909s
user    0m2.628s
sys     0m1.683s

 

7.3 --oltp-nontrx-mode=delete 删除测试
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=delete --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
[root@xx bin]# time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=delete --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 10000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            140000
write:                           50000
other:                           20000
total:                           210000
transactions:                        10000  (330.13 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 190000 (6272.53 per sec.)
other operations:                    20000  (660.27 per sec.)

Test execution summary:
total time:                          30.2908s
total number of events:              10000
total time taken by event execution: 30.2212
per-request statistics:
min:                                  1.84ms
avg:                                  3.02ms
max:                                 13.14ms
approx.  95 percentile:               6.02ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   30.2212/0.00
real    0m30.328s
user    0m2.416s
sys     0m1.658s
7.4 --oltp-nontrx-mode=update_nokey 修改测试 No Index
--oltp-nontrx-mode=STRING       mode for non-transactional test {select, update_key, update_nokey, insert, delete} [select]
time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=update_nokey --mysql-db=test  --oltp-table-size=193040000 --max-requests --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run 

[root@xx bin]# time sysbench --test=oltp --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --oltp-nontrx-mode=update_nokey --mysql-db=test  --oltp-table-size=193040000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 10000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            140000
write:                           50000
other:                           20000
total:                           210000
transactions:                        10000  (319.11 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 190000 (6063.01 per sec.)
other operations:                    20000  (638.21 per sec.)

Test execution summary:
total time:                          31.3376s
total number of events:              10000
total time taken by event execution: 31.2650
per-request statistics:
min:                                  1.83ms
avg:                                  3.13ms
max:                                 13.44ms
approx.  95 percentile:               6.11ms

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   31.2650/0.00
real    0m31.367s
user    0m2.422s
sys     0m1.680s

7.4 总结
--oltp-nontrx-mode=STRING       mode for non-transactional test {select, update_key, update_nokey, insert, delete} [select]
--max-requests=N           limit for total number of requests [10000]
如果这里不指定--max-requests,默认为10000,就是执行1W次操作就结束了,如果要做压力测试的话,就需要手动设置此值。
--num-threads=N            number of threads to use [1] (clients to access mysql db)
这里指的是有多少个mysql clents来访问mysql服务器,用show full processlist就可以看到大概的记录数。

7.5 接下来重新压力测试,设置2000W request测试。

录入测试数据 2000W
time sysbench --test=oltp --oltp-test-mode=nontrx --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=insert --mysql-db=test  --oltp-table-size=20000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock prepare
Creating table 't1'...
Creating 20000000 records in table 't1'...

real    8m46.287s
user    0m8.048s
sys     0m0.428s

 

7.6开始测试 纯粹写
nohup time sysbench --test=oltp --oltp-test-mode=nontrx --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=insert --mysql-db=test  --oltp-table-size=20000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run >3.log &
time sysbench --test=oltp --oltp-test-mode=nontrx --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=insert --mysql-db=test  --oltp-table-size=20000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run  1> 2.log

OLTP test statistics:
queries performed:
read:                            0
write:                           20007732
other:                           0
total:                           20007732
transactions:                        20007732 (8556.00 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 20007732 (8556.00 per sec.)
other operations:                    0      (0.00 per sec.)

Test execution summary:
total time:                          2338.4451s
total number of events:              20007732
total time taken by event execution: 1169095.8829
per-request statistics:
min:                                  0.35ms
avg:                                 58.43ms
max:                                317.35ms
approx.  95 percentile:              83.54ms

Threads fairness:
events (avg/stddev):           40015.4640/13.47
execution time (avg/stddev):   2338.1918/0.02

900.69user 667.19system 39:07.55elapsed 66%CPU (0avgtext+0avgdata 128176maxresident)k
0inputs+16outputs (0major+8251minor)pagefaults 0swaps
7.7 混合读写测试
nohup time sysbench --test=oltp --oltp-test-mode=complex --mysql-table-engine=innodb --mysql-user=root --db-driver=mysql --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=select --mysql-db=test  --oltp-table-size=20000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock run >4.log &

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 500

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 20000000
Threads started!
Done.

OLTP test statistics:
queries performed:
read:                            280001652
write:                           100000590
other:                           40000236
total:                           420002478
transactions:                        20000118 (1150.48 per sec.)
deadlocks:                           0      (0.00 per sec.)
read/write requests:                 380002242 (21859.10 per sec.)
other operations:                    40000236 (2300.96 per sec.)

Test execution summary:
total time:                          17384.1703s
total number of events:              20000118
total time taken by event execution: 8691857.1553
per-request statistics:
min:                                 28.97ms
avg:                                434.59ms
max:                              14634.16ms
approx.  95 percentile:             856.09ms

Threads fairness:
events (avg/stddev):           40000.2360/119.14
execution time (avg/stddev):   17383.7143/0.06

5142.95user 4422.48system 4:49:50elapsed 55%CPU (0avgtext+0avgdata 256224maxresident)k
2440inputs+16outputs (15major+16290minor)pagefaults 0swaps

 avg:    434.59ms

看到这里,大leader说这个数值偏高了,在game领域,最好<10ms,并且他猜测我是local测试的,local500个线程也许会
   对db server有一些压力的,建议我remote测试访问测试下看

7.8.1 混合读写测试 远程读写数据准备,远程跑后台进程准备数据。
nohup time sysbench --test=oltp --oltp-test-mode=nontrx --mysql-table-engine=innodb --mysql-host=xxxx.xx --mysql-user=xxxxx--mysql-password=hRhsYzbm  --db-driver=mysql --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=select --mysql-db=test  --oltp-table-size=20000000 --oltp-table-name=t1  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock prepare > 6.log &
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Creating table 't1'...
Creating 20000000 records in table 't1'...
8.48user 1.96system 40:23.30elapsed 0%CPU (0avgtext+0avgdata 12224maxresident)k
0inputs+8outputs (0major+854minor)pagefaults 0swaps

7.8.2 混合读写测试 远程读写
nohup time sysbench --test=oltp --oltp-test-mode=complex --mysql-table-engine=innodb --mysql-host=xxxxxx.xx.com --mysql-user=chunman --mysql-password=hRhsYzbm --db-driver=mysql --mysql-db=test --oltp-table-name=t1  --num-threads=500 --max-requests=20000000  --oltp-nontrx-mode=select --oltp-table-size=20000000  --mysql-socket=/opt/mysql/product/mysql/mysql3306.sock  run >5.log &

ALERT: Error: failed to determine table 't1' type!
ALERT: MySQL error:
FATAL: failed to get database capabilities!
0.00user 0.00system 0:00.01elapsed 64%CPU (0avgtext+0avgdata 9856maxresident)k
0inputs+8outputs (0major+706minor)pagefaults 0swaps

报这个错误,是因为数据没有,7.8.1没有执行,需要重新录入测试数据。

anyShare分享到:
Comments (0) Trackbacks (0)

Sorry, the comment form is closed at this time.

Trackbacks are disabled.

site