前端性能监控系统ShowSlow

Posted 2020-06-17

  作者：zhanhailiang 日期：2014-11-14

1. 简单介绍

ShowSlow是开源的前端性能监控系统，提供了下面功能：

• 前端性能指标数据收集功能：ShowSlow原生提供了数据收集工具DOM Monster!，但也支持通过YSlow，PageSpeed等第三方工具将性能数据上报给服务端完毕收集（其server端提供了针对多达8种不同工具上报的数据收集器dommonster，dynatrace，events，har，metric，pagespeed，webpagetest，yslow）；
• 前端性能指标数据展示系统；

2. 安装与配置

1). 安装与配置

安装ShowSlow：

[root@~/software]# wget http://www.showslow.org/downloads/showslow_1.2.2.tar.bz2
[root@~/software]# tar jxf showslow_1.2.2.tar.bz2
[root@~/software]# mv showslow_1.2.2 /usr/local/wwwroot/
[root@/usr/local/wwwroot]# ln -s showslow_1.2.2 showslow
[root@/usr/local/wwwroot]# ln -s /usr/local/php/bin/php /usr/local/bin/php
[root@/usr/local/wwwroot/showslow]# make
# generating crc32 hashes of all assets that should be versioned
find ./ -type f | grep -v -E ‘^./(timeline|timeplot|ajax|users|svn-assets|flot\\/examples)/‘ | grep -E ‘\\.(png|jpg|js|gif|ico)$‘ | sort | xargs -n10 php crc32.php | sed -e ‘s/\\t\\.\\//\\t/‘ > asset_versions.tsv
php dbupgrade.php
Nothing to upgrade from v.34 to v.34.
cd users && make
make[1]: Entering directory `/usr/local/wwwroot/showslow_1.2.2/users‘
php depcheck.php
Using PHP version 5.3.29 ... OK
PHP mysqli extension is loaded ... OK
PHP mcrypt extension is loaded ... OK
PHP curl extension is loaded ... OK
PHP mbstring extension is loaded ... OK
Found configuration file ../users_config.php
php dbupgrade.php
Nothing to upgrade from v.31 to v.31.
php aggregatepoints.php
[*** SUCCESS ***] Installation and upgrade of Startup API completed successfully
make[1]: Leaving directory `/usr/local/wwwroot/showslow_1.2.2/users‘

改动ShowSlow DB配置：

[root@/usr/local/wwwroot/showslow]# cp config.sample.php config.php

配置Nginx:

server {
    listen       80; 
    server_name  showslow;
    root   /usr/local/wwwroot/showslow;

    location / { 
        root   /usr/local/wwwroot/showslow;
        index  index.php index.html index.htm;
    }   

    if (!-e $request_filename) {
        rewrite ^(.*)$ /index.php$1 last;
    }   

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #   
    #error_page   500 502 503 504  /50x.html;
    #location = /50x.html {
    #    root   html;
    #}  

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #   
    location ~ .*\\.php(\\/.*)*$ {
        root           /usr/local/wwwroot/showslow;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;
    }   
}

配置beacon：

1). 使用DOM Monster!

訪问http://showslow/beacon/dommonster/，将DOM Monster!收藏为书签。然后訪问http://m.vip.com，点击书签“DOM Monster!”。例如以下：

点击“Send to Show Slow“将性能监控数据上报到server端存储；

2). 配置YSlow上报数据：

改动about:config：

extensions.yslow.beaconUrl = http://showslow/beacon/yslow/
extensions.yslow.beaconInfo = grade
extensions.yslow.optinBeacon = true

然后配置“Autorun YSlow each time a web page is loaded”：

刷新页面后。抓包就能够看到性能监控数据上报HTTP请求：

假设看到响应“HTTP/1.1 204 Data accepted”，就表示数据存储成功。

说明：

• 对于YSlow上报的数据标准请參考http://yslow.org/user-guide/#yslow_beacon
• YSlow配置值extensions.yslow.defaultRuleset相应面板上的Rulesets，相应server端配置例如以下：

  /usr/local/wwwroot/showslow/beacon/yslow/index.php
  /usr/local/wwwroot/showslow/global.php

3). 其他工具配置方法类似，略。

3. 数据展示系统

4. 应用场景及缺陷分析

1. 若使用DOM Monster做为前端数据收集器，通过改动showslow/beacon/dommonster/dom-monster/src/dommonster.js和showslow/beacon/dommonster/index.php，能够实现自己主动化上报，代码改动请见：

改动DOM Monster支持自己主动上报监控数据
=======
1. 改动beacon/dommonster/index.php；去掉confirm二次确认弹出框。

https://github.com/billfeller/showslow/blob/master/beacon/dommonster/index.php

2. 改动beacon/dommonster/dom-monster/src/dommonster.js；支持自己主动上报数据。

https://github.com/billfeller/dom-monster/blob/master/src/dommonster.js

但其本身所得的监控粒度很粗放。其统计表结构例如以下：

--
-- Table structure for table `dommonster`
--
 
CREATE TABLE IF NOT EXISTS `dommonster` (
`id` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘Measurement ID‘,
  `version` VARCHAR(255) DEFAULT NULL COMMENT ‘Version of DOM Monster bookmarklet‘,
  `timestamp` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ‘Measurement time‘,
  `url_id` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘URL ID‘,
  `elements` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘number of elements [number]‘,
  `nodecount` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘number of DOM nodes [number]‘,
  `textnodes` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘number of Text nodes [number]‘,
  `textnodessize` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘size of Text nodes [bytes]‘,
  `contentpercent` DECIMAL(5,2) UNSIGNED NOT NULL COMMENT ‘content percentage [percentage]‘,
  `average` DECIMAL(10,1) UNSIGNED NOT NULL COMMENT ‘average nesting depth [number]‘,
  `domsize` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘serialized DOM size [bytes]‘,
  `bodycount` BIGINT(20) UNSIGNED DEFAULT NULL COMMENT ‘DOM tree serialization time [ms]‘
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8;

2. 若使用YSlow等第三方工具确实实现收集细粒度的监控数据。以YSlow为例，其统计表结构例如以下：

--
-- Table structure for table `yslow2`
--
 
CREATE TABLE IF NOT EXISTS `yslow2` (
`id` BIGINT(20) UNSIGNED NOT NULL COMMENT ‘Entry id‘,
  `timestamp` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ‘Measurement timestamp‘,
  `ip` INT(4) UNSIGNED DEFAULT ‘0‘ COMMENT ‘IP address of the agent‘,
  `user_agent` text NOT NULL COMMENT ‘User agent string‘,
  `url_id` BIGINT(20) UNSIGNED NOT NULL DEFAULT ‘0‘,
  `w` BIGINT(20) UNSIGNED NOT NULL DEFAULT ‘0‘ COMMENT ‘size of the page in bytes‘,
  `o` SMALLINT(5) UNSIGNED NOT NULL DEFAULT ‘0‘ COMMENT ‘overall YSlow grade calculated for this profile‘,
  `r` SMALLINT(6) UNSIGNED NOT NULL DEFAULT ‘0‘ COMMENT ‘total amount of requests with empty cache‘,
  `i` text NOT NULL COMMENT ‘testing profile used‘,
  `lt` BIGINT(20) UNSIGNED NOT NULL DEFAULT ‘0‘ COMMENT ‘page load time‘,
  `ynumreq` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Make fewer HTTP requests‘,
  `ycdn` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Using CDN‘,
  `yexpires` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Expires Headers‘,
  `yemptysrc` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Avoid Empty Image src‘,
  `ycompress` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Gzip components‘,
  `ycsstop` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘CSS at the top‘,
  `yjsbottom` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘JS at the bottom‘,
  `yexpressions` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘CSS expressions‘,
  `yexternal` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Make javascript and CSS external‘,
  `ydns` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Reduce DNS lookups‘,
  `yminify` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Minify JavaScript and CSS‘,
  `yredirects` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Avoid URL redirects‘,
  `ydupes` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Remove duplicate JavaScript and CSS‘,
  `yetags` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Configure entity tags (ETags)‘,
  `yxhr` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Make AJAX cacheable‘,
  `yxhrmethod` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Use GET for AJAX requests‘,
  `ymindom` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Reduce the number of DOM elements‘,
  `yno404` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Avoid HTTP 404 (Not Found) error‘,
  `ymincookie` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Reduce cookie size‘,
  `ycookiefree` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Use cookie-free domains‘,
  `ynofilter` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Avoid AlphaImageLoader filter‘,
  `yimgnoscale` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Do not scale images in HTML‘,
  `yfavicon` SMALLINT(3) UNSIGNED DEFAULT NULL COMMENT ‘Make favicon small and cacheable‘,
  `details` text COMMENT ‘Beacon details‘
) ENGINE=InnoDB AUTO_INCREMENT=31 DEFAULT CHARSET=utf8;

可是因为对工具的依赖，其本身无法实现自己主动化监控用户真实的应用场景。

针对移动端的性能监控。眼下因为其本身依赖的大多数高级工具绝大多数仅仅有PC端的版本号，在移动端缺乏对应的工具支持。所以假设想使用ShowSlow作为前端性能监控平台。须要单独实现数据收集系统，而仅仅是将ShowSlow当作展示系统使用。

5. 參考

• http://www.showslow.org/Main_Page
• http://www.showslow.org/Installation_and_configuration
• http://yslow.org/user-guide/#yslow_beacon
• http://www.cnblogs.com/fnng/archive/2011/09/24/2186708.html
• http://mir.aculo.us/dom-monster/