collapsed_forwarding 插件初探

Posted 2020-09-15

 

0、Bug描述

ats-x.x.x 使用 Read While Writer + Open Read Retry Timeout 实现合并回源 和 icp回源会产生冲突，导致合并回源效果不好

在ats-6.2中有一个 collapsed_forwarding 插件，将这个插件编译好部署在 ats-x.x.x 中，之前还担心会不兼容，效果还好；

1、编译方式

collapsed_forwarding 文件夹下只有：

1 [[email protected]10.110.26.73 collapsed_forwarding]# pwd
2 /path/trafficserver-6.2.1/plugins/experimental/collapsed_forwarding
3 [[email protected]10.110.26.73 collapsed_forwarding]# ll
4 total 52
5 -rw-rw-r-- 1 1000 1000 10941 Dec 17 02:14 collapsed_forwarding.cc
6 -rw-rw-r-- 1 1000 1000   996 Dec 17 02:14 Makefile.am
7 -rw-rw-r-- 1 1000 1000 31281 Dec 17 02:14 Makefile.in
8 -rw-rw-r-- 1 1000 1000  3023 Dec 17 02:14 README
9 [[email protected]10.110.26.73 collapsed_forwarding]#

 

直接make编译会报错，于是我采用ats自带的编译工具 tsxs 进行编译：

• /path/ats/bin/tsxs -I ../../../proxy/api -L ../../../lib -ldl -o collapsed_forwarding.so collapsed_forwarding.cc
• 将编译好的 collapsed_forwarding.so 拷贝到 /path/ats/libexec/trafficserver/ 目录下

2、需要改的配置

• collapsed_forwarding 是基于 open_write_fail_action 和 Read While Writer 的，所以需要在在主配置文件中开启相应的功能；

• 因为这个插件是 remap mode 所以需要在remap.config 中配置使用；

• 关于background fill，详见

• 这个插件需要2个参数，我测试的时候使用了：delay=50 + retries=10 ， 后续可根据测试效果进一步修改；

• 没有在 plugin.config 中启用；

remap.config

1 regex_map ... 2 ... 3 @plugin=/path/ats/libexec/trafficserver/collapsed_forwarding.so @pparam=--delay=50 @pparam=--retries=10

record.config

1 # active the collapsed_forwarding plugin
2 CONFIG proxy.config.http.cache.open_write_fail_action INT 1
3 CONFIG proxy.config.cache.enable_read_while_writer INT 1
4 CONFIG proxy.config.http.redirection_enabled INT 1
5 CONFIG proxy.config.http.number_of_redirections INT 10
6 CONFIG proxy.config.http.redirect_use_orig_cache_key INT 1
7 CONFIG proxy.config.http.background_fill_active_timeout INT 0
8 CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.000000

 

3、原理解释

 1 ////////////////////////////////////////////////////////////////////////////////
 2 // collapsed_forwarding::
 3 //
 4 // ATS plugin to allow collapsed forwarding of concurrent requests for the same
 5 // object. This plugin is based on open_write_fail_action feature, which detects
 6 // cache open write failure on a cache miss and returns a 502 error along with a
 7 // special @-header indicating the reason for 502 error. The plugin acts on the
 8 // error by using an internal redirect follow back to itself, essentially blocking
 9 // the request until a response arrives, at which point, relies on read-while-writer
10 // feature to start downloading the object to all waiting clients. The following
11 // config parameters are assumed to be set for this plugin to work:
12 ////////////////////////////////////////////////////////////////////////////////////
13 // proxy.config.http.cache.open_write_fail_action        1 /////////////////////////
14 // proxy.config.cache.enable_read_while_writer           1 /////////////////////////
15 // proxy.config.http.redirection_enabled                 1 /////////////////////////
16 // proxy.config.http.number_of_redirections             10 /////////////////////////
17 // proxy.config.http.redirect_use_orig_cache_key         1 /////////////////////////
18 // proxy.config.http.background_fill_active_timeout      0 /////////////////////////
19 // proxy.config.http.background_fill_completed_threshold 0 /////////////////////////
20 ////////////////////////////////////////////////////////////////////////////////////
21 // Additionally, given that collapsed forwarding works based on cache write
22 // lock failure detection, the plugin requires cache to be enabled and ready.
23 // On a restart, Traffic Server typically takes a few seconds to initialize
24 // the cache depending on the cache size and number of dirents. While the
25 // cache is not ready yet, collapsed forwarding can not detect the write lock
26 // contention and so can not work. The setting proxy.config.http.wait_for_cache
27 // may be enabled which allows blocking incoming connections from being
28 // accepted until cache is ready.
29 ////////////////////////////////////////////////////////////////////////////////////
30 // This plugin currently supports only per-remap mode activation.
31 ////////////////////////////////////////////////////////////////////////////////////

 

• collapsed_forwarding
  • open_write_fail_action
  • read-while-writer

Open Write Fail Action
In addition to the open read retry settings TS supports a new setting proxy.config.http.cache.open_write_fail_action that allows to further reduce multiple concurrent requests hitting the origin for the same object by either returning a stale copy, in case of hit-stale or an error in case of cache miss for all but one of the requests.

• proxy.config.http.cache.open_write_fail_action

4、测试效果

 1 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 2 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 3 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 4 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 5 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 6 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 7 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 8 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
 9 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
10 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
11 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
12 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT
13 10.110.26.73 HTTP/1.1 200 TCP_MISS:PARENT_HIT
14 10.110.26.73 HTTP/1.1 200 TCP_HIT:NONE
15 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
16 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
17 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
18 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
19 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
20 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
21 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
22 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
23 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
24 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
25 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE

 

也测试了 icp 回源，也是OK的！

5、参考文献

• [email protected]

• Collapsed Forwarding Plugin

• Open Write Fail Action

• HTTP Redirection