运维实战 CDN内容分发网络

Posted 洛冰音

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了运维实战 CDN内容分发网络相关的知识,希望对你有一定的参考价值。

常见的业务流程

Client -> DNS -> CDN -> Proxy -> Webserver

安装Varnish

yum install -y varnish-4.0.5-1.el7.x86_64.rpm varnish-libs-4.0.5-1.el7.x86_64.rpm jemalloc-3.6.0-1.el7.x86_64.rpm jemalloc-devel-3.6.0-1.el7.x86_64.rpm

curl 172.25.5.1
vim default.vcl 
systemctl reload varnish.service 
curl 172.25.5.1
curl -I 172.25.5.1

编辑主配置文件

##添加默认后端服务器
##Server2上有安装好的nginx
cd /etc/varnish/
vim default.vcl

backend default {
    .host = "172.25.5.2";
    .port = "80";
}

##修改程序环境中的监听端口为80方便测试
vim varnish.params

VARNISH_LISTEN_PORT=80

使用宿主机进行测试

[root@Server1 varnish]# curl 172.25.5.1
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

image-20210505021132598

image-20210505021156411

增加缓存设置

##为响应添加X-Cache首部,显示缓存是否命中
vim default.vcl

if (obj.hits > 0) {
	set resp.http.X-Cache = "HIT from westos cache";
    }
    else {
    	set resp.http.X-Cache = "MISS from westos cache";
       	}
    return (deliver);
}


systemctl reload varnish

使用宿主机进行测试

[root@Server1 varnish]# curl -I 172.25.5.1
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Fri, 16 Apr 2021 07:02:45 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Fri, 16 Apr 2021 02:41:15 GMT
ETag: "6078f94b-264"
X-Varnish: 5 32771
Age: 6
Via: 1.1 varnish-v4
X-Cache: HIT from westos cache
Connection: keep-alive

image-20210505021144397

增加负载均衡及健康检查

  • 首先需要在Server3再启用一台Nginx服务器

健康检查

probe backend_healthcheck {
    .url = "/bbs/index.html";
    .window = 3;
    .threshold = 2;
    .interval = 3s;
}

backend Server2 {
    .host = "172.25.5.2";
    .port = "80";
    .probe = backend_healthcheck;
}

backend Server3 {
    .host = "172.25.5.3";
    .port = "80";
    .probe = backend_healthcheck;
}

根据不同的域名, 分发到不同的后端主机

if (req.http.host ~ "^(www.)?westos.org") {
                set req.http.host = "www.westos.org";
                set req.backend_hint = Server2;
                #return(pass);
        }
        elseif (req.http.host ~ "^bbs.westos.org") {
                set req.backend_hint = Server3;
                #return(pass);
        }
        else {
                return(synth(404,"Not in cache"));
        }

实现了如下效果

  • 当访问www.westos.org时, 调度到web2, 也就是Server2主机
  • 当访问bbs.westos.org时, 调度到web2, 也就是Server3主机
  • 当访问其他王志时, 返回404报错

设置负载均衡

import directors;

sub vcl_init {
        new web_cluster = directors.round_robin();
        web_cluster.add_backend(Server2);
        web_cluster.add_backend(Server3);
}

sub vcl_recv {

  	if (req.http.host ~ "^(www.)?westos.org") {
		set req.http.host = "www.westos.org";
                set req.backend_hint = web_cluster.backend();
                return(pass);
        }
}

directors模块包含round_robin, fallback, random, hash四种负载均衡模式,可以视情况决定使用哪种…

这里采用的是round_robin.

实验效果

##当访问www.westos.org时轮调的效果
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server3
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server2
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server3
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server2
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server3
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server2
[root@foundation5 ~]# curl www.westos.org/bbs/index.html
Server3
##当关闭Server3后健康检查的结果
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2
[root@foundation5 ~]# curl www.westos.org
Server2

自定义缓存文件的缓存时长

sub vcl_backend_response { 
    if (bereq.url ~ "\\.(jpg|jpeg|gif|png)$") {
        set beresp.ttl = 7200s;
    }
    if (bereq.url ~ "\\.(html|css|js)$") {
        set beresp.ttl = 1200s;
    }
    if (beresp.http.Set-Cookie) { 
        return(deliver);
    }
}

使服务端可以获取客户端的原始IP

不知道您有没有发现

如果按照这样操作,所有的请求都是CDN发往Server端的,也就是说对Server端来说看到的IP永远是CDNIP

如果服务端需要根据访问量等信息做业务分析, 这时候如果不做设置, 则服务端获取到的永远都是CDNIP, 无法满足业务需求.

##为发往后端主机的请求添加X-Forward-For首部
if (req.http.X-Forward-For) {    
        set req.http.X-Forward-For = req.http.X-Forward-For + "," + client.ip;
    } else {
        set req.http.X-Forward-For = client.ip;
    }
##同时禁止服务端缓存隐私信息
if (req.url ~ "(?i)^/(login|admin)") {
        return(pass);
    }

##Nginx设置
server {
        listen 80;
        server_name www.westos.org;
        real_ip_header    X-Forwarded-For;
        real_ip_recursive on;
        set_real_ip_from 172.25.5.0/24;

        location / {
                root /www;
                index index.html;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_pass http://westos;
        }
}

查看日志可以看到取到了真实IP

[root@Server3 nginx]# tail -f logs/access.log 

image-20210505023503003

以上是关于运维实战 CDN内容分发网络的主要内容,如果未能解决你的问题,请参考以下文章

企业运维之 CDN 内容分发网络

运维必学:全局负载均衡与CDN内容分发

全局负载均衡(GSLB)和内容分发网络(CDN)原理及实战

百度智能云实战——静态文件CDN加速

摩杜云将出席2021亚太内容分发大会暨CDN峰会

使用动态 url 暂停和恢复从 CDN(内容分发网络)下载大文件