nginx+tomcat负载均衡+动静分离

Posted 2020-07-21 神一样的存在

1、服务器A安装ng，服务器B、C安装tomcat；

2、服务器A建立/data/www目录，用于发布静态文件；

3、ng无动静分离配置；

user root root;
worker_processes 8;
pid /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 102400;
events
{
use epoll;
worker_connections 102400;
}
http
{
  include       mime.types;
  default_type  application/octet-stream;
  fastcgi_intercept_errors on;
  charset  utf-8;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 4k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
   
  keepalive_timeout 60;
   
  tcp_nodelay on;
  client_body_buffer_size  512k;
 
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;
   
  gzip on;
  gzip_min_length  1k;
  gzip_buffers     4 16k;
  gzip_http_version 1.1;
  gzip_comp_level 2;
  gzip_types       text/plain application/x-javascript text/css application/xml;
  gzip_vary on;
   
###2012-12-19 change nginx logs
log_format  main  \'$http_x_forwarded_for - $remote_user [$time_local] "$request" \'
              \'$status $body_bytes_sent "$http_referer" \'
              \'"$http_user_agent"  $request_time $remote_addr\';
               
upstream web_app {
 server 192.168.203.130:10086 weight=1 max_fails=2 fail_timeout=30s;
 server 192.168.203.131:10087 weight=1 max_fails=2 fail_timeout=30s;
}
 
####chinaapp.sinaapp.com
server {
    listen 80;
    server_name  chinaapp.sinaapp.com;
    index index.jsp index.html index.htm;
    # 发布目录:/data/www
    root  /data/www;
     
    location /
    {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_set_header Host  $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://web_app;
        expires      3d;
    }
  
  }
 
}

View Code

4、ng动静分离配置；

user root root;
worker_processes 8;
pid /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 102400;
events
{
use epoll;
worker_connections 102400;
}
http
{
  include       mime.types;
  default_type  application/octet-stream;
  fastcgi_intercept_errors on;
  charset  utf-8;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 4k;
  large_client_header_buffers 4 32k;
  client_max_body_size 300m;
  sendfile on;
  tcp_nopush     on;
   
  keepalive_timeout 60;
   
  tcp_nodelay on;
  client_body_buffer_size  512k;
 
  proxy_connect_timeout    5;
  proxy_read_timeout       60;
  proxy_send_timeout       5;
  proxy_buffer_size        16k;
  proxy_buffers            4 64k;
  proxy_busy_buffers_size 128k;
  proxy_temp_file_write_size 128k;
   
  gzip on;
  gzip_min_length  1k;
  gzip_buffers     4 16k;
  gzip_http_version 1.1;
  gzip_comp_level 2;
  gzip_types       text/plain application/x-javascript text/css application/xml;
  gzip_vary on;
   
###2012-12-19 change nginx logs
log_format  main  \'$http_x_forwarded_for - $remote_user [$time_local] "$request" \'
              \'$status $body_bytes_sent "$http_referer" \'
              \'"$http_user_agent"  $request_time $remote_addr\';
               
upstream web_app {
 server 192.168.203.130:10086 weight=1 max_fails=2 fail_timeout=30s;
 server 192.168.203.131:10087 weight=1 max_fails=2 fail_timeout=30s;
}
 
####chinaapp.sinaapp.com
server {
    listen 80;
    server_name  chinaapp.sinaapp.com;
    index index.jsp index.html index.htm;
    # 发布目录:/data/www
    root  /data/www;
     
    location /
    {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_set_header Host  $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://web_app;
        expires      3d;
    }
    location ~ .*\\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
    {
        root /data/www; 
        #expires定义用户浏览器缓存的时间为3天，如果静态页面不常更新，可以设置更长，这样可以节省带宽和缓解服务器的压力
        expires      3d;
     
    }
  
  }
 
}

View Code

5、安装ng的时候不用指定编译参数，直接./configure --prefix=/usr/local/nginx & make & make install就行；

6、这里直接访问域名依然会请求tomcat，没搞清楚为什么，只有域名后面带有具体静态文件才会从ng请求；

7、下载https://github.com/jcoleman/tomcat-redis-session-manager，修改相关参数，编译，将下面的9个文件放到tomcat的lib目录下；

gradle.build修改成为：

apply plugin: \'java\'
apply plugin: \'maven\'
apply plugin: \'signing\'

group = \'com.orangefunction\'
version = \'2.0.0\'

repositories {
  mavenCentral()
}

compileJava {
  sourceCompatibility = 1.7
  targetCompatibility = 1.7
}

dependencies {
  compile group: \'org.apache.tomcat\', name: \'tomcat-catalina\', version: \'7.0.67\'
  compile group: \'redis.clients\', name: \'jedis\', version: \'2.5.2\'
  compile group: \'org.apache.commons\', name: \'commons-pool2\', version: \'2.2\'
  //compile group: \'commons-codec\', name: \'commons-codec\', version: \'1.9\'

  testCompile group: \'junit\', name: \'junit\', version: \'4.+\'
  testCompile \'org.hamcrest:hamcrest-core:1.3\'
  testCompile \'org.hamcrest:hamcrest-library:1.3\'
  testCompile \'org.mockito:mockito-all:1.9.5\'
  testCompile group: \'org.apache.tomcat\', name: \'tomcat-coyote\', version: \'7.0.67\'
}

task javadocJar(type: Jar, dependsOn: javadoc) {
  classifier = \'javadoc\'
  from \'build/docs/javadoc\'
}

task sourcesJar(type: Jar) {
  from sourceSets.main.allSource
  classifier = \'sources\'
}

artifacts {
  archives jar

  archives javadocJar
  archives sourcesJar
}

//signing {
//  sign configurations.archives
//}

task copyJars(type: Copy) {
  from configurations.runtime
  into \'dist\'  
}

uploadArchives {
  repositories {
    mavenDeployer {
      beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }

      //repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/") {
      //  authentication(userName: sonatypeUsername, password: sonatypePassword)
      //}
      //repository(url: "https://oss.sonatype.org/content/repositories/snapshots") {
      //  authentication(userName: sonatypeUsername, password: sonatypePassword)
      //}

      pom.project {
        name \'tomcat-redis-session-manager\'
        packaging \'jar\'
        description \'Tomcat Redis Session Manager is a Tomcat extension to store sessions in Redis\'
        url \'https://github.com/jcoleman/tomcat-redis-session-manager\'

        issueManagement {
          url \'https://github.com:jcoleman/tomcat-redis-session-manager/issues\'
          system \'GitHub Issues\'
        }

        scm {
          url \'https://github.com:jcoleman/tomcat-redis-session-manager\'
          connection \'scm:git:git://github.com/jcoleman/tomcat-redis-session-manager.git\'
          developerConnection \'scm:git:[e-mail]:jcoleman/tomcat-redis-session-manager.git\'
        }

        licenses {
          license {
            name \'MIT\'
            url \'http://opensource.org/licenses/MIT\'
            distribution \'repo\'
          }
        }

        developers {
          developer {
            id \'jcoleman\'
            name \'James Coleman\'
            email \'[e-mail]\'
            url \'https://github.com/jcoleman\'
          }
        }
      }
    }
  }
}

View Code

8、编译完之后，上面的倒数第三那个是在build目录下面；

9、修改tomcat配置文件：

<Context>
      <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
      <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
           host="redis主机ip地址"
           port="6379"
           database="0"
           maxInactiveInterval="60" />
</Context>

View Code

10、上面的host是redis的地址，如果tomcat和redis没有在同一台机器上，那么tomcat的启动是比较慢的，需要耐心等候，不是报错；

11、一般来说，session中我们记录用户的信息，如果记录单个属性，那么没什么说得 ，如果session的属性是用户对象，那么该对象需要实现Serializable接口；

12、异常记录：由于我的数据库装载笔记本实体机上面，而ng，tomcat1，tomcat2分别装在vm的3个不同的机器里面的，所以导致vm里面的机器连接实体机的数据库时报错，无法连接，我们需要为数据库增加访问权限：

GRANT ALL PRIVILEGES ON *.* TO \'root\'@\'%\' IDENTIFIED BY \'123456\' WITH GRANT OPTION; 
FLUSH   PRIVILEGES; 

13、另外：tomcat启动真的很慢，需要1分钟，搞不懂为什么，我这里把文件做了动静分离的，把静态文件放在ng的/data/www目录下面，而ajax请求会被做负载均衡。