Docker全新的容器集群项目——SwarmKit初步测试

Posted chinagissoft

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Docker全新的容器集群项目——SwarmKit初步测试相关的知识,希望对你有一定的参考价值。

题记

对于Docker容器集群来说,比较成熟的方案包括Swarm、Mesos、k8s和Google的Kubernetes,特别是后者得到了更多厂商的使用和推广,但是kubernetes相比较来说技术门槛较高,让很多用户望而却步,幸好,Docker在今年的6月7号开源发布了原生的集群管理工具SwarmKit,主要提供容器集群以及编排能力,那赶紧尝试一下,看看有什么好的功能。


SwarmKit框架




SwarmKit中有两种角色,Manager和Worker。Manager主要管理节点、调度任务。Worker主要通过Executor来执行任务,当前缺省的Executor为Docker Container Executor。包含了一下特点:

(1)内建分布式存储,不要额外的数据库
(2)支持Rolling update
(3)容器HA,支持Zero applicaton downtime
(4)通过TLS保证通讯安全


接下来就赶紧安装一下。

             “ SwarmKit 环境部署

部署环境

VMWare Workstation 12 

Ubuntu 14.04


先制作一个通用的Ubuntu14.04虚拟机,然后安装Docker、安装Swarmkit包,然后进行虚拟机复制,生成三台集群环境,分别为一个Manager和两个Worker。

1、安装Ubuntu 14.04

2、安装Docker

curl -sSL https://get.docker.io | bash

3、由于SwarmKit采用Go语言开发,所以需要部署一个go环境,下载go包

下载地址:http://www.golangtc.com/download

我这里下载的是go1.6.2.linux-amd64.tar.gz

4、解压缩到/usr/local下面即可

然后添加环境变量 至 /etc/profile


export GOROOT=/usr/local/go
export PATH=/root/go/src/github.com/docker/swarmkit/bin:/usr/local/go/bin:$PATH
export GOPATH=/root/go
export SWARM_SOCKET=/tmp/controller/swarm.sock
注意:我目前的环境变量包含了所有的

环境变量生效之后,直接输入go即可看到命令提示。


5、安装git包

apt-get install git

6、输入如下命令下载swarmkit包

$ go get github.com/docker/swarmkit
注意:需要配置GOPATH环境变量,如上所述:/root/go,相关代码会下载到该目录下

7、进入/root/go/src/github.com/docker/swarmkit目录,make即可

root@controller:~/go/src/github.com/docker/swarmkit# pwd
/root/go/src/github.com/docker/swarmkit
root@controller:~/go/src/github.com/docker/swarmkit# ls
agent  bin          ca          cmd          CONTRIBUTING.md  Godeps    ioutils  log          Makefile  NOMENCLATURE.md  protobuf   vendor
api    BUILDING.md  circle.yml  codecov.yml  doc.go           identity  LICENSE  MAINTAINERS  manager   picker           README.md  version

8、在该目录的bin文件夹下可以看到生成的二进制文件

root@controller:~/go/src/github.com/docker/swarmkit/bin# ll
total 76092
drwxr-xr-x  2 root root     4096 Jun 30 10:00 ./
drwxr-xr-x 17 root root     4096 Jun 30 10:00 ../
-rwxr-xr-x  1 root root 12929120 Jun 30 10:00 protoc-gen-gogoswarm*
-rwxr-xr-x  1 root root 18044776 Jun 30 10:00 swarm-bench*
-rwxr-xr-x  1 root root 19219800 Jun 30 10:00 swarmctl*
-rwxr-xr-x  1 root root 27708144 Jun 30 10:00 swarmd*

swarmd是一个swarmkit daemon程序,用来运行manager和worker。

swarmctl是一个命令行工具,用来访问swarm manger

我们需要将这四个文件分别拷贝到不同manage和worker节点的/usr/bin里面,当然,也可以放在上述的环境变量里面。

9、建议在该机器下载一个测试镜像。

10、关闭虚拟机,复制。


              “ SwarmKit 具体使用 

目前我已经准备好了三台虚拟机,里面的环境都是一样的。


Manager:192.168.14.244/controller

Worker:192.168.14.80/worker1

Worker:192.168.14.223/worker2

确保manager和所有worker都可以实现ssh无密码访问


1、在manager节点执行启动命令swarmd -d /tmp/controller --listen-control-api /tmp/controller/swarm.sock --hostname controller

root@controller:~# swarmd -d /tmp/controller --listen-control-api /tmp/controller/swarm.sock --hostname controller
Warning: Specifying a valid address with --listen-remote-api may be necessary for other managers to reach this one.
INFO[0000] Listening for connections                     addr=[::]:4242 proto=tcp
INFO[0000] Listening for local connections               addr=/tmp/controller/swarm.sock proto=unix
INFO[0000] 2dacc8ed9d604bff became follower at term 0
INFO[0000] newRaft 2dacc8ed9d604bff [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
INFO[0000] 2dacc8ed9d604bff became follower at term 1
INFO[0000] 2dacc8ed9d604bff is starting a new election at term 1
INFO[0000] 2dacc8ed9d604bff became candidate at term 2
INFO[0000] 2dacc8ed9d604bff received vote from 2dacc8ed9d604bff at term 2
INFO[0000] 2dacc8ed9d604bff became leader at term 2
INFO[0000] raft.node: 2dacc8ed9d604bff elected leader 2dacc8ed9d604bff at term 2
INFO[0000] node is ready

注意:你可以使用nohup命令启动后台运行


然后分别在worker机器上运行加入集群的命令

swarmd -d /tmp/work1 --hostname work1 --join-addr 192.168.13.244:4242

root@worker1:~# swarmd -d /tmp/work1 --hostname work1 --join-addr 192.168.13.244:4242
Warning: Specifying a valid address with --listen-remote-api may be necessary for other managers to reach this one.
INFO[0000] Waiting for TLS certificate to be issued...
INFO[0000] Downloaded new TLS credentials with role: swarm-worker.
INFO[0000] node is ready


swarmd -d /tmp/work2 --hostname work2 --join-addr 192.168.13.244:4242

root@worker2:~# swarmd -d /tmp/work2 --hostname work2 --join-addr 192.168.13.244:4242
Warning: Specifying a valid address with --listen-remote-api may be necessary for other managers to reach this one.
INFO[0000] Waiting for TLS certificate to be issued...
INFO[0000] Downloaded new TLS credentials with role: swarm-worker.
INFO[0000] node is ready

这样三台机器都加入了统一的SwarmKit集群管理中了。接下来我们就可以通过swarmctl命令对docker集群进行管理了,这里面需要添加SWARM_SOCKET环境变量,参看上述。


2、查看目前的集群节点情况

root@controller:~# swarmctl node ls
ID                         Name        Membership  Status  Availability  Manager Status
--                         ----        ----------  ------  ------------  --------------
5a9hk2li71cx0fuzgi42wsrvc  work1       ACCEPTED    READY   ACTIVE
a84vwflvs2ao4awtxor4ybaa6  work2       ACCEPTED    READY   ACTIVE
azwpol92a8mrk1mssqw2ry7p7  controller  ACCEPTED    READY   ACTIVE        REACHABLE *

可以看到目前所有的状态都是正常状态


3、创建一个ubuntu服务

root@controller:~# swarmctl service create --name ubuntu --image ubuntu:14.04
1601akon3w1t1m7i9el4pn9gl

注意:如果集群各个节点包含镜像,启动比较快,如果不包含,各个节点如果联网,自行去docker hub里面下载镜像,这个可能会花费时间。这也是我建议为什么先下载一个镜像的原因。


查看目前的服务情况

root@controller:~# swarmctl service ls
ID                         Name    Image         Replicas
--                         ----    -----         --------
1601akon3w1t1m7i9el4pn9gl  ubuntu  ubuntu:14.04  0/1

通过inspect查看服务的详细情况。

root@controller:~# swarmctl service inspect ubuntu
ID                : 1601akon3w1t1m7i9el4pn9gl
Name              : ubuntu
Replicas          : 0/1
Template
 Container
  Image           : ubuntu:14.04

Task ID                      Service    Slot    Image           Desired State    Last State                  Node
-------                      -------    ----    -----           -------------    ----------                  ----
9qflavdz3okfqtael5558o8ey    ubuntu     1       ubuntu:14.04    RUNNING          PREPARING 34 seconds ago    controller


Last State是PREPARING,说明controller还没有启动ubuntu容器,因为本地还没有镜像,需要从镜像仓库拉取。


4、更新服务

SwarmKit提供的swarmctl service  update功能可以对已有的服务的信息进行更新,例如镜像版本,实例参数(CPU、内存、端口号、网络、卷...)、副本数、标签、环境变量等进行更新

root@controller:~# swarmctl service update ubuntu
Error: no changes detected
Usage:
  swarmctl service update <service ID> [flags]

Flags:
      --args value                  container args (default [])
      --bind value                  define a bind mount (default [])
      --command value               override entrypoint (default [])
      --constraint value            Placement constraint (node.labels.key==value) (default [])
      --cpu-limit string            CPU cores limit (e.g. 0.5)
      --cpu-reservation string      number of CPU cores reserved (e.g. 0.5)
      --env value                   container env (default [])
      --image string                container image
      --label value                 service label (key=value) (default [])
      --memory-limit string         memory limit (e.g. 512m)
      --memory-reservation string   amount of reserved memory (e.g. 512m)
      --name string                 service name
      --network string              network name
      --ports value                 ports (default [])
      --replicas uint               number of replicas for the service (only works in replicated service mode) (default 1)
      --restart-condition string    condition to restart the task (any, failure, none) (default "any")
      --restart-delay string        delay between task restarts (default "5s")
      --restart-max-attempts uint   maximum number of restart attempts (0 = unlimited)
      --restart-window string       time window to evaluate restart attempts (0 = unbound) (default "0s")
      --update-delay string         delay between task updates (0s = none) (default "0s")
      --update-parallelism uint     task update parallelism (0 = all at once)
      --volume value                define a volume mount (default [])

5、节点管理

root@controller:~# swarmctl service inspect ubuntu
ID                : 1601akon3w1t1m7i9el4pn9gl
Name              : ubuntu
Replicas          : 0/3
Template
 Container
  Image           : ubuntu:14.04

Task ID                      Service    Slot    Image           Desired State    Last State                  Node
-------                      -------    ----    -----           -------------    ----------                  ----
e4xaodj9v2sest4ix141wgu4t    ubuntu     1       ubuntu:14.04    ACCEPTED         ACCEPTED now                work2
0yfsmmpvq51clsywnfjn077va    ubuntu     2       ubuntu:14.04    RUNNING          PREPARING 17 minutes ago    work1
706418wj5je0h7jtirvqmhlfi    ubuntu     3       ubuntu:14.04    ACCEPTED         ACCEPTED now                controller

我们可以通过swarmctl node drain work2命令将work2设置为不可用状态

(如果希望激活,使用swarmctl node activate work2命令即可)

同时我们可以查看到状态改变

root@controller:~# swarmctl node drain work2
root@controller:~# swarmctl service inspect ubuntu
ID                : 1601akon3w1t1m7i9el4pn9gl
Name              : ubuntu
Replicas          : 0/3
Template
 Container
  Image           : ubuntu:14.04

Task ID                      Service    Slot    Image           Desired State    Last State                  Node
-------                      -------    ----    -----           -------------    ----------                  ----
2znvxhamkbvj2eqf5q3xxin7d    ubuntu     1       ubuntu:14.04    RUNNING          PREPARING 6 seconds ago     controller
0yfsmmpvq51clsywnfjn077va    ubuntu     2       ubuntu:14.04    RUNNING          PREPARING 19 minutes ago    work1
890be0l7eapjep5v8k70b2pto    ubuntu     3       ubuntu:14.04    ACCEPTED         ACCEPTED 2 seconds ago      controller
我们可以看到

任务已经迁移到controller和work1上了。


后面可以看到controller的实例以ubuntu.3.xxxxx;work1的实例以ubuntu.2.xxxxx;work2的实例名以ubuntu.1.xxxxx,这是因为我执行了(swarmctl service update redis --replicas 3)副本操作。


我们看到在controller里面已经可以看到原来work2的实例了

root@controller:~# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                          PORTS               NAMES
a159185201f8        ubuntu:14.04        "/bin/bash"         5 seconds ago        Exited (0) 4 seconds ago                            ubuntu.1.58zd7f87vldcici33i272v20o
0507647074e1        ubuntu:14.04        "/bin/bash"         8 seconds ago        Exited (0) 6 seconds ago                            ubuntu.3.dwbyeoyk9bgu07u2wz3utoput
50a5aea24ed2        ubuntu:14.04        "/bin/bash"         17 seconds ago       Exited (0) 16 seconds ago                           ubuntu.1.9dwu5e3tb9cte2w2fnp7lyd1r
f8244e6e4d55        ubuntu:14.04        "/bin/bash"         23 seconds ago       Exited (0) 22 seconds ago                           ubuntu.3.0dipyu39lyhvqtc68bq5tfld2
7019fbed41f1        ubuntu:14.04        "/bin/bash"         31 seconds ago       Exited (0) 29 seconds ago                           ubuntu.1.2kjo445p3l2fucgo82e06gy1e
a63eac32c424        ubuntu:14.04        "/bin/bash"         33 seconds ago       Exited (0) 31 seconds ago                           ubuntu.3.anr48rmhqyh98v2xmwvafb60x
928882228a57        ubuntu:14.04        "/bin/bash"         46 seconds ago       Exited (0) 45 seconds ago                           ubuntu.3.bymam97t2g6libynguczc6lm8
6ec7ad7a6bd2        ubuntu:14.04        "/bin/bash"         48 seconds ago       Exited (0) 47 seconds ago                           ubuntu.1.ct3fc130hki4keq6ax0uq0at6
98e9728e9000        ubuntu:14.04        "/bin/bash"         56 seconds ago       Exited (0) 55 seconds ago                           ubuntu.3.arjd0dtjefhofd5xnxw48loli
bcf0708aae66        ubuntu:14.04        "/bin/bash"         59 seconds ago       Exited (0) 58 seconds ago                           ubuntu.1.augk3ferw6tvwe8qs6fk3wtt2
57db27be5bbd        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.3.6tkx22dhta5gtmkbpk83osj4k
6fdf69e49e98        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.4qamwbaezfp3iigo0kkwzh5my
3715774ddda2        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.3.5h2ohspdob81pueb74xmrr9q9
22071586b2a0        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.f4lmuk63l20kp165qe6uovo1n
797aa652b3da        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.3.2pyygimy8ilaub70gltsa96d0
22143a2f7795        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.dvrjxce69qa382otiw225gmac
1a26f50f87fa        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.3.byb6g2iuehmvgif7ss0nwnk0d
f43232d8a05a        ubuntu:14.04        "/bin/bash"         2 minutes ago        Exited (0) About a minute ago                       ubuntu.1.8zucjezgh65dv58jckgt0ycjj

后面我们将work2激活后,看到work2的实例又启用了

root@worker2:~# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                          PORTS               NAMES
9424a0b3dd72        ubuntu:14.04        "/bin/bash"         10 seconds ago       Exited (0) 9 seconds ago                            ubuntu.1.ezptv6gxoalsqdawl9uz2anwk
6dd10817d0cf        ubuntu:14.04        "/bin/bash"         20 seconds ago       Exited (0) 19 seconds ago                           ubuntu.1.8lnrs64khjyqj69cnt03o20cs
a71f20118aba        ubuntu:14.04        "/bin/bash"         32 seconds ago       Exited (0) 31 seconds ago                           ubuntu.1.2i81636zz5vpkzjxs0emswmys
ef939614adcc        ubuntu:14.04        "/bin/bash"         42 seconds ago       Exited (0) 41 seconds ago                           ubuntu.1.0b66k4exrehqc1zatoalmvfki
e06a3e77986f        ubuntu:14.04        "/bin/bash"         52 seconds ago       Exited (0) 52 seconds ago                           ubuntu.1.41gzjjru89aym5yxx56f1li77
c9a5e2ecacfd        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.5hwusgfhb20fkzo1nloragkia
0785dd33cdf1        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.71qca8ze2rpb94p3zv86y4u0e
6fc4a1a657aa        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.0vujqtkoz7bl3rtui2kpexjhm
b2c046e8515c        ubuntu:14.04        "/bin/bash"         About a minute ago   Exited (0) About a minute ago                       ubuntu.1.54elpfbr77jaq420mar5hn3vi

结语

目前SwarmKit由于刚开源,肯定有大量的问题存在,而且都还不能进入生产系统,但是作为直接集成到Docker Engine的优势比较明显,我个人在使用过程中还发现了不少问题。(问题可能是我理解不同)

例如:为什么我频繁刷新某个机器的docker实例,相关的Contrainer_ID并不一样,频繁变化,如果我全部启动后,contrainer_id固定下来了,但是类似上述的迁移还发生了变化。


另外,我理解的集群管理应该是我创建一个镜像的服务,应该根据内部的调度算法,创建某个容器实例在某个集群节点上,(上述看到的很多,是因为我更新了服务副本为3,执行了swarmctl service update redis --replicas 3),

但是为什么每个集群节点都包含9个容器实例,难道是根据我的虚拟机配置2vCPU+2vG,直接占满资源么?


后面还需要对该工具进一步研究,不过我还是比较看好该工具在容器集群的发展前景的。













以上是关于Docker全新的容器集群项目——SwarmKit初步测试的主要内容,如果未能解决你的问题,请参考以下文章

Docker Swarm

docker技术剖析--docker1.12版本+swarmkit

docker swarm英文文档学习-2-关键概念

Docker Swarm群集项目案例

小规模容器编排使用Docker Swarm不香么,用个锤子的kubernetes

docker swarm集群