Jusene's Blog

Docker三剑客之一 Docker Swarm

字数统计: 2.2k阅读时长: 11 min
2017/11/14 Share

Docker Swarm

Docker Swarm是多个Docker主机封装为单个大型的虚拟Docker主机,是Docker官方推出的Docker集群解决方案,快速打造一套容器云平台。

安装和使用Docker Swarm

consul服务发现

  1. 下载镜像

Docker官方已经提供了swarm镜像,需要在所有被swarm管理的docker主机上下载该镜像。

1
~]# docker run --rm swarm -v
2
swarm version 1.2.8 (48d86b1)
  1. 配置节点

添加docker daemon的网络监听,-H 0.0.0.0:2375

1
~]# vim /usr/lib/systemd/system/docker.service
2
.......
3
ExecStart=/usr/bin/dockerd-current \
4
			-H 0.0.0.0:2375-H unix:///var/run/docker.sock \
5
			.....
  1. 启动集群

Docker Swarm需要使用到服务发现服务,Swarm支持以下几种方式:Docker Hub、本地文件、Etcd、Consul、Zookeeper和手动指定IP地址信息等。

  • 启动consul服务后端

    1
    ~]# docker run -p 8500:8500 --rm consul  agent -server -client=0.0.0.0 -bootstrap            
    2
    bootstrap = true: do not enable unless necessary
    3
    ==> Starting Consul agent...
    4
    ==> Consul agent running!
    5
               Version: 'v1.0.0'
    6
               Node ID: 'c4011f35-a4c5-f124-1409-fbb2dbf09d91'
    7
             Node name: 'd4ca61c1fb93'
    8
            Datacenter: 'dc1' (Segment: '<all>')
    9
                Server: true (Bootstrap: true)
    10
           Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600)
    11
          Cluster Addr: 172.17.0.2 (LAN: 8301, WAN: 8302)
    12
               Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
    13
    14
    ==> Log data will now stream in as it occurs:
    15
    16
        2017/11/17 16:27:27 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:c4011f35-a4c5-f124-1409-fbb2dbf09d91 Address:172.17.0.2:8300}]
    17
        2017/11/17 16:27:27 [INFO] raft: Node at 172.17.0.2:8300 [Follower] entering Follower state (Leader: "")
    18
        2017/11/17 16:27:27 [INFO] serf: EventMemberJoin: d4ca61c1fb93.dc1 172.17.0.2
    19
        2017/11/17 16:27:27 [INFO] serf: EventMemberJoin: d4ca61c1fb93 172.17.0.2
    20
        2017/11/17 16:27:27 [INFO] consul: Adding LAN server d4ca61c1fb93 (Addr: tcp/172.17.0.2:8300) (DC: dc1)
    21
        2017/11/17 16:27:27 [INFO] consul: Handled member-join event for server "d4ca61c1fb93.dc1" in area "wan"
    22
        2017/11/17 16:27:27 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
    23
        2017/11/17 16:27:27 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
    24
        2017/11/17 16:27:27 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
    25
        2017/11/17 16:27:35 [ERR] agent: failed to sync remote state: No cluster leader
    26
        2017/11/17 16:27:36 [WARN] raft: Heartbeat timeout from "" reached, starting election
    27
        2017/11/17 16:27:36 [INFO] raft: Node at 172.17.0.2:8300 [Candidate] entering Candidate state in term 2
    28
        2017/11/17 16:27:36 [INFO] raft: Election won. Tally: 1
    29
        2017/11/17 16:27:36 [INFO] raft: Node at 172.17.0.2:8300 [Leader] entering Leader state
    30
        2017/11/17 16:27:36 [INFO] consul: cluster leadership acquired
    31
        2017/11/17 16:27:36 [INFO] consul: New leader elected: d4ca61c1fb93
    32
        2017/11/17 16:27:36 [INFO] consul: member 'd4ca61c1fb93' joined, marking health alive
    33
        2017/11/17 16:27:37 [INFO] agent: Synced node info
  • 启动swarm管理节点

    1
    ~]# docker run --rm -p 4000:4000 swarm manage -H :4000 --replication --advertise 10.211.55.43:4000 consul://10.211.55.43:8500 
    2
    time="2017-11-17T16:27:42Z" level=info msg="Initializing discovery without TLS" 
    3
    time="2017-11-17T16:27:42Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp 
    4
    time="2017-11-17T16:27:42Z" level=info msg="Leader Election: Cluster leadership lost" 
    5
    time="2017-11-17T16:27:42Z" level=info msg="Leader Election: Cluster leadership acquired"

    为了提高可用性我们还可以启动从管理节点:

    1
    docker run -p 4000:4000 --rm swarm manage -H :4000 --replication --advertise 10.211.55.6:4000 consul://10.211.55.43:8500
    2
    time="2017-08-26T07:50:57Z" level=info msg="Initializing discovery without TLS" 
    3
    time="2017-08-26T07:50:57Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp 
    4
    time="2017-08-26T07:50:57Z" level=info msg="Leader Election: Cluster leadership lost" 
    5
    time="2017-08-26T07:50:57Z" level=info msg="New leader elected: 10.211.55.43:4000"
  • 启动工作节点

    1
    ~]# docker run -d swarm join --advertise 10.211.55.6:2375 consul://10.211.55.43:8500
    2
    aa0d9197e16f9621785297893577ef1588135ff6519f6441dd6c4d631d767a51
    3
    ~]# docker run -d swarm join --advertise 10.211.55.43:2375 consul://10.211.55.43:8500 
    4
    48e577ed3b9b4841bfb211416c4021582d5dec8172edc303a585d8126d91f511
    5
    ~]# docker run --rm  swarm list consul://10.211.55.43:8500
    6
    time="2017-11-17T16:37:27Z" level=info msg="Initializing discovery without TLS" 
    7
    10.211.55.43:2375
    8
    10.211.55.6:2375
    9
    ~]# docker -H 10.211.55.43:4000 info
    10
    Containers: 6
    11
     Running: 4
    12
     Paused: 0
    13
     Stopped: 2
    14
    Images: 6
    15
    Server Version: swarm/1.2.8
    16
    Role: primary
    17
    Strategy: spread
    18
    Filters: health, port, containerslots, dependency, affinity, constraint, whitelist
    19
    Nodes: 2
    20
     INIT: 10.211.55.6:2375
    21
      └ ID: IHRY:ZZ3W:CGPH:YCAW:L2UY:ROM3:7NH2:RSQG:GSXM:UM42:5GVQ:URCX|10.211.55.6:2375
    22
      └ Status: Healthy
    23
      └ Containers: 3 (1 Running, 0 Paused, 2 Stopped)
    24
      └ Reserved CPUs: 0 / 2
    25
      └ Reserved Memory: 0 B / 1.883 GiB
    26
      └ Labels: kernelversion=3.10.0-327.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), ostype=linux, storagedriver=devicemapper
    27
      └ UpdatedAt: 2017-11-17T16:54:23Z
    28
      └ ServerVersion: 1.12.6
    29
     node2: 10.211.55.43:2375
    30
      └ ID: LBNW:K6OR:XKKB:EZLI:W4JO:WCAB:O44H:KYN7:4W2Y:VB6I:LIJK:PINL|10.211.55.43:2375
    31
      └ Status: Healthy
    32
      └ Containers: 3 (3 Running, 0 Paused, 0 Stopped)
    33
      └ Reserved CPUs: 0 / 2
    34
      └ Reserved Memory: 0 B / 1.883 GiB
    35
      └ Labels: kernelversion=3.10.0-327.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), ostype=linux, storagedriver=devicemapper
    36
      └ UpdatedAt: 2017-11-17T16:54:28Z
    37
      └ ServerVersion: 1.12.6

我们可以指定docker服务到swarm的管理接口,这个swarm集群会像普通docker一样工作。

1
~]# docker -H 10.211.55.43:4000 run --rm centos ping 8.8.8.8
2
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
3
64 bytes from 8.8.8.8: icmp_seq=1 ttl=127 time=52.4 ms
4
64 bytes from 8.8.8.8: icmp_seq=2 ttl=127 time=76.8 ms
5
64 bytes from 8.8.8.8: icmp_seq=4 ttl=127 time=80.2 ms
6
64 bytes from 8.8.8.8: icmp_seq=5 ttl=127 time=52.2 ms
7
64 bytes from 8.8.8.8: icmp_seq=6 ttl=127 time=55.3 ms

这个过程通过swarm的管理接口调度后端的docker主机进行运行,所以如果swarm 集群出现故障,被调度运行的docker容器也可以一样的运行。

Docker Hub

Docker Hub是一种更加方便创建swarm集群的方法。

  • 创建集群id

    1
    ~]# docker run --rm swarm create
    2
    2425280328ade901f1d3f21315a00063
    3
    Token based discovery is now deprecated and might be removed in the future.
    4
    It will be replaced by a default discovery backed by Docker Swarm Mode.
    5
    Other mechanisms such as consul and etcd will continue to work as expected.

    国内的网是不推荐这么做,而且从docker hub上看见这个将在不久后废弃。

  • 配置管理节点

    1
    ~]# docker run --rm -p 4000:4000 swarm manage  -H :4000 --addr=10.211.55.6:4000 token://2425280328ade901f1d3f21315a00063                                                   
    2
    time="2017-08-26T08:29:41Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp
  • 配置工作节点

    1
    ~]# docker run --rm swarm join -addr=10.211.55.43:2375 token://2425280328ade901f1d3f21315a00063
    2
    time="2017-11-17T17:08:01Z" level=info msg="Registering on the discovery service every 1m0s..." addr="10.211.55.43:2375" discovery="token://2425280328ade901f1d3f21315a00063"
  • 查看

    1
    ~]# docker run --rm swarm list token://2425280328ade901f1d3f21315a00063
    2
    10.211.55.43:2375
    3
    ~]# docker -H 10.211.55.6:4000 run --rm centos ping 8.8.8.8
    4
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    5
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=127 time=56.7 ms
    6
    64 bytes from 8.8.8.8: icmp_seq=4 ttl=127 time=51.8 ms
    7
    ...
    8
    ~]# docker -H 10.211.55.6:4000 ps -a
    9
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
    10
    d3c36bbea89c        swarm               "/swarm join -addr=10"   11 weeks ago        Up 6 minutes        2375/tcp            node2/cocky_swirles

服务发现后端

  • token://{token}: 使用Docker Hub提供的服务
  • file://path/to/file: 使用本地文件
  • consul://{ip:port}: 使用consul服务
  • etcd://{ip:port}: 使用etcd服务
  • zk://{ip:port}: 使用zookeeper服务
  • [nodes://]{ip1},{ip2}: 手动指定节点的地址

使用文件:

1
tee /tmp/cluster_info << EOF
2
10.211.55.6:2375
3
10.211.55.43:2375
1
docker run -d -p 4000:4000 -v /tmp/cluster_info:/tmp/cluster_info swarm manage -H :4000 file:///tmp/cluster_info

Swarm中的调度器

swarm目前支持三种调度器:spread,binpack,random

在swarm manage的时候可以使用-strategy来指定调度器,默认spread。

  • spread:如果节点配置相同,选择一个正在运行容器数量最小的节点。
  • binspread:尽可能地把所以容器放在一台节点上运行。
  • random:直接随机分配,不考虑节点状态。

Swarm中的过滤器

swarm的调度器可以按照指定的调度策略自动分配到容器节点,但可能有时也需要干预下,让i/o频繁的尽量到ssd的容器中等,目前支持5种过滤器:Constraint,Affinity,Port,Dependency,Health。

  • Constraint
    通过绑定到节点的键对值来过滤服务运行的相对应的节点标签:
    1
    docker daemon --label ser=db -H 0.0.0.0:2375....

在swarm启动容器的时候采用-e constartint:key=value 来过滤匹配到的swarm的工作节点。

  • Affinty

可以在启动一个容器的时候,将它分配到一个已经真正运行的容器的同一台swarm工作节点上。

1
docker -H 10.211.55.43:4000 run -d -e affinity:container==mysql nginx...
  • Port

端口为判断条件

  • Dependency

    数据卷或链接为判断条件

CATALOG
  1. 1. Docker Swarm
  2. 2. 安装和使用Docker Swarm
    1. 2.1. consul服务发现
    2. 2.2. Docker Hub
    3. 2.3. 服务发现后端
  3. 3. Swarm中的调度器
  4. 4. Swarm中的过滤器