Jusene's Blog

Docker三剑客之一 Docker Swarm

docker
字数统计: 2.2k阅读时长: 11 min
2017/11/14 Share

Docker Swarm

Docker Swarm是多个Docker主机封装为单个大型的虚拟Docker主机，是Docker官方推出的Docker集群解决方案，快速打造一套容器云平台。

安装和使用Docker Swarm

consul服务发现

  1. 下载镜像

Docker官方已经提供了swarm镜像，需要在所有被swarm管理的docker主机上下载该镜像。

1
~]# docker run --rm swarm -v
2
swarm version 1.2.8 (48d86b1)
  1. 配置节点

添加docker daemon的网络监听，-H 0.0.0.0:2375

1
~]# vim /usr/lib/systemd/system/docker.service
2
.......
3
ExecStart=/usr/bin/dockerd-current \
4
			-H 0.0.0.0:2375-H unix:///var/run/docker.sock \
5
			.....
  1. 启动集群

Docker Swarm需要使用到服务发现服务，Swarm支持以下几种方式：Docker Hub、本地文件、Etcd、Consul、Zookeeper和手动指定IP地址信息等。

  • 启动consul服务后端

    1
    ~]# docker run -p 8500:8500 --rm consul  agent -server -client=0.0.0.0 -bootstrap            
    2
    bootstrap = true: do not enable unless necessary
    3
    ==> Starting Consul agent...
    4
    ==> Consul agent running!
    5
               Version: 'v1.0.0'
    6
               Node ID: 'c4011f35-a4c5-f124-1409-fbb2dbf09d91'
    7
             Node name: 'd4ca61c1fb93'
    8
            Datacenter: 'dc1' (Segment: '<all>')
    9
                Server: true (Bootstrap: true)
    10
           Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, DNS: 8600)
    11
          Cluster Addr: 172.17.0.2 (LAN: 8301, WAN: 8302)
    12
               Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
    13
    14
    ==> Log data will now stream in as it occurs:
    15
    16
        2017/11/17 16:27:27 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:c4011f35-a4c5-f124-1409-fbb2dbf09d91 Address:172.17.0.2:8300}]
    17
        2017/11/17 16:27:27 [INFO] raft: Node at 172.17.0.2:8300 [Follower] entering Follower state (Leader: "")
    18
        2017/11/17 16:27:27 [INFO] serf: EventMemberJoin: d4ca61c1fb93.dc1 172.17.0.2
    19
        2017/11/17 16:27:27 [INFO] serf: EventMemberJoin: d4ca61c1fb93 172.17.0.2
    20
        2017/11/17 16:27:27 [INFO] consul: Adding LAN server d4ca61c1fb93 (Addr: tcp/172.17.0.2:8300) (DC: dc1)
    21
        2017/11/17 16:27:27 [INFO] consul: Handled member-join event for server "d4ca61c1fb93.dc1" in area "wan"
    22
        2017/11/17 16:27:27 [INFO] agent: Started DNS server 0.0.0.0:8600 (udp)
    23
        2017/11/17 16:27:27 [INFO] agent: Started DNS server 0.0.0.0:8600 (tcp)
    24
        2017/11/17 16:27:27 [INFO] agent: Started HTTP server on [::]:8500 (tcp)
    25
        2017/11/17 16:27:35 [ERR] agent: failed to sync remote state: No cluster leader
    26
        2017/11/17 16:27:36 [WARN] raft: Heartbeat timeout from "" reached, starting election
    27
        2017/11/17 16:27:36 [INFO] raft: Node at 172.17.0.2:8300 [Candidate] entering Candidate state in term 2
    28
        2017/11/17 16:27:36 [INFO] raft: Election won. Tally: 1
    29
        2017/11/17 16:27:36 [INFO] raft: Node at 172.17.0.2:8300 [Leader] entering Leader state
    30
        2017/11/17 16:27:36 [INFO] consul: cluster leadership acquired
    31
        2017/11/17 16:27:36 [INFO] consul: New leader elected: d4ca61c1fb93
    32
        2017/11/17 16:27:36 [INFO] consul: member 'd4ca61c1fb93' joined, marking health alive
    33
        2017/11/17 16:27:37 [INFO] agent: Synced node info

  • 启动swarm管理节点

    1
    ~]# docker run --rm -p 4000:4000 swarm manage -H :4000 --replication --advertise 10.211.55.43:4000 consul://10.211.55.43:8500 
    2
    time="2017-11-17T16:27:42Z" level=info msg="Initializing discovery without TLS" 
    3
    time="2017-11-17T16:27:42Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp 
    4
    time="2017-11-17T16:27:42Z" level=info msg="Leader Election: Cluster leadership lost" 
    5
    time="2017-11-17T16:27:42Z" level=info msg="Leader Election: Cluster leadership acquired"

    为了提高可用性我们还可以启动从管理节点：

    1
    docker run -p 4000:4000 --rm swarm manage -H :4000 --replication --advertise 10.211.55.6:4000 consul://10.211.55.43:8500
    2
    time="2017-08-26T07:50:57Z" level=info msg="Initializing discovery without TLS" 
    3
    time="2017-08-26T07:50:57Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp 
    4
    time="2017-08-26T07:50:57Z" level=info msg="Leader Election: Cluster leadership lost" 
    5
    time="2017-08-26T07:50:57Z" level=info msg="New leader elected: 10.211.55.43:4000"

  • 启动工作节点

    1
    ~]# docker run -d swarm join --advertise 10.211.55.6:2375 consul://10.211.55.43:8500
    2
    aa0d9197e16f9621785297893577ef1588135ff6519f6441dd6c4d631d767a51
    3
    ~]# docker run -d swarm join --advertise 10.211.55.43:2375 consul://10.211.55.43:8500 
    4
    48e577ed3b9b4841bfb211416c4021582d5dec8172edc303a585d8126d91f511
    5
    ~]# docker run --rm  swarm list consul://10.211.55.43:8500
    6
    time="2017-11-17T16:37:27Z" level=info msg="Initializing discovery without TLS" 
    7
    10.211.55.43:2375
    8
    10.211.55.6:2375
    9
    ~]# docker -H 10.211.55.43:4000 info
    10
    Containers: 6
    11
     Running: 4
    12
     Paused: 0
    13
     Stopped: 2
    14
    Images: 6
    15
    Server Version: swarm/1.2.8
    16
    Role: primary
    17
    Strategy: spread
    18
    Filters: health, port, containerslots, dependency, affinity, constraint, whitelist
    19
    Nodes: 2
    20
     INIT: 10.211.55.6:2375
    21
      └ ID: IHRY:ZZ3W:CGPH:YCAW:L2UY:ROM3:7NH2:RSQG:GSXM:UM42:5GVQ:URCX|10.211.55.6:2375
    22
      └ Status: Healthy
    23
      └ Containers: 3 (1 Running, 0 Paused, 2 Stopped)
    24
      └ Reserved CPUs: 0 / 2
    25
      └ Reserved Memory: 0 B / 1.883 GiB
    26
      └ Labels: kernelversion=3.10.0-327.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), ostype=linux, storagedriver=devicemapper
    27
      └ UpdatedAt: 2017-11-17T16:54:23Z
    28
      └ ServerVersion: 1.12.6
    29
     node2: 10.211.55.43:2375
    30
      └ ID: LBNW:K6OR:XKKB:EZLI:W4JO:WCAB:O44H:KYN7:4W2Y:VB6I:LIJK:PINL|10.211.55.43:2375
    31
      └ Status: Healthy
    32
      └ Containers: 3 (3 Running, 0 Paused, 0 Stopped)
    33
      └ Reserved CPUs: 0 / 2
    34
      └ Reserved Memory: 0 B / 1.883 GiB
    35
      └ Labels: kernelversion=3.10.0-327.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), ostype=linux, storagedriver=devicemapper
    36
      └ UpdatedAt: 2017-11-17T16:54:28Z
    37
      └ ServerVersion: 1.12.6

我们可以指定docker服务到swarm的管理接口，这个swarm集群会像普通docker一样工作。

1
~]# docker -H 10.211.55.43:4000 run --rm centos ping 8.8.8.8
2
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
3
64 bytes from 8.8.8.8: icmp_seq=1 ttl=127 time=52.4 ms
4
64 bytes from 8.8.8.8: icmp_seq=2 ttl=127 time=76.8 ms
5
64 bytes from 8.8.8.8: icmp_seq=4 ttl=127 time=80.2 ms
6
64 bytes from 8.8.8.8: icmp_seq=5 ttl=127 time=52.2 ms
7
64 bytes from 8.8.8.8: icmp_seq=6 ttl=127 time=55.3 ms

这个过程通过swarm的管理接口调度后端的docker主机进行运行，所以如果swarm 集群出现故障，被调度运行的docker容器也可以一样的运行。

Docker Hub

Docker Hub是一种更加方便创建swarm集群的方法。

  • 创建集群id

    1
    ~]# docker run --rm swarm create
    2
    2425280328ade901f1d3f21315a00063
    3
    Token based discovery is now deprecated and might be removed in the future.
    4
    It will be replaced by a default discovery backed by Docker Swarm Mode.
    5
    Other mechanisms such as consul and etcd will continue to work as expected.

    国内的网是不推荐这么做，而且从docker hub上看见这个将在不久后废弃。

  • 配置管理节点

    1
    ~]# docker run --rm -p 4000:4000 swarm manage  -H :4000 --addr=10.211.55.6:4000 token://2425280328ade901f1d3f21315a00063                                                   
    2
    time="2017-08-26T08:29:41Z" level=info msg="Listening for HTTP" addr=":4000" proto=tcp

  • 配置工作节点

    1
    ~]# docker run --rm swarm join -addr=10.211.55.43:2375 token://2425280328ade901f1d3f21315a00063
    2
    time="2017-11-17T17:08:01Z" level=info msg="Registering on the discovery service every 1m0s..." addr="10.211.55.43:2375" discovery="token://2425280328ade901f1d3f21315a00063"

  • 查看

    1
    ~]# docker run --rm swarm list token://2425280328ade901f1d3f21315a00063
    2
    10.211.55.43:2375
    3
    ~]# docker -H 10.211.55.6:4000 run --rm centos ping 8.8.8.8
    4
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    5
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=127 time=56.7 ms
    6
    64 bytes from 8.8.8.8: icmp_seq=4 ttl=127 time=51.8 ms
    7
    ...
    8
    ~]# docker -H 10.211.55.6:4000 ps -a
    9
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
    10
    d3c36bbea89c        swarm               "/swarm join -addr=10"   11 weeks ago        Up 6 minutes        2375/tcp            node2/cocky_swirles

服务发现后端

  • token://{token}: 使用Docker Hub提供的服务
  • file://path/to/file: 使用本地文件
  • consul://{ip:port}: 使用consul服务
  • etcd://{ip:port}: 使用etcd服务
  • zk://{ip:port}: 使用zookeeper服务
  • [nodes://]{ip1},{ip2}: 手动指定节点的地址

使用文件：

1
tee /tmp/cluster_info << EOF
2
10.211.55.6:2375
3
10.211.55.43:2375
1
docker run -d -p 4000:4000 -v /tmp/cluster_info:/tmp/cluster_info swarm manage -H :4000 file:///tmp/cluster_info

Swarm中的调度器

swarm目前支持三种调度器：spread，binpack，random

在swarm manage的时候可以使用-strategy来指定调度器，默认spread。

  • spread：如果节点配置相同，选择一个正在运行容器数量最小的节点。
  • binspread：尽可能地把所以容器放在一台节点上运行。
  • random：直接随机分配，不考虑节点状态。

Swarm中的过滤器

swarm的调度器可以按照指定的调度策略自动分配到容器节点，但可能有时也需要干预下，让i/o频繁的尽量到ssd的容器中等，目前支持5种过滤器：Constraint,Affinity,Port,Dependency,Health。

  • Constraint
    通过绑定到节点的键对值来过滤服务运行的相对应的节点标签：
    1
    docker daemon --label ser=db -H 0.0.0.0:2375....

在swarm启动容器的时候采用-e constartint:key=value 来过滤匹配到的swarm的工作节点。

  • Affinty

可以在启动一个容器的时候，将它分配到一个已经真正运行的容器的同一台swarm工作节点上。

1
docker -H 10.211.55.43:4000 run -d -e affinity:container==mysql nginx...
  • Port

端口为判断条件

  • Dependency

    数据卷或链接为判断条件

原文作者：Zhang Jusene

原文链接：http://jusene.github.io/2017/11/14/docker-swarm/

发表日期：November 14th 2017, 11:21:37 pm

更新日期：November 30th 2019, 2:10:00 am

版权声明：本文采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可

CATALOG
  1. 1. Docker Swarm
  2. 2. 安装和使用Docker Swarm
    1. 2.1. consul服务发现
    2. 2.2. Docker Hub
    3. 2.3. 服务发现后端
  3. 3. Swarm中的调度器
  4. 4. Swarm中的过滤器
Total : 190
2020
2019
2018
2017
python boot troubshooting docker drbd ssh web ELK kubernetes Kubernetes git ansible tools dns corosync consul Django etcd security ftp Go bash bigdata grub heartbeat Java kickstart kvm lvs mongodb mysql mesh nc nfs mogilefs openresty upgrade puppet rabbitmq saltstack rsync screen socat svn tcpdump fastdfs haproxy iscsi keepalived cache openssl vpn optimization prometheus redis RPM cncf supervisord kernel JAVA
python Linux troubshooting Docker docker storage nginx Nosql kubernetes Flask Kubernetes ansible Shell cluster Django Go bigdata boot Java hypervisor mysql mesh tools openresty puppet rabbitmq saltstack optimization cncf varnish tomcat JAVA