标签：资源 tst exp capture stay proc tor trail min
架构图
常见的CNI网络插件：
Flannel三种模型：
host-gw模型即网关模式，在服务器直接添加一条静态路由即可，效率高，各节点必须在同一网段
10.4.7.21通过172.7.22.0这条静态路由连接10.4.7.22这台主机再连接172.7.21网段，反之亦是
/opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}‘
/opt/etcd/etcdctl set /coreos.com/network/config # 查看
/opt/etcd/etcdctl rm /coreos.com/network/config # 删除
在不同网段，可以用VxLAN模式，主机A会生成一个flannel.1网卡，通过封装头从flannel.1网卡出去通过flanne隧道传出，从flannel.1网卡传入拆包，到达目标主机网卡到指向的静态路由，效率低
/opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN"}}‘直接路由模式，结合了VxLAN和host-gw模型
/opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN","Directrouting": true}}‘| 主机名 | IP | 角色 | 节点 |
|---|---|---|---|
| hdss21-host.com | 10.4.7.21 | Flannel | Node |
| hdss22-host.com | 10.4.4.22 | Flannel | Node |
在hdss21-host.com操作，hdss22-host.com操作类似
cd /opt/src
rz ==> flannel-v0.12.0-linux-amd64.tar.gz
mkdir /opt/flanneld-v0.12.0
tar xf flannel-v0.12.0-linux-amd64.tar.gz -C /opt/flanneld-v0.12.0
ln -s /opt/flanneld-v0.12.0 /opt/flannelcat > /opt/flannel/subnet.env <<‘eof‘
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false
eof不同地方
172.4.7.21 配置 FLANNEL_SUBNET=172.7.21.1/24
172.4.7.22 配置 FLANNEL_SUBNET=172.7.22.1/24
mkdir /opt/flannel/certs
scp hdss7-200:/opt/certs/ca.pem /opt/flannel/certs/
scp hdss7-200:/opt/certs/client.pem /opt/flannel/certs/
scp hdss7-200:/opt/certs/client-key.pem /opt/flannel/certs/cat > /opt/flannel/flanneld.sh <<‘eof‘
#!/bin/sh
./flanneld --public-ip=10.4.7.21 --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 --etcd-keyfile=./certs/client-key.pem --etcd-certfile=./certs/client.pem --etcd-cafile=./certs/ca.pem --iface=ens33 --subnet-file=./subnet.env --healthz-port=2401
eof
chmod +x /opt/flannel/flanneld.sh
mkdir -p /data/logs/flanneld不同地方：
172.4.7.21 配置 -public-ip=10.4.7.21
172.4.7.22 配置 -public-ip=10.4.7.22
--iface=ens33 配置为服务器网卡名
cat > /etc/supervisord.d/flanneld.ini << ‘eof‘
[program:flanneld-7-21]
command=/opt/flannel/flanneld.sh ; the program (relative uses PATH, can take args)
numprocs=1 ; number of processes copies to start (def 1)
directory=/opt/flannel ; directory to cwd to before exec (def no cwd)
autostart=true ; start at supervisord start (default: true)
autorestart=true ; retstart at unexpected quit (default: true)
startsecs=22 ; number of secs prog must stay running (def. 1)
startretries=3 ; max # of serial start failures (default 3)
exitcodes=0,2 ; ‘expected‘ exit codes for process (default 0,2)
stopsignal=QUIT ; signal used to kill process (default TERM)
stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
user=root ; setuid to this UNIX account to run the program
redirect_stderr=false ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log ; stdout log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4 ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB ; number of bytes in ‘capturemode‘ (default 0)
stdout_events_enabled=false ; emit events on stdout writes (default false)
stderr_logfile=/data/logs/flanneld/flanneld.stderr.log ; stderr log path, NONE for none; default AUTO
stderr_logfile_maxbytes=64MB ; max # logfile bytes b4 rotation (default 50MB)
stderr_logfile_backups=4 ; # of stderr logfile backups (default 10)
stderr_capture_maxbytes=1MB ; number of bytes in ‘capturemode‘ (default 0)
stderr_events_enabled=false
eof不同地方：[program:flanneld-7-21]
hdss21-host.com操作，任意一台etcd机器操作都可以
/opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}‘
supervisorctl update
supervisorctl status
[root@hdss7-22 ~]# route -n # 在22主机上可看到添加了一条10.4.7.21的静态路由
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 10.4.7.21 255.255.255.0 UG 0 0 0 ens33
172.7.22.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
[root@hdss7-21 ~]# route -n # 在21主机上可看到添加了一条10.4.7.22的静态路由
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
172.7.22.0 10.4.7.22 255.255.255.0 UG 0 0 0 ens33
[root@hdss7-21 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
my-nginx-h88nr 1/1 Running 1 3h46m 172.7.22.2 hdss7-22.host.com <none> <none>
my-nginx-vrlsk 1/1 Running 0 3h46m 172.7.21.2 hdss7-21.host.com <none> <none>
[root@hdss7-21 ~]# ping 172.7.22.2
[root@hdss7-22 ~]# ping 172.7.21.2[root@hdss7-21 ~]# supervisorctl stop flanneld
[root@hdss7-22 ~]# supervisorctl stop flanneld
[root@hdss7-22 ~]# route del -net 172.7.21.0/24 gw 10.4.7.21
[root@hdss7-21 ~]# route del -net 172.7.22.0/24 gw 10.4.7.22
[root@hdss7-21 ~]# /opt/etcd/etcdctl rm /coreos.com/network/config
[root@hdss7-21 ~]# /opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN"}}‘
[root@hdss7-21 ~]# /opt/etcd/etcdctl get /coreos.com/network/config
[root@hdss7-21 ~]# supervisorctl start flanneld
[root@hdss7-22 ~]# supervisorctl start flanneld
[root@hdss7-21 ~]# ifconfig flannel.1
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.7.21.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::803:ff:fe6d:1f33 prefixlen 64 scopeid 0x20<link>
ether 0a:03:00:6d:1f:33 txqueuelen 0 (Ethernet)
RX packets 10 bytes 2234 (2.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14 bytes 892 (892.0 B)
TX errors 0 dropped 8 overruns 0 carrier 0 collisions 0
[root@hdss7-22 etcd]# ifconfig flannel.1
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.7.22.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::d0c2:4bff:fe29:df00 prefixlen 64 scopeid 0x20<link>
ether d2:c2:4b:29:df:00 txqueuelen 0 (Ethernet)
RX packets 14 bytes 892 (892.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 2234 (2.1 KiB)
TX errors 0 dropped 8 overruns 0 carrier 0 collisions 0
[root@hdss7-22 etcd]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 172.7.21.0 255.255.255.0 UG 0 0 0 flannel.1
172.7.22.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
[root@hdss7-21 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
172.7.22.0 172.7.22.0 255.255.255.0 UG 0 0 0 flannel.1
[root@hdss7-21 ~]# ping 172.7.22.2
[root@hdss7-22 ~]# ping 172.7.21.2hdss7-21主机操作为例，7-22操作类似
supervisorctl stop flanneld
[root@hdss7-21 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
172.7.22.0 172.7.22.0 255.255.255.0 UG 0 0 0 flannel.1
route del -net 172.7.22.0/24 gw 172.7.22.0 # 删除旧flannel路由
/opt/etcd/etcdctl rm /coreos.com/network/config # 删除旧数据
/opt/etcd/etcdctl set /coreos.com/network/config ‘{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN","Directrouting": true}}‘ #设置新数据
/opt/etcd/etcdctl get /coreos.com/network/config
supervisorctl start flanneld
[root@hdss7-21 ~]# route -n # 由于服务器同网段直接路由模型使用host-gw模型
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.4.7.254 0.0.0.0 UG 100 0 0 ens33
10.4.7.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.7.21.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
172.7.22.0 10.4.7.22 255.255.255.0 UG 0 0 0 ens33
[root@hdss7-21 ~]# ping 172.7.22.2二进制部署K8s集群第18节addons之flannel三种模型安装部署详解
标签：资源 tst exp capture stay proc tor trail min
原文地址：https://blog.51cto.com/yht1990/2539920
