云原生网络排障利器

Posted 2021-04-23 shok

最近在学习网络架构，于是遇到了许多网络问题，深陷其中，无法自拔。相信很多小伙伴和我一样，下面就提供一个针对网络的解忧器。

netshoot: a Docker + Kubernetes network trouble-shooting swiss-army container

                    dP            dP                           dP
                    88            88                           88
88d888b. .d8888b. d8888P .d8888b. 88d888b. .d8888b. .d8888b. d8888P
88'  `88 88ooood8   88   Y8ooooo. 88'  `88 88'  `88 88'  `88   88
88    88 88.  ...   88         88 88    88 88.  .88 88.  .88   88
dP    dP `88888P'   dP   `88888P' dP    dP `88888P' `88888P'   dP

Purpose: Docker and Kubernetes network troubleshooting can become complex. With proper understanding of how Docker and Kubernetes networking works and the right set of tools, you can troubleshoot and resolve these networking issues. The netshoot container has a set of powerful networking tshooting tools that can be used to troubleshoot Docker networking issues. Along with these tools come a set of use-cases that show how this container can be used in real-world scenarios.

Network Namespaces: Before starting to use this tool, it's important to go over one key topic: Network Namespaces. Network namespaces provide isolation of the system resources associated with networking. Docker uses network and other type of namespaces (pid,mount,user..etc) to create an isolated environment for each container. Everything from interfaces, routes, and IPs is completely isolated within the network namespace of the container.

Kubernetes also uses network namespaces. Kubelets creates a network namespace per pod where all containers in that pod share that same network namespace (eths,IP, tcp sockets...etc). This is a key difference between Docker containers and Kubernetes pods.

Cool thing about namespaces is that you can switch between them. You can enter a different container's network namespace, perform some troubleshooting on its network's stack with tools that aren't even installed on that container. Additionally, netshoot can be used to troubleshoot the host itself by using the host's network namespace. This allows you to perform any troubleshooting without installing any new packages directly on the host or your application's package.

• Container's Network Namespace: If you're having networking issues with your application's container, you can launch netshoot with that container's network namespace like this:

  $ docker run -it --net container:<container_name> nicolaka/netshoot

• Host's Network Namespace: If you think the networking issue is on the host itself, you can launch netshoot with that host's network namespace:

  $ docker run -it --net host nicolaka/netshoot

• Network's Network Namespace: If you want to troubleshoot a Docker network, you can enter the network's namespace using nsenter. This is explained in the nsenter section below.

Kubernetes

If you want to spin up a throw away container for debugging.

$ kubectl run tmp-shell --rm -i --tty --image nicolaka/netshoot -- /bin/bash

And if you want to spin up a container on the host's network namespace.

$ kubectl run tmp-shell --rm -i --tty --overrides='{"spec": {"hostNetwork": true}}' --image nicolaka/netshoot -- /bin/bash

Network Problems

Many network issues could result in application performance degradation. Some of those issues could be related to the underlying networking infrastructure(underlay). Others could be related to misconfiguration at the host or Docker level. Let's take a look at common networking issues:

• latency

• routing

• DNS resolution

• firewall

• incomplete ARPs

To troubleshoot these issues, netshoot includes a set of powerful tools as recommended by this diagram.

Included Packages: The following packages are included in netshoot. We'll go over some with some sample use-cases.

apache2-utils
bash
bind-tools
bird
bridge-utils
busybox-extras
calicoctl
conntrack-tools
ctop
curl
dhcping
drill
ethtool
file
fping
httpie
iftop
iperf
iproute2
ipset
iptables
iptraf-ng
iputils
ipvsadm
jq
libc6-compat
liboping
mtr
net-snmp-tools
netcat-openbsd
netgen
nftables
ngrep
nmap
nmap-nping
openssl
py-crypto
py2-virtualenv
python2
scapy
socat
strace
tcpdump
tcptraceroute
termshark
tshark
util-linux
vim
websocat

Sample Use-cases

iperf

Purpose: test networking performance between two containers/hosts.

Create Overlay network:

$ docker network create -d overlay perf-test

Launch two containers:

以上是关于云原生网络排障利器的主要内容，如果未能解决你的问题，请参考以下文章
 
 云原生架构系列：微服务架构的四大金刚利器
 阿里云ACM：云原生配置管理利器
 云原生利器 -- Nexus3
 buildah 云原生时代的镜像打包利器
 云原生，智慧营销破圈新利器
 系统排障掉坑背锅？亚马逊云科技要为开发者研发提效了