google/cadvisor:latest image monitor container run error

Posted 2023-03-29

【中文标题】google/cadvisor:latest image monitor container run error

【英文标题】：

【发布时间】：2017-02-14 20:51:39

【问题描述】：

我在 bugzilla.redhat.com 的上述“devicemapper”上发现了以下错误。它让我安装了“device-mapper-persistent-data”包。但是，当我安装它时，我收到了以下错误。

我尝试使用this link解决错误，但是我仍然没有设法解决。

这是我尝试过的：

1. update the libcontainer dependency
2. look at all cgroup mounts

这是错误：

[root@slave712 ~]# docker run \
--volume=/:/rootfs:ro \
--volume=/var/run:/var/run:rw \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:ro \
--publish=8080:8080 \
--detach=true \
--name=cadvisor \
google/cadvisor:latest
Unable to find image 'google/cadvisor:latest' locally
latest: Pulling from google/cadvisor
e110a4a17941: Pull complete 
e17fa94aae07: Pull complete 
7097ce2382c0: Pull complete 
Digest:     sha256:3b1f3714a27d40bf413d5fde0429ad6410bc14fcdae5a3478daa34cc701901fa
Status: Downloaded newer image for google/cadvisor:latest
a54c09247fd6572f96365dced4178f0cec8696329ed39c0349991f39ab6d660d

[root@slave712 ~]# docker logs cadvisor

I1001 02:41:16.237284 1 storagedriver.go:50] Caching stats in memory for 2m0s
I1001 02:41:16.238324 1 manager.go:140] cAdvisor running in container: "/docker/a54c09247fd6572f96365dced4178f0cec8696329ed39c0349991f39ab6d660d"
W1001 02:41:16.379106 1 manager.go:148] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I1001 02:41:16.428790 1 fs.go:116] Filesystem partitions: map[/dev/mapper/centos-root:mountpoint:/var/lib/docker/devicemapper major:253 minor:0 fsType:xfs blockSize:0 /dev/sda1:mountpoint:/rootfs/boot major:8 minor:1 fsType:xfs blockSize:0 /dev/mapper/docker-253:0-402876333-34a64b9159b501f40b949fc1ae894ee7fc1985645d65a63f4dfe689c32919e43:mountpoint:/ major:253 minor:3 fsType:xfs blockSize:0]
I1001 02:41:16.655301 1 manager.go:195] Machine: NumCores:4 CpuFrequency:3192628 MemoryCapacity:1025396736 MachineID:f40694ca9e71417fb1af85dcf66dfc39 SystemUUID:564DBE01-1C91-8E79-5C04-6FC1D390B5EB BootID:9008dc53-91fe-4510-b199-8c26241bc0b5 Filesystems:[Device:/dev/mapper/docker-253:0-402876333-34a64b9159b501f40b949fc1ae894ee7fc1985645d65a63f4dfe689c32919e43 Capacity:10725883904 Type:vfs Inodes:10484736 HasInodes:true Device:/dev/mapper/centos-root Capacity:105011822592 Type:vfs Inodes:102600704 HasInodes:true Device:/dev/sda1 Capacity:206221312 Type:vfs Inodes:204800 HasInodes:true] DiskMap:map[253:0:Name:dm-0 Major:253 Minor:0 Size:105063120896 Scheduler:none 253:1:Name:dm-1 Major:253 Minor:1 Size:2097152000 Scheduler:none 253:2:Name:dm-2 Major:253 Minor:2 Size:107374182400 Scheduler:none 253:3:Name:dm-3 Major:253 Minor:3 Size:10737418240 Scheduler:none 2:0:Name:fd0 Major:2 Minor:0 Size:4096 Scheduler:deadline 8:0:Name:sda Major:8 Minor:0 Size:107374182400 Scheduler:deadline] NetworkDevices:[Name:br-92607f6f2739 MacAddress:02:42:9a:50:1f:2b Speed:0 Mtu:1500 Name:eno16777736 MacAddress:00:0c:29:90:b5:eb Speed:1000 Mtu:1500] Topology:[Id:0 Memory:1073205248 Cores:[Id:0 Threads:[0] Caches:[] Id:1 Threads:[1] Caches:[]] Caches:[Size:6291456 Type:Unified Level:3] Id:1 Memory:0 Cores:[Id:0 Threads:[2] Caches:[] Id:1 Threads:[3] Caches:[]] Caches:[Size:6291456 Type:Unified Level:3]] CloudProvider:Unknown InstanceType:Unknown InstanceID:None
I1001 02:41:16.659951 1 manager.go:201] Version: KernelVersion:3.10.0-327.el7.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:1.12.1 CadvisorVersion:v0.24.0 CadvisorRevision:0cdf491
E1001 02:41:16.698803 1 factory.go:291] devicemapper filesystem stats will not be reported: unable to find thin_ls binary
I1001 02:41:16.698900 1 factory.go:295] Registering Docker factory
W1001 02:41:16.698962 1 manager.go:244] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I1001 02:41:16.698982 1 factory.go:54] Registering systemd factory
I1001 02:41:16.699503 1 factory.go:86] Registering Raw factory
I1001 02:41:16.700006 1 manager.go:1082] Started watching for new ooms in manager
W1001 02:41:16.715521 1 manager.go:272] Could not configure a source for OOM detection, disabling OOM events: unable to find any kernel log file available from our set: [/var/log/kern.log /var/log/messages /var/log/syslog]
I1001 02:41:16.730767 1 manager.go:285] Starting recovery of all containers
I1001 02:41:19.183769 1 manager.go:290] Recovery completed
F1001 02:41:19.254307 1 cadvisor.go:151] Failed to start container manager: inotify_add_watch /sys/fs/cgroup/cpuacct,cpu: no such file or directory

[root@slave712 ~]# docker version
Client:
Version: 1.12.1
API version: 1.24
Go version: go1.6.3
Git commit: 23cf638
Built:

OS/Arch: linux/amd64

Server:
Version: 1.12.1
API version: 1.24
Go version: go1.6.3
Git commit: 23cf638
Built:

OS/Arch: linux/amd64

[root@slave712 ~]# mount -l
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=490460k,nr_inodes=122615,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/mapper/centos-root on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
/dev/sda1 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=100140k,mode=700)
/dev/mapper/centos-root on /var/lib/docker/devicemapper type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

[root@slave712 ~]# uname -a
Linux slave712 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

[root@slave712 ~]# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

【参考方案1】：

对我来说，它适用于下一个 docker-compose 设置。确实，我还没有完全测试过。我会尽快更新帖子。但如果它对某人有帮助，那就完美了。

  cadvisor:
  image: google/cadvisor:latest
  container_name: monitoring_cadvisor
  restart: unless-stopped
  volumes:
    - /:/rootfs:ro
    - /var/run/docker.sock:/var/run/docker.sock:rw
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
  expose:
    - 8080
  ports:
    - 8084:8080

我更新了 docker.sock 路径。没有直接链接，cadvisor 无法找到容器。

干杯

【参考方案2】：

您似乎遇到了您提到的link 中已解决的问题。该修复尚未包含在 cAdvisor 版本中，因此在您运行 docker 映像 google/cadvisor:latest 时不包含该修复（假设您没有构建自己的映像并使用它进行标记）。

您可以尝试从 master 构建和运行自己的 docker 映像，或者等待下一个补丁版本（应该很快），其中将包含该修复。

编辑：

该修复程序现已合并到 cAdvisor v0.24.1（docker 映像 google/cadvisor:v0.24.1）中

【讨论】：

嗯，让人很流氓

我正在使用 Ubuntu 操作系统，docker 版本为“17.05.0-ce”，cadvisor 镜像版本为“google/cadvisor:v0.24.1”和 influx docker 镜像“tutum/influxdb :0.8.8”。我在运行 cadvisor 时遇到错误：“W0913 17:19:43.993296 1 manager.go:148] 无法连接到 Rkt api 服务：rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt:拒绝连接 ”。我可以在浏览器中看到 cadvisor 并且 influxdb 也已启动。但我无法在 influxdb 中看到 cadvisor 的统计数据。我关注了brianchristner.io/how-to-setup-docker-monitoring

@Raji 您可以忽略 rkt 错误（除非您使用的是 rkt）。另外，我建议更新到更新的 cAdvisor 版本。

感谢蒂姆的回复。当我在浏览器 (hostmachine:8083) 中打开 influxdb 时，我没有看到来自 cadvisor 的任何数据。我检查了“统计”选项和“测量”选项。 'stats' 显示了一些关于 '_internal' 数据库的数据，尽管我选择了 'cadvvisor' 数据库。和测量是空的。我看错了吗？