如何将LINUX远程服务器的日志写入本地的文件里，求大神。。。。在线等

Posted 2023-04-17

假设有12台服务器，主机名字依次为portal[01-12].ea.com. 你可以在本地可以用root账户不带密码的ssh登陆到这些机器上面。Account.http log 位于/var/log/httpd/http_access.log 请写一个脚本来收集HTTP LOGS 来自所有的服务器并写入本地的/tem/http_logs.

#!/bin/bash
for i in 01..12
do
echo "=======HTTP LOG From Host Portal$i=========" >> /tem/http_logs
ssh root@portal$i.ea.com 'cat /var/log/httpd/http_access.log' >> /tem/http_logs
done

追问

有个小问题
ssh root@portal$i.ea.com 'cat /var/log/httpd/http_access.log' >> /tem/http_logs
done
这是写入本地的/tem/http_logs对吧, 那我要是写入远程的那个机器里的/tem/http_logs要怎么写?

追答

把ssh那行的后单引号放到最后

追问

ssh root@portal$i.ea.com cat /var/log/httpd/http_access.log >> '/tem/http_logs'
done
这样对么?

追答

ssh root@portal$i.ea.com 'cat /var/log/httpd/http_access.log >> /tem/http_logs'

追问

不好意思 我理解错误..
那我这句
ssh root@portal$i.ea.com cat /var/log/httpd/http_access.log >> '/tem/http_logs'
引号在最后面的 是什么意思?或者还是不能执行?

追答

能，跟开始一样，放在本地

参考技术A scp 比较简单吧 自己动手吧追问

Scp root@portal[01-12].ea.com: /var/log/httpd/http_access.log/tem/http_logs.
那怎么写入呢？

如何将流利的位输入日志写入本地主机系统日志服务器

【中文标题】如何将流利的位输入日志写入本地主机系统日志服务器

【英文标题】：How to write fluent bit input logs to localhost syslog server

【发布时间】：2021-09-20 15:56:17

【问题描述】：

我正在从 docker 容器化应用程序收集日志。我能够将日志带到 stdout 输出插件，但是当我尝试 syslog 输出插件时，它不会在 syslog 服务器上写入。 下面是配置文件。

[SERVICE]
    Parsers_File        /etc/td-agent-bit/parsers.conf

[INPUT]
    Name                forward

[Output]
    name                syslog
    match               *
    host                127.0.0.1
    port                514
    mode                udp
    syslog_format       rfc5424
    syslog_hostname_key hostname
    syslog_appname_key  appname
    syslog_procid_key   procid
    syslog_message_key  log

容器应用的日志属性设置为

logging:
  driver: fluentd
  options:
    fluentd-address: localhost:24224
    tag: logs

运行流畅位后/opt/td-agent-bit/bin/td-agent-bit -c fluent.conf

[2021/09/20 08:47:16] [ warn] [engine] failed to flush chunk '8481-1632152835.361162854.flb', retry in 7 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:47:23] [ warn] [engine] chunk '8481-1632152835.361162854.flb' cannot be retried: task_id=0, input=forward.0 > output=syslog.0
[2021/09/20 08:47:26] [ warn] [engine] failed to flush chunk '8481-1632152845.361118393.flb', retry in 6 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:47:32] [ warn] [engine] chunk '8481-1632152845.361118393.flb' cannot be retried: task_id=0, input=forward.0 > output=syslog.0
[2021/09/20 08:47:36] [ warn] [engine] failed to flush chunk '8481-1632152855.361556013.flb', retry in 8 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)

谁能告诉这里出了什么问题？

即使对于简单的 cpu 输入插件 syslog 也不起作用。喜欢

/opt/td-agent-bit/bin/td-agent-bit -i cpu -o syslog

输出

[2021/09/20 08:53:43] [ info] [cmetrics] version=0.2.1
[2021/09/20 08:53:43] [ info] [output:syslog:syslog.0] setup done for 127.0.0.1:514
[2021/09/20 08:53:43] [ info] [sp] stream processor started
[2021/09/20 08:53:48] [ warn] [engine] failed to flush chunk '8765-1632153224.515974981.flb', retry in 10 seconds: task_id=0, input=cpu.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:53:53] [ warn] [engine] failed to flush chunk '8765-1632153228.516869744.flb', retry in 6 seconds: task_id=1, input=cpu.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:53:58] [ warn] [engine] chunk '8765-1632153224.515974981.flb' cannot be retried: task_id=0, input=cpu.0 > output=syslog.0

【参考方案1】：

我遇到了类似的问题，因为 fluentbit pod 本身是 127.0.0.1。我创建了另一个 rsyslog pod 并使用它的 IP 从 fluentbit 获取日志。 https://artifacthub.io/packages/helm/rsyslog/rsyslog