一般日志量比较大，会先收集到kafka然后再进行消费。
安装
- yum -y install https://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/7.14.0/filebeat-7.14.0-x86_64.rpm
配置filebeat
- vim /etc/filebeat/filebeat.yml
- filebeat.inputs:
- - type: log
- paths:
- - /var/log/*.log
- - /var/log/messages
- fields:
- type: "systemlog"
- log_topic: "systemlog"
- fields_under_root: true
- exclude_lines: ["^DBG"]
- exclude_files: [".gz$"]
- name: 192.168.122.100
- output.kafka:
- enabled: true
- hosts: ["192.168.122.200:9092"]
- topic: "%{[log_topic]}"
- partition.round_robin:
- reachable_only: true
- worker: 2
- required_acks: 1
- compression: gzip
- max_message_bytes: 10000000
服务管理
- systemctl enable filebeat
- systemctl start filebeat
总结
filebeat是Go写的，部署起来很方便，没什么依赖，批量部署简单。
