I’m using EVE JSON output. Enable EVE from Service – Suricata – Edit interface mappingEVE Output Settings
EVE JSON Log [x]
EVE Output Type: File
Install Filebeat FreeBSD package
https://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/
Find beats-x.x.x.txz
curl -o beats-x.x.x.txz https://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/beats-x.x.x.txz pkg add beats-x.x.x.txz
Download modules
https://www.elastic.co/downloads/past-releases
Download same version Filebeat LINUX 64-BIT
curl -o filebeat-x.x.x-linux-x86_64.tar.gz https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-x.x.x-linux-x86_64.tar.gz curl -o filebeat-x.x.x-linux-x86_64.tar.gz.sha512 https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-x.x.x-linux-x86_64.tar.gz.sha512 sha256 -c filebeat-x.x.x-linux-x86_64.tar.gz.sha512 tar zxvf filebeat-x.x.x-linux-x86_64.tar.gz mv filebeat-x.x.x-linux-x86_64/modules.d /var/db/beats/filebeat/ mv filebeat-x.x.x-linux-x86_64/module /var/db/beats/filebeat/
Configure Filebeat
vi /usr/local/etc/filebeat.yml
filebeat.inputs:
- type: log
paths:
- "/var/log/suricata/*/eve.json*"
fields_under_root: true
fields:
tags: ["suricata","json"]
- type: log
paths:
- "/var/syslog-ng/default.log"
fields_under_root: true
fields:
tags: ["pfsense"]
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
output.logstash:
hosts: ["LOGSTASHIP:5044"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
logging.to_syslog: false
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat.log
keepfiles: 7
mkdir /var/log/filebeat
/usr/local/sbin/filebeat -c /usr/local/etc/filebeat.yml test config
Enable Filebeat to run on boot
ln -s /usr/local/etc/rc.d/filebeat /usr/local/etc/rc.d/filebeat.sh echo "filebeat_enable=yes" >> /etc/rc.conf.local echo "filebeat_conf=/usr/local/etc/filebeat.yml" >> /etc/rc.conf.local
Logstash
vi 01-inputs.conf
input {
beats {
port => 5044
}
}
vi 12-suricata.conf
filter {
if "suricata" in [tags] {
json {
source => "message"
}
date {
match => [ "timestamp", "ISO8601" ]
}
if ![geoip] and [src_ip] !~ /^(10\.|192\.168\.)/ {
geoip {
add_tag => [ "GeoIP" ]
source => "src_ip"
}
}
}
}
vi 30-outputs.conf
output {
if "pfsense" in [tags] {
elasticsearch {
hosts => localhost
index => "pfsense-%{+YYYY.MM}"
}
}
if "suricata" in [tags] {
elasticsearch {
hosts => localhost
index => "suricata-%{+YYYY.MM}"
}
}
}
Hi Villekri, I like your post on how to send suricata logs to ELK using Filebeat. This is a significant issue among people using PFsense. Have you experienced any issues with your method of setting up Filebeat??
LikeLike
Hi, i did have some unstability. I had a lot of settings and packages so i don’t know if it had anything to do with Filebeat. Haven’t had the time to check what was the reason i just reinstalled whole pfsense and not using Filebeat nor many other packages at the moment. e.g. i used syslog-ng package to get pfsense logs into files which Filebeat can read ( /var/syslog-ng/default.log ).
If i had to guess i would say problem is in Suricata can’t handle the traffic since my Netgate box is not very powerful.
Otherwise Filebeat was working as i expected.
LikeLike
#—————-inputs——————–
filebeat.inputs:
– type: log
enabled: true
fields_under_root: true
tail_files: true
paths:
– /var/log/*.log
#—————-input tags—————-
tags:
– fw
– pfsense
– fw_logs
#—————-inputs——————–
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
#—————-Elastic_Output——————–
output.elasticsearch:
hosts: [“http://host:9200”]
username: “username_gooes_here”
password: “Password_goes_here”
processors:
– add_host_metadata: ~
– add_cloud_metadata: ~
#——————————————
LikeLike