Suricata logs to Logstash with Filebeat on pfSense 2.4

I’m using EVE JSON output. Enable EVE from Service – Suricata – Edit interface mappingEVE Output Settings

EVE JSON Log [x]
EVE Output Type: File

Install Filebeat FreeBSD package

https://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/

Find beats-x.x.x.txz

curl -o beats-x.x.x.txz https://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/beats-x.x.x.txz
pkg add beats-x.x.x.txz

Download modules

https://www.elastic.co/downloads/past-releases

Download same version Filebeat LINUX 64-BIT

curl -o filebeat-x.x.x-linux-x86_64.tar.gz https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-x.x.x-linux-x86_64.tar.gz
curl -o filebeat-x.x.x-linux-x86_64.tar.gz.sha512 https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-x.x.x-linux-x86_64.tar.gz.sha512
sha256 -c filebeat-x.x.x-linux-x86_64.tar.gz.sha512
tar zxvf filebeat-x.x.x-linux-x86_64.tar.gz
mv filebeat-x.x.x-linux-x86_64/modules.d /var/db/beats/filebeat/
mv filebeat-x.x.x-linux-x86_64/module /var/db/beats/filebeat/

Configure Filebeat

vi /usr/local/etc/filebeat.yml

filebeat.inputs:
- type: log
  paths:
    - "/var/log/suricata/*/eve.json*"
  fields_under_root: true
  fields:
    tags: ["suricata","json"]
- type: log
  paths:
    - "/var/syslog-ng/default.log"
  fields_under_root: true
  fields:
    tags: ["pfsense"]

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

output.logstash:
  hosts: ["LOGSTASHIP:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

logging.to_syslog: false
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat.log
  keepfiles: 7
mkdir /var/log/filebeat
/usr/local/sbin/filebeat -c /usr/local/etc/filebeat.yml test config

Enable Filebeat to run on boot

ln -s /usr/local/etc/rc.d/filebeat /usr/local/etc/rc.d/filebeat.sh

echo "filebeat_enable=yes" >> /etc/rc.conf.local
echo "filebeat_conf=/usr/local/etc/filebeat.yml" >> /etc/rc.conf.local

Logstash

vi 01-inputs.conf

input {
  beats { 
    port => 5044 
  }
}

vi 12-suricata.conf

filter {
  if "suricata" in [tags] {
    json {
      source => "message"
    }
    date {
      match => [ "timestamp", "ISO8601" ]
    }
    if ![geoip] and [src_ip] !~ /^(10\.|192\.168\.)/ { 
      geoip {
        add_tag => [ "GeoIP" ] 
        source => "src_ip" 
      }
    }
  }
}

vi 30-outputs.conf

output {
    if "pfsense" in [tags] {
        elasticsearch {
            hosts => localhost
            index => "pfsense-%{+YYYY.MM}"
        }
    }
    if "suricata" in [tags] {
        elasticsearch {
            hosts => localhost
            index => "suricata-%{+YYYY.MM}"
        }
    }
}

3 thoughts on “Suricata logs to Logstash with Filebeat on pfSense 2.4

  1. Hi Villekri, I like your post on how to send suricata logs to ELK using Filebeat. This is a significant issue among people using PFsense. Have you experienced any issues with your method of setting up Filebeat??

    Like

    1. Hi, i did have some unstability. I had a lot of settings and packages so i don’t know if it had anything to do with Filebeat. Haven’t had the time to check what was the reason i just reinstalled whole pfsense and not using Filebeat nor many other packages at the moment. e.g. i used syslog-ng package to get pfsense logs into files which Filebeat can read ( /var/syslog-ng/default.log ).
      If i had to guess i would say problem is in Suricata can’t handle the traffic since my Netgate box is not very powerful.
      Otherwise Filebeat was working as i expected.

      Like

    2. #—————-inputs——————–
      filebeat.inputs:
      – type: log
      enabled: true
      fields_under_root: true
      tail_files: true
      paths:
      – /var/log/*.log
      #—————-input tags—————-
      tags:
      – fw
      – pfsense
      – fw_logs
      #—————-inputs——————–
      filebeat.config.modules:
      path: ${path.config}/modules.d/*.yml

      reload.enabled: false

      setup.template.settings:
      index.number_of_shards: 1
      #—————-Elastic_Output——————–
      output.elasticsearch:
      hosts: [“http://host:9200”]
      username: “username_gooes_here”
      password: “Password_goes_here”
      processors:
      – add_host_metadata: ~
      – add_cloud_metadata: ~
      #——————————————

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.