Logstash filter

martin
Contributor III
Contributor III

I'm using logstash for reading logs and parsing them. I have created a filter to process the JAMFSoftwareServer.log and would like to share this with you.

input {
  file {
    type => "jss-syslog"
    path => "/var/log/jss/JAMFSoftwareServer.log"
    codec => multiline {
      pattern => "^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} "
      negate => true
      what => "previous"
    }
  }
}

filter {
  if [type] == "jss-syslog" {
    grok {
      match => [ "message", "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9]),%{INT} [%{LOGLEVEL:jss_severity}s*] [%{DATA:jss_thread}s*] [%{DATA:jss_process}s*] - %{GREEDYDATA:jss_message}" ]
    }
    grok {
      match => [ "path", "/var/log/jss/JAMFSoftwareServer.log" ]
    }
  }
}

Logstash will output the tImestamp, severity (info, warn, error), thread, process/service/function and message. You can use all these values to further analyse the JAMF Software Server. In my case we output the values to elasticsearch.

If you want to know more about logstash have a look at http://logstash.net/ and http://www.elasticsearch.org/.

5 REPLIES 5

monogrant
Contributor

Curious if you've update this in the last 2 years @martin

martin
Contributor III
Contributor III

Hi @monogrant,

Currently I'm using Filebeat (next- generation Logstash Forwarder) in order to sent the log file to Logstash. DigitalOcean has (as always) a great article in how to setup ELK.

Let filebeat sent /var/log/jss/JAMFSoftwareServer.log to Logstash. I created a jss-syslog type in /etc/logstash/conf.d/jss-syslog.conf:

filter {
  if [type] == "jss-syslog" {
    grok {
      match => { "message" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9]),%{INT} [%{LOGLEVEL:jss_severity}s*] [%{DATA:jss_thread}s*] [%{DATA:jss_process}s*] - %{GREEDYDATA:jss_message}" }
      add_field => { "received_at" => "%{@timestamp}" }
      add_field => { "received_from" => "%{host}" }
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
    multiline {
        negate => true
        pattern => "^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} "
        what => "previous"
    }
  }
}

Don't forget to add jss-syslog in the filebeat config file.

monogrant
Contributor

Appreciate the update. I'm using an ELK stack for a bunch of different servers and have always has a grok parse error on my JSS logs. This will be handy!

jbrush
New Contributor

@martin I know this is a super old thread, curious if you are doing anything with this still?

We are moving to jamf cloud and security needs me to pass this info to our QRadar instance and this was one of few results on the matter. Going to pull the data I need from the API.

DorisMulcahey
New Contributor

@martin and @jbrush - Wondering if you got your jamf cloud logs into ELK.  I need to get our jamf pro logs aggregated into our company-wide logstream.