Insider 2017-12: 3.13; Splunk HEC; Application Adapters; Graylog;

Dear syslog-ng users,

This is the 64th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news.

NEWS

syslog-ng 3.13 released

The latest version of syslog-ng, 3.13 is now available. It now parses collected messages automatically using application adapters and can easily forward name-value pairs using the enterprise-wide message model. Support for Graylog and the GELF message format was also added. There are many more smaller features and bug fixes. For a complete list check the release announcements.

Sending logs to Splunk through HTTP

For quite some time, Splunk has recommended to collect syslog messages using syslog-ng, save them to files, and send them to Splunk using forwarders. Unless you have a very high message rate, the HTTP destination of syslog-ng can greatly simplify this logging architecture. Instead of writing messages to files and reading them by a forwarder, syslog-ng can forward messages to Spunk HTTP Event Collector (HEC) directly, using HTTP or HTTPS connections. And if you parse messages using syslog-ng, you can send the resulting name-value pairs to Splunk in JSON format and be able to search them instantly.

Sending logs from Logstash to syslog-ng

Logstash adds a new syslog header to log messages before forwarding them to a syslog server. In the case of syslog messages, it is problematic as there will be two syslog headers in the message. Using syslog-ng for everything logging related in an Elasticsearch environment can considerably simplify your architecture. Still, there are situations, when Filebeats and Logstash are already deployed and you need some logs from Logstash in syslog-ng. Learn how you can remove the extra syslog header.

Application Adapters & Enterprise-wide Message Model

Do you want to simplify parsing your log messages? Try the new “application adapter” and “enterprise-wide message model” frameworks in syslog-ng: you can automatically parse log messages and forward the results to another syslog-ng instance. Optionally, you can also include the original, raw message that you can forward unmodified to a SIEM system for further analysis.

Learn how to use these new features.

Graylog as destination in syslog-ng

Version 3.13 of syslog-ng introduced a graylog2() destination and a GELF (Graylog Extended Log Format) template to make sending syslog messages to Graylog easier. You can also use them to forward simple name-value pairs where the name starts with a dot or underscore. If names of your name-value pairs include dots other than the first character, you should use JSON formatting directly instead of the GELF template and send logs to a raw tcp port in Graylog, which can then extract fields from nested JSON.

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

Leave a Reply