The MongoDB destination receives a face-lift

Reasons behind the migration

We have migrated to the official mongo-c-driver binding for providing the MongoDB destination driver in syslog-ng 3.8. Previously in syslog-ng 3.7.x and earlier, libmongo-client provided this binding, mandating its own special syntax.

This change will facilitate future-proof and more fine-grained configuration. MongoDB 3 is not officially supported or being tested yet, but this kind of connection should theoretically enable easy MongoDB 3 support in the future.

What to do when using legacy syntax

If you have used legacy syntax in your configuration file, syslog-ng will substitute the given deprecated options to form a URI. Note that certain aspects of semantics could also differ between the two drivers.

Continue reading

The grouping-by() parser in syslog-ng 3.8

Until recently, the correlation and aggregation of information from multiple messages was within the domain of the PatternDB parser. The limitation of this implementation is that it only worked for data extracted by PatternDB. There are now many more parsers: the CSV parser for columnar data, the JSON parser for logs in JSON format or the recently introduced key=value parser. Now I want to introduce you to a new parser, called grouping-by(). It can correlate and aggregate information independent from PatternDB. Read more about it at https://czanik.blogs.balabit.com/2016/04/the-grouping_by-parser-in-syslog-ng-3-8/

Transferring Conserver Logs to Elasticsearch

If your organization manages Linux, AIX, HP-UX or Solaris servers in-house, chances are your system administrators at least occasionally need low-level access to those devices. Typically, administrators use some kind of serial console—for example, traditional serial port, Serial-over-LAN or Intelligent Platform Management Interface (IPMI). Managing and auditing console access is not trivial, so many organizations rely on the Conserver application to create session logs when accessing these servers via the serial console. These logs can be useful for various reasons—for example, maintenance or troubleshooting (to review why something crashed), security (to find out who did what—connecting user names to actual users) or compliance (to provide detailed session logs).

This article covers the following:

  • How to parse and process serial console logs using syslog-ng Open Source Edition (Balabit).
  • How to send the logs to Elasticsearch (Elastic), so you get a complete, searchable audit trail of the console access.
  • How to integrate the console logs into a real-time monitoring system using Riemann.

Read more at http://www.linuxjournal.com/content/transferring-conserver-logs-elasticsearch