Tag Archives: syslog-ng

New release: syslog-ng OSE 3.8.1

I am happy to announce that after almost a year of development, syslog-ng 3.8.1 has been released. This is the first stable release from the syslog-ng 3.8 branch. Version 3.8.1 adds some long-awaited features to syslog-ng and improves the performance of name-value-pair related operations. Read more at https://www.balabit.com/blog/syslog-ng-hits-major-milestone-with-version-3-8-1-release/

Meet syslog-ng at the OpenS– USE Conference 2016

Come and meet Peter Czanik and learn about syslog-ng at the openS– USE Conference 2016 in Nuremberg!

syslog-ng at the OpenS-- USE conference

Visit us at the OpenS– USE conference

Attend our presentation on June 24, Friday, at 13:30 in the Roter Salon.

If you have any questions, contact Peter on LinkedIn or as @PCzanik on Twitter, or just find him any time at the conference in the next four days.

The MongoDB destination receives a face-lift

Reasons behind the migration

We have migrated to the official mongo-c-driver binding for providing the MongoDB destination driver in syslog-ng 3.8. Previously in syslog-ng 3.7.x and earlier, libmongo-client provided this binding, mandating its own special syntax.

This change will facilitate future-proof and more fine-grained configuration. MongoDB 3 is not officially supported or being tested yet, but this kind of connection should theoretically enable easy MongoDB 3 support in the future.

What to do when using legacy syntax

If you have used legacy syntax in your configuration file, syslog-ng will substitute the given deprecated options to form a URI. Note that certain aspects of semantics could also differ between the two drivers.

Continue reading

The grouping-by() parser in syslog-ng 3.8

Until recently, the correlation and aggregation of information from multiple messages was within the domain of the PatternDB parser. The limitation of this implementation is that it only worked for data extracted by PatternDB. There are now many more parsers: the CSV parser for columnar data, the JSON parser for logs in JSON format or the recently introduced key=value parser. Now I want to introduce you to a new parser, called grouping-by(). It can correlate and aggregate information independent from PatternDB. Read more about it at https://czanik.blogs.balabit.com/2016/04/the-grouping_by-parser-in-syslog-ng-3-8/

Transferring Conserver Logs to Elasticsearch

If your organization manages Linux, AIX, HP-UX or Solaris servers in-house, chances are your system administrators at least occasionally need low-level access to those devices. Typically, administrators use some kind of serial console—for example, traditional serial port, Serial-over-LAN or Intelligent Platform Management Interface (IPMI). Managing and auditing console access is not trivial, so many organizations rely on the Conserver application to create session logs when accessing these servers via the serial console. These logs can be useful for various reasons—for example, maintenance or troubleshooting (to review why something crashed), security (to find out who did what—connecting user names to actual users) or compliance (to provide detailed session logs).

This article covers the following:

  • How to parse and process serial console logs using syslog-ng Open Source Edition (Balabit).
  • How to send the logs to Elasticsearch (Elastic), so you get a complete, searchable audit trail of the console access.
  • How to integrate the console logs into a real-time monitoring system using Riemann.

Read more at http://www.linuxjournal.com/content/transferring-conserver-logs-elasticsearch

Writing into HDFS with syslog-ng video and setup instructions

The goal of this article is summarizing the necessary steps for a simple ‘syslog-ng and hadoop’ setup. The demo at the end of this summary demonstrates how it works. This video was recorded for the Hortonworks Data Platform certification process and as a result both syslog-ng PE 5.3+ and OSE 3.7+ are Hortonworks HDP certified.

Continue reading

New releases: syslog-ng OSE 3.7.1 and incubator 0.5

The first syslog-ng 3.7 release is now available together with a matching incubator 0.5 release. Major new features include Java and Python destinations migrated from the incubator, and a collection of Java based “big data” destinations: Elasticsearch, Hadoop and Kafka. For a complete list of news, check the release notes available at https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.1

syslog-ng Insider – January 2013

Dear syslog-ng users,

This is the 20th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.
Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com


Video: a short introduction to syslog-ng

There is a video series in the work to introduce the basics of syslog-ng. The first video is online now, you can watch it.

New videos will be published on the same Youtube channel. You are very welcome to subscribe to receive notifications about further syslog-ng videos and other interesting content.

syslog-ng OSE configurator

syslog-ng OSE configurator is a web application, which can generate a syslog-ng OSE configuration with only a few mouse clicks. Right now it can only deal with global options, sources, destinations and of course with logpath. There are plans to add filters and templates in the future.
For more information and links to sources and the configurator, check the authors blog

syslog-ng 3.4 RC2 is released

Version 3.4 RC2 was released today. Compared to the RC1 release, it has some minor bugfixes, mainly related to fixing compile problems of different features and platforms. This also involves an update to the bundled ivykis library from version 0.30 to 0.36.
For a summary about what’s new in the 3.4 releases, check Bazsi’s blog at http://bazsi.blogs.balabit.com/2012/12/syslog-ng-3-4beta1-released/.

At the time of this writing there are RC1 packages for Debian, openS– USE, Ubuntu, FreeBSD ports are updated and there is a Fedora source rpm. These should be updated to RC2 in the coming days.
OpenS– USE Factory already features syslog-ng 3.4 RC1 and will be updated to RC2 after openS– USE 12.3beta1 is out.





syslog-ng Insider – June 2012

Dear syslog-ng users,

This is the 14th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.
Your feedback and news tips about the next issue is welcome at


New features in syslog-ng 3.4

The CEE/Lumberjack project might be very silent recently, but is still a good excuse to demonstrate some of the new features of syslog-ng 3.4. These make implementing structured logging (and this way CEE) possible by adding a JSON parser, marker detection, channels and junctions and a flexible use of blocks, so complex configurations can be combined in a block and easily reused in many configs. For details and examples check: http://bazsi.blogs.balabit.com/2012/05/cee-prototype-and-a-show-case-for-the-new-3-4-features/
Version 3.4 also merges many features from syslog-ng PE, which can be followed in git commit messages. These include the SYSUPTIME macro, AM/PM related macros, test cases, support for Cisco sequence numbers, etc.

Message rate alerting in SSB

Even though syslog-ng Store Box neither is nor aims to be a full-blown SIEM solution, it can be and is indeed often used to detect anomalies, identify possible threats, and find problems within an organization’s IT infrastructure. One important thing to note is that it is not only the contents of log messages that carry information about what happens in the network but their volume too.

Read how message rate alerting works in SSB.

syslog-ng 3.3 has a new maintainer

As Bazsi, lead developer of syslog-ng announced on the syslog-ng mailing list, the stable version has now a new maintainer. He is Gergely Nagy, or better known as Algernon, who coded some interesting new features for syslog-ng, including a MongoDB destination, and a JSON output and parser (for 3.4). This change will leave Bazsi more time for 3.4 developments and also speed up merging bugfixes to the 3.3 line.
Announcement: https://lists.balabit.hu/pipermail/syslog-ng/2012-May/018885.html
Algernon’s plans: http://algernon.blogs.balabit.com/2012/05/hats-and-sticks/

syslog-ng community forum

For those, who prefer to use web based forums instead of mailing lists, BalaBit provides now a community forum. Right now there are over forty users and their number is growing every day. If you want to read about interesting topics, or could help fellow users using a forum instead of the mailing list, please visit the forum.





syslog-ng Insider – May 2012

Dear syslog-ng users,

This is the 13th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.
Your feedback and news tips about the next issue is welcome at


Have you tried to add custom information to log messages, fix mis-formatted logs or anonymize logs?

The next long-time-supported release of SSB version 3 LTS is about to be released. This release includes a switch to 64-bit architecture, a huge performance improvement in the indexing/searching feature for a large number of message and search patterns and a couple of new features, too. The following post plans to introduce those new features to you. The updated User’s Manual will contain a detailed description of them — this post is written more to serve as teaser and to highlight some of the use cases we had in mind when we’d planned the features, and, of course, to ask for your feedback about them.

syslog-ng participates in GSoC

This year syslog-ng participates in GSoC under the umbrella of openS– USE. We have one student accepted, who will work on syslog-ng’s mongodb destination.

MongoDB howto

One of the major reasons to update to 3.3 other than threading is MongoDB. It allows great flexibility when using patterns and storing parsed data from logs. The syslog-ng documentation covers most information necessary to use MongoDB, but this HowTo compiles all these into a single document and extends it with features from the upcoming syslog-ng 3.4 version.