insider 2014-02: FIPS; GSoC; Incubator; Conferences

Dear syslog-ng users,

This is the 31th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

FIPS

Certain departments of the US government and those conducting business on certain regulated markets are required to use tools utilizing cryptography certified according to the FIPS 140-2 standard to handle sensitive data. As logs often contain sensitive information, this regulation can apply to the log management system, too.

Starting with the 5.0.4 release, we are offering a version of syslog-ng Premium Edition to customers that is built and shipped with the OpenSSL FIPS Object Module and as thus usable in environments where such validation is required. (For more information about the OpenSSL FIPS Object Module, see http://openssl.com/fips/.) This is a separate version and the original version is available that uses the standard OpenSSL library. It is currently available on major Linux platforms and has the same functionality as the non-compliant version apart from a few limitations listed in the documentation.
Please visit https://www.balabit.com/lp/fips if for more information.

Google Summer of Code (GsoC)

Based on the success of last years syslog-ng Gsoc participation (in co-operation with the openS– USE project), this year we plan to participate in GsoC again with a number of syslog-ng related projects. The current list of ideas range from new sources and destination through under the hood improvements to integration with configuration management systems.

syslog-ng incubator updates

The syslog-ng incubator is a collection of tools and modules which are not (yet) part of the official repository.

Since last month there are many smaller changes and brand new features. The master branch goes now hand in hand with syslog-ng’s master branch, which is currently 3.6 pre-alpha code. If you are working on syslog-ng 3.5, use the 3.5/master branch of syslog-ng-incubator.

On the feature side the Lua destination was enhanced, a monitoring source was added, together with a Graphite output. Among the Lua examples one can also find a script to push logs into ElasticSearch.

To try the new features, check out https://github.com/balabit/syslog-ng-incubator from git. You can read more about the Lua destination at https://talien.blogs.balabit.com/2014/02/lua-the-undiscovered-country/

syslog-ng at conferences

In the past few weeks we visited three conferences. While there we talked about syslog-ng to many people, some during organized presentations, and many more between presentations and social events of the conferences.

A look back at FOSDEM and Config Management Camp: https://czanik.blogs.balabit.com/2014/02/fosdem-2014-and-config-management-camp/

The next confirmed event is: Open Source Data Center Conference, 8-10 April, Berlin, Germany: http://www.netways.de/osdc/, Giving a talk titled “Monitoring with syslog-ng, Riemann and Kibana” There will be two developers present, so if anyone attends, and has Incubator, Lua, Riemann, etc. related questions, we’ll be happy to answer.

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2014-01: 3.6 pre-alpha ; incubator ; PCI-DSS

Dear syslog-ng users,

This is the 30th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng at conferences

This year, we plan to present syslog-ng in many conferences, or just participate conferences where many syslog-ng users are expected to show up. If you want to meet someone from the syslog-ng team, come to one of our presentations or drop an e-mail so we can find each other. We are also very interested to hear suggestions, where syslog-ng should be presented!
Here is a list of confirmed events, which will be updated regularly in later newsletters:

  • FOSDEM, 1-2 February, Brussels, Belgium: https://fosdem.org/2014/, giving a talk titled: “Babelfish for DevOps: syslog-ng”
  • Fedora, JBoss and RedHat developers conference, 7-9 February, Brno, Czech Republic: http://devconf.cz/, Participating only
  • Open Source Data Center Conference, 8-10 April, Berlin, Germany: http://www.netways.de/osdc/, Giving a talk titled “Monitoring with syslog-ng, Riemann and Kibana”

Check out syslog-ng 3.6 pre-alpha!

While syslog-ng 3.6 still did not even reach an alpha release, it received already a lot of development. So, while it might still eat your logs for lunch, those who are interested in where syslog-ng is heading should check out syslog-ng 3.6 from git. Unlike previous syslog-ng versions, it is developed in a unified git repository without a version string attached.

While most changes are under the hood, there are also some user visible features like pseudofile destination.

Nodejs support was also added to syslog-ng: use the widespread winston logging API, and syslog-ng will process its JSON formatted messages.

And if you look at the stats you can see a healthy growth of the code base and in the number of contributors. Thank you for your support!

syslog-ng incubator

The syslog-ng incubator is a collection of tools and modules which are not (yet) part of the official repository. It has some very interesting code in it, like a riemann or an RSS destination, but until now it was completely undocumented. Not any more: http://asylum.madhouse-project.org/blog/2013/12/29/the-incubator/.
The Incubator also includes a Lua destination, which makes it possible to write simple destination drivers without a line of C. It is still a work in progress, but is an important step into writing modules in other languages.
And to make your life easier packages are available in Debian testing, Ubuntu Trusty and for openS– USE in the 3rd party repositories.

PCI DSS 3.0 Continues to Emphasize the Importance of Log Management

The Payment Card Industry Security Standards Council recently released the Data Security Standard 3.0, three years after the prior version. As one of the most important international data security standards, the latest release was eagerly awaited by IT security practitioners. Clarifications make up the bulk of the changes but the standards council changed most of the 12 major requirements to include modified or additional sub-requirements. With PCI DSS 3.0 the standards council has reiterated that log management is a critical part of security best practices. You can read more about it.

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-11: syslog-ng 3.5 released; Logging to Hadoop; EoL

Dear syslog-ng users,

This is the 29th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng 3.5 is released

Beginning last week syslog-ng 3.5 was finally declared stable, so 3.5.1 was released. Source is available from https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/download if you want to compile syslog-ng yourself. Packages and ports for different Linux distributions and FreeBSD are already available from our 3rd party page.

New features of 3.5 were detailed here many times, so here are just the headlines:

  • Redis and Stomp destinations
  • multiline messages
  • type hinting

For details, check the ChangeLog or algernon’s blog.

Logging to Hadoop

Big data is gaining a momentum also in the logging world, so we made some experiments how syslog-ng can log to Hadoop. Even without a dedicated Hadoop destination driver one can already send logs to Hadoop and analyze them using Hadoop tools. For details, check https://tiborbenke.blogs.balabit.com/2013/11/the-syslog-ng-in-the-hadoop-era/

syslog-ng EoL policy

The syslog-ng End of Life policy was updated recently, based on the experiences of syslog-ng stable maintainer, Gergely Nagy. Practically there are four different branches of syslog-ng maintained at any time, representing different level of maturity and support. More details are available at http://asylum.madhouse-project.org/blog/2013/10/22/syslog-ng-eol-dates/.

And following the URLs below you can read about how this affects different Linux distributions and FreeBSD:

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-10: syslog-ng 3.5 beta releases; syslog-ng PE with Windows support; GSoC summaries

Dear syslog-ng users,

This is the 28th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng 3.5 beta2 and beta3 are released

The second and third beta of syslog-ng 3.5 were released during the past few weeks. These brought mostly bugfixes, but also some new features. Beta2 brought us a new redis destination and extended type hinting. Beta3 finally compiles again on FreeBSD.
As usual, testing is greatly appreciated as it helps to iron out problems before the final release! Sources are available from git or as tgz.
For third-party binary packages for various Linux and UNIX platforms, visit this page. SLES / openS– USE packages are already available and Debian/Ubuntu packages are expected to be available in the coming days.

syslog-ng PE with full Windows support

The Premium Edition of syslog-ng now offers full Windows support with its latest release, version 5LTS. Until now, only the syslog-ng Agent for Windows could be installed on Windows platforms. With the latest version, syslog-ng Premium Edition can be installed on client but also as server, so even a homogenous Windows environment can benefit from using syslog-ng. Installing a separate Linux machine as relay is no longer necessary in remote offices, as a Windows machine can do the job.
Compression support in RLTP was also introduced, which can save valuable bandwidth.

Read the product managers blog about the new release.

syslog-ng GSoC: code merged

The syslog-ng GSoC finished a few weeks ago, but it does not mean, that the work is over. The code had to be updated to work with the latest syslog-ng 3.5 sources. The new redis destination is already merged and available as part of the syslog-ng 3.5 beta2 sources. The new mysql destination is heading now to the “incubator” project and waits there for some additional polish, before it is merged syslog-ng master.

A summary blog post of their work is available at http://petrovicsgyula.blogspot.com/ and http://tichygsoc.blogspot.hu/ A blog post about the syslog-ng GSoC from the mentors (and algernon’s) point of view can also be read.

syslog-ng 3.5 documentation is now feature complete

The syslog-ng 3.5 documentation is now feature complete and available on our website.
It is available in many forms:

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-09: syslog-ng 3.5 beta release; incubator project; GSoC updates

Dear syslog-ng users,

This is the 27th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng 3.5 beta1 is released

The first beta of syslog-ng 3.5 was released today. It has many internal rework and also many user visible new features. Here you can read just the headlines, for a complete list and details please visit the NEWS file or this blog post.

  • Stomp destination
  • Multiline support
  • Blacklist support
  • and many more

As usual, testing is greatly appreciated as it helps to iron out problems before the final release!
Sources are available from git or as tgz. For third-party binary packages for various Linux and UNIX platforms, visit https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/3rd-party. Debian/Ubuntu and openS– USE packages are expected to be available in the coming days.

syslog-ng incubator project

The syslog-ng module incubator (Incubator henceforth) is a collection of tools and modules for syslog-ng that for one reason or the other, are not part of the official repository. This serves both as a staging ground for experimental modules, and as a repository of plugins that are not aimed at upstream inclusion. It’s also an example of a third party syslog-ng module.
Sources are available at https://github.com/algernon/syslog-ng-incubator

syslog-ng GSoC: finished successfully!

The coding time for Google Summer of Code is over and both of our students finished their projects successfully. The redis destination is to merged to syslog-ng 3.5 before the beta2 release and the high performance mysql destination is expected to follow it soon. A summary blog post of their work is due to appear next week at http://petrovicsgyula.blogspot.com/ and http://tichygsoc.blogspot.hu/

syslog-ng 3.5 draft documentation

A draft version of syslog-ng 3.5 documentation was also released today. It contains some of the new features. It is available in many forms:

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-08: New releases; GSoC updates; Dedicated OSE developer; Observe, Hack, Make 2013;

Dear syslog-ng users,

This is the 26th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng 3.4.3 and 3.3.11 are released

Both the current (3.4) and previous (3.3) stable syslog-ng releases received maintenance updates. Sources are available from git or as tgz from https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/download
For third-party binary packages for various Linux and UNIX platforms, visit https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/3rd-party for 3rd party binary packages for various distributions and UNIX variants.

syslog-ng Agent for Windows 5.0

There is a major new version of syslog-ng Agent available, version 5.0. Under the hood it builds now on the same code base as the UNIX version of syslog-ng. The new highlights include support for the Reliable Log Transfer Protocol (RLTP), flow control and support for the Windows Server 2012 and Windows 8 platforms.

Long-time syslog-ng contributor Gergely Nagy now working full-time on syslog-ng Open Source Edition

If you’ve been following the mailing list, the community around syslog-ng or the commits in git you most certainly recognize the name of Gergely Nagy. He’s been working at BalaBit for several years and we’re happy to announce that from now on his dedicated role is to work on syslog-ng OSE full-time.
You can read his thoughts about the change on his blog.

syslog-ng GSoC: it’s half-time!

The coding for Google Summer of Code is at half-time. Our students passed half-time evaluations and are working hard on a faster MySQL destination and on a brand new redis destination. You can follow their progress from their related blogs:

There is also a plan to provide test packages for those really adventurous in the coming weeks 🙂

syslog-ng JSON HowTo

JSON is gaining popularity, not just in Web 2.0 but in all fields of IT. As it’s an easy way of storing and transmitting name value pairs, JSON emitting and parsing is now part of syslog-ng.
The HowTo describing how it works in syslog-ng is available at http://asylum.madhouse-project.org/blog/2013/07/29/json-howto/

syslog-ng developers at Observe, Hack, Make 2013

Laser harp, Wikileaks, lockpicking, NSA, SIM card exploiting, LHC, counter-cryptanalysis, 24 pull requests, neutrino detectors, open source Bach, weird lights, quadrocopters, electric music cranked up to 11, workshops, lectures, discussions, and a lot more! We’ve been Observing, Hacking and Making!
You can read more about it at this post.

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-07: syslog-ng 3.4.2 released; PatternDB update; GSoC; RSS destination

Dear syslog-ng users,

This is the 25th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng 3.4.2 is released

Last week syslog-ng 3.4.2 was released, the first bugfix release in the 3.4 series. The most important fix is for a hang when suppress() was used. A full list of changes is available in git.
Sources are available from git or as tgz.
For third-party binary packages for various Linux and UNIX platforms, visit this page for 3rd party binary packages for various distributions and UNIX variants.

PatternDB git moved and updated

The BalaBit patterndb git moved recently to github. It’s available now at https://github.com/balabit/syslog-ng-patterndb instead of our server. That way the patterns are available next to the syslog-ng sources, and are also faster to download. We have also updated, reorganized and extended the available patterns. For details, read this post.

syslog-ng is participating GSoC again

Just as last year, syslog-ng is participating in the Google Summer of Code under the umbrella of the openS– USE project. Two of our candidates have the opportunity to code all summer long as GsoC students. Gyula Petrovics will work on a faster MySQL destination, which works without libdbi. The other student, Tihamér Petrovics, will add a redis destination, which can not only store logs but also provide counters and help with statistics.

Related blogs:

RSS destination

RSS destination is a neat little feature of syslog-ng. It can store up to 100 pieces log lines, and can serve them as an Atom feed.
This destination works as a FIFO, so if it is full, the last incoming log line kicks out the first one. Now you are able to read the alerts from your machines in your favorite RSS reader (well, unfortunately not in Google Reader).
Available in the feature/rss-destination branch.

SHORT NEWS

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-06: syslog-ng configurator on Android; Using syslog-ng with Splunk; EU data protection and logging

Dear syslog-ng users,

This is the 24th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng configurator app for Android

There is now a new configurator app available for Android, developed as a hobby project by one of the syslog-ng team members. The focus of the application is to create a syslog-ng.conf wich provides optimum performance based on a number of questions.
For more details and download locations read the authors blog.

syslog-ng and Splunk

We often receive questions, how to use syslog-ng and Splunk together in a logging infrastructure. We collected the most popular usage scenarios into a white paper, together with example configurations to make testing and integration even more easy.

syslog-ng is participating GSoC again

Just as last year, syslog-ng is participating Google Summer of Code under the umbrella of the openS– USE project. We have candidates for developing a native mysql destination, a redis destination, a XMPP (jabber) destination and log signing, which is a big improvement from last year, where we only had a single candidate for a similar number of development projects. The application process is closed now and there are still a couple of weeks to go, before the final list of approved students is announced. http://news.opensuse.org/2013/04/25/opensuse-hedgewars-and-owncloud-are-moving-gsoc-along-participate-and-submit-your-proposals-fast/

Big changes ahead for EU data protection regulation

This summer will most likely bring big changes in the regulation of Data Protection in the European Union. We collected these proposed changes and also how syslog-ng and proper central log management can help to comply with these regulations.

Compiling syslog-ng with MS SQL support on RHEL / CentOS & Co.

During the past few months many people asked, how to log from syslog-ng to MS SQL on RHEL or CentOS. If you cannot buy syslog-ng PE, follow these steps to compile all the necessary components yourself and configure the MS SQL part.

SHORT NEWS

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2013-04: syslog-ng presentation at LOADays; New maintenance releases; Comment the Adminguide online

Dear syslog-ng users,

This is the 23th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

syslog-ng presentation at LOADays

Peter Czanik of BalaBit gave a presentation at LOADays, a conference focusing on system administrators. It was about the major new features of the syslog-ng 3.X series including a few short demos of these features. Participants were most interested in patterndb, correlation and AMQP and asked about upcoming features.
The conference page is available at: http://loadays.org/
Blog about the event: http://czanik.blogs.balabit.com/2013/04/czp-loadays/

New maintenance releases

While most people are interested in new features, the syslog-ng team is also working on maintenance releases which fix problems in already released software. A
new maintenance version was released for the 3.3 series
a few days ago and an other one is expected to arrive soon for the 3.4 series.

Comment the Adminguide online!

As you might know, we publish The syslog-ng Open Source Edition Administrator guide in three format: PDF, single-page HTML, and many-page HTML. The many-page HTML version of the OSE 3.3 and 3.4 guides have a new feature: online commenting. That means that you can easily give us feedback on any section of the adminguide. For example, you did not understand how X feature works? Let us know! You have found a typo? Please add a comment at the bottom of the page so we can correct it! You have a better, real-life configuration example instead of the one in the guide? Add it as a comment! Or even worse, there are no examples where it would be needed? If you already got a working example, please share it with us!
Try it at https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.4-guides/en/syslog-ng-ose-v3.4-guide-admin/html/ch01s01.html

BOOK OFFERING:

Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management

Effectively analyzing large volumes of diverse logs can pose many challenges. The recently published Logging and Log Management book helps to simplify this complex process using practical guidance and real-world examples. Packed with information you need to know for system, network and security logging. Log management and log analysis methods are covered in detail, including approaches to creating useful logs on systems and applications, log searching and log review.
Chapter 5 describes you what is syslog-ng, and you also find some useful examples for deployment and configuration.
http://www.amazon.com/Logging-Log-Management-Authoritative-Understanding/dp/1597496359

SHORT NEWS

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

ARCHIVE

http://insider.blogs.balabit.com/

syslog-ng Insider – March 2013

Dear syslog-ng users,

This is the 22th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.
Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com

FEATURED NEWS

Video: networking scenarios and filters

There is a video series in the work to introduce the basics of syslog-ng. The third video is online now, which explains networking scenarios and filters. You can watch it at

Masking credit card numbers in log messages with syslog-ng

Compliance with different regulations is becoming more and more important recently also in logging. One of these, PCI-DSS requires credit card numbers to be masked in logs. Our CTO describes, how it can be achieved using the freshly released syslog-ng 3.4 at http://marci.blogs.balabit.com/2013/02/masking-credit-card-numbers-in-log-messages-with-syslog-ng/.

Brand new syslog-ng Superhero T-shirt available

The syslog-ng Superhero T-shirt was designed for open source fan geeks. Can’t wait more to get one? Simply tell us which version of syslog-ng (OSE, PE or SSB) you use, and be our public syslog-ng reference. Please send an email to Peter Czanik for more information and don’t forget to include your t-shirt size. 😉

SHORT NEW

  • openS– USE 12.3 was released yesterday and is the first Linux distribution to include syslog-ng 3.4.1

NEW RELEASES

ARCHIVE

http://insider.blogs.balabit.com/