Community met GSoC 2015 participants via IRC

Channel statistics

#syslog-ng@freenode
Event started at 2015.06.04 17:00 CET
42 nick’s on channel (+1 joined later)
14 active participants (at least 1 comment)

Chat around 5 GSoC topics

htrap – My project is to create a Riak Destination for Syslog-ng:
https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap)

Krishna_ – My project is – Syslog-ng server monitoring with mobile app:
https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-syslog-ng-server-monitoring-with-mobile-app-(Krishna41)

battila7 – My project is: syslog-ng as a command line tool:
https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Syslog-ng-as-a-command-line-tool-%28battila7%29

mamenyaka – I am working on the Qt syslog-ng configuration editor:
https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-syslog-ng-configuration-editor-using-Qt-(mamenyaka)

asdwsda – and my project is to extend java language binding for syslog-ng:
https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-syslog-ng-for-Java-(szpeter)

Short summary

Community members detailed their use case expectations, asked different questions and started brainstorming.

  • SHRREAK would be the next level of SHREK (Syslog-ng Riemann Elasticsearch Kibana) stack.
    A stands for Android for monitoring syslog-ng servers. Stat
    istics will be available also
    2nd R stands for Riak which would store events instead of ES (ES would be used only for indexing)
  • A feature proposal arrived – Qt config editor should be able to parse existing files
  • CLI example:

    throw-logs | syslog-ng-cli –parser db –dbparser /var/lib/pdb.xml –source stdin –destination stdout’

  • htrap blogs his experiences about GSoC: http://thetechtrap.com It turned out that cor edevelopers of syslog would welcome articles on https://syslog-ng.org
  • squid generated JSON parsing was discussed with syslog-ng meanwhile

insider 2015-03: GsoC; Hadoop; 3.7 documentation; kafka; grok;

Dear syslog-ng users,

This is the 40th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news.

NEWS

syslog-ng in Google Summer of Code 2015

This year we participate in GSoC again, with a focus on making
the life of our users and contributors easier. Our main projects for this year include:

  • Python and Java language bindings to make code contribution easier
  • Develop a Qt-based graphical configuration editor to help new users to get started
  • A data-flow visualization tool to help in debugging

If you are a student and willing to spend the summer coding syslog-ng, or know someone who could participate, check our detailed project and idea list.

You can reach mentors by direct e-mail, by IRC on channel #syslog-ng
on FreeNode, or on our mailing list.

syslog-ng Hadoop support

With the release of syslog-ng PE 5F3 support for Hadoop arrived. It will also be part of the upcoming syslog-ng OSE 3.7 release. This enables syslog-ng to write log messages to HDFS. https://jluby.blogs.balabit.com/2015/03/10/syslog-ng-and-big-data-streaming-into-your-data-lake/

syslog-ng 3.7 beta is coming

The first beta of syslog-ng OSE 3.7 is expected to arrive in the coming weeks. It has many smaller and larger changes, like the Java destination is migrated from incubator to syslog-ng core.

insider 2014-09: 3.6 beta; eCSI; DevOps; anonymization; GSoC;

Dear syslog-ng users,

This is the 37th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

Beta testing of syslog-ng 3.6 started

Beta testing period of syslog-ng 3.6 started two weeks ago with a beta1 release. It was quickly followed by a beta2 release, fixing mostly portability. Some of the new features are coming from the syslog-ng incubator, like Riemann support or the graphite template function. The journal source and many enhancements to existing features are coming from the syslog-ng PE team. Also, the @cim prefix can be used as a local “rich” log transport on /dev/log, and messages will be parsed by the JSON parser if available.

Beta2 is already available in the FreeBSD ports in sysutils/syslog-ng-devel. For RPM distributions, check https://czanik.blogs.balabit.com/2014/09/journal-source-in-syslog-ng-3-6-beta1/.

BalaBit syslog-ng team takes over the syslog-ng OSE development

algernon, the full-time syslog-ng OSE developer at BalaBit decided to look for new challenges.
First of all, thank you very much for all your hard work, bugfixes nifty new features, and everything else you did for syslog-ng OSE! We all wish you the best, and hope you’ll have a great time outside BalaBit as well 🙂
To keep the development, maintenance, and releases of syslog-ng OSE on track, the developer team of syslog-ng Premium Edition will take over the tasks related to syslog-ng OSE: they will manage bugfixes, patches, pull requests, and also the general development of syslog-ng OSE. Naturally, this does not affect the current or future openness of syslog-ng OSE in any way: Your contribution is as welcome as ever.
For more details about this change, see algernon’s blog post.

eCSI training

BalaBit provides now a free training, which provides a fresh insight on logmanagement. The first level introduces the listener to compliance, planning an infrastructure and logs in forensics situations. Once your knowledge is tested, you can go to the next level and learn about IT security and eCSI.

syslog-ng, riemann, collectd-notifications, elasticsearch

How to build an event-based infrastructure to push structured messages to different subsystems for alerting, reporting and storage. Using syslog-ng, each message is normalized into a structured event, optionally correlated with other messages, and conditionally routed to systems. Read more at:
http://devops.com/features/guide-modern-monitoring-alerting/

Data Privacy, Anonymization and Log Data

Strong data privacy laws are arriving slowly but surely in Europe, which also affects logging. Raw data contains too much information, on the other hand anonymized data does not have enough information to handle a security incident. Read about a possible solution and how syslog-ng can help.

Google Summer of Code: success

Google Summer of Code ended a few weeks ago. All of our students successfully completed their projects. We would like to thank for the hard work of students and their mentors, and Google for the opportunity!

You can read more about the completed projects.

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2014-08: EPEL; graphite; PCI DSS

Dear syslog-ng users,

This is the 36th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

EPEL 7 now contains syslog-ng

RHEL 7 was released over a month ago and CentOS 7 not much later, but one piece of software was still missing: syslog-ng. Not any more. EPEL, which stands for Extra Packages for Enterprise Linux, is a software collection containing additional packages for Enterprise Linux and derivatives. Now its latest version, EPEL 7 also contains syslog-ng, version 3.5.

Introducing syslog-ng PE 5F1

The latest version of the syslog-ng Premium Edition, 5F1, adds support for the popular NoSQL database MongoDB. Along with support for MongoDB, we have added support for Java Script Object Notation (JSON), a text-based open standard designed for human-readable data interchange.

Performance monitoring using syslog-ng and graphite

For most of its history, syslog-ng could only be used for collecting, processing and storing log messages. Not any more. The Redis and Riemann destinations are already a step into the direction of metrics-based monitoring, and the monitoring source combined with Graphite template support are the next.
https://czanik.blogs.balabit.com/2014/07/how-to-setup-syslog-ng-quickly-for-performance-monitoring-using-graphite-inside-docker/

Introducing syslog-ng store box 3F2

We recently released a new version of our log management appliance, the syslog-ng Store Box. 3F2 is the latest feature release and includes one major new feature and a major improvement to an existing one. First, we have added a RESTful API which opens up all sorts of possibilities for accessing log data in SSB. Second, we have revamped the search interface on the web-based user interface making searching and troubleshooting much easier.

syslog-ng incubator 0.3.3 released

The syslog-ng incubator is a set of tools and modules for syslog-ng, which are not (yet) available in the official release. This version of incubator works with the latest stable syslog-ng (v3.5.5+) and fixes many problems of the initial 0.3 incubator release.

Log management and the Verizon 2014 PCI Compliance Report

Recently, the eagerly anticipated Verizon Data Breach Investigations Report for 2014 was published. With more than 63,000 security incidents, 1,300 confirmed data breaches and 50 contributing global organizations, it provides the most comprehensive insight to state of IT security around the world. Drawing on data from the Data Breach Investigation Report, Verizon also publishes a lesser known but very interesting report on the state of compliance of with the Payment Card Industry Data Security Standard (PCI DSS), perhaps the most widely-adopted security standards globally. Read, what requirements PCI DSS has towards log management/.

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.

insider 2014-06: syslog-ng PE news; ISO27001

Dear syslog-ng users,

This is the 35th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.

FEATURED NEWS

Handing over syslog-ng maintenance

Bazsi (Balázs Scheidler) started the syslog-ng project many years ago and maintained it ever since. He handed over stable branches to algernon (Gergely Nagy) a few years ago and now also the master branch, where current development is happening.
You can read the original announcement at https://bazsi.blogs.balabit.com/2014/05/handing-over-syslog-ng-maintenance/

JSON and MongoDB support added to syslog-ng PE

The two major new features of the syslog-ng PE 5.1.1 release are JSON support (both parsing and emitting JSON formatted messages) and MongoDB destination. You might notice, that these features were available in OSE for a while. They were cleaned up, bug fixed and enhanced by the PE team, and many quality assurance tests were added. The resulting code is merged back to upcoming OSE 3.6 version with some of the bugfixes ported back to 3.5 and even to 3.4 in some cases.
You can read more about what is new.

Moving from BugZilla to GitHub Issues

As development of syslog-ng moved to GitHub a while ago and we started to use the more convenient GitHub Issues for bugtracking the current syslog-ng BugZilla will depreciated. No new issues will be allowed in Bugzilla from the 13th of June, but existing issues will remain there.
For more details check the announcement at https://lists.balabit.hu/pipermail/syslog-ng/2014-June/021458.html

Python and Perl support in incubator

Perl and python support were already introduced in last months newsletter, now it’s available as a release. Other new features include support for getent template function and enhanced graphite template.
Source code is available at https://github.com/balabit/syslog-ng-incubator and there are compiled packages available for Debian, Fedora, openS– USE, Ubuntu. FreeBSD will be updated after additional bugfixes.

ISO27001 and Log Management

PCI DSS wasn’t the only standard to be updated recently. A new version of ISO27001, an information security standard first published in 2005, was released last September. ISO27001:2013 provides a framework for implementing an Information Security Management System (ISMS). The new version has been modified to align better with other ISO standards. Ten new controls have been added with an emphasis on measuring the effectiveness of the ISMS. Just as with PCI DSS, we decided ISO27001’s importance and broad adoption merited a technical white paper dedicated exclusively to how log management and more specifically, the syslog-ng application and the syslog-ng Store Box, can meet the standard’s requirements.
You can download the whitepaper at https://pages.balabit.com/iso-27001-compliance-and-log-management.html

NEW RELEASES

Your feedback and news tips about the next issue is welcome at documentation(at)balabit.com.