Need to filter based on domain portion of query log entry

Q & ANeed to filter based on domain portion of query log entry
Robert Webb asked 2 months ago

So I have a situation where I need to take a list of domains, ie. google.com, and send those entries from Bind’s query log to a separate log file. Given that the fqdn is delivered in the MSG section, how can I parse out google.com out of http://www.google.com and match to my list?
 
Once I have the parsing and filter done, the sending to a specific file I am good with.
 
The list file would only have “google.com” in it.

1 Answers
Róbert Fekete Staff answered 2 months ago

Hi, since Bind query logs are not really structured, you can either:
 * Use the pattern database feature of syslog-ng to write a pattern that matches these messages, and extract the needed data. This sample database might actually work for you: https://github.com/balabit/syslog-ng-patterndb/blob/master/applications/bind9/named.xml For details on pattern databases, see https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/chapter-patterndb.html Note that you might need to use a further parser to split the unneeded part of the fqdn (for example, a csv parser, or the substr template function)
 * If you need a lot of custom processing for these logs, it might be easier to write a custom python parser ( https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/python-parser.html ). Note that Python parser requires a recent version of syslog-ng.
HTH, 
Robert