Installing / Configuring liblognorm.

Intro to liblognorm

"liblognorm" is a log normalization library. That is, you can feed liblognorm a "log message" (think syslog or event log) and it can extract useful information from the message. Liblognorm was created by Rainer Gerhards of "Rsyslog" fame in mid-2010. It uses a simple rule syntax to extract useful information from a log message. Actually, a better way to think about it is that it uses a "masking" approach to extract information. "liblognorm" is based of the Mitre Common Event Expression Language which has not been standardized yet. While testing with Sagan, we've found liblognorm to be extremely stable, but it should be noted that since CEE is not standardized yet and it's a fairly young project, liblognorm is expected to go through changes making it a "moving target".

For a simple example of how liblognorm works, let's examine the following log message:

Invalid user bobuser from 192.168.0.1 on port 14121

Even though this is a simple example, we can already see some information we might want to extract using liblognorm. The username, TCP/IP address and TCP port number in particular. To do this, we can create a simple "rule base" for liblognorm to use to extract this information. For example:

Invalid user %username:word% from %src-ip:ipv4% on port %src-port:number%

Sagan uses liblognorm to extract information from log messages in order to better correlate the events. Sagan also has other methods of parsing, such as "parse_src_ip", "parse_dst_ip" or "parse_port", but liblognorm is by far the best at extracting useful information. Sagan rules ship with a standard rules and liblognorm "rulebase" files. These files end with the extension of .rulebase and are intended for liblognorm use. Sagan only uses liblognorm in certain cases, where rulebases or "masks" have been created from known log events to correlate information. That is, Sagan doesn't use liblognorm against every incoming log line because sometimes the log line might not contact information we need. In some dynamic log messages, it's better to use the "parse_ip" or "parse_port" flags within a rule.

You can certainly disable liblognorm at compile time, but it's advised against as liblognorm adds a lot of functionality to Sagan.

Compiling and installing liblognorm from the 'Github' tree.

$ git clone https://github.com/rsyslog/libfastjson
$ cd libfastjson
$ ./autogen.sh
$ ./configure && make && sudo make install

[Note: Sagan & liblognorm no longer use libjson-c! liblognorm 1.1.3+ & Sagan 1.1.0+ use libfastjson]

In order to install from the Github resource, open a terminal and enter these commands in this order:

$ git clone https://github.com/rsyslog/libestr
$ cd libestr
$ autoreconf -vfi
$ ./configure
$ make && sudo make install

Once libestr is installed, continue with liblognorm.

$ git clone https://github.com/rsyslog/liblognorm/
$ cd liblognorm
$ autoreconf -vfi
$ ./configure --disable-docs
$ make && sudo make install

Once you completed these steps, you can ./configure Sagan with liblognorm support. By re-running the Sagan ./configure, it'll detect that liblognorm is installed and use it.

-- ChampClark - 2016-05-04