Why Sagan?

In mid 2009, Quadrant Information Security. staff monitored an attacker breaking into a network and modifying logs. The attacker was attempting to "hide" traces of the break-in. Nothing new there. However, the company that had hired us had a centralized logging system. The problem was, it wasn't in real time and the attacker could modify the logs before they were sent to the centralized logging server! In order to retrieve evidence from the centralized logging system, you had to know what you were looking for. For the average administrator, this might not be an easy task.

The thought occurred to build a real time monitoring system that would tell the administrator when "bad things" are happening.

There's already plenty of software to do this type of task. However, most of it is "near real time". That's the type of software the attacker above took advantage of.

The other issue is what to do with the attack data once you detect it? At Quadrant, we're fans of Sourcefire's Snort IDS/IPS (Intrusion Detection/Prevention System) software. Rather than having "Yet another console" to monitor, we decided to put the data into a Snort database as a separate "sensor ID". This has since been been expanded to Unified2 output support, the Prelude framework and many others.

If you don't use Sourcefire's Snort, you probably should. However, even if you don't, don't worry, Sagan supports multiple output formats.

What does Quadrant, Inc use Sagan for?

Quadrant Information Security is a 24/7 managed network security monitoring company. We use Sagan to monitor security events from:

    • Routers (Cisco, etc)
    • Managed network switches
    • Firewalls (Sonicwall, Fortigate, etc)
    • IDS/IPS systems (Cisco, Fortigate, etc)
    • Linux and Unix systems (services, kernel messages, etc)
    • Windows based networks (Event logs, etc)
    • Wireless access points (Cisco, D-Link, etc)
    • Host based IDS systems (HIDS) ( AIDE, OSSEC, etc)
    • Detection of rogue devices on networks (via Arpalert, etc)
    • Much, much more..

      From Quadrant's stand point, Sagan gives us a broad range of devices, services, applications that we can monitor. For example, if your organization is a "Cisco shop" and you don't want to deploy Snort based IDS/IPS sensors, it really doesn't matter to our staff. We can monitor the Cisco devices just as we would a Snort based IDS/IPS solution.

What license does Sagan use?

We believe in open source. Sagan is licensed under the GNU General Public License version 2. You're welcome to test, deploy and use it as you see fit! However, if you need professional Sagan support, 24/7 managed network security services or have other security related needs, consider talking to Quadrant Information Security! You can contact us at 1-800-538-9357 (U.S.A - 24/7).

What operating systems does Sagan run on?

Sagan was primarily developed to run on Unix and Linux based systems. Sagan has been tested on various Linux platforms (Gentoo, Ubuntu) and FreeBSD OpenBSD, etc. Please let us know if you've successfully used Sagan on other platforms. Sagan may compile under Microsoft Windows, but we don't have any plans at this time to port it.

Why the name "Sagan"?

Sagan is named after my dog (a mutt). You can see a picture of her in the top left hand corner of this page. Sagan, the dog, is named after Carl Sagan. One of my favorite teachers/scientists. The software is named after both wink

Why is Sagan multi-threaded? / Why does Sagan use pthreads?

Mostly to prevent I/O blocking. On some systems, we receive millions of logs per day that need to be checked for "bad things". We cannot afford to "drop" any log entries, so when an event needs to be logged, a "thread" is created. This allows Sagan to continue processing event/syslog data in real time without dropping events.

Can I use Barnyard2 with Sagan? (Does Sagan support Unified2 output?)

Sagan version 0.1.8 added support for the Unified2 output format. This means using you can use Barnyard2 with Sagan. Sagan with unified2 output support and Barnyard2 is practically identical to how Snort uses unified2 with Barnyard2. The main advantages of using Sagan with unified2 and Barnyard2 is that the logging processes is seperated from the detection engine. Barnyard2 and the unified2 output format were created to separate the Snort IDS/IPS engine from having to deal with logging. That had a lot to do with I/O blocking. Without separating the processes, Snort would sometimes "drop" packets. This isn't a "Snort" issue, I'm simply explaining the existence of Barnyard2. Since Sagan is multi-threaded, I/O blocking isn't an issue. However, Barnyard2 and Unified2 have other advantages than precenting I/O blocking.

In particular, the ability to "queue events". For example, imagine you're logging to a remote database. If that database where to go down at any point, when an events is triggered, there would be no way for Sagan to store the event. Basically, the event would be "lost". This is not good. With Barnyard reading the Unified2 output, if the database connection is lost, Sagan can continue "recording events" and Barnyard2 will insert them when the database connection is re-established.

Barnyard2 also has the ability to record information in many other formats. Sagan + Unified2 + Barnyard2 == many forms of output. For example, Barnyard2 can write to Microsoft MS-SQL, Oracle SQL, MySQL, PostgreSQL and can use ODBC drivers. Barnyard2 also supports the Prelude Framework and Sguil support.

There are current no plans for Sagan to support the older "Unified" format, only Unified2

For a simple Barnyard2 example configuration, see the Barnyard2HowTo page. For general information Unified2 information, see the SaganHOWTO.

How does Sagan do the event correlation with Snort IDS/IPS events?

It should be pointed out, when using Sagan with a Snort database event's are kept separate from IDS/IPS events. That is, while your IDS/IPS system might be reporting as "sensor ID: #1", Sagan will startup as "sensor ID: #2". This way you can isolate between Sagan events and Snort events without an issue.

Sagan simply logs to the database in a way that if a Snort event correlates with it, it will. That is, just because there might not be a Snort event, doesn't mean Sagan won't log "bad things".

Sagan attempts to correlate events using these methods:

    • Time stamp - This should be fairly obvious.
    • Destination TCP/IP address - This being the machine that is being targeted or has a problem.
    • Source TCP/IP address - The source of the attacker or problem. This could be the machine directly, or the TCP/IP address of the attacker. Sagan attempts to pull the source of the attack or problem when it can.
    • Destination TCP/IP port - In many cases, we can tie the "service" reporting to a TCP/IP port. For example, we know OpenSSH runs on port 22 (normally), so we log it that way.
    • Source TCP/IP port: Sagan attempts to pull the remote TCP/IP port information, when it can. However, not all services report this.
    • Protocol used - For example, if the service being attacked or has a problem is OpenSSH, we can assume the protocol is TCP. So we log the information in the TCP database.
    • Classification - For example, with Snort a SMTP session with the command "expn root" (smtp.rules) is classified as an "attempted-recon". Sagan can see the same event, and will classify it the same ("attempted-recon").

How does Sagan use "liblognorm" for correlation?

Sagan uses liblognorm to gather information from log messages for better correlation of events. For more information about liblognorm, how it's used, how to compile and install please see the LibLogNorm page.

How does Sagan deal with Snort database information like Window size? (Offset, Chksum, Seq. Number)

That data is not applicable here. We're dealing with "logs", so there is no "Window size", etc. These values are set to zero or null. Sagan only populates the following fields:

    • sid
    • cid
    • ip_src
    • ip_dst
    • ip_proto

    • sid
    • cid
    • tcp_sport
    • tcp_dport

    • sid
    • cid
    • udp_sport
    • udp_dport
    • udp_len

    • sid
    • cid
    • icmp_type (always set to type 8)
    • icmp_code (always set to code 8)

    • sid
    • cid
    • data_payload
Any other fields are set to zero as they don't apply. If you think this is an error, or have ideas about how to use the other data fields, please let us know.

I don't get remote TCP/IP addresses from {insert service/application here} (For example, OpenSSH) via the Sagan rule flag "parse_ip" or "parse_port".

First off, Sagan can't always get remote TCP/IP address because they aren't always applicable. For example, if you're monitoring a Microsoft Windows environment and Sagan alerts you when an application crashes, there will not be a TCP/IP addresses tied to that event. In that case, Sagan "assumes" that the message it's receiving was generated by that machine.

Network services are a bit difference. For example, let's say you have OpenSSH running on 10.0.0.1 and a centralized logging server at 10.0.0.50. If an attacker attempts a brute force attack from 192.168.0.1 to 10.0.0.1, normally the centralized logging server will see the "source" of the message from 10.0.0.1 with a destination of 10.0.0.50.

.... And that is true, but it doesn't help the network administrator or security staff...

Sagan will attempt to identify the true attacker TCP/IP address, port being targeted, the protocol (TCP/UDP/etc) and in some cases the remote port. This way, the event shows the attacker from 192.168.0.1, and he's attacking 10.0.0.1.

This is useful information you can use. If the service is attempting to resolve (DNS lookup) the IP address of the attacker and put that in the event/syslog data, this needs to be disabled! For example, if your OpenSSH log entries look like this:

Jan 01 12:00:00 xhost sshd[44]: Failed keyboard-interactive/pam for invalid user work from dev.example.com port 58261 ssh2 

Sagan won't like this. There's no way for Sagan to determine the remote system's address. Also, from a logging stand point, you want the TCP/IP address, not what a DNS server "told" you. In this example, it'd mean in your "sshd_config" on the server you have the "UseDNS" flag enabled. Disable it, and your log entries will look like:

Jan 01 12:00:00 xhost sshd[44]: Failed keyboard-interactive/pam for invalid user work from 10.0.0.1 port 58261 ssh2 

This Sagan can deal with! Fortunately, the majority of services report TCP/IP addresses rather than using DNS lookup information.

Another way Sagan can extract useful information is via the "liblognorm" library. This actually works much better than "parse_ip" and "parse_port" in many cases. For more information, please see the LibLogNorm page.

Do I have to run Sagan on all workstations/machines/systems?

You can. It depends on how you want to use Sagan. If you're looking to catch "bad things" happening on say, your personal laptop, you can configure Sagan to run locally. This way, when something bad happens, you can be notified by a popup message (via xmessage) or what not. Building Sagan without MySQL/PostgreSQL/libesmtp saves a considerable amount of RAM and it'll give the desired results for notifying you on the local workstation.

However....

Sagan was originally designed to be run on a secure, centralized log server. This way, all systems within your network (network switches, routers, Windows server, Linux boxes, etc) can send events to a centralized server which Sagan can examine, correlate and warn you of potential bad things happening. This also makes Sagan easier to manage instead of having to manage workstations with Sagan installed, including rule set updates, you have a central point of management.

If you Google around the 'net', there's plenty of documentation to help you get started.

Does Sagan only work with syslog-ng and Rsyslog?

Sagan deals with log input in two different formats. The first is via a FIFO. When Sagan is in this mode, it'll watch for events through the FIFO. The second mode is via Standard input (stdin) . In the syslog-ng world, this is known as the "program" mode in the syslog-ng.conf file. Syslog-ng will work with both formats.

Rsyslog will work fine with Sagan, however, only in FIFO/named pipe mode. This is typically fine, as it's usually preferred that Sagan run in FIFO mode. For more information about configuration of Sagan with Rsyslog or Syslog-ng, see our SaganHOWTO.

It's likely that Sagan will work with other syslog daemons. If the syslog daemon in question can format the message properly, so Sagan can "read" it, in either program or FIFO mode, then it'll likely work.

If you find another syslog daemon that Sagan works with, please let us know!

Is Sagan meant as a replacement for Splunk / php-syslog-ng / etc... ?

No. Sagan performs a different role. Actually, these types of software can work in conjunction with Sagan and you'll benefit from it. Sagan simply identifies "bad things" in your network and has the ability to correlate that data with your Snort IDS/IPS data.

I'm not a Snort user, can I still benefit from Sagan?

Yes. Sagan has multiple output formats, with more to come. See the SaganTODO list for upcoming formats we'll be supporting. Right now, Sagan can e-mail you when "bad things" happen (via libesmtp), create "Snort like" alert log files or call an external program.

Calling an external program is interesting. This gives you the power to write your own routine(s) in the language of your choosing to deal with events how you see fit. When using this function, external calls are threaded. This insures that your program wouldn't be blocking any I/O and everything remains "real time".

Can I use my Snort console {insert application} with Sagan? (For example, BASE, Snorby, proprietary console, etc)

Yes. If your preferred console deals with standard Snort databases and tables, then it should work fine. The same goes for report generation tools and utilities that manage rule sets.

I have a lot of Microsoft Windows servers that I need to monitor. What's the best Event log to Syslog daemon for Windows?

In Quadrant's, Inc's research and development of Sagan, we found that Curtis Smith's (Purdue University) adaptation of the Event log to Syslog service to work quite well. For more information, see:

Eventlog-to-Syslog (Eventlog to Syslog Service for Windows (2k, XP, 2k3, 2k8+)

Sagan gives me the message, "Warning: inet_pton() error, but continuing...". Why is that?

This typically means that Sagan is receiving a 'host name' and not an 'IP address'. This means that your receiving server is using DNS, when is shouldn't be. (Hint: "use_dns(no);" in the syslog-ng.conf).

Sagan gives me the message, "Sagan received a malformed (host|date|tag|etc)". Why is that?

You shouldn't see the message very often. When it does happen, it means that Sagan didn't receive a fully formated syslog message. Typically this might happen on startup of Sagan where 'half' a message was received. Sagan will recover from this by filling in the missing information and continuing to the next message. If you are receiving a lot of these messages, then it likely means that a Sagan is receiving improperly formated messages. Look into your configuration and examine the output from the FIFO or syslog program.

Sagan isn't building correct under FreeBSD/OpenBSD/etc. What's the problem?

From the SaganHOWTO :

"If you are a FreeBSD or OpenBSD user, you might need to do the following. In most cases, the required libraries get installed in the /usr/local/lib and include files into the /usr/local/include directory. If you are getting an error in BSD stating the libraries and/or include files can't be found and you are sure they've been installed, try the following. This assumes you're using the 'bash' shell."

LDFLAGS=-L/usr/local/lib CFLAGS=-I/usr/local/include ./configure

Sagan isn't building under CentOS. What do I do?

Please refer to the CentOS page. Ibrahim Lopez Rivera wrote a some notes about how he got it working.
Jay Lopez's 'Installing Sagan on CentOS-5'

How do I use oinkmaster with Sagan rules?

Pretty much the exact same why you use it with Snort rules. However, you'll need to add the "\.rulebase$|" to your update_files (for liblognorm rulebase files). For a simple example configuration file, see the SaganOinkmaster page.

I get "fatal error: libpq-fe.h: No such file or directory" when building with PostgreSQL support. How do I fix this?

This means the PostgreSQL include/header files can't be found. Use the "--with-postgresql-includes" to fix this. For example, "--with-postgresql-includes=/usr/include/postgresql". Make sure you point it to the location of your libpq-fe.h file.

I have a question that isn't covered here, what do I do?

Join the Sagan mailing list and ask! The mailing list is low volume, and by asking in that forum, you might be helping other people with a similar problem.

You can also see if we're on IRC. You can point your favorite IRC client at:

Server: irc.2600.net
Channel: #sagan

If you don't have an IRC client installed, you can use the online client.

-- ChampClark - 2010-06-23

Topic revision: r12 - 2013-03-04 - ChampClark
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback