11 Tools You Should Be Monitoring With Syslog

Syslog has been around for four decades, and it’s a well-used tool in every DevOps and admin tool kit. The syslog format began as part of the Sendmail project, and it has since become a ubiquitous logging protocol used by hundreds of applications and supported out of the box by most major operating systems. This battle-tested logging format provides all the pieces you need to create actionable log messages and diagnose problems with your apps and services.

Because syslog is supported by most dev and admin tools (either directly or with third-party tools), using it is one of the easiest ways to start capturing log messages. Here’s a rundown of 11 tools you should be monitoring with syslog.

1. Docker

Running your app in containers makes it easy to scale as your needs grow. Logs from Docker containers can be routed to a syslog server using a logspout container, which collects logs from other containers using the json-file or journald logging drivers.

For example, here’s how to run a new logspout container directing all log messages to a remote syslog server at SolarWinds^® Papertrail^™:

$ docker run—name=”logspout” \
•	volume=/var/run/docker.sock:/var/run/docker.sock \ gliderlabs/logspout \ syslog+tls://logs.papertrailapp.com:55555

2. MySQL

Understanding the speed of queries is crucial for keeping your MySQL servers running at peak performance. MySQL writes various entries to log files, and one of these files is the slow query log, which contains queries taking a long time. Starting with MySQL 5.1.20, you can natively send log entries to syslog servers (via the system logger) using the syslog command-lineparameters when starting your MySQL server.

If you’re using an earlier version of MySQL, you can send logs to a syslog server like Papertrail using the remote_syslog2 daemon in Papertrail.

3. HAProxy

HAProxy logs contain a wealth of information related to connections and requests, and they’re the first place to look when you need to troubleshoot issues with your load balancer. The log format used is completely customizable, allowing you to include things like the number of bytes read by the client, the HTTP method of the request, and timestamps. Sending your logs to a syslog server allows you to keep your valuable logs in a single place, making analysis quicker.

To direct HAProxy logs to syslog, you need to edit the haproxy.cfg configuration file and add a new global log configuration option. Additionally, you need to add a log global directive to the defaults section. Here’s an example of how you can do this:

global
log 127.0.0.1:514 local0

defaults
log global

It’s also a good idea to add the log-send-hostname to your global section, which sets the HAProxy server in the hostname field of the syslog header. This makes it much easier to later match log messages with specific senders.

4. Redis

Redis is another popular tool designed to natively support writing log messages to syslog. You can configure Redis to send log messages to a syslog server by uncommenting the syslog-enabled option in the redis.conf configuration file. If you need to override the syslog identity from the default redis string, you can uncomment and customize the syslog-ident option too. Here’s an example showing you how to do this:

syslog-enabled yes
syslog-ident customhostname

5. IIS

Monitoring web access logs is a daily task for DevOps teams, and even though Windows has mature support for logging with the Windows Event Log API and Event Viewer, consolidating your IIS logs in one place by sending them to a syslog server makes it easier to holistically analyze the data inside.

IIS supports two different ways of transmitting logs to syslog: using nxlog or Eventlog-to-Syslog.

6. Node.js

No code is perfect, especially when it’s still in development. Extracting log messages from services written in Node.js is something all developers need to do. Luckily, you can send log messages directly from Node.js apps by using the Winston logger with the Papertrail transport. And because it’s written by the folks at Papertrail, it works especially well with their software. This transport allows you to configure exactly where and how messages are delivered. You can send messages with TLS encryption, and you can even configure the transport to retry on connection failure.

Here’s an example showing you how to use Winston with Node.js to send messages to syslog:

var winston = require(‘winston’);
require(‘winston-papertrail’).Papertrail;

var winstonPapertrail = new winston.transports.Papertrail({
	host: ‘logs.papertrailapp.com’,
	port: 12345
})

var logger = new winston.Logger({
	transports: [winstonPapertrail]
});

logger.info(‘this is my message’);

7. VMware

One of the best practices when running VMware is to make sure your logs are sent to a remote destination. This prevents an unrecoverable event from happening to your virtual machine and causing data loss. ESXi provides native support for syslog, allowing you to send logs to a remote syslog server.

(Note: You must make sure the firewall on the ESXi host is configured to allow the syslog connection through.)

You can configure remote syslog using the following:

$ esxcli system syslog config set—loghost=’ssl://hostname:1514’
$ esxcli system syslog reload

8. Amazon EC2

Amazon CloudWatch is the usual method for capturing logs generated by other Amazon services. If some of your services are running outside of AWS, however, you can easily find yourself in a position where your app and infrastructure logs are stored in two separate places. Fortunately, you can simplify things by sending your logs to a remote destination.

Using a logspout container—the same method discussed in the Docker entry—you can route your EC2 logs to a remote syslog server like Papertrail.

9. Fastly

During operation, Fastly captures detailed information about the HTTP requests it receives and can send them in real time to a remote syslog server. Inspecting logs is an important part of optimizing the performance of Fastly services. To configure Fastly to send logs to a remote destination, you need to set up a new logging endpoint. This article shows you how.

10. Apache

The log messages generated by web servers like Apache can be used to detect things such as errors affecting your users, which pages are the most frequently visited, and whether unauthorized users are trying access private areas of your site. Apache logs can be sent to a syslog server by installing and enabling the mod_syslog module. Afterward, all you need to do is add an ErrorLog directive with your chosen syslog facility. Here’s an example showing you how to do this:

ErrorLog syslog:user

11. Cloud Foundry

Cloud Foundry aggregates the logs created by your apps, and these logs can be streamed to an external log management service using Cloud Foundry’s drain API. To extract logs, you need to set up a communication channel between your Cloud Foundry deployment and the syslog server. Here’s an example of how to do this:

$ cf cups my-logs -l syslog-tls://logsN.papertrailapp.com:XXXXX
$ cf bind-service <appname> my-logs
$ cf restart <appname>

A Cloud-Based Syslog Viewer

Monitoring your log messages using syslog makes it easy to extract useful information from your tools. But what do you do once have this information? To make it easy to search your logs, you need a tool capable of holistically analyzing them without slowing down as the volume of your logs grow. The best tool for this is a cloud-based syslog viewer like Papertrail. If you haven’t tried it and want to see it in action, head over to papertrail.com and request a demo!