Make Your Logs Work for You

The days of logging in to servers and manually viewing log files are over. SolarWinds® Papertrail™ aggregates logs from applications, devices, and platforms to a central location.

View Technology Info

FEATURED TECHNOLOGY

Troubleshoot Fast and Enjoy It

SolarWinds® Papertrail™ provides cloud-based log management that seamlessly aggregates logs from applications, servers, network devices, services, platforms, and much more.

View Capabilities Info

FEATURED CAPABILITIES

Aggregate and Search Any Log

SolarWinds® Papertrail™ provides lightning-fast search, live tail, flexible system groups, team-wide access, and integration with popular communications platforms like PagerDuty and Slack to help you quickly track down customer problems, debug app requests, or troubleshoot slow database queries.

View Languages Info

FEATURED LANGUAGES

TBD - APM Integration Title

TBD - APM Integration Description

TBD Link

APM Integration Feature List

TBD - Built for Collaboration Title

TBD - Built for Collaboration Description

TBD Link

Built for Collaboration Feature List

Join SolarWinds Day October 19 to see the next evolution in observability. Register now.

Blog > The Syslog Staying Power

The Syslog Staying Power

Posted by By Papertrail Team on August 24, 2021

Some classics never go out of style, like a good pair of boat shoes or cowboy boots, depending on where you live. In the logging world, syslog is this classic. For more than 30 years, the syslog protocol has been a standard for logging. When we talk to users about what type of logs they collect and how they send them to SolarWinds® Papertrail, syslog always comes up. 

“Our application logs and server system logs are sent to Papertrail. We use various libraries for sending logs and alerts from applications. Servers are all sending logs via rsyslog.” Tyler, Word Press Developer

Advantages of Syslog

Here are some of the reasons why Papertrail users continue to rely on this classic. 

Broad Compatibility

Syslog works with a wide variety of both network devices as well as application.  Everything from printers and routers to platforms such as Unix/Linux, BSD Unix, and macOS supports syslog. 

Built-In Transport

From the beginning, syslog has offered the ability to route log messages over a network connection. This means you’re not limited to logging messages to a local file—you can direct syslog messages to a logging server. For Papertrail users, syslog is a highly versatile logging protocol capable of directing logs from a multitude of different devices and applications to Papertrail. 

Layered Architecture

The layered architecture in the syslog protocol separates the transport, application, and content layers. This design allows you to choose your preferred transport protocol independently from application or content constraints.

The layered architecture in Syslog separates the transport, application and content layers.
The syslog layered architecture

Standard Message Format

The syslog message format provides a structured framework with enough flexibility to support vendor-specific extensions while maintaining broad compatibility. In a syslog message, you’ll find three elements:

  • Header: version, time stamp, host name, priority, application, process, and message ID
  • Structured data: data blocks in a specific format
  • Message: content encoded in Unicode Transformation Format (UTF-8) and a tag identifying the process that triggered the message

Beyond the requirement for a source tag and UTF-8 encoding, messages can be configured for a variety of needs. Check out this article for best practices creating syslog messages.

Drawbacks of Syslog

Papertrail users also mentioned some of the shortcomings of syslog and how they overcome them. 

One-Way Communication

One of the major drawbacks is its pure simplex protocol, which means there’s no acknowledgement for message delivery. When you couple the one-way communication offered by the simplex protocol with UDP, there’s a potential for lost data. You can minimize the risk of losing data by switching to TCP. Unlike UDP, TCP can recognize when a link is congested and can slow the sending rate.

Potential Security Weaknesses 

There’s no built-in authentication on syslog messages, especially when used with UDP. As a result, it’s possible for one machine to impersonate another machine and send bogus log events. This means it’s also susceptible to replay attacks. Fortunately, you can address this concern by switching to TCP. Another big security advantage of using TCP is the ability to use TLS encryption. For more information on TLS, check out the documentation.

Papertrail Users Love Syslog

A large part of what makes Papertrail a perfect log management  choice  for syslog events is the built-in separation between the application or device generating the message and the system storing the message. When you combine this flexible design with syslog’s broad compatibility and the easy setup and intuitive search capabilities of Papertrail, you end up with a simple yet powerful logging solution. 

Syslog is a popular choice among Papertrail users.

Papertrail users comment frequently on how fast and easy it is to configure remote syslog to send logs to Papertrail. All they need to do is add the Papertrail instance and port number to the end of a config file.

The most frequent question we hear when users are configuring syslog is where they can find the Papertrail instance and port information. There are two ways you can view the instance and port details. You can either go to the Dashboard view and select Add Systems or select the Quick Start option under the Support menu and select Add Systems. Both options will display the instance and port information needed for configuring remote syslog. 

The Papertrail Quick Start wizard provides the instance and port information need to setup syslog.
Papertrail instance and port information

The instance and port information are displayed in the yellow bar at the top of the Add Systems screen. 

Manual Setup

You can manually configure remote syslog by editing the syslog daemon configuration file and adding the Papertrail instance and port number. Here are the basic configuration steps, but there are even more details in the documentation

Rsyslog

Edit /etc/rsyslog.conf. by adding this to the end of the file:

*.*                   	@loghostname.papertrailapp.com:XXXXX

Where loghostname is the prefix of your Papertrail instance and XXXXX is the port number.

Syslogd

Edit /etc/syslog.conf. by adding this to the end of the file:

*.*                   	@loghostname.papertrailapp.com:XXXXX

Where loghostname is the prefix of your Papertrail instance and XXXXX is the port number.

Syslog-ng

Edit either /etc/syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf.

Create a logging destination for Papertrail by adding this block to the configuration file:

destination d_papertrail {
	udp("logsN.papertrailapp.com" port(XXXXX));
};

Where loghostname is the prefix of your Papertrail instance and XXXXX is the port number.

Next, find lines starting with source and paste configuration entries pointing your sources at the new Papertrail destination.

log { source(s_startctllog); destination(d_papertrail); };
log { source(s_startctludp); destination(d_papertrail); };
log { source(s_startctltcp); destination(d_papertrail); };

Automation Options

Papertrail offers several automated configuration options in the Quick Start menu under Support. You just need to select the type of log files and the source, and it provides a customized configuration file and step-by-step instructions.

Papertrail quick start wizard.
The Quick Start menu provides step-by-step instructions

If you happen to be working in Linux, you can also automate the setup of the remote_syslog2 logging daemon with Chef, Puppet, and Salt. Links to the cookbooks, modules, and formulas are below:

chef-papertrail

puppet-papertrail

salt-papertrail

Summing It Up

Syslog is a protocol applications and devices use to send event data to a central location. With broad compatibility and great flexibility in both the message structure and transportation options, syslog remains a popular choice among Papertrail users. Setting up syslog with Papertrail is quick and easy, and it can be done manually, with a customized script provided by the Quick Start menu or through automation tools such as Check, Puppet, and Salt.

Searching and tailing syslog messages in Papertrail.
Tailing syslog messages in Papertrail

If you haven’t used Papertrail with syslog messages give it a try and let us know what you think.