Operating a modern web application requires having a good handle on what’s going on with your application’s log data. But this doesn’t mean you need to keep the logs pulled up all the time. Sometimes, you want the flexibility to send important logged events to any number of places. The integration of SolarWinds® Papertrail™ and AWS Simple Notification Service (SNS) can help you make this happen.
In this post, we’ll cover how to set up an automated alert to send a notification to an SNS topic. The alert will be configured to trigger on a custom search query of any new logged events across your entire stack.
Step One: Create a Node.js Application for Logging
For the purposes of this article, we’ll use an extremely simple Node.js/Express application designed to log at an INFO level for the only route it matches on and return an ERROR-level log for any other request. To get started, create a project folder and initialize it. Though you can use npm or yarn for your package manager, we’ll use yarn. We’ll also add the express package to our project.
In our project folder, we create a file called index.js. Our file has the following contents:
Note our server only has one successful route—on the root path (/)—and any other route will log (and respond with) an error.
Now, we could start our server with the simple node index.js command. However, since we’re planning to send our logs to Papertrail, we’ll start our server with this command instead:
This will dump the console output into a file called logger.log in our project’s working directory. Feel free to send the output somewhere else if it’s more convenient for you. Assuming your application runs and logs as you’d expect, let’s move on.
Step Two: Configure Papertrail
Sign up for a Papertrail account. As soon as you configure your account, you’ll learn how to send log data to the Papertrail servers. If you choose to aggregate application log files, then the setup screen will look like this:
Choose the platform you’re running on. Then, install remote_syslog2. Finally, update the command you received for sending your logs to Papertrail, pointing to the full path of the logger.log file we just created (replacing /path/to/your/file.log in the example).
With Papertrail logging set up, you can visit any route for your Node.js app in your browser. If remote_syslog2 runs without an error, then the setup page should update with a notice a log has been received. If you go to the Events page, you’ll start to see logs from your Node.js app appear. Visit several routes for your Node.js app and see how your Events page updates.
In a full implementation of Papertrail, you could set up system logs and application logs from many different sources and Papertrail would indicate the source and origin of all the logs it aggregates. However, for our demo, we just need this simple example.
Step Three: Set Up Your SNS Alert
Alerts in Papertrail are based on searches across your aggregated log data. Saving a search is simple. When you type something in the search box at the bottom of the Events page, Papertrail automatically filters results down to match your search:
When you click on the bookmark icon, you’ll see the option to save your search.
Our search is pretty simple, but once we save it, we have many options for what to do next.
If you scroll down a little more, you’ll see even more options.
If we click on the SNS alert under Webhooks, we see the Papertrail alert creation interface. We have several options for how to configure our alert.
We also see we need to provide some details for our SNS topic.
We’ll paste in the access key ID and secret access key from our AWS IAM settings. Next, we’ll need to create an SNS topic, but it doesn’t need to be anything fancy.
Notice we’ve chosen a Standard Type SNS topic. If you really want the FIFO guarantees, you can choose this, but you’ll also need to configure SQS for your subscription. Our demo will involve a simple email subscription. You can add complexity to your own integration as you see fit.
Once we’ve configured our SNS topic, we can add the ARN and the region code into our Papertrail configuration. Finally, we’ll save the alert.
Next, we’ll set up a subscriber for the SNS topic. For simplicity, we’ll set up our SNS subscription as an email alert.
Step Four: Test
We’ve connected all the pieces, and we’re about ready to test. Let’s briefly recap how all the pieces fit together:
- Our Express server listens on the / path but will emit an
ERROR-level log message for requests to any other path. - The server will write log messages to a file called
logger.log. - r
emote_syslog2streams the contents of our log file to Papertrail. - A custom query in Papertrail detects incoming
ERROR-level log messages. When it detects one, it will send an alert. - The alert we’ve configured sends the log message to an AWS SNS topic.
- Our subscriber for the SNS topic sends an email with the information from the log message.
With our server running, we simply need to visit a bad path to trigger an ERROR-level log message.
The error is logged. Papertrail detects it and sends an alert to SNS. Within moments, we receive an email like this:
We did it! We’ve successfully set up the basic integration between Papertrail and AWS SNS.
Conclusion
The Papertrail and AWS SNS integration we’ve demonstrated is simple and straightforward, but it unlocks a lot of potential. You can set up subscribers to SNS topics to perform a whole host of tasks, from provisioning resources through CloudFormation stacks and invoking Lambda.
