Make Your Logs Work for You

The days of logging in to servers and manually viewing log files are over. SolarWinds® Papertrail™ aggregates logs from applications, devices, and platforms to a central location.

View Technology Info

FEATURED TECHNOLOGY

Troubleshoot Fast and Enjoy It

SolarWinds® Papertrail™ provides cloud-based log management that seamlessly aggregates logs from applications, servers, network devices, services, platforms, and much more.

View Capabilities Info

FEATURED CAPABILITIES

Aggregate and Search Any Log

SolarWinds® Papertrail™ provides lightning-fast search, live tail, flexible system groups, team-wide access, and integration with popular communications platforms like PagerDuty and Slack to help you quickly track down customer problems, debug app requests, or troubleshoot slow database queries.

View Languages Info

FEATURED LANGUAGES

TBD - APM Integration Title

TBD - APM Integration Description

TBD Link

APM Integration Feature List

TBD - Built for Collaboration Title

TBD - Built for Collaboration Description

TBD Link

Built for Collaboration Feature List

Tips from the Team

How to Live Tail Docker Logs

START FREE TRIAL

Fully Functional for 14 Days

Last updated: December 15, 2023

Docker has improved how we develop, package, and deploy applications. It allows us to encapsulate logic and dependencies in a container ready to run on any platform. However, each container produces logs, which can be challenging to manage. Logs give a record of events happening in a system and are very useful when debugging. Thus, we need ways to access the container logs. Docker logs tail is a command that allows us to follow specific log outputs of our container. 

In this article, we’ll talk a lot about Docker logs. You’ll learn how to display Docker logs in your console. Also, you’ll learn tips and tricks related to displaying logs. Lastly, this article will discuss how to configure Docker to send logs to SolarWinds® Papertrail™. 

Let’s learn first about the importance of Docker logs. 

What Are Docker Logs? 

Docker logs are any output data the container writes to STDOUT (standard output) or STDERR (standard error). These logs are helpful for debugging and tracing issues in your application running in a container. However, if you run a container in detached mode, you can’t see the logs in your console. This is because detached mode refers to running a container in the background of your terminal. Therefore, you won’t see any logging or other output from your Docker container. To display these logs, you can use the command docker logs with different options like –tail

Docker logs are important for developers and DevOps engineers. Logs are most useful when debugging or analyzing a problem because they offer insights into what went wrong. Thus, developers and engineers can solve issues faster. 

In addition, you can apply trend analysis to Docker logs to detect anomalies. For example, SolarWinds® Observability offers this feature for any set of data, including logs. This lets you detect anomalies faster and shift from reactive to proactive monitoring. 

The next section goes into detail on how to open a stream of Docker logs. 

How to Tail Docker Logs? 

Imagine we’re running a container and want to access the logs for this container. How can we accomplish this task? 
First, we can use the following command to check for the currently running containers: 

docker ps -a 

This command prints a list of the running containers. You’ll see information about the containers, like their IDs, used images, start commands, how long the containers have been running, and many other variables. 
For us, the most important parameter is the container ID, which we will use in the next step. 

CONTAINER ID    IMAGE                COMMAND                 CREATED      STATUS 

fcae22fa4144    xyz/my-image:latest  "/opt/bin/entrypoint-"  2 hours ago  Up 2 minutes 

155206af2f03    abc/my-image:latest  "/entrypoint.sh mysql"  2 hours ago  Up 2 minutes 

Now that we’re sure our container is running, let’s use the container ID to see all of its logs. 
Enter the following command in your command-line interface (CLI), replacing <container ID> with your container’s ID: 

docker logs <container ID>

Although this will show us the logs, it won’t allow us to view continuous log output. In Docker jargon, we refer to creating a continuous stream of log output as tailing logs. To tail the logs for our container, we can use the follow option. 

docker logs --follow <container ID>

Where Are Docker Logs Stored by Default? 

Docker logs are stored in a JSON file on the Docker host by default. The default log file directory for Docker is /var/lib/docker/containers/ on the host machine. Each container’s logs are stored in a separate directory under this location. The naming format of the log files is container_id-json.log, where container_id is a unique identifier for each container. Below is an example file path, but you’ll replace the container id with yours. 

/var/lib/docker/containers/<container_id>/<container_id>-json.log 

Next, let’s explore more exciting tricks related to displaying Docker logs. 

Other Docker Logging Tricks

Here are three useful logging tricks you can access through your CLI. 

#1: Display Only the Latest Lines 

In some cases, you don’t want to see all logs for a container. Perhaps something happened and you want to quickly verify the latest 100 logs for your container. In this case, you can use the tail option to specify the number of logs you want to see: 

docker logs --tail 100 <container ID>

#2: Stream Logs Until a Specific Point in Time

Docker provides the option to only stream logs from a specific period of time. For example, logs written during the first three seconds when the container was active can tell you if the container started successfully. In this case, you don’t have to create a never-ending stream of logs. Here, you can use the until option in combination with the follow option. The until option allows you to specify a time span for which the container should print logs to your CLI. 

docker logs --follow --until=3s

Note that you can use different notations to designate the timescale. For example, to see the logs for the last 30 minutes, use the following command: 

docker logs --follow --until=30m

#3: Stream Logs From a Specific Point in Time

The opposite action is also possible with Docker CLI options. Let’s say you want to see the logs from a specific point in time up until now. The since option helps with this task. 

docker logs --since 2023-12-02 <container ID>

Note that the accepted format here is YYYY-MM-DDTHH:MM:SS. This means you can specify a very accurate timestamp from which you want to display logs, or a less specific timestamp, as shown in the example above. 

Alternatives to Docker Logs Tail 

Using the docker logs tail command to track your logs isn’t the only way you can track your logs. There are other ways and other docker commands you can use. They include: 

  1. Directly accessing the log files: by knowing where logs are stored by default, you navigate to the location. For example, this is how you can use the tail command to view the logs of your container. 
     
    tail -f /var/lib/docker/containers/<container_id>/<container_id>-json.log 
  1. Using Docker desktop GUI: in the docker desktop, you navigate to the containers section, select your container, and then click on the logs tab to view and tail the logs. 
  1. Using docker-compose: if you’re managing multiple containers, you can use docker-compose -f <service_name> to view and tail logs. 
  1. Use logging drivers: docker supports various logging drivers like Syslog, journald, or fluentd that you can configure to tail and view logs. 
  1. Use third-party logging tools: you can leverage tools like Logspout, Logstash, or Papertrail to collect, tail, and analyze logs from your containers. 

How to Log Docker Logs to Papertrail

In this section, we’ll give you a simple example of how you can configure Docker to send logs to Papertrail
First of all, you can run a logspout container, which allows you to configure an address to send logs to. The below example starts a logspout container configured to send logs to Papertrail. 

docker run --restart=always -d \
    -v=/var/run/docker.sock:/var/run/docker.sock gliderlabs/logspout  \
    syslog+tls://logsN.papertrailapp.com:XXXXX

Alternatively, you can configure the syslog driver to tell the container where to send logs. Make sure to change the syslog-address property to your custom Papertrail address. 

docker run --log-driver=syslog
    --log-opt syslog-address=udp://logsN.papertrailapp.com:XXXXX 
    image-name

You can find more information in Papertrail tutorial
Next, let’s dive into best practices for logging in Docker. 

Frustration-free log management. (It’s a thing.)

Aggregate, organize, and manage your logs with Papertrail

4 Best Practices When Logging in Docker

Here’s a shortlist of four best practices related to logging in Docker containers, which you can apply to your organization. 

Export Logs to Persistent Storage 

You can easily create and destroy containers. However, every time a container restarts, you lose all the data it holds. Therefore, never store application-specific data in your container. 
For the same reason, you should take good care of your logs. Logs can be stored persistently to a volume, but it’s even better to store them long-term. For example, you can pipe logs to a local hard drive or you can send them to a log management platform. Both options allow you to save your logs long-term and use them for future analysis. 

Consider Using a Logging Container 

A logging container helps you scale your logging. The idea is that you pipe your logging output from multiple containers to a logging container. Next, your logging container takes care of saving the logs to persistent storage or a log management platform. 

Also, you can spin up multiple logging containers to scale your logging service when you decide to host more containers. It’s a flexible and easy solution for handling log output. 

Log Data to Standard Text: Output Channels 

To be able to aggregate logs from your containers, you need to make sure the applications running in those containers log data to STDOUT or STDERR. Those are both standard channels for logging output messages or error messages. Docker is configured to automatically pick up data from both outputs. If you log data to a file inside your container, you risk losing all this data when the container crashes or restarts. Therefore, if you don’t want to lose any important logging data, it’s important to log to STDOUT or STDERR. 

Log Data as JSON Format 

Docker supports the JSON logging format, and logging data in this format is recommended. You might wonder why. The reason is that Docker itself stores logs as JSON files; therefore, it’s optimized to handle JSON data. 

For this reason, many Node.js logging libraries, such as Bunyan or Winston, prefer to log data using the JSON format. 

Now, let’s summarize what we’ve learned from this article. 

Final Words on Docker Logs 

As you’ve seen, Docker provides multiple CLI options to display logs in different formats. For example, you can display logs for a specified point in time. In addition, the follow option is one of the most used Docker options because it allows developers to live tail logs for specific containers. 

Lastly, you can configure Docker to transport logs to a log management platform, such as Papertrail. This also helps to visualize your logs. Papertrail allows you to easily monitor logs and provides developers with the ability to create alarms to warn them in case an anomaly is detected. 

Want to give it a try? You can sign up for a free trial of Papertrail now and see just how it can work for you. 


This post was written by Michiel Mulders. Michiel is a passionate blockchain developer who loves writing technical content. Besides that, he loves learning about marketing, UX psychology, and entrepreneurship. When he’s not writing, he’s probably enjoying a Belgian beer!

Aggregate, organize, and manage your logs

  • Collect real-time log data from your applications, servers, cloud services, and more
  • Search log messages to analyze and troubleshoot incidents, identify trends, and set alerts
  • Create comprehensive per-user access control policies, automated backups, and archives of up to a year of historical data
Start Free Trial

Fully Functional for 30 Days

Let's talk it over

Contact our team, anytime.