Make Your Logs Work for You

The days of logging in to servers and manually viewing log files are over. SolarWinds® Papertrail™ aggregates logs from applications, devices, and platforms to a central location.

View Technology Info

FEATURED TECHNOLOGY

Troubleshoot Fast and Enjoy It

SolarWinds® Papertrail™ provides cloud-based log management that seamlessly aggregates logs from applications, servers, network devices, services, platforms, and much more.

View Capabilities Info

FEATURED CAPABILITIES

Aggregate and Search Any Log

SolarWinds® Papertrail™ provides lightning-fast search, live tail, flexible system groups, team-wide access, and integration with popular communications platforms like PagerDuty and Slack to help you quickly track down customer problems, debug app requests, or troubleshoot slow database queries.

View Languages Info

FEATURED LANGUAGES

TBD - APM Integration Title

TBD - APM Integration Description

TBD Link

APM Integration Feature List

TBD - Built for Collaboration Title

TBD - Built for Collaboration Description

TBD Link

Built for Collaboration Feature List

START FREE TRIAL

Fully Functional for 14 Days

Lately, I’ve noticed developers are in a hurry to deliver the next big product idea. And most of the time, the product includes tasks such as user authentication, database connectivity, debugging, and performance monitoring. Frequently, developers have built these modules for other projects. But these modules weren’t architected to be reused, so the developers have to build them again. This means spending time repeating code, testing again, and debugging again. To address this issue, one framework in Python already has these types of modules and enables you to focus on your main task. Its name is Django, and in this article, I’ll show you how to start working with it. I’ll also go over its strong points and its weaknesses.

What Is Django?

Django is a web framework written in Python. Its main goal is to allow you to quickly deploy a working website or web application. To achieve this goal, the framework includes built-in modules enabling it to perform different tasks. Some examples of these include database connectivity, logging, authentication, and even HTML views. In addition, there are plenty of third-party modules to further extend its capabilities.

If you’d like to see an example of a high-visibility website built in Django a for thousands of visitors, look at the NASA website. For an example of a Django website built to serve API requests, look at Instagram or YouTube.

Working With Django

Creating an initial Django setup is easy, as most package managers already include a Django package. To install the package, you only need to invoke the package manager with the package name in the command line. If this isn’t the case for you, you can follow the instructions in the Django documentation, which will show you how to use the Python command line to perform the install.

If you don’t use containers, you should install a virtual environment tool for Python. In this way, you won’t mix Python libraries specific to your Django project and those required to perform unrelated functions or even testing.

Since Django uses Python for its development, being familiar with the language will make it easier to work with. You have all the tools available in Python to create complex projects. However, Django is not without some development concerns, which I’ll review in the next section.

Pitfalls of Working With Django

Django is a useful tool, but this doesn’t mean development will always flow smoothly. Next are some pitfalls to look at when using this framework in a project.

Production Hosting Environment

It doesn’t matter what environment you develop in; you can always configure it to suit your needs. The issue lies when the project is ready and you need to deploy it to production.

Any number of hosting companies exist. However, not all of them allow access to the console or access to add additional software. And even when they do, software installation is restricted to specific elements. This affects your use of Django—if you’re not able to install Python, or if it isn’t already available in your system, you simply can’t use Django. And  if you aren’t very familiar with system administration, configuring Django’s production environment will take additional time.

Django’s ORM

One of the features of working with Django is the ability to use a variety of database platforms. Accordingly, Django uses its own object relational mapping (ORM). As a result, you have great flexibility in the way you work with your preferred database system. You can even change your database manager without changing too much code. However, it’s difficult to use the ORM when you need to use a database-specific feature or to create a complex query.

Some reports need to handle multiple tables, conditions, grouping, paging, or database-specific commands. And yes, you can implement them in Django’s ORM, but if you know your database won’t change, direct SQL statements is a faster solution.

Debugging

Django is built on Python, so it’s easy to use Python’s default logging system. On the negative side, in this setup, logging is limited to a single machine. When a single developer is working on a feature, this isn’t an issue. But this changes when the issue is in production and multiple engineers are looking at different issues on the same system. Log analysis also provides information on performance, error trends, and metrics.

Front-End Development

Django is a web development framework, so it includes front-end and back-end development features. For the front end, Django uses views to provide application output. It also provides integration with front-end tools. Even so, front-end development is now as complex as back-end development, so it’s not a good idea to keep both in the same project.

Workarounds for Django

Though may find some issues when working with Django, it doesn’t mean there’s no solution for them. Here are some recommendations.

Deployment to Production

Let’s say you need to get a server where you have full control and administrative privileges. Server maintenance may be the price for a quick development process and deployment, but it’s not the only option. Python’s popularity means hosting companies can also include Python in the offerings, improving the production deployment process. Additionally, cloud providers already offer solutions designed to take the burden off server administration, and you just need to focus on the proper ways for scaling the application.

Working With Databases

The preferred method to work with databases is using Django’s ORM solution. But for special cases, this isn’t the best approach. Django also offers options to pass straight SQL queries and provides parameterization, which allows you to avoid SQL injection while still taking advantage of the ORM.

Logs and Debugging

The solution to review logs and allow debugging is to send them outside of Django’s environment. This avoids wasting your production system’s resources on tasks not directly related to the application. Analyzing logs is a time- and resource-consuming task, so it’s better to allocate specific resources for it. It’s better when you centralize log management.

By centralizing log management, you’ll develop a clearer view of your application’s resource consumption, thus defining metrics to guide your application development. You can focus on fixing the most critical issues and discover features providing the most value. In short, this lets you define your application’s future development.

Besides, you can also add all your infrastructure’s devices (firewalls, routers, databases, and other applications). Then, you can define metrics to fire alerts when necessary. All this information gives you the ability to make smarter decisions.

Keep in mind it’s up to you to configure log level verbosity. Here’s an example to configure logging in the setting.py configuration file:

import os

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/django/debug.log',
        },
    },
    'root': {
        'handlers': ['file'],
        'level': 'WARNING',
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            'propagate': False,
        },
    },
}

In the above code, Django will log the level specified in the environment variable DJANGO_LOG_LEVEL; otherwise, it defaults to INFO. DEBUG, WARNING, ERROR, and CRITICAL are additional log levels allowing you to see even more detailed information. Here, messages are passed to the file handler for later analysis. There are more examples in the documentation.

SolarWinds® Papertrail is a solution built to perform all the tasks I mentioned above. It easily integrates with Django’s log configuration, as shown here. Additionally, you can include many different log sources. You also don’t need to worry about managing the infrastructure for handling logs. Take the tour and try it for yourself.

No Front End

My experience tells me an application’s front end is complex, and there aren’t workarounds for this, so I suggest you just integrate what’s necessary. For example, generate all static files (images, JavaScript libraries, parsed CSS files, etc.) in an earlier step in your deployment process and then deploy the whole application.

Another option is to have a server for the front end and just use Django on the back end, with zero dependencies between them. Then, it’ll be easier to update components as required. In the end, it all depends on the application needs and architecture.

Final Words

Django is a useful tool to quickly develop websites and web applications. Still, it doesn’t mean that working with Django is hassle-free. You’ll encounter some issues related to production deployment, database management, logging, and back-end or front-end development. But there are ways to address these problems, making Django the framework of choice for several big companies. Feel like debugging something interesting? Try Papertrail!

This post was written by Juan Pablo Macias Gonzalez. Juan is a computer systems engineer with experience in back-end, front-end, database, and system administration.

Aggregate, organize, and manage your logs

  • Collect real-time log data from your applications, servers, cloud services, and more
  • Search log messages to analyze and troubleshoot incidents, identify trends, and set alerts
  • Create comprehensive per-user access control policies, automated backups, and archives of up to a year of historical data
Start Free Trial

Fully Functional for 30 Days

Let's talk it over

Contact our team, anytime.