Motivation

Managing log files in production environments can be challenging – files can grow too large, consume too much disk space, or become difficult to manage over time.

Standard logging requires multiple imports and complex configurations for basic log rotation. The RotatingFileHandler has limited flexibility, rotating logs based on file size and number of backups only.

import logging
from logging.handlers import RotatingFileHandler

# Complex setup for rotation
logger = logging.getLogger(__name__)
handler = RotatingFileHandler('app.log', maxBytes=1024*1024, backupCount=5)
logger.addHandler(handler)

For time-based rotation, compression, or specific log retention, you need custom code, making logging setup more complex and harder to maintain.

Introduction to Loguru

Loguru is a Python logging library that simplifies logging configuration and provides powerful features out of the box. Install it using:

pip install loguru

As covered in Simplify Python Logging with Loguru and 3 Tools to Track Python Code, Loguru provides basic logging and improved error tracking. In this post, we’ll cover its file logging capabilities with rotation, retention, and compression.

File Logging Features

Loguru makes file logging management simple with built-in support for rotation, retention, and compression:

Basic file logging with rotation based on size:

from loguru import logger

# Rotate when file reaches 500 MB
logger.add("app.log", rotation="500 MB")

for i in range(1000000):
    logger.info(f"Logging message {i}")

The rotation="500 MB" parameter configures log rotation. When the log file reaches 500MB, Loguru will:

• Close the current log file
• Rename it with a timestamp or incremental number
• Create a new empty log file with the original name to continue logging

Time-based rotation and retention:

# Create new file every day and keep logs for a week
logger.add("app.log", rotation="12:00", retention="1 week")

In this code:

• rotation="12:00": Rotate the log file daily at 12:00 PM (noon). The current log file is closed, renamed with a timestamp, and a new empty log file is created.
• retention="1 week": Automatically delete log files older than 1 week to prevent accumulation and save disk space.

Automatic compression of rotated files:

# Compress rotated files automatically
logger.add("app.log", rotation="100 MB", compression="zip")

compression="zip" automatically compresses rotated log files into ZIP archives. The active log file remains uncompressed.

Output:

app.log           # Current log file
app.2024-02-15_12-00-00.log.zip  # Compressed rotated log
app.2024-02-14_12-00-00.log.zip  # Previous day's compressed log

Conclusion

Loguru’s file logging features simplify log file management by providing built-in support for rotation, retention, and compression. This eliminates the need for custom code and makes it easier to maintain logs in production environments.

Link to Loguru