Managing Logs with Logrotate

When working with virtual servers or dedicated servers, it is important to track events and errors for system monitoring and diagnostics.

However, large log files create several challenges. They are difficult to view and process, consume significant CPU and memory resources, and when the disk is full, they can cause application failures.

Log rotation helps solve these problems by automatically archiving current entries and creating new files for incoming data.

In Linux, the Logrotate utility is used to manage log rotation. In this article, we will cover its operation principles and configuration for specific requirements.

---

Logrotate is usually pre-installed in most Linux distributions and is automatically run via the cron scheduler. To check the utility and its version, run the command:

logrotate --version

The output will display version information and basic settings:

logrotate 3.21.0
    Default mail command: /usr/bin/mail
    Default compress command: /bin/gzip
    Default uncompress command: /bin/gunzip
    Default compress extension: .gz
    Default state file path: /var/lib/logrotate/status
    ACL support: yes
    SELinux support: yes

The output indicates which tools Logrotate uses for log compression and notifications, as well as the path to the state file, which stores the rotation history.

---

The Logrotate configuration has a hierarchical structure. Global default parameters are set in the main configuration file /etc/logrotate.conf, while configurations for individual applications are located in the /etc/logrotate.d/ directory.

Example content of /etc/logrotate.conf for Ubuntu:

/etc/logrotate.conf
# rotate log files weekly
weekly

# use the adm group by default, since this is the owning group
# of /var/log/.
su root adm

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
#dateext

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

According to this configuration:

• Log rotation occurs weekly (weekly).
• New empty log files are created after rotation (create).
• Archived copies of logs are kept for the last 4 weeks (rotate 4).
• The su root adm directive ensures rotation runs as user root and group adm, preventing access issues with system logs.
• Compression (compress) is disabled by default but can be enabled to save disk space.
• The include /etc/logrotate.d line tells Logrotate to load all configuration files from the /etc/logrotate.d directory. This allows a modular approach where packages define their own log rotation rules without modifying global settings.

For the full list of directives, see the manual:

man logrotate

---

The global configuration file only sets default directives that apply to all logs defined in included configurations (for example, in /etc/logrotate.d).

When viewing the contents of this directory, you can find configuration files for various installed services.

The structure of files for individual applications is similar to the global configuration but applies only to specific logs:

/etc/logrotate.d/nginx
/var/log/nginx/*.log {
    # These directives apply only to Nginx logs
    daily
    missingok
    rotate 14
    compress
    notifempty
    # other Nginx-specific settings
}

Parameters inside these files override global values from /etc/logrotate.conf. If a directive is not specified in an application’s configuration, the value from the global file is used.

---

By default, Logrotate uses a time-based approach, where log files are rotated on a fixed schedule. The following intervals are supported:

• hourly
• daily
• weekly
• monthly
• yearly

When you specify one of these intervals, Logrotate checks whether the specified time has passed since the last rotation. If yes — the log is rotated.

---

Logrotate is often configured system-wide to run once a day via /etc/cron.daily/logrotate. If you use the hourly option for a specific log file, you need to ensure Logrotate runs more frequently. To do this, move the corresponding cron job to the hourly path:

sudo mv /etc/cron.daily/logrotate /etc/cron.hourly  # On Ubuntu

If Logrotate needs to run even more frequently, a custom cron job may be required.

---

For services that generate unpredictable log volumes, a purely time-based rotation schedule can lead to very large log files or many small files during low activity periods.

In this case, you can use size-based rotation. The size directive tells Logrotate to rotate a log file only when it reaches a certain size, regardless of the time schedule. For example:

/etc/logrotate.d/app
/var/log/app/*.log {
  daily    # now ignored
  size 50M # the last option takes effect
}

This configuration ensures that the specified log file rotates only when it exceeds 50 MB, even if the daily interval has already passed. Sizes can be specified in bytes (default), kilobytes (k), megabytes (M), or gigabytes (G).

---

For more flexible control, Logrotate provides minsize and maxsize directives, which work together with time-based schedules:

• minsize — rotation occurs only if the file reaches the minimum size while respecting the time interval:

/etc/logrotate.d/app
/var/log/app/*.log {
  daily
  minsize 50M
}

Here the log rotates daily, but only if its size is at least 50 MB. If the file is smaller, rotation will occur later when the condition is met.

• maxsize — rotation occurs immediately when the file exceeds the specified size, even if the scheduled time has not yet arrived. Rotation also occurs at the scheduled time if the maxsize limit has not been exceeded:

/etc/logrotate.d/app
/var/log/app/*.log {
  daily
  maxsize 50M
}

These options prevent too frequent rotation of small files (minsize) and prevent large files from growing too much before the next scheduled rotation (maxsize).

---

Logrotate supports two main methods for handling active log files: create and copytruncate.

/var/log/app/*.log {
  create
  # You can also specify permissions, owner, and group:
  # create 644 <appuser> <group>
}

• create (default method) renames the active file, e.g., app.log → app.log.1, and creates a new empty file with the original name. The new file inherits attributes from the old file, but they can be specified explicitly. This method is preferred as it minimizes the risk of log loss if the application can immediately write to the new file.

• copytruncate copies the content of the current file to the rotated file and then truncates the original file:

/etc/logrotate.d/app
/var/log/app/*.log {
  copytruncate
}

This method allows applications to continue writing to the same file descriptor.

Logrotate also supports the copy directive, which copies the file without truncating the original — useful for specific scenarios but does not replace create or copytruncate for continuous logging.

---

By default, rotated files receive sequential numbers, e.g., app.log.1, app.log.2, etc. This preserves order but does not indicate creation date.

The dateext and dateformat directives allow adding timestamps:

/etc/logrotate.d/app
/var/log/app/*.log {
    dateext
    dateformat # default -%Y%m%d for daily, -%Y%m%d%H for hourly
}

If a log rotates multiple times a day (size-based or hourly), you can use a Unix timestamp for uniqueness:

/var/log/app/*.log {
    dateformat -%Y%m%d-%s  # produces app.log-20250625-1748181385
}

The dateyesterday directive sets the date to the previous day instead of the current day — useful if processing occurs after midnight but the data belongs to the previous day.

---

Compression saves disk space. The compress directive uses gzip:

/etc/logrotate.d/app
/var/log/app/*.log {
    compress
}

Other compressors can be used with compresscmd, uncompresscmd, and compressext:

/etc/logrotate.d/app
/var/log/app/*.log {
    compress
    compresscmd /usr/bin/bzip2
    uncompresscmd /usr/bin/bunzip2
    compressext .bz2
}

The delaycompress directive prevents immediate compression of the last rotated file, useful if the program continues writing to the old file after rotation.

/etc/logrotate.d/app
/var/log/app/*.log {
    compress
    delaycompress
}

compressoptions allows setting the gzip compression level from -1 (fast, low compression) to -9 (slow, maximum compression):

/etc/logrotate.d/app
/var/log/app/*.log {
    compress
    compressoptions -9
}

---

If rotation does not work, check:

• permissions
• configuration errors
• time conflicts

The Logrotate state file (/var/lib/logrotate/status) contains information about the last rotation:

sudo cat /var/lib/logrotate/status

For a more detailed analysis, use debug mode:

sudo logrotate --debug /etc/logrotate.conf