Logrotate is a system utility that controls the automatic rotation and compression of log files. If files have not been rotated, compressed and periodically deleted during long-term server operation, they may sooner or later take up all available memory space.

    Logrotate is installed by default on Debian 9 and is configured to handle log rotation for all installed packages and applications.

    Checking the version of Logrotate:

    logrotate --version  
    

    The output of the command will be

    [email protected]:~# logrotate --version  
    logrotate 3.11.0  
    

    The default configuration of Logrotate is stored in two paths:

    • The file /etc/logrotate.conf stores some of the default configurations. It also contains archiving templates for non-system files.
    • File /etc/logrotate.d/ is designed to store third-party parameters that the administrator sets himself. Rotation templates for system utilities are stored here.

    Let's take a look at the Logrotate configuration file /etc/logrotate.d for the apt package manager:

    cat /etc/logrotate.d/apt  
    

    The output of the command will be

    [email protected]:~# cat /etc/logrotate.d/apt  
    /var/log/apt/term.log {
      rotate 12
      monthly
      compress
      missingok
      notifempty
    }
    
    /var/log/apt/history.log {
      rotate 12
      monthly
      compress
      missingok
      notifempty
    }
    

    This file contains configuration blocks for two different log files in /var/log/apt/: term.log and history.log. Both blocks have the same options. Any options not specified in these configuration blocks inherit the default values or values set in /etc/logrotate.conf

    Parameters set for apt logs:

    • rotate 12 - specifies that the utility saves the last twelve logs;
    • monthly - update once a month;
    • compress - archive command; by default it uses the standard Linux utility gzip; if you want to change it to something else, specify the keys after the command;
    • missingok - do not write an error message if there is no log file;
    • notifempty - do not rotate empty log file.

    Two options can be used to manage log files for applications:

    • Create a new Logrotate configuration file and place it in /etc/logrotate.d/. It will run daily as the root user along with all other standard LogRotate tasks.
    • Create a new configuration file and run it with the default LogRotate settings in Debian.

    Create configuration in /etc/logrotate.d/

    As an example, let's configure updates for a server that writes logs to access.log and error.log, located in the /var/log/app/ directory

    To add the configuration directory /etc/logrotate.d/, open a new file:

    nano /etc/logrotate.d/app  
    
    /var/log/example-app/*.log {
        daily
        missingok
        rotate 14
        compress
        notifempty
        create 0640 www-data www-data
        sharedscripts
        postrotate
            systemctl reload example-app
        endscript
    }
    

    Description of the directives:

    • create 0640 www-data www-data - this command will create a new empty log file after rotation with specified permissions (0640), owner ( www-data) and groups (www-data);
    • sharedscripts - this option means that any scripts added to the configuration will only run once per run after the files have been compressed, not for each individual updated file. Because our configuration will match two log files (access.log and error.log), the script specified in postrotate will only be run once;
    • postrotate to endscript - the script in this block will be run after the log file is updated. In the example, the application is rebooted

    After configuring it according to the requirements, you can run the test with the command:

    sudo logrotate /etc/logrotate.conf --debug  
    
    [email protected]:~# logrotate /etc/logrotate.conf --debug  
    reading config file /etc/logrotate.conf  
    including /etc/logrotate.d  
    reading config file app  
    error: app:13 lines must begin with a keyword or a filename (possibly in double quotes)  
    reading config file apt  
    reading config file dpkg  
    reading config file rsyslog  
    Reading state from file: /var/lib/logrotate/status  
    Allocating hash table for state file, size 64 entries  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    Creating new state  
    
    Handling 9 logs  
    
    rotating pattern: /var/log/example-app/*.log after 1 days (14 rotations)  
    empty log files are not rotated, old logs are removed  
    No logs found. Rotation not needed.  
    
    rotating pattern: /var/log/apt/term.log monthly (12 rotations)  
    empty log files are not rotated, old logs are removed  
    considering log /var/log/apt/term.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    
    rotating pattern: /var/log/apt/history.log monthly (12 rotations)  
    empty log files are not rotated, old logs are removed  
    considering log /var/log/apt/history.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    
    rotating pattern: /var/log/dpkg.log monthly (12 rotations)  
    empty log files are not rotated, old logs are removed  
    considering log /var/log/dpkg.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    
    rotating pattern: /var/log/alternatives.log monthly (12 rotations)  
    empty log files are not rotated, old logs are removed  
    considering log /var/log/alternatives.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    
    rotating pattern: /var/log/syslog  
     after 1 days (7 rotations)
    empty log files are not rotated, old logs are removed  
    considering log /var/log/syslog  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not a day ago yet)
    
    rotating pattern: /var/log/mail.info  
    /var/log/mail.warn
    /var/log/mail.err
    /var/log/mail.log
    /var/log/daemon.log
    /var/log/kern.log
    /var/log/auth.log
    /var/log/user.log
    /var/log/lpr.log
    /var/log/cron.log
    /var/log/debug
    /var/log/messages
     weekly (4 rotations)
    empty log files are not rotated, old logs are removed  
    considering log /var/log/mail.info  
      log /var/log/mail.info does not exist -- skipping
    considering log /var/log/mail.warn  
      log /var/log/mail.warn does not exist -- skipping
    considering log /var/log/mail.err  
      log /var/log/mail.err does not exist -- skipping
    considering log /var/log/mail.log  
      log /var/log/mail.log does not exist -- skipping
    considering log /var/log/daemon.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not a week ago yet)
    considering log /var/log/kern.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not week ago yet)
    considering log /var/log/auth.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not week ago yet)
    considering log /var/log/user.log  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not week ago yet)
    considering log /var/log/lpr.log  
      log /var/log/lpr.log does not exist -- skipping
    considering log /var/log/cron.log  
      log /var/log/cron.log does not exist -- skipping
    considering log /var/log/debug  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not a week ago yet)
    considering log /var/log/messages  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not week ago yet)
    not running postrotate script, since no logs were rotated  
    
    rotating pattern: /var/log/wtmp monthly (1 rotations)  
    empty log files are rotated, old logs are removed  
    considering log /var/log/wtmp  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    
    rotating pattern: /var/log/btmp monthly (1 rotations)  
    empty log files are rotated, old logs are removed  
    considering log /var/log/btmp  
      Now: 2020-05-21 16:52
      Last rotated at 2020-05-21 06:00
      log does not need rotating (log has been rotated at 2020-5-21 6:0, that is not months ago yet)
    

    As a result the logrotate utility will be called, pointing to the standard configuration file in debug mode.
    The console will display information about which files logrotate is processing at the moment. The standard Logrotate configuration will be performed once a day, including new configuration.

    Creating a LogRotate configuration

    In this example, we have an application running under user testing, generating logs that are stored in the /home/testing/logs/ directory. We need to make these logs rotate on an hourly basis, so we need to set it up outside the /etc/logrotate.d structure presented in Debian.

    Let's create a configuration file in our directory with a text editor

    nano /home/testing/logrotate.conf  
    

    Then insert the following configuration:

    /home/testing/logrotate.conf
    /home/testing/logs/*.log {
        hourly
        missingok
        rotate 24
        compress
        create
    }
    

    file

    Save and close the file

    This configuration will rotate files hourly, compressing and saving the twenty-four old logs and creating a new log file to replace the rotated one.

    It is necessary to customize the configuration to suit your application.

    Let's make a log file to check that it works:

    cd ~  
    
    mkdir logs  
    
    touch logs/access.log  
    

    Since the logs belong to testing we don't need to use sudo. However, we do need to specify a status file. This file records what logrotate saw and did last time, so it knows what to do the next time it runs

    We will ask Logrotate to put the status file directly in our home directory for this example. We can specify anywhere that is available and convenient:

    logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force  
    

    Output

    reading config file /home/testing/logrotate.conf  
    
    Handling 1 logs  
    
    rotating pattern: /home/testing/logs/*.log hourly (24 rotations)  
    empty log files are rotated, old logs are removed  
    considering log /home/testing/logs/access.log  
      log does not need rotating
    
    

    -verbose will print detailed information about what Logrotate is doing. This is the first time LogRotate has seen this log file, so as far as we know, the file is zero hours old and should not be rotated.

    If we look at the status file, we can see that Logrotate has recorded startup information:

    cat /home/testing/logrotate-state  
    

    The output is

    [email protected]:~# cat /home/testing/logrotate-state  
    logrotate state -- version 2  
    
    

    Logrotate noted that it saw the logs, and when it last looked at them rotate. If you run the same command, one hour later, the log is rotated as expected.

    If you want to force LogRotate to rotate the log file, then you must use the -force flag:

    logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state --verbose --force  
    

    Next, you need to configure a cron job to run Logrotate every hour. Open a user's crontab:

    crontab -e  
    

    This opens a text file. The file probably already has some comments explaining the expected basic syntax
    Move the cursor to a new blank line at the end of the file and add the following:

    14 * * * * * /usr/sbin/logrotate /home/testing/logrotate.conf --state /home/testing/logrotate-state  
    

    file

    This task will run on the 14th minute of every hour, every day.