How to configure container log rotation for the Docker daemon

Rancher 2.x
RKE
Docker

Task

As the default setting on Docker is to log using the json-file log driver, without a container log limit, this can lead to disk-fill events on nodes. This article provides steps to configure any nodes running Docker to have a limited container log size and rotate out older container logs.

Pre-requisites

• Node(s) running Docker, using the json-file log driver
• Permission to edit the /etc/docker/daemon.json and to restart the Docker daemon

Warning

• You must restart the Docker daemon for the changes to take effect for newly created containers
• As the container logging configuration for pre-existing container is immutable, existing containers do not use the new logging configuration and will need to be redeployed to take on this new configuration.

Resolution

1. Edit the Docker daemon configuration file:

   $ vim /etc/docker/daemon.json

2. Add the following lines to the file, to configure a maximum container log file size of 10MB and maintain only 10 of these before deleting the oldest:

   {  
     "log-driver": "json-file",  
     "log-opts": {
       "max-size": "10m",
       "max-file": "10"  
     }
   }

3. Restart the docker daemon to apply the settings to new containers (see Warnings above):

   $ systemctl restart docker

Tips

You could include this Docker daemon container log rotation configuration in your build/connfiguration management systems, to ensure this is automatically applied to nodes on provisioning, removing any requirement for manual configuration.

Further reading

Docker JSON file log driver documentation