Most services on Linux will generate log files, usually stored in /var/log/. These are just simple text files, but they can get really, really large—to the point where you could run your server out of space.
What Does Logrotate Do?
Logrotate helps manage log files, and prevents them from piling up. It will automatically archive the current log file, make a new empty one, and delete the really old archives after it has rotated a few times. The default time might be set really high (nginx defaults to one year of logs), so you may run into issues if you don’t edit your config files.
The size of your files will vary by service, and you’ll want to configure logrotate accordingly. If you want to find how much space each service is taking up with logs, you can use the du command:
Nearly all log files will be in /var/log/, so du will total up all the log files, and display them in a nice list (courtesy of sort -hr for ‘human-readable’):
This only shows directories, so you’ll need to use ls -sh to find the sizes of individual files.
Usually, things like web servers (apache2, nginx, etc.) and other apps that receive a lot of interaction will generate a lot of log files, thought the exact size will depend on how much data they’re actually logging to disk.
Installing Logrotate (If You Don’t Have It Already)
By default, you probably won’t have to do anything to setup logrotate, as it’s installed by default on many distros, and services that generate a lot of log files (nginx, apache, etc) will usually have preconfigured logrotate config.
You can check if logrotate is installed with:
Or install it from your distro’s package manager if it isn’t. For Debian-based systems like Ubuntu, that would be:
Logrotate will automatically install its default global config, but if you’ve installed it after installing your web server, you may have to reinstall your web server if the default config for that service isn’t generated.
How Logrotate Works
While logrotate will be preconfigured for popular services like nginx and apache2, you may want to set up configuration manually, or edit the default config.
The global config for logrotate is stored in /etc/logrotate.conf, but you’ll override much of it with per-app config files, stored in the /etc/logrotate.d/ directory. Let’s take a look at nginx’s default configuration in /etc/logrotate.d/nginx, which you can use as a template for adding new apps to logrotate:
The first line starts the block, and tells logrotate which files to look for. The wildcard *.log will match any files ending with the .log extension in the nginx log folder. You can also specify multiple directories here to include them with the same configuration.
weekly will rotate logs once a week. If you’d like to rotate them based on size, you can instead use size 25M to rotate them once they reach a certain limit, 25 megabytes in this example.
missingok will prevent logrotate from throwing errors if a log file gets deleted by anything other than logrotate.
rotate 52 will keep 52 different log files. In this case, since it will rotate once a week, logrotate will keep an entire years worth of log files before deleting old ones. You could change this to rotate 4 to only keep a month’s worth of log files.
compress will compress old log files to save space. This causes issues with some apps continuing to write to the log file while logrotate is still doing its thing, so you can add the delaycompress flag to leave a buffer of one old log file before compressing.
notifempty is another check flag to make sure logrotate is only running if new log files are being written to. The create flag makes these new log files, with specific permissions. The syntax is create [mode] [owner] [group].
You’ll want to make sure these match whatever your current log files are configured as, which you can do with ls -la:
Note that you’ll need to convert the mode to octal, so -rw-r—– becomes 0640. The owner for nginx here is www-data, and the group is adm.
sharedscripts specifies that the prerotate and postrotate hooks will only be ran once per rotation, and not once for each log file rotated. These hooks are scripts you can call before and after rotation. By default, these will run a few scripts to configure nginx for having log files switched. If the service you’re configuring can’t hotload new log files, you’ll want to stop and restart it in these hooks. The hook doesn’t have to interact with the service though; for example, you could use the prerotate hook to back up your log files to AWS S3 with s3cmd before they get deleted.
Logrotate will manage running itself, so you won’t need to worry about starting the service on boot or configuring a cron job. Just place config files for each app in /etc/logrotate.d/, and never worry about logs again.
