Huge syslog file in production
We have a rails app with sidekiq in an Ubuntu 18.04 machine hosted in AWS.
We've recently started to run out of space in our disk. I've then noticed that we have a huge /var/log/syslog file, that reaches more than 10gb every day. When I run "tail -f" on it, I basically see non-stop queries running from ActiveRecord calls.
Is this expected? Is it normal for the syslog file to get this big because of the queries being recorded?
I'm not sure if all rails apps are like this or if I have to change some configuration option.
If you're running Puma with systemd or something, it will write to syslog by default. That will need to be logrotated more often so it doesn't fill the disk.
The logrotate location for syslog is
/etc/logrotate.d/rsyslog and you can add a line in there for setting a max file size.
maxsize 500M to rotate it after it becomes 500MB. You can also tweak how often the syslog is rotated as well.
You can also use lograge or a similar tweak to the Rails logs to minimize what they write.
Hi Chris, thanks for the reply.
That's exactly it. I'm running Puma with systemd. I believe setting logrotate to something like 500M solves the issue for me, thank you very much.
One final question though: I already use lograge in my application, but as far as I know it logs to $APP_PATH/log/production.log
I've read the lograge documentation but couldn't find it: how do I suppress the sql statements being written to /var/log/syslog?
Awesome. You can use the RAILS_LOG_TO_STDOUT env var to have Rails not write to the normal log file. You may also need to tweak the ActiveRecord logger if you need to turn it off.