Reducing apache log files size

When a domain have a lot of traffic, its access log files at /var/sentora/logs/domains/xxxx-access is growing fast enough to take large place.

An easy way to reduce it is to tell apache to not log the most used request which are known as fully safe and which have not some interest for you

 

1) To enable this you will have first to do a small config change :

- open phpmyadmin, goto sentora_core database, then open x_settings table.

- look for line with parameter named "access_log_format" (usually line 85).

- Its value is "combined". Add at its end  " env=!nolog" to make it becoming :

combined env=!nolog

 

2) You can now add some filter in apache config for virtual domains concerned.
In sentora panel, goto menu Admin -> Module admin -> Apache Config. 
For each the domain which you want to filter some logs :

- open it to override parameters (at the bottom of the page),

- in the override box, add lines to set "nolog" when requests must not be logged.

They are looking like

SetEnvIf [parameter] [REGEXP condition] nolog

Refers at apache documentation for SetEnvIf for exact syntax.

Exemple :

SetEnvIf Request_URI "^/(Images/|Img).+$" nolog
will disable logging request for any file in directory "/Images/" or with path starting by "/Imgxxxxx"

 

3) Once you have setup all your filters, force the rewrite of vhost file

.. by checking the check box named "Force Update" in apache config page, just above the override tools.

Then wait until the next daemon run to rewrite the vhost file with new settings and reload apache.

 

WARNINGS:

1) Be sure that requests you will filter out are fully safe : you will be no longer able to analyse a filtered request if it was used to hack your server.
-> Usually you can filter out requests returning static pages, like robot.txt file, images files, etc.
-> Avoid to filter php pages that handle parameters, except if you know very well what you do.

2) Ensure the filter you write is working as you expect. An error can screw-up the whole vhost file, or may discard all logs for the whole domain.
-> Take time to monitor the log file (ie with tail -f [logfile]) while you are doing some request to be filtered and some to be not filtered.

Advise:

To help you to setup a correct filter, you can first to add it (same line) in the .htaccess file placed at the root of the domain, while you monitor the log file.
When the filter is ready and thoroughly tested, apply it in the override box for the domain.

Note : you might think that you can leave it working in .htaccess file, but remember that the vhosts file is loaded, parsed and "compiled" only once when apache is (re) loaded, while each .htaccess file is loaded and parsed on each request, inducing a lot of more overhead.

 

PenWant to help ? Click here to report mistake or to send complement to add.