How to Configure Apache Logging for Better Debugging & Analysis
How to Configure Apache Logging for Better Debugging & Analysis. In this post we introduce Apache, types of Apache logging and after that how to configure Apache logging for better debugging and analysis.
Let’s start the article How to Configure Apache Logging for Better Debugging & Analysis
What is Apache?
Apache, also known as the Apache HTTP server. A free, open source and cross platform web server. Importantly, Apache is one of the most popular and widely used web servers, because of its unique features like security, flexibility, performance, and efficiency.
Types of Apache Logging
To record the events handled by the Apache server, we use logging. Hence, Apache logs contain all the actions of the Apache server, like requests coming from clients, responses sent by the web server, and actions taken inside the Apache server. It also stores information about the error occurring within the process, the time taken to process the request by the server, resource mapping process, etc. We improve debugging and analysis by configuring the Apache logs. Generally, Apache logs are of two types: access logs and error logs.
1. Access Logs: The access logs record information about the requests coming from the clients to the web server. This information contains the success rate of incoming requests, the time taken by the web server to respond to each request, time taken by the Apache server to process the request, etc.
2. Error Logs: As the name suggests, error logs record information about errors encountered by the Apache server while processing a client’s request, like missing a file, sending a scrappy request, etc.
Apache Logging: Configure Apache Logging for Better Debugging & Analysis
In this section, we explain different types of Apache log formats and how to implement it in Apache web server.
Prerequisites
- A server running Linux operating system.
- A root user or a user with sudo privileges.
Install Apache Web Server
apt install apache2 -y
After the successful installation, start and enable the Apache service with the following command.
systemctl start apache2
systemctl enable apache2
dnf install httpd -y
Once the Apache package is installed, start and enable the Apache service with the following command.
systemctl start httpd
systemctl enable httpd
Find the Apache Log Files
Access and error log files are stored by default on the default log directory. The location of Apache logs depends on the operating system in use.
In the CentOS, RHEL and Fedora Linux operating system, the Apache access and error logs are located at /var/log/httpd/access.log and /var/log/httpd/error.log.
For Debian and Ubuntu operating system, the Apache access and error log files are located at /var/log/apache2/access.log and /var/log/apache2/error.log.
A typical access logs looks like,
12.180.48.071 - - [30/May/2023:19:12:06 +0000] 808840 "GET /incentiveService/incentive/profit?userId=20253434839371 HTTP/1.1" 500 17 "-" "Apache-HttpClient/4.2.6 (java 1.5)"
And, a typical error logs looks like:
[Wed May 30 19:04:13 2023] [error] [client 50.1.104.025] File does not exist: /var/www/linuxhelp.io
The location of access and error logs are configured using the CustomLog directive, like
CustomLog “/var/log/httpd/access.log”
Or
CustomLog “/var/log/apache2/access.log”
View the Apache Log Files
tail -f /var/log/apache2/access.log
See the access log in the following screen.
If you want to see the entire log file, use the cat command as shown below:
cat /var/log/apache2/access.log
To view the Apache error log, run the following command.
tail -f /var/log/apache2/error.log
This shows Apache error log in the following screen.
Understand and Configure Apache Logging Behaviour
The Apache logging framework is highly configurable and lets you adjust logging behaviour both globally and for each virtual host. To change the Apache logging behaviour, there are several directives. Commonly used directives are log level and log format directives, which help in configuring the logs for better analysis. Let’s discuss some ways to configure Apache logs for better debugging.
LogLevel Directive
When events are logged to a specific destination, the log level directive decides the severity level of these logs. The term severity shows how crucial the event log is and ranges from “Emerg” to “Trace8” level. As we move from “Emerg” to “Trace8”, the importance and stability of event logs increase.
These are the list of Apache log levels that you use with your Apache server.
- trace1 – trace8: used to trace messages.
- debug: debugging purposes.
- info: produces informational messages.
- notice: normal but significant conditions.
- warn: produces warnings.
- error: shows you error conditions.
- crit: produces critical messages that requires prompt action.
- alert: displays errors that require immediate action.
- emerg: system is unusable.
LogFormat Directive
The layout and formatting of event logs is done by the LogFormat directive. By default, Apache uses Common Log Format (CLF), but you customise the format string to update fields present in each log. Generally, the Custom log directive helps in changing the location of log files. In the Linux operating system, Apache logs are found either in the /var/log/apache2 or /var/log/httpd directory.
After filename, we also define a LogFormat string that applies a format string to the file. Below is the configuration that uses a common format to write logs in the logs/access_log directory:
LogFormat “%h %l %u %t \ %r \ ” %>s %b“ common
CustomLog “logs/access_log” common
The description of elements present in the above log format is as follows:
- %h -represents the remote host IP address that made the request to Apache web server.
- %l – represents the value that we are expecting to get logged in case the logging directory is not accessible.
- %u – represents the user identifier derived from HTTP authentication.
- %t – represents the date and time when a request is made within the proper time zone.
- “\ %r \” -represents the first line of the request within double quotes.
- %>s – represents the status code received by the client. It plays an important role in deciding whether a request is a success or a failure.
- %b -shows the size of the response object sent to the client.
Nicknames Assigning
Nicknames are assigned to the LogFormat string that is used with the CustomLog directive to build certain types of logs. By doing this, we reuse the same log format for other files too, thus reducing the effort to redefine the format every time. It is crucial when we are using different log files for multiple hosts.
Let’s make a sample log format and name it as “host_format”. Then create a CustomLog directive designed to note the logs using host_format.
LogFormat "%v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" host_format
CustomLog /var/log/apache2/host.log host_format
JSON Formatting
When logs are stored in text format, we read them easily. But the situation becomes difficult when we need to use log management tools to read logs, as these tools only read logs defined in a specific format. Thus, we need to store logs in a structured format like JSON. JSON is very flexible. Basically, it stores any data type and structure. So, formatting the logs in JSON format is a good option, as using this we apply analysis to the logs.
Below is the example of LogFormat build to store logs in JSON format:
LogFormat "{ \"remoteIP\":\"%a\", \"host\":\"%V\", \"request\":\"%U\", \"time\":\"%t\", \"status\":\"%>s\", \"userAgent\":\"%{User-agent}i\", \"referer\":\"%{Referer}i\" }"
Configure Apache Access Log
Well, Apache allows you to configure different types of log format that helps you to identify and troubleshoot Apache server. In this section, we show you different method to configure Apache access log.
Configure a Custom Log Format
Define the custom log format by editing the Apache main configuration file /etc/apache2/apache2.conf.
nano /etc/apache2/apache2.conf
Add the following lines to configure your own custom log format.
LogFormat "%t %H %m %U %q %I %>s %O %{ms}T" custom
Save and close the file then restart the Apache service to apply the changes.
systemctl restart apache2
After defining the Apache custom log format, edit your website virtual host configuration file.
nano /etc/apache2/sites-enabled/website.conf
Add the following line:
CustomLog ${APACHE_LOG_DIR}/access.log combined
Save and close the file then restart the Apache service to apply the changes. Next, make the following request to your server.
curl --head http://localhost
You should see the following response.
Next, run the following command to view the Apache access log.
tail /var/log/apache2/access.log
You should see the following screen.
Configure a JSON Log Format
Additionally, you may also configure the JSON logging format in Apache web server. It produces the structured logging output in your terminal. To define the JSON log format, edit the Apache configuration file and add the following line:
LogFormat "{ \"timestamp\":\"%t\", \"protocol\":\"%H\", \"method\":\"%m\", \"request\":\"%U\", \"query\":\"%q\", \"request_size_in_bytes\":\"%I\", \"status_code\":\"%>s\", \"response_size_in_bytes\":\"%O\", \"time_taken_ms\":\"%{ms}T\" }" json
Save and close the file then restart the Apache service to implement the changes. This produces the Apache access log in the following format.
Configure Apache Error Log
The Apache error log helps to find out and troubleshoot web server related error in your system. YUse the different log format to define where the error log is transported.
Configure Apache Common Log
Configure the Apache common log using the ErrorLog directive in your Apache virtual host configuration file.
Let’s edit the Apache default virtual host configuration file.
nano /etc/apache2/sites-available/000-default.conf
Add the following line:
ErrorLog ${APACHE_LOG_DIR}/error.log
Save and close the file then restart the Apache service to apply the changes.
systemctl restart apache2
Now, verify the Apache error log using the following command.
tail -f /var/log/apache2/error.log
You should see the Apache log in the following screen.
Configure Apache Custom Log
Moreover, also customize Apache error log as per your requirement using the ErrorLogFormat directive. First, edit the Apache main configuration file using the following command.
nano /etc/apache2/apache2.conf
Add the following line:
ErrorLogFormat "[%{u}t] [%l] [pid %P:tid %T] [client\ %a] %M"
Save and close the file then restart the Apache service to apply the changes.
systemctl restart apache2
The above configuration produces the Apache error log in the following format.
How to Configure Apache Logging for Better Debugging & Analysis Conclusion
The above article gives a brief introduction to Apache Logs and how to configure them. This helps you to debug the issues quickly. We may even apply analytics to derive the outcomes from logs. We also showed you how to customize your Apache access and error log formats, such as common log, JSON log and custom log. Please read the Apache official documentation page to find more about Apache logging and debugging information.
