Apache Access & Error Logs

Apache is a popular web server, like most applications, it is responsible for a lot of activity and as such records a lot of those activities in logs. Logs are nothing more than text files that record specified activity. Similar to NGINX, it generates two very specific logs: Error.log and Access.log. This article will dive into each one respectively.

Log & Configuration Locations

Before you get started, it's best to orient yourself with where things are located. Below are the log locations for Apache according to the different Linux distributions you might be using:

Name	Description
Red Hat, CentOS, or Fedora Linux	/var/log/httpd/
Debian and Ubuntu	/var/log/apache2/
FreeBSD	/var/log/
Configuration File	/etc/httpd/conf /etc/apache/ /etc/apache2/

Apache Access Logs

The Apache access log exists to record the activity of all the visitors to your site. You can use these logs to see what pages are being visited, what files are being requested, where users are coming from, and the list goes on. It also contains sensitive information like the users IP, the date and time of the request, and even the type of application being used to access the site (e.g., Chrome vs Firefox, Mobile vs Desktop). This log is especially helpful when you're trying to better understand your visitor behavior.

Apache logs have two very common log formats that can be set via your Apache configuration file.

1. Common Log Format
2. Combined Log Format

When configuring your configuration file the log formatting directives are used in combination with the LogFormat option. It looks something like this:

LogFormat "%t %h %m \"%r\"" custom

The above line tells that the “%t %h %m \”%r\”” format should be used and assigned to an alias called custom. After that, we can use the custom alias when defining Apache logging. For example:

CustomLog "logs/access_log" custom

This tells Apache to write all the logs to access_log while using the formatting as defined by the custom LogFormat.

Directive	Meaning
%t	time of the request
%h	Remote hostname
%m	HTTP method
%r	The first line of the request

There is a more complete list of directives you can use on the Apache Custom Log Formats page.

1. Common Access Log

The common log format is one of the two more common formatting options. It's formatting is:

LogFormat "%h %l %u %t \"%r\" %>s %b" common

Which translates to something like this:

Directive	Meaning
%h	IP address of the remote host that made the request
%l	remote log name provided by identd
%u	user identifier determined by the HTTP authentication
%t	date and time of the request with the time zone
\”%r\”	first line of the request inside double quotes
%>s	status code reported to the client.
%b	size of the object sent to the client

So this is how an access log using the common log format will look:

101.12.34.3 - - [10/May/2022:22:22:12 -0000] "GET /trunc.png HTTP/1.1" 200 1023

Using the formatting, this is what we can extract from the log:

Directive	Meaning	Sample
%h	IP address of the remote host that made the request	101.12.34.3
%l	remote log name provided by identd	"-", what you get when it can't be found
%u	user identifier determined by the HTTP authentication	"-", what you get when it can't be found
%t	date and time of the request with the time zone	10/May/2022:22:22:12 -0000
\”%r\”	first line of the request inside double quotes	"GET /trunc.png HTTP/1.1"
%>s	status code reported to the client.	200
%b	size of the object sent to the client	1023

2. Combined Log Format

The combined log format is another common log format, it's usually set as default on today's modern distributions. It's formatting is:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined

Which translates to something like this:

Directive	Meaning
%h	IP address of the remote host that made the request
%l	remote log name provided by identd
%u	user identifier determined by the HTTP authentication
%t	date and time of the request with the time zone
\”%r\”	first line of the request inside double quotes
%>s	status code reported to the client.
%b	size of the object sent to the client
\"%{Referer}i\"	Logs Referer on all requests
\"%{User-agent}i\""	Logs User-agent

So this is how an access log using the combined log format will look:

101.12.34.3 - - [10/May/2022:22:22:12 -0000] "GET /trunc.png HTTP/1.1" 200 1023 "http://www.trunc.org/index.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.74 Safari/537.36 Edg/79.0.309.43"

Using the formatting, this is what we can extract from the log:

Directive	Meaning	Sample
%h	IP address of the remote host that made the request	101.12.34.3
%l	remote log name provided by identd	"-", what you get when it can't be found
%u	user identifier determined by the HTTP authentication	"-", what you get when it can't be found
%t	date and time of the request with the time zone	10/May/2022:22:22:12 -0000
\”%r\”	first line of the request inside double quotes	"GET /trunc.png HTTP/1.1"
%>s	status code reported to the client.	200
%b	size of the object sent to the client	1023
\"%{Referer}i\"	Logs Referer on all requests	"http://www.trunc.org/index.php"
\"%{User-agent}i\""	Logs User-agent	"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.74 Safari/537.36 Edg/79.0.309.43"

Apache Error Logs

The Apache error log exists to record any issues, glitches, that the application might encounter. Maybe there is an error in a configuration file, or a conflict with a dependency. Maybe the application is forced to reboot, or fails to initialize. This is the log you want to analyze when something goes wrong as it usually contains a bit more detail and can help point you in the right direction. It's also extremely helpful to spend time analyzing the warnings, they can be indicative a problem.

1. Error Log Format & Sample

By default, all error logs have a standard format and are set in the Apache configuration file using the ErrorLog directive. The configuration looks like this:

ErrorLog "/var/log/httpd-error.log"

The formatting for Error Logs is as follows:

ErrorLogFormat “[%{u}t] [%-m:%l] [pid %P:tid %T] [client\ %a] %M”

Which translates to:

Directive	Meaning
%{u}t	The current time including microseconds
%-m	the module that produced the error
%l	the level of the log event
%P	process identifier
%T	thread identifier
%M	the actual log message

There is a more complete list of directives you can use on the Apache Custom Log Formats page.

So this is how an access log using the combined log format will look:

[Thu May 26 09:32:23.811033 2022] [core:error] [pid 23508:tid 3215462132] [client 10.10.10.10] File does not exist: /usr/local/apache2/htdocs/favicon.ico

Using the formatting, this is what we can extract from the log:

Directive	Meaning	Sample
%{u}t	The current time including microseconds	[Thu May 26 09:32:23.811033 2022]
%-m	the module that produced the error	-
%l	the level of the log event	[core:error]
%P	process identifier	pid 23508
%T	tid 3215462132
%M	the actual log message	[client 10.10.10.10] File does not exist: /usr/local/apache2/htdocs/favicon.ico

2. Error Log Severity

Speaking of which, the NGINX error logs provide 8 unique severity levels you should be aware of.

Name	Description
1. emerg	Emergency messages when your system may be unstable.
2. alert	Alert messages of serious issues.
3. crit	Critical issues that need to be taken care of immediately.
4. error	An error has occurred. Something went wrong while processing a page.
5. warn	A warning messages that you should look into it.
6. notice	A simple log notice that you can ignore.
7. info	Just an information messages that you might want to know.
8. debug	Debugging information used to pinpoint the location of error.
9. trace1 – trace8	Used for very low-level information logging that you can probably skip.

Posted in   log-guide   apache-logs     by Daniel Cid (@dcid)