Apache stores details of HTTP requests in its Access Log as defined by the CustomLog directive. The log can be used to check for malicious requests such as hacking or defacement attempts.

Scalp! is a security log analyzer for Apache. It automates reading the Apache log files and perform threat analysis.

Scalp! uses rulesets provided by PHP-IDS project and is available for download from GitHub.

Steps to perform threat analysis on Apache log using Scalp:

  1. Download Scalp from GitHub.

    $ git clone https://github.com/neuroo/apache-scalp scalp Cloning into 'scalp'... remote: Enumerating objects: 11, done. remote: Total 11 (delta 0), reused 0 (delta 0), pack-reused 11 Unpacking objects: 100% (11/11), 11.85 KiB | 346.00 KiB/s, done.

    Related: How to install Git on Ubuntu

  2. Download signature file from PHPIDS project on GitHub. 

    $ git clone https://github.com/PHPIDS/PHPIDS phpids Cloning into 'phpids'... remote: Enumerating objects: 11281, done. remote: Total 11281 (delta 0), reused 0 (delta 0), pack-reused 11281 Receiving objects: 100% (11281/11281), 4.16 MiB | 2.81 MiB/s, done. Resolving deltas: 100% (5636/5636), done.

  3. Split Apache log file if longer than 10000 lines. 

    $ split -l 10000  /var/log/apache2/access_log

  4. Analyze Apache log file using Scalp and PHPIDS signature.

    $ sudo python scalp/scalp.py --log /var/log/apache2/access_log --filters phpids/lib/IDS/default_filter.xml Password: Loading XML file 'phpids/lib/IDS/default_filter.xml'... Processing the file '/var/log/apache2/access_log'... Scalp results: 	Processed 1318 lines over 1318 	Found 6 attack patterns in 0.425544 s Generating output in /home/user/access_log_scalp_*

    More options for Scalp: 

    Scalp the apache log! by Romain Gaucher - http://rgaucher.info usage:  ./scalp.py [--log|-l log_file] [--filters|-f filter_file] [--period time-frame] [OPTIONS] [--attack a1,a2,..,an]                    [--sample|-s 4.2]    --log       |-l:  the apache log file './access_log' by default    --filters   |-f:  the filter file     './default_filter.xml' by default    --exhaustive|-e:  will report all type of attacks detected and not stop                      at the first found    --tough     |-u:  try to decode the potential attack vectors (may increase                      the examination time)    --period    |-p:  the period must be specified in the same format as in                      the Apache logs using * as wild-card                      ex: 04/Apr/2008:15:45;*/Mai/2008                      if not specified at the end, the max or min are taken    --html      |-h:  generate an HTML output    --xml       |-x:  generate an XML output    --text      |-t:  generate a simple text output (default)    --except    |-c:  generate a file that contains the non examined logs due to the                      main regular expression; ill-formed Apache log etc.    --attack    |-a:  specify the list of attacks to look for                      list: xss, sqli, csrf, dos, dt, spam, id, ref, lfi                      the list of attacks should not contains spaces and comma separated                      ex: xss,sqli,lfi,ref    --ignore-ip|-i:  specify the list of IP Addresses to look exclude                      the list of IP Addresses should be comma separated and not contain spaces                      This option can be used in conjunction with --ignore-ip    --ignore-subnet|-n:  specify the list of Subnets to look exclude                      the list of Subnets should be comma separated and not contain spaces                      This option can be used in conjunction with --ignore-subnet    --output    |-o:  specifying the output directory; by default, scalp will try to write                      in the same directory as the log file    --sample    |-s:  use a random sample of the lines, the number (float in [0,100]) is                      the percentage, ex: --sample 0.1 for 1/1000

  5. Review generated output. 

    $ cat /home/user/access_log_scalp_*  # # File created by Scalp! by Romain Gaucher - http://code.google.com/p/apache-scalp # Apache log attack analysis tool based on PHP-IDS filters # Scalped file: access_log Creation date: Sat-08-Feb-2020  Attack type: files Attack Cross-Site Scripting (xss) Attack Cross-Site Request Forgery (csrf) Attack Spam (spam) Attack Local File Inclusion (lfi)  	### Impact 5 	127.0.0.1 - - [08/Feb/2020:07:31:12 +0800] "GET /windows/start HTTP/1.1" 200 4541 	Reason: "Detects specific directory and path traversal"  	127.0.0.1 - - [08/Feb/2020:07:31:16 +0800] "GET /windows/burn-iso-image HTTP/1.1" 200 4941 	Reason: "Detects specific directory and path traversal"  	127.0.0.1 - - [08/Feb/2020:07:31:17 +0800] "GET /_media/windows/burn-iso-image/windows7-burn-iso-explorer-write-to-disc.png?w=400&tok=e21342 HTTP/1.1" 200 16104 	Reason: "Detects specific directory and path traversal"  	127.0.0.1 - - [08/Feb/2020:07:31:17 +0800] "GET /windows/disable-system-restore HTTP/1.1" 200 4679 	Reason: "Detects specific directory and path traversal"  	127.0.0.1 - - [08/Feb/2020:07:31:18 +0800] "GET /windows/firewall-command HTTP/1.1" 200 5511 	Reason: "Detects specific directory and path traversal"  	127.0.0.1 - - [08/Feb/2020:07:31:19 +0800] "GET /windows/restore-mbr HTTP/1.1" 200 5323 	Reason: "Detects specific directory and path traversal"  Attack SQL Injection (sqli) Attack type: format string Attack Remote File Execution (rfe) Attack Denial Of Service (dos) Attack Directory Traversal (dt) Attack Information Disclosure (id)