Listing all MASSCAN traffic port 80 Apache/Nginx access.log

[Asphyxia]

You have either Apache or Nginx most likely, just navigate to your appropriate web server access logs for example

cd /var/log/nginx

What files do we have to inspect? ls -la

Mostly .log files for this Nginx system. I want to grep all the logs for any traffic containing masscan!

We could use grep like

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'scan'

If the above is giving too much info, we can be more specific as to include the exact word 'masscan' instead:

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'masscan'

Even better, we can then get a count on IP addresses as I have in prior postings (example 1).

Here is a list of all the IPs sorted using our former grep method also:

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'masscan' | cut -d ' ' -f 1

Now to get a count of duplicate lines we can simply use the uniq command:

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'masscan' | cut -d ' ' -f 1 | sort | uniq -c

This will get every occurence of masscan, even ones only occurring once. Want to see IP addresses that send more than one request as they could perhaps pose a greater threat? Easy, how about we just leave out results only showing 1.

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'masscan' | cut -d ' ' -f 1 | sort | uniq -c | grep -v " 1 /var"

Now for example I may want to copy this list over to termbin, for a quick reference to share with a friend.

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i 'masscan' | cut -d ' ' -f 1 | sort | uniq -c | grep -v " 1 /var" | nc termbin.com 9999

Results https://termbin.com/q99y (mirror: https://pastebin.com/raw/d2K6iXa4 )

      2 /var/log/nginx/access.log:173.212.233.69
      2 /var/log/nginx/access.log:173.249.16.234
      2 /var/log/nginx/access.log:173.249.60.176
      2 /var/log/nginx/access.log:174.138.7.207
      2 /var/log/nginx/access.log:178.33.122.173
      2 /var/log/nginx/access.log:51.68.137.11
      2 /var/log/nginx/access.log:91.194.90.159

Now we know these IP addresses appear to be scanning IPv4 addresses across the Internet by using masscan and they are trying multiple requests.

Let's say we wanted to make some sort of beautiful web panel to showcase these scanning IP addresses.. we could narrow down searches into specific suspect IP addresses to see more closely what they are doing,

find /var/log/nginx -type f -print | grep '.log' | xargs grep -i '178.33.122.173'

Will show us this result:

/var/log/nginx/access.log:178.33.122.173 - - [20/Nov/2019:22:24:33 +0000] "GET / HTTP/1.0" 200 612 "-" "masscan/1.0 (https://github.com/robertdavidgraham/masscan)"
/var/log/nginx/access.log:178.33.122.173 - - [20/Nov/2019:22:24:36 +0000] "GET / HTTP/1.0" 200 612 "-" "masscan/1.0 (https://github.com/robertdavidgraham/masscan)"

This shows us the scanner simply checked twice.

Another important thing to make note of is these are looking to be primarily discovery scans. We will cover in more detail how to detect more malicious scans and behavior. We will even pursue establishing a honeypot to look for new attacks and figuring out how they work.

Bash help sources:

List file names based on a filename pattern and file content?

How can I use grep command to search file name based on a wild card "LMN2011*" listing all files with this as beginning? I want to add another check on those file content. If file conten...

stackoverflow.com

How to get the count of duplicate strings in a set using grep, uniq and awk in unix?

I have a very large set of strings, one on every line of a file. Many strings occur more than one times in the file at different locations. I want a frequency count of the strings using unix comm...

stackoverflow.com