通过查看nginx 日志进行分析

Published on with 0 views and 0 comments

1.根据访问 ip 统计 UV；

[root@blog_server logs]# awk '{print $1}'  access.log|sort | uniq -c |wc -l
138

2.统计访问 URL 统计 PV；

[root@blog_server logs]# awk '{print $7}' access.log|wc -l
637

3.查询访问最频繁的 URL;

[root@blog_server logs]# awk '{print $7}' access.log|sort | uniq -c |sort -n -k 1 -r|more
     55 /solo/console/article/
     52 /
     46 /solo/console/markdown/2html
     17 /solo/articles/2019/03/16/1552701584725.html
     ……从高往低一次排列

4.查询访问最频繁的 ip

[root@blog_server logs]# awk '{print $1}' access.log|sort | uniq -c |sort -n -k 1 -r|head -n 10

    351 49.4.136.242
     47 103.254.69.246
     34 118.31.188.179
     10 65.154.226.126
      9 203.34.152.133
      8 70.42.131.170
      8 65.155.30.101
      8 65.154.226.109
      8 222.82.54.223
      4 203.208.60.115

5.查看 nginx 并发连接数；

[root@blog_server logs]# netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
ESTABLISHED 6
TIME_WAIT 13

返回值说明：
CLOSED //无连接是活动的或正在进行
LISTEN //服务器在等待进入呼叫
SYN_RECV //一个连接请求已经到达，等待确认
SYN_SENT //应用已经开始，打开一个连接
ESTABLISHED //正常数据传输状态/当前并发连接数
FIN_WAIT1 //应用说它已经完成
FIN_WAIT2 //另一边已同意释放
ITMED_WAIT //等待所有分组死掉
CLOSING //两边同时尝试关闭
TIME_WAIT //另一边已初始化一个释放
LAST_ACK //等待所有分组死掉

6.根据时间段查看相应的连接日志

[root@blog_server logs]# cat access.log | sed -n '/19\/Mar\/2019:08/,/19\/Mar\/2019:15/p'|more

109.92.203.214 - - [19/Mar/2019:08:11:09 +0800] "GET / HTTP/1.1" 200 11279 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
118.31.188.179 - - [19/Mar/2019:08:11:26 +0800] "GET /solo/articles/2019/03/08/1552013816021.html HTTP/1.1" 200 31397 "-" "Sym/3.4.7; +https://github.com/b3log/symphony"
118.31.188.179 - - [19/Mar/2019:08:11:26 +0800] "GET /solo/articles/2019/03/08/1552013816021.html HTTP/1.1" 200 31397 "-" "Sym/3.4.7; +https://github.com/b3log/symphony, Sym/3.4.7; +https://github.com/b3log/symphony"
118.31.188.179 - - [19/Mar/2019:08:11:45 +0800] "GET /solo/articles/2019/03/16/1552701584725.html HTTP/1.1" 200 31967 "-" "Sym/3.4.7; +https://github.com/b3log/symphony"
118.31.188.179 - - [19/Mar/2019:08:11:45 +0800] "GET /solo/articles/2019/03/16/1552701584725.html HTTP/1.1" 200 31998 "-" "Sym/3.4.7; +https://github.com/b3log/symphony, Sym/3.4.7; +https://github.com/b3log/symphony"
101.132.106.40 - - [19/Mar/2019:08:17:02 +0800] "GET / HTTP/1.1" 200 11279 "-" "Mozilla/5.0 (Linux; Android 4.1.1; Nexus 7 Build/JRO03D))"
39.107.121.196 - - [19/Mar/2019:08:18:33 +0800] "GET / HTTP/1.1" 200 11279 "-" "Mozilla/5.0 (Linux; Android 4.1.1; Nexus 7 Build/JRO03D))"
196.50.11.186 - - [19/Mar/2019:08:25:50 +0800] "GET / HTTP/1.1" 200 11279 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
118.31.188.179 - - [19/Mar/2019:08:29:04 +0800] "GET /solo/articles/2019/03/15/1552644110343.html HTTP/1.1" 200 30990 "-" "Sym/3.4.7; +https://github.com/b3log/symphony"
118.31.188.179 - - [19/Mar/2019:08:29:05 +0800] "GET /solo/articles/2019/03/15/1552644110343.html HTTP/1.1" 200 30990 "-" "Sym/3.4.7; +https://github.com/b3log/symphony, Sym/3.4.7; +https://github.com/b3log/symphony"
80.82.70.187 - - [19/Mar/2019:08:43:11 +0800] "GET http://www.baidu.com/cache/global/img/gs.gif HTTP/1.1" 404 1109 "-" "Mozilla"
203.34.152.133 - - [19/Mar/2019:08:56:29 +0800] "GET /admin/ HTTP/1.1" 404 1084 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPa
th.3; KB974488)"
……

7.统计状态码：

$ awk '{print $9}' /usr/local/nginx/logs/access.log |sort|uniq -c |sort -nr|head -10

    706 200
     42 404
     11 302
      2 400
      2 173

8.统计 nginx 流量：

$ grep "03/May/2019" /usr/local/nginx/logs/access.log | awk '{sum+=$10}END{print sum}'

19381073

9. 查看请求出现 404 次数多少的 ip

[root@localhost~]# cat /usr/local/nginx/logs/access.log | awk '{if($9="404"){ip_code[$1" "$9]++}} END{for (i in ip_code){print i,ip_code[i]}}'

192.168.88.98 404 18
192.168.89.234 404 2

10. 统计每个 ip 访问状态码数量

cat /usr/local/nginx/logs/access.log | awk '{ip_code[$1" "$9]++} END{for (i in ip_code){print i,ip_code[i]}}' |sort -k1 -rn|head -n10

223.104.3.193 200 10
223.104.101.243 200 10
216.144.240.130 200 1
203.99.54.238 200 1
193.106.30.234 157 2
185.209.0.12 157 1
185.208.149.91 200 1
185.176.27.114 200 1
183.136.190.62 200 2
173.244.36.70 400 1
标题：通过查看nginx 日志进行分析
作者：cuijianzhe
地址：https://cuijianzhe.github.io/articles/2019/03/19/1552967425056.html

↑ ↓