ELK 设置dashboard样例

转自:https://mp.weixin.qq.com/s/Jyeefwe60mztzESxGVOl_A

 

新上线的系统可以做通过打印的日志做些基本监控图表,以下为做的几个监控图表,抛砖引玉,日志系统使用Elasticsearch+Kibana+Logstash

软件名 版本
ElasticSearch 1.7.x
Logstash 1.5.x
Kibana 4.1.x

 

样例及配置如下:

 


 

1.可以统计各个接口的请求数量

 

配置


 

2.可以查看接口响应时长的分布情况

 

 

配置:

 

 


 

3.可以查看请求数量分布情况

配置:

 


 

4.查看具体接口的响应时间

配置:

 


 

5.系统错误码数量

 

 

配置

 

6.服务器logstash配置

input {

heartbeat {

interval => 30

add_field => { “kafka_topic_name” => “logstash-heartbeat” }

}

file {

path => “/opt/xxx/logs/tomcat/app/tomcat_auth_01/auth-log.log”

add_field => { “type” => “tomcat_auth_log” }

add_field => { “kafka_topic_name” => “topic.tomcat.applog” }

add_field => { “es_index_prefix” => “tomcat-applog” }

 

codec => multiline {

pattern => “^%{TIMESTAMP_ISO8601}”

negate => true

what => “previous”

}

}

file {

path => “/opt/xxx/logs/tomcat/sys/tomcat_auth_01/tomcat_auth_01.log”

add_field => { “type” => “tomcataccess_auth_log” }

add_field => { “kafka_topic_name” => “topic.tomcataccess.log” }

add_field => { “es_index_prefix” => “tomcataccess-log” }

}

 

filter {

mutate {

replace => ["host", "10.0.0.51"]

}

 

if [type] == “tomcataccess_auth_log” {

grok {

match => ["message", '(?<clientip>%{IPORHOST}%{DATA}|%{IPORHOST}|-) %{USER:ident} %{USER:auth} \[%{HTTPDATE:time}\] “(?:%{WORD:method} %{DATA:request}(?:\?%{NOTSPACE:query_string}|%{SPACE})(?: HTTP/%{NUMBER:httpversion:float})?|%{DATA:rawrequest})” %{WORD:method2} %{NUMBER:http_status_code:int} %{NUMBER:body_bytes:int} %{NUMBER:duration:int}’]

remove_field => ["ident","auth","method2"]

}

 

#clientip字段有多个ip时,获取第一个ip

if ‘,’ in [clientip] {

mutate {

gsub => ["clientip", ",.*+", ""]

}

}

 

}

}

 

output {

kafka {

codec => “json”

topic_id => “%{kafka_topic_name}”

bootstrap_servers => “XXXX:9092″

}

}

 

7.app log配置

<!– 文件输出日志 (文件大小策略进行文件输出,超过指定大小对文件备份) –>

<appender name=”FILE” class=”ch.qos.logback.core.rolling.RollingFileAppender”>

<File>/opt/xxx/logs/tomcat/app/${tomcat_app_base}/auth-log.log</File>

<rollingPolicy class=”ch.qos.logback.core.rolling.TimeBasedRollingPolicy”>

<FileNamePattern>

/opt/xxx/logs/tomcat/app/${tomcat_app_base}/auth-log.%d{yyyy-MM-dd}.log

</FileNamePattern>

</rollingPolicy>

<layout>

<Pattern>%-20(%d{HH:mm:ss.SSS} [%thread]) %-5level %logger{32} -[%X{sequenceId}]%msg%n</Pattern>

</layout>

</appender>

 

 

原文链接:,转发请注明来源!

Leave a Reply

You must be logged in to post a comment.