码迷,mamicode.com
首页 > 其他好文 > 详细

ELK测试

时间:2019-02-17 12:28:35      阅读:109      评论:0      收藏:0      [点我收藏+]

标签:package   number   arp   elastic   RKE   internet   ted   yml   json格式   

ELK 日志分析系统

        ELK 日志分析系统
            1.0 ELK 介绍
            1.1 ELK 安装准备工作
            1.2 es 安装
            1.3 es配置
            1.4 es测试
            1.5 Kibana安装
            1.6 logstash安装
            1.7 logstash配置解析rsyslog文件
            1.8 kibana查看日志
            1.9 nginx日志收集
            2.0 beats采集日志

1.0 ELK 介绍

官网https://www.elastic.co/cn/

中文指南https://www.gitbook.com/book/chenryn/elk-stack-guide-cn/details

ELK Stack (5.0版本之后)? Elastic Stack == (ELK Stack + Beats)

ELK Stack包含:ElasticSearch、Logstash、Kibana

ElasticSearch是一个搜索引擎,用来搜索、分析、存储日志。它是分布式的,也就是说可以横向扩容,可以自动发现,索引自动分片,总之很强大。文档https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html

Logstash用来采集日志,把日志解析为json格式交给ElasticSearch。

Kibana是一个数据可视化组件,把处理后的结果通过web界面展示

Beats在这里是一个轻量级日志采集器,其实Beats家族有5个成员
早期的ELK架构中使用Logstash收集、解析日志,但是Logstash对内存、cpu、io等资源消耗比较高。相比 
Logstash,Beats所占系统的CPU和内存几乎可以忽略不计

x-pack对Elastic Stack提供了安全、警报、监控、报表、图表于一身的扩展包,是收费的

1.1 ELK 安装准备工作

环境准备:192.168.137.30、192.168.137.40、192.168.137.45
// 三台机器均安装Elaelasticsearch(后续简称es)、jdk8,设置hosts 
主节点:
192.168.137.30
数据节点:
192.168.137.40、192.168.137.45
所有节点均安装jdk环境
yum install -y java-1.8.0-openjdk

1.2 es 安装

官方文档 https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html
三台机器均都要执行以下操作
[root@linux-node3 ~]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@linux-node3 ~]# cat /etc/yum.repos.d/elastic.repo

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[root@linux-node3 ~]# yum install -y elasticsearch
或者方法安装
[root@linux-node3 ~]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0.rpm
[root@linux-node3 ~]# rpm -ivh elasticsearch-6.0.0.rpm

1.3 es配置

elasticsearch配置文件/etc/elasticsearch和/etc/sysconfig/elasticsearch 
参考https://www.elastic.co/guide/en/elasticsearch/reference/6.0/rpm.html 
主节点:192.168.137.30编辑配置文件

[root@linux-node3 ~]# cat /etc/elasticsearch/elasticsearch.yml 
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
cluster.name: linux-node3.com
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
node.name: linux-node3.com
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
 node.master: true
 node.data: false

#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.137.30
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
discovery.zen.ping.unicast.hosts: ["192.168.137.30", "192.168.137.40", "192.168.137.45"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
[root@linux-node3 ~]#

同理修改数据节点配置:

[root@linux-node4 ~]# cat /etc/elasticsearch/elasticsearch.yml 
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
cluster.name: linux-node3.com
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
node.name: linux-node4.com
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
 node.master: false
 node.data: true

#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.137.40
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
discovery.zen.ping.unicast.hosts: ["192.168.137.30", "192.168.137.40", "192.168.137.45"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
[root@linux-node4 ~]# 

三台均启动服务

[root@linux-node3 ~]# systemctl start elasticsearch
[root@linux-node3 ~]# ps -aux |grep elasticsearch
elastic+   3140 23.7 45.9 1482312 459248 ?      Ssl  14:29   0:00 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
root       3168  5.0  0.0 112680   716 pts/1    S+   14:29   0:00 grep --color=auto elasticsearch
[root@linux-node3 ~]# 
[root@linux-node3 ~]# netstat -lntnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      966/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1095/master         
tcp        0      0 192.168.137.30:27017    0.0.0.0:*               LISTEN      1006/mongod         
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      1006/mongod         
tcp6       0      0 192.168.137.30:9200     :::*                    LISTEN      1422/java           
tcp6       0      0 :::8080                 :::*                    LISTEN      1185/java           
tcp6       0      0 :::80                   :::*                    LISTEN      961/httpd           
tcp6       0      0 192.168.137.30:9300     :::*                    LISTEN      1422/java           
tcp6       0      0 :::22                   :::*                    LISTEN      966/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1095/master         
[root@linux-node3 ~]# 
// 依次启动数据节点。端口分别为9200,9300
[root@linux-05 ~]# ss -ltnp |grep -E ‘9200|9300‘
LISTEN     0      128      ::ffff:192.168.137.45:9200                    :::*                   users:(("java",pid=2758,fd=118))
LISTEN     0      128      ::ffff:192.168.137.45:9300                    :::*                   users:(("java",pid=2758,fd=108))
[root@linux-05 ~]#
[root@linux-node4 ~]# ss -ltnp |grep -E ‘9200|9300‘
LISTEN     0      128      ::ffff:192.168.137.40:9200                    :::*                   users:(("java",pid=3257,fd=119))
LISTEN     0      128      ::ffff:192.168.137.40:9300                    :::*                   users:(("java",pid=3257,fd=110))

1.4 es测试

健康检查

[root@linux-node3 ~]# curl ‘192.168.137.30:9200/_cluster/health?pretty‘
{
  "cluster_name" : "linux-node3.com",
  "status" : "green",   //健康状态
  "timed_out" : false,
  "number_of_nodes" : 3,     //3个节点
  "number_of_data_nodes" : 2,  //2个数据节点
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
[root@linux-node3 ~]#

集群详细信息

[root@linux-node3 ~]# curl ‘192.168.137.30:9200/_cluster/state?pretty‘ 
{
  "cluster_name" : "linux-node3.com",
  "compressed_size_in_bytes" : 355,
  "version" : 5,
  "state_uuid" : "RBH5dvstTyqgHVVSdfNi_Q",
  "master_node" : "d1yLa9f9RfSPwUXPvm_lqQ",
  "blocks" : { },
  "nodes" : {
    "d1yLa9f9RfSPwUXPvm_lqQ" : {
      "name" : "linux-node3.com",
      "ephemeral_id" : "DGs6lBiaQvaJlmyasez-TA",
      "transport_address" : "192.168.137.30:9300",
      "attributes" : { }
    },
    "pyOddTkYRN6fRjWjb-ehBw" : {
      "name" : "linux-05.com",
      "ephemeral_id" : "X8oa-yozSxqVmb6Dp2fhAQ",
      "transport_address" : "192.168.137.45:9300",
      "attributes" : { }
    },
    "mf7rEM3oScqEOqNFniEJfA" : {
      "name" : "linux-node4.com",
      "ephemeral_id" : "eZ4jATDJRDyv3rmnup3zfg",
      "transport_address" : "192.168.137.40:9300",
      "attributes" : { }
    }
  },
  "metadata" : {
    "cluster_uuid" : "3_2FFY-XTPexeDEZ6MXR1Q",
    "templates" : { },
    "indices" : { },
    "index-graveyard" : {
      "tombstones" : [ ]
    }
  },
  "routing_table" : {
    "indices" : { }
  },
  "routing_nodes" : {
    "unassigned" : [ ],
    "nodes" : {
      "mf7rEM3oScqEOqNFniEJfA" : [ ],
      "pyOddTkYRN6fRjWjb-ehBw" : [ ]
    }
  },
  "restore" : {
    "snapshots" : [ ]
  },
  "snapshots" : {
    "snapshots" : [ ]
  },
  "snapshot_deletions" : {
    "snapshot_deletions" : [ ]
  }
}
[root@linux-node3 ~]#

1.5 Kibana安装

主节点安装:kibana
[root@linux-node3 ~]#  yum install -y kibana //会很慢
[root@linux-node ~]# wget https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-x86_64.rpm
[root@linux-node3 ~]# rpm -ivh kibana-6.0.0-x86_64.rpm 
准备中...                          ################################# [100%]
正在升级/安装...
1:kibana-6.0.0-1                   ################################# [100%]
[root@linux-node3 ~]# 
[root@linux-node3 ~]# grep -v "^#" /etc/kibana/kibana.yml 
server.port: 5601  //监听端口
server.host: "192.168.137.30"

elasticsearch.url: "http://192.168.137.30:9200" //es访问地址

logging.dest: /var/log/kibana.log

[root@linux-node3 ~]#
[root@linux-node3 log]# touch kibana.log
[root@linux-node3 log]# chmod 777 kibana.log 
[root@linux-node3 log]# systemctl restart kibana
[root@linux-node3 log]# ps aux | grep kibana
kibana     1626 39.5 11.6 1121852 116968 ?      Ssl  17:09   0:04 /usr/share/kibana/bin/../node/bin/node --no-warnings /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml
root       1638  0.0  0.0 112680   980 pts/0    R+   17:10   0:00 grep --color=auto kibana
[root@linux-node3 log]# netstat -lntnp | grep nod
tcp        0      0 192.168.137.30:5601     0.0.0.0:*               LISTEN      1626/node           
[root@linux-node3 log]#
浏览器里访问 http://192.168.137.30:5601

1.6 logstash安装

数据节点安装: logstash
[root@linux-node4 ~]# yum install -y  logstash 或者使用
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.0.0.rpm
rpm -ivh logstash-6.0.0.rpm 
[root@linux-node4 ~]# rpm -ivh logstash-6.0.0.rpm 
准备中...                          ################################# [100%]
正在升级/安装...
1:logstash-1:6.0.0-1               ################################# [100%]
Using provided startup.options file: /etc/logstash/startup.options
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Successfully created system startup script for Logstash

1.7 logstash配置解析rsyslog文件

[root@linux-node4 ~]# cat /etc/logstash/conf.d/syslog.conf

 input {
  syslog {
    type => "system-syslog"
    port => 10514  
  }
}
output {
  stdout {
    codec => rubydebug
  }
}

[root@linux-node4 ~]# 
检测配置文件是否有错
[root@linux-node4 ~]# cd /usr/share/logstash/bin
[root@linux-node4 bin]# ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf --config.test_and_exit
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash‘s logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK
启动logstash
vim /etc/rsyslog.conf//在#### RULES下面增加一行
*.* @@192.168.137.40:10514
[root@linux-node4 ~]# systemctl restart rsyslog
[root@linux-node4 ~]# netstat -lnp |grep 10514
tcp6       0      0 :::10514                :::*                    LISTEN      3708/java           
udp        0      0 0.0.0.0:10514           0.0.0.0:*                           3708/java
[root@linux-node4 bin]#  ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf

OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash‘s logs to /var/log/logstash which is now configured via log4j2.properties
{
          "severity" => 6,
           "program" => "rsyslogd",
           "message" => "[origin software=\"rsyslogd\" swVersion=\"7.4.7\" x-pid=\"3768\" x-info=\"http://www.rsyslog.com\"] start\n",
              "type" => "system-syslog",
          "priority" => 46,
         "logsource" => "linux-node4",
        "@timestamp" => 2017-12-14T10:07:03.000Z,
          "@version" => "1",
              "host" => "192.168.137.40",
          "facility" => 5,
    "severity_label" => "Informational",
         "timestamp" => "Dec 14 18:07:03",
    "facility_label" => "syslogd"
}
{
          "severity" => 6,
           "program" => "systemd",
           "message" => "Stopping System Logging Service...\n",
              "type" => "system-syslog",
          "priority" => 30,
         "logsource" => "linux-node4",
        "@timestamp" => 2017-12-14T10:07:03.000Z,
          "@version" => "1",
              "host" => "192.168.137.40",
          "facility" => 3,
    "severity_label" => "Informational",
         "timestamp" => "Dec 14 18:07:03",
    "facility_label" => "system"
}
{
          "severity" => 6,
           "program" => "systemd",
           "message" => "Starting System Logging Service...\n",
              "type" => "system-syslog",
          "priority" => 30,
         "logsource" => "linux-node4",
        "@timestamp" => 2017-12-14T10:07:03.000Z,
          "@version" => "1",
              "host" => "192.168.137.40",
          "facility" => 3,
    "severity_label" => "Informational",
         "timestamp" => "Dec 14 18:07:03",
    "facility_label" => "system"
}
{
          "severity" => 6,
           "program" => "systemd",
           "message" => "Started System Logging Service.\n",
              "type" => "system-syslog",
          "priority" => 30,
         "logsource" => "linux-node4",
        "@timestamp" => 2017-12-14T10:07:03.000Z,
          "@version" => "1",
              "host" => "192.168.137.40",
          "facility" => 3,
    "severity_label" => "Informational",
         "timestamp" => "Dec 14 18:07:03",
    "facility_label" => "system"
}
{
          "severity" => 5,
               "pid" => "654",
           "program" => "polkitd",
           "message" => "Unregistered Authentication Agent for unix-process:3761:2437779 (system bus name :1.47, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8) (disconnected from bus)\n",
              "type" => "system-syslog",
          "priority" => 85,
         "logsource" => "linux-node4",
        "@timestamp" => 2017-12-14T10:07:03.000Z,
          "@version" => "1",
              "host" => "192.168.137.40",
          "facility" => 10,
    "severity_label" => "Notice",
         "timestamp" => "Dec 14 18:07:03",
    "facility_label" => "security/authorization"
}

// 注:启动后不能敲命令,屏幕上查看到日志输出

1.8 kibana查看日志

数据节点配置日志收集:
启动logstash
[root@linux-node4 ~]# cat /etc/logstash/conf.d/syslog.conf

input {
  syslog {
    type => "system-syslog"
    port => 10514  
  }
}
output {
  elasticsearch {
  hosts => ["192.168.137.30:9200"]
  index => "system-syslog-%{+YYYY.MM}" 
  }
}

[root@linux-node4 ~]# chown -R logstash /var/lib/logstash
[root@linux-node4 ~]# systemctl start logstash
[root@linux-node4 ~]#
[root@linux-node4 ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      963/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1238/master         
tcp6       0      0 192.168.137.40:9200     :::*                    LISTEN      13890/java          
tcp6       0      0 :::10514                :::*                    LISTEN      14164/java          
tcp6       0      0 192.168.137.40:9300     :::*                    LISTEN      13890/java          
tcp6       0      0 :::22                   :::*                    LISTEN      963/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1238/master         
tcp6       0      0 127.0.0.1:9600          :::*                    LISTEN      14164/java 
// 默认是127.0.0.1:9600,修改
[root@linux-node4 ~]# grep -v "^#" /etc/logstash/logstash.yml 
path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d/*.conf


http.host: "192.168.137.40"
path.logs: /var/log/logstash

[root@linux-node4 ~]#
[root@linux-node4 ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      965/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1224/master         
tcp6       0      0 192.168.137.40:9200     :::*                    LISTEN      2215/java           
tcp6       0      0 :::10514                :::*                    LISTEN      5450/java           
tcp6       0      0 192.168.137.40:9300     :::*                    LISTEN      2215/java           
tcp6       0      0 :::22                   :::*                    LISTEN      965/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1224/master         
tcp6       0      0 192.168.137.40:9600     :::*                    LISTEN      5450/java
主节点查看索引信息
[root@linux-node3 ~]# curl ‘192.168.137.30:9200/_cat/indices?v‘ //可以获取索引信息
health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana               Z7JVUVlLSRSu5xqySplQ5w   1   1          1            0      6.9kb          3.4kb
green  open   system-syslog-2017.12 c9ZmYijTTYSMLMIESb3N4Q   5   1          1            0     24.5kb         12.2kb
[root@linux-node3 ~]#

    [root@linux-node3 ~]# curl -XGET ‘192.168.137.30:9200/indexname?pretty‘ 
{  // 获指定索引详细信息
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index",
        "resource.type" : "index_or_alias",
        "resource.id" : "indexname",
        "index_uuid" : "_na_",
        "index" : "indexname"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index",
    "resource.type" : "index_or_alias",
    "resource.id" : "indexname",
    "index_uuid" : "_na_",
    "index" : "indexname"
  },
  "status" : 404
}

[root@linux-node3 ~]#
curl -XDELETE ‘localhost:9200/logstash-xxx-*‘ 可以删除指定索引
浏览器访问192.168.137.40:5601,到kibana配置索引
左侧点击“Managerment”-> “Index Patterns”-> “Create Index Pattern”
Index pattern这里需要根据前面curl查询到的索引名字来写,否则下面的按钮是无法点击
输入:system-syslog-2017.12 或者system-syslog-*

1.9 nginx日志收集

[root@linux-node4 ~]# cat /etc/logstash/conf.d/nginx.conf

input {
  file {
    path => "/tmp/elk_access.log"
    start_position => "beginning"
    type => "nginx"
  }
}
filter {
    grok {
        match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:http_verb} %{NOTSPACE:http_request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_http_request})\" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    stdout { codec => rubydebug }
    elasticsearch {
        hosts => ["192.168.137.40:9200"]
	index => "nginx-test-%{+YYYY.MM.dd}"
  }
}

[root@linux-node4 ~]# cd /usr/share/logstash/bin
[root@linux-node4 bin]# ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/nginx.conf --config.test_and_exit
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash‘s logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK 
//没有nginx的需要安装nginx
[root@linux-node4 ~]# yum -y install nginx
[root@linux-node4 ~]# cat /etc/nginx/conf.d/elk.conf

server {
            listen 80;
            server_name elk.linux.com;

            location / {
                proxy_pass      http://192.168.137.30:5601;
                proxy_set_header Host   $host;
                proxy_set_header X-Real-IP      $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            }
            access_log  /tmp/elk_access.log main2;
        }

[root@linux-node4 ~]#
配置日志
vim /etc/nginx/nginx.conf//增加如下内容

log_format main2 ‘$http_host $remote_addr - $remote_user [$time_local] "$request" ‘
                      ‘$status $body_bytes_sent "$http_referer" ‘
                      ‘"$http_user_agent" "$upstream_addr" $request_time‘;

[root@linux-node4 ~]# systemctl start nginx
[root@linux-node4 ~]# ps -ef | grep nginx
root       2916      1  0 14:48 ?        00:00:00 nginx: master process /usr/sbin/nginx
nginx      2917   2916  0 14:48 ?        00:00:00 nginx: worker process
root       2919   2732  0 14:48 pts/4    00:00:00 grep --color=auto nginx
[root@linux-node4 ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      2916/nginx: master  
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      972/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1247/master         
tcp6       0      0 :::80                   :::*                    LISTEN      2916/nginx: master  
tcp6       0      0 192.168.137.40:9200     :::*                    LISTEN      2214/java           
tcp6       0      0 :::10514                :::*                    LISTEN      2304/java           
tcp6       0      0 192.168.137.40:9300     :::*                    LISTEN      2214/java           
tcp6       0      0 :::22                   :::*                    LISTEN      972/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1247/master         
tcp6       0      0 192.168.137.40:9600     :::*                    LISTEN      2304/java           
[root@linux-node4 ~]#
绑定hosts 192.168.137.40 elk.linux.com
浏览器访问,检查是否有日志产生
[root@linux-node4 ~]# systemctl restart logstash  

主节点查看获取的索引
[root@linux-node3 ~]# curl ‘192.168.137.30:9200/_cat/indices?v‘ 
health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana               Z7JVUVlLSRSu5xqySplQ5w   1   1          2            0     20.4kb         10.2kb
green  open   system-syslog-2017.12 c9ZmYijTTYSMLMIESb3N4Q   5   1        127            0    723.8kb        317.6kb
green  open   nginx-test-2017.12.18 w3j3J-wXT6eXzaVf6ycmBg   5   1         20            0     42.7kb           466b
[root@linux-node3 ~]#
// 检查是否有nginx-test开头的索引生成 
如果有,才能到kibana里去配置该索引
左侧点击“Managerment”-> “Index Patterns”-> “Create Index Pattern”
Index pattern这里写nginx-test-*
之后点击左侧的Discover

2.0 beats采集日志

官网:https://www.elastic.co/cn/products/beats
优点:可扩展,支持自定义构建
数据节点:linux-05.com
[root@linux-05 ~]# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-x86_64.rpm
[root@linux-05 ~]# rpm -ivh filebeat-6.0.0-x86_64.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
1:filebeat-6.0.0-1                 ################################# [100%]
[root@linux-05 ~]#
编辑配置文件
[root@linux-05 ~]# grep  -v "^#" /etc/filebeat/filebeat.yml  | grep -v "#" |grep -v "^$"

filebeat.prospectors:
- type: log
  paths:
    - /var/log/messages
output.console:
  enable: true

/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml //可以在屏幕上看到对应的日志信息

[root@linux-05 ~]# /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml 
^C[root@linux-05 ~]# /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml 
{"@timestamp":"2017-12-18T07:32:01.785Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.0.0"},"prospector":{"type":"log"},"beat":{"name":"linux-05.com","hostname":"linux-05.com","version":"6.0.0"},"message":"Dec 18 12:30:01 linux-05 systemd: Started Session 6 of user root.","source":"/var/log/messages","offset":66}
{"@timestamp":"2017-12-18T07:32:01.785Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.0.0"},"offset":133,"message":"Dec 18 12:30:01 linux-05 systemd: Starting Session 6 of user root.","prospector":{"type":"log"},"beat":{"version":"6.0.0","name":"linux-05.com","hostname":"linux-05.com"},"source":"/var/log/messages"}...........

再编辑配置文件
vim /etc/filebeat/filebeat.yml //增加或者更改

filebeat.prospectors:
- input_type: log 
  paths:
    - /var/log/messages
output.elasticsearch:
  hosts: ["192.168.137.30:9200"]

[root@linux-05 ~]# systemctl start  filebeat
[root@linux-05 ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      933/sshd            
tcp        0      0 192.168.137.45:27017    0.0.0.0:*               LISTEN      1061/mongod         
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      1061/mongod         
tcp6       0      0 :::3306                 :::*                    LISTEN      1454/mysqld         
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd           
tcp6       0      0 192.168.137.45:9200     :::*                    LISTEN      888/java            
tcp6       0      0 192.168.137.45:9300     :::*                    LISTEN      888/java            
tcp6       0      0 :::22                   :::*                    LISTEN      933/sshd            
[root@linux-05 ~]# ps aux |grep filebeat
root       5123  0.0  1.2 277436 12324 ?        Ssl  15:45   0:00 /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebea
root       5140  0.0  0.0 112652   964 pts/3    R+   15:46   0:00 grep --color=auto filebeat
[root@linux-05 ~]#


拓展
x-pack 收费,免费  http://www.jianshu.com/p/a49d93212eca
https://www.elastic.co/subscriptions
Elastic stack演进  http://70data.net/1505.html
基于kafka和elasticsearch,linkedin构建实时日志分析系统 http://t.cn/RYffDoE  
使用redis http://blog.lishiming.net/?p=463
ELK+Filebeat+Kafka+ZooKeeper 构建海量日志分析平台  https://www.cnblogs.com/delgyd/p/elk.html
http://www.jianshu.com/p/d65aed756587

  

ELK测试

标签:package   number   arp   elastic   RKE   internet   ted   yml   json格式   

原文地址:https://www.cnblogs.com/zhaobin-diray/p/10390420.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!