ELK安装笔记

3421eeee

1、安装JDK

1 2 3 4 5 6 7 8 9 10 11 12

rpm -ivh jdk-8u101-linux-x64.rpm Preparing...                ########################################### [100%]    1:jdk1.8.0_101           ########################################### [100%] Unpacking JAR files...     tools.jar...     plugin.jar...     javaws.jar...     deploy.jar...     rt.jar...     jsse.jar...     charsets.jar...     localedata.jar...

检测java版本

1 2 3 4

java -version java version "1.8.0_101" Java(TM) SE Runtime Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

2、安装redis

1 2 3 4 5 6 7 8 9

yum install -y tcl gcc mkdir /usr/local/redis tar zxvf redis-2.8.20.tar.gz \cp -rf redis-2.8.20/* /usr/local/redis/ cd /usr/local/redis make MALLOC=libc make install cd utils/ ./install_server.sh   #所有选项默认

查看redis监控端口

1 2 3

netstat -tnlp |grep redis tcp        0      0 0.0.0.0:6379                0.0.0.0:*                   LISTEN      1978/redis-server * tcp        0      0 :::6379                     :::*                        LISTEN      1978/redis-server *

3、安装logstansh

1 2 3 4 5

rpm -ivh  logstash-2.4.0.noarch.rpm Preparing...                ########################################### [100%]    1:logstash               ########################################### [100%] echo "PATH=$PATH:/opt/logstash/bin" >> /etc/profile source /etc/profile

测试logstash

1 2 3 4 5

logstash -e "input {stdin{}} output {stdout{}}" hello Settings: Default pipeline workers: 1 Pipeline main started 2016-09-18T09:10:32.369Z localhost.localdomain hello

3.1、测试redis缓存（分两个终端运行b/c两步）

a、新建logstash配置文件

1 2 3 4 5 6 7 8 9 10 11

mkdir /opt/logstash/conf vi output_redis.conf input { stdin { } }    #手动输入数据 output {                    stdout { codec => rubydebug }  #页面debug信息     redis {         host => '127.0.0.1'         data_type => 'list'         key => 'redis'     } }

b、查看redis是否缓存数据

1 2	cd /usr/local/redis-2.8.20/src/ redis-cli monitor

c、启动logstansh(重启一个终端)

1 2 3 4 5 6 7 8 9 10 11 12 13

logstash -f output_redis.conf --verbose hello starting agent {:level=>:info} starting pipeline {:id=>"main", :level=>:info} Settings: Default pipeline workers: 1 Starting pipeline {:id=>"main", :pipeline_workers=>1, :batch_size=>125, :batch_delay=>5, :max_inflight=>125, :level=>:info} Pipeline main started {        "message" => "hello",       "@version" => "1",     "@timestamp" => "2016-09-18T09:14:55.288Z",           "host" => "localhost.localdomain" }

d、测试成功

1 2 3

redis-cli monitor OK 1474190709.219548 [0 127.0.0.1:36399] "rpush" "redis" "{\"message\":\"hello\",\"@version\":\"1\",\"@timestamp\":\"2016-09-18T09:25:07.911Z\",\"host\":\"localhost.localdomain\"}"

四、安装elasticsearch
1、elasticsearch的安装

1 2 3 4 5 6 7 8 9 10

rpm -ivh elasticsearch-2.4.0.rpm warning: elasticsearch-2.4.0.rpm: Header V4 RSA/SHA1 Signature, key ID d88e42b4: NOKEY Preparing...                ########################################### [100%] Creating elasticsearch group... OK Creating elasticsearch user... OK    1:elasticsearch          ########################################### [100%] ### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using chkconfig sudo chkconfig --add elasticsearch ### You can start elasticsearch service by executing sudo service elasticsearch start

2、修改elasticsearch配置文件

1 2	vi /etc/elasticsearch/elasticsearch.yml network.host: 172.16.1.224

3、查看elasticsearch是否启动

1 2 3

netstat -tnlp |grep java tcp        0      0 ::ffff:172.16.1.224:9200    :::*                        LISTEN      1345/java            tcp        0      0 ::ffff:172.16.1.224:9300    :::*                        LISTEN      1345/java

4、测试logstansh和elasticsearch是否能结合使用
a.新建logstansh配置文件elasticsearch.conf  

1 2 3 4 5 6 7

cd /opt/logstash/conf/ vi elasticsearch.conf input { stdin {} }    #手动输入 output {     elasticsearch { hosts => "127.0.0.1" }        stdout { codec=> rubydebug }   #页面debug信息 }

b.启动elasticsearch.conf配置文件

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

logstash -f elasticsearch.conf --verbose hello starting agent {:level=>:info} starting pipeline {:id=>"main", :level=>:info} Settings: Default pipeline workers: 1 Using mapping template from {:path=>nil, :level=>:info} Attempting to install template {:manage_template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"string", "index"=>"not_analyzed"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"float"}, "longitude"=>{"type"=>"float"}}}}}}}, :level=>:info} New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["127.0.0.1"], :level=>:info} Starting pipeline {:id=>"main", :pipeline_workers=>1, :batch_size=>125, :batch_delay=>5, :max_inflight=>125, :level=>:info} Pipeline main started {        "message" => "hello",       "@version" => "1",     "@timestamp" => "2016-09-18T09:41:44.603Z",           "host" => "localhost.localdomain" }

c.查看elasticsearch是否获取到了"hello elasticsearch"

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

curl http://localhost:9200/_search?pretty {   "took" : 41,   "timed_out" : false,   "_shards" : {     "total" : 5,     "successful" : 5,     "failed" : 0   },   "hits" : {     "total" : 1,     "max_score" : 1.0,     "hits" : [ {       "_index" : "logstash-2016.09.18",       "_type" : "logs",       "_id" : "AVc8rFYwCkn6K6s_ltue",       "_score" : 1.0,       "_source" : {         "message" : "hello",         "@version" : "1",         "@timestamp" : "2016-09-18T09:41:44.603Z",         "host" : "localhost.localdomain"       }     } ]   } }

4、安装elasticsearch插件

elasticsearch有很多插件：http://www.searchtech.pro/elasticsearch-plugins

elasticsearch-head插件安装

1 2 3 4 5 6 7

./plugin install mobz/elasticsearch-head -> Installing mobz/elasticsearch-head... Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ... Downloading .........DONE Verifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ... NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify) Installed head into /usr/share/elasticsearch/plugins/head

5、查看elasticsearch-head插件显示的页面

http://172.16.1.224:9200/_plugin/head/

五、kibana安装
1、安装kibana

1 2 3

rpm -ivh kibana-4.6.1-x86_64.rpm Preparing...                ########################################### [100%]    1:kibana                 ########################################### [100%]

修改kibana配置文件，把下面这行改成elasticsearc的访问路径

1 2	vi /opt/kibana/config/kibana.yml elasticsearch.url: "http://172.16.1.224:9200"

2 启动kibana

1 2 3 4 5 6 7 8 9 10 11 12 13

/opt/kibana/bin/kibana& [2] 1441 [iyunv@localhost elk]#   log   [18:06:27.275] [info][status][plugin:kibana@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.324] [info][status][plugin:elasticsearch@1.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch   log   [18:06:27.387] [info][status][plugin:kbn_vislib_vis_types@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.400] [info][status][plugin:markdown_vis@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.407] [info][status][plugin:metric_vis@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.412] [info][status][plugin:spyModes@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.420] [info][status][plugin:statusPage@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.425] [info][status][plugin:table_vis@1.0.0] Status changed from uninitialized to green - Ready   log   [18:06:27.432] [info][listening] Server running at http://0.0.0.0:5601   log   [18:06:32.411] [info][status][plugin:elasticsearch@1.0.0] Status changed from yellow to yellow - No existing Kibana index found   log   [18:06:35.448] [info][status][plugin:elasticsearch@1.0.0] Status changed from yellow to green - Kibana index ready

3、测试kinaba
访问页面:http://172.16.1.224:5601/