參考博客：linux下ElasticSearch.6.2.2集群安裝與head、Kibana、X-Pack..插件的配置安裝參考博客：ELK5.5.1 插件安裝實踐紀要（head/bigdesk/kopf/cerebo/中文分詞插件）參考博客：ELK構建MySQL慢日誌收集平臺詳解參考博客：針 ...

1. 主機規劃

主機名稱	IP信息	內網IP	操作系統	安裝軟體	備註：運行程式
mini01	10.0.0.11	172.16.1.11	CentOS 7.4 2G記憶體	jdk、elasticsearch、kibana	插件head、bigdesk、cerebro
mini02	10.0.0.12	172.16.1.12	CentOS 7.4 4G記憶體	jdk、elasticsearch、logstash
mini03	10.0.0.13	172.16.1.13	CentOS 7.4 4G記憶體	jdk、elasticsearch、logstash
mini04	10.0.0.14	172.16.1.14	CentOS 7.4	jdk、logstash

添加hosts信息，保證每台都可以相互ping通

1 [root@mini01 ~]# cat /etc/hosts  
2 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
3 ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
4 
5 10.0.0.11    mini01
6 10.0.0.12    mini02
7 10.0.0.13    mini03
8 10.0.0.14    mini04
9 10.0.0.15    mini05

windows的hosts文件也追加如下信息

1 c:\windows\system32\drivers\etc
2 ########################################## 追加信息如下：
3 10.0.0.11    mini01
4 10.0.0.12    mini02
5 10.0.0.13    mini03
6 10.0.0.14    mini04
7 10.0.0.15    mini05

2. 添加用戶賬號

1 # 使用一個專門的用戶，避免直接使用root用戶
2 # 添加用戶、指定家目錄並指定用戶密碼
3 useradd -d /app yun && echo '123456' | /usr/bin/passwd --stdin yun
4 # sudo提權
5 echo "yun  ALL=(ALL)       NOPASSWD: ALL" >>  /etc/sudoers
6 # 讓其它普通用戶可以進入該目錄查看信息
7 chmod 755 /app/

3. Jdk【java8】

3.1. 軟體安裝

 1 [yun@mini01 software]# pwd
 2 /app/software
 3 [yun@mini01 software]# tar xf jdk1.8.0_112.tar.gz 
 4 [yun@mini01 software]# ll
 5 total 201392
 6 drwxr-xr-x 8   10  143      4096 Dec 20 13:27 jdk1.8.0_112
 7 -rw-r--r-- 1 root root 189815615 Mar 12 16:47 jdk1.8.0_112.tar.gz
 8 [yun@mini01 software]# mv jdk1.8.0_112/ /app/
 9 [yun@mini01 software]# cd /app/
10 [yun@mini01 app]# ll
11 total 8
12 drwxr-xr-x  8   10   143 4096 Dec 20 13:27 jdk1.8.0_112
13 [yun@mini01 app]# ln -s jdk1.8.0_112/ jdk
14 [yun@mini01 app]# ll
15 total 8
16 lrwxrwxrwx  1 root root    13 May 16 23:19 jdk -> jdk1.8.0_112/
17 drwxr-xr-x  8   10   143 4096 Dec 20 13:27 jdk1.8.0_112

3.2. 環境變數

 1 [root@mini01 ~]$ pwd
 2 /app
 3 [root@mini01 ~]$ ll -d jdk*  # 選擇jdk1.8  
 4 lrwxrwxrwx 1 yun yun   11 Mar 15 14:58 jdk -> jdk1.8.0_112
 5 drwxr-xr-x 8 yun yun 4096 Dec 20 13:27 jdk1.8.0_112
 6 [root@mini01 profile.d]$ pwd
 7 /etc/profile.d
 8 [root@mini01 profile.d]$ cat jdk.sh # java環境變數   
 9 export JAVA_HOME=/app/jdk
10 export JRE_HOME=/app/jdk/jre
11 export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATH
12 export PATH=$JAVA_HOME/bin:$PATH
13 
14 [root@mini01 profile.d]# source /etc/profile
15 [root@mini01 profile.d]$ java -version  
16 java version "1.8.0_112"
17 Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
18 Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

4. elasticsearch部署

4.1. 軟體部署

1 [yun@mini01 software]$ pwd
2 /app/software
3 [yun@mini01 software]$ tar xf elasticsearch-6.3.2.tar.gz 
4 [yun@mini01 software]$ mv elasticsearch-6.3.2 /app/
5 [yun@mini01 software]$ cd /app/
6 [yun@mini01 ~]$ ln -s elasticsearch-6.3.2/ elasticsearch

4.2. 環境變數

所有安裝es的機器都要有該配置

1 [root@mini01 profile.d]# pwd
2 /etc/profile.d
3 [root@mini01 profile.d]# cat es.sh 
4 export ES_HOME="/app/elasticsearch"
5 export PATH=$ES_HOME/bin:$PATH
6 
7 [root@mini01 profile.d]# logout
8 [yun@mini01 es-data]$ source /etc/profile  # 重加在環境變數

4.3. 配置修改

由於node.name使用了變數，所以所有集群的該配置都可以一樣

 1 [yun@mini01 config]$ pwd
 2 /app/elasticsearch/config
 3 [yun@mini01 config]$ vim elasticsearch.yml
 4 ………………
 5 # ---------------------------------- Cluster -----------------------------------
 6 #
 7 # Use a descriptive name for your cluster:
 8 #
 9 #cluster.name: my-application
10 # 集群名稱
11 cluster.name: zhang-es
12 #
13 # ------------------------------------ Node ------------------------------------
14 #
15 # Use a descriptive name for the node:
16 #
17 #node.name: node-1
18 # 節點名稱
19 node.name: ${HOSTNAME}
20 
21 #
22 # Add custom attributes to the node:
23 ………………
24 # Path to directory where to store the data (separate multiple locations by comma):
25 #
26 #path.data: /path/to/data
27 # 該目錄需要創建
28 path.data: /app/es-data
29 
30 #
31 # Path to log files:
32 #
33 #path.logs: /path/to/logs
34 # 該目錄需要創建
35 path.logs: /app/es-data/logs
36 ………………
37 # Lock the memory on startup:
38 #
39 # 鎖定記憶體
40 bootstrap.memory_lock: true
41 #
42 ………………
43 # Set the bind address to a specific IP (IPv4 or IPv6):
44 #
45 #network.host: 192.168.0.1
46 # 綁定
47 network.host: 0.0.0.0
48 
49 #
50 # Set a custom port for HTTP:
51 #
52 http.port: 9200
53 #
54 ………………
55 # Elasticsearch performs poorly when the system is swapping the memory.
56 #
57 #discovery.zen.ping.unicast.hosts: ["host1", "host2"]
58 # 集群節點ip或者主機
59 discovery.zen.ping.unicast.hosts: ["mini01", "mini02", "mini03"]
60 
61 #
62 # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
63 #
64 # 防止腦裂配置，註意在多master時，這個值應該等於 Math.floor(master候選節點數/2)+1
65 # 意思是master候選節點的數目最少達到多少個，才去選舉master
66 # 沒有這個配置,在多master時容易造成腦裂,出現多個集群
67 # 這裡只有一個master就設置成1即可  【預設為1】
68 #discovery.zen.minimum_master_nodes: 
69 ………………
70 #action.destructive_requires_name: true
71 
72 # 下麵兩行配置為haad和bigdesk插件配置，各伺服器一致
73 http.cors.enabled: true
74 http.cors.allow-origin: "*"

4.4. 啟動es程式

 1 # -d 後臺運行   -p 指定pid文件
 2 [yun@mini01 ~]$ elasticsearch -d -p /app/elasticsearch/es.pid  # 添加了環境變數，所以可以在任何地方啟動程式 
 3 [yun@mini01 ~]$ cat elasticsearch/es.pid 
 4 2637
 5 [yun@mini01 ~]$ netstat -lntup | grep '9200'
 6 (Not all processes could be identified, non-owned process info
 7  will not be shown, you would have to be root to see it all.)
 8 tcp6       0      0 :::9200                 :::*                    LISTEN      2637/java   
 9 [yun@mini01 ~]$ kill -9 2637   # 停止es
10 [yun@mini01 ~]$ ps -ef | grep 'ela'
11 yun        3263   1807  0 20:56 pts/0    00:00:00 grep --color=auto ela

4.5. 瀏覽器訪問

1 http://mini01:9200/    
2 http://mini02:9200/    
3 http://mini03:9200/

4.6. 啟動報錯與解決

1 # 啟動報錯如下：
2 [3] bootstrap checks failed
3 [1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]
4 [2]: memory locking requested for elasticsearch process but memory is not locked
5 [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

4.6.1. 解決1和2

 1 # 將 soft nofile和hard nofile從 65535 改為 131070，如下：
 2 # 添加soft memlock和hard memlock
 3 [yun@mini01 ~]$ tail /etc/security/limits.conf   # 需要退出重新登錄才生效
 4 #*               hard    rss             10000
 5 #@student        hard    nproc           20
 6 #@faculty        soft    nproc           20
 7 #@faculty        hard    nproc           50
 8 #ftp             hard    nproc           0
 9 #@student        -       maxlogins       4
10 
11 # End of file
12 * soft nofile 131070
13 * hard nofile 131070
14 * soft memlock unlimited
15 * hard memlock unlimited

4.6.2. 解決3

1 [root@mini01 ~]# vim /etc/sysctl.conf  # 追加如下信息
2 ………………
3 
4 vm.max_map_count=655360
5 [root@mini01 ~]# sysctl -p  # 生效

5. ES的XPack 過期新 License 更新

當我們直接訪問ES的索引，出現如下信息時，表示license已經過期，需要重新更新

其中涉及ES的賬號信息------elastic：是可以 build 這個項目的超級用戶，對應預設的密碼為：changeme

命令行訪問

 1 [yun@mini02 ~]$ curl -XGET -u elastic:changeme 'http://mini01:9200/_xpack/license'
 2 {
 3   "license" : {
 4     "status" : "expired",
 5     "uid" : "59bc0e32-685b-48a9-bfdb-ddd373f672ab",
 6     "type" : "trial",
 7     "issue_date" : "2018-06-03T08:56:33.376Z",
 8     "issue_date_in_millis" : 1528016193376,
 9     "expiry_date" : "2018-07-03T08:56:33.376Z",
10     "expiry_date_in_millis" : 1530608193376,
11     "max_nodes" : 1000,
12     "issued_to" : "zhang-es",
13     "issuer" : "elasticsearch",
14     "start_date_in_millis" : -1
15   }
16 }
17 [yun@mini02 ~]$ curl -XGET -u elastic:changeme 'http://mini01:9200/_cat/indices'
18 {"error":{"root_cause":[{"type":"security_exception","reason":"current license is non-compliant for [security]","license.expired.feature":"security"}],"type":"security_exception","reason":"current license is non-compliant for [security]","license.expired.feature":"security"},"status":403}

瀏覽器訪問

1 http://mini01:9200/_cat/indices

獲取License

1 https://register.elastic.co/marvel_register

需要你輸入你對應的郵箱等基本信息，隨便填寫，但是郵箱和國家必須是真實的，將對應新下載的 license 上傳到對應集群的某一臺伺服器上即可，我將我下載的一堆很長名字的 license 修改為簡單的名稱即：

mv xxxxx-license.json license.json

更新 license

我們更新最新的 license 不需要重啟相應的 ElasticSearch 節點，只需要通過一個命令就可以動態的生效

 1 [yun@mini02 ~]$ curl -XPUT -u elastic:changeme 'http://mini01:9200/_xpack/license?acknowledge=true' -H "Content-Type: application/json" -d @license.json 
 2 {"acknowledged":true,"license_status":"valid"}
 3 [yun@mini02 ~]$ curl -XGET -u elastic:changeme 'http://mini01:9200/_xpack/license'
 4 {
 5   "license" : {
 6     "status" : "active",
 7     "uid" : "aad141e1-c24b-453c-92d1-0fdf5ac63540",
 8     "type" : "basic",
 9     "issue_date" : "2018-09-07T00:00:00.000Z",
10     "issue_date_in_millis" : 1536278400000,
11     "expiry_date" : "2019-09-07T23:59:59.999Z",
12     "expiry_date_in_millis" : 1567900799999,
13     "max_nodes" : 100,
14     "issued_to" : "zhang lia (myself)",
15     "issuer" : "Web Form",
16     "start_date_in_millis" : 1536278400000
17   }
18 }

之後就可以正常訪問了

1 [yun@mini02 ~]$ curl http://mini01:9200/_cat/indices
2 green open logstash-2018.08.21         MoHGSrCBQgyYrA5PLcHePg 5 1     9 0  74.5kb  37.2kb
3 green open nginx-access-log-2018.08.25 TJRUOCELRPaNBLj_t943Ww 5 1   121 0 652.8kb 321.3kb
4 green open httpd-access-log-2018.08.31 21NENLdBTNu49oIg9bIlnw 5 1   573 0 739.6kb 409.6kb
5 green open index-demo                  cTz0lN39SmSQfOLAM89GRg 5 1     2 0  17.9kb   8.9kb
6 green open system-rsyslog-2018.08      zHmPivsQS72dtkQzVhIFBQ 5 1   154 0 605.9kb 302.9kb
7 ………………

ELK-elasticsearch-6.3.2部署