2.kafka 分散式集群安裝

Kafka集群安裝
主節點h201，從節點h202、h203
1.安裝jdk1.8
[hadoop@h201 ~]$ /usr/jdk1.8.0_144/bin/java -version

2.安裝zookeeper
kafka集群需要 zookeeper支持
[hadoop@h201 ~]$ tar -zxvf zookeeper-3.4.5-cdh5.5.2.tar.gz

[hadoop@h201 zookeeper-3.4.5-cdh5.5.2]$ mkdir data
[hadoop@h201 zookeeper-3.4.5-cdh5.5.2]$ mkdir log

[hadoop@h201 conf]$ vi zoo.cfg
tickTime=2000
#心跳的時間間隔
initLimit=10
#初始化連接時最長能忍受多少個心跳時間間隔數
syncLimit=5
#Leader 與Follower 之間發送消息，請求和應答時間長度

dataDir=/home/hadoop/zookeeper-3.4.5-cdh5.5.2/data
dataLogDir=/home/hadoop/zookeeper-3.4.5-cdh5.5.2/data
clientPort=2181
server.1=192.168.8.201:2888:3888
server.2=192.168.8.202:2888:3888
server.3=192.168.8.203:2888:3888

[hadoop@h201 ~]$ scp -r zookeeper-3.4.5-cdh5.5.2 h202:/home/hadoop/
[hadoop@h201 ~]$ scp -r zookeeper-3.4.5-cdh5.5.2 h203:/home/hadoop/

[hadoop@h201 ~]$ echo "1" > zookeeper-3.4.5-cdh5.5.2/data/myid
[hadoop@h202 ~]$ echo "2" > zookeeper-3.4.5-cdh5.5.2/data/myid
[hadoop@h203 ~]$ echo "3" > zookeeper-3.4.5-cdh5.5.2/data/myid

3.kafka安裝 （集群安裝模式）
[hadoop@h201 ~]$tar -zxvf kafka_2.12-0.10.2.1.tgz
[hadoop@h201kafka_2.12-0.10.2.1]$ mkdir kafkalogs
[hadoop@h201kafka_2.12-0.10.2.1]$ cd config
[hadoop@h201 config]$ vi server.properties

#broker.id=0
#每台節點唯一
#zookeeper.connect=localhost:2181
#關閉kafka自帶的zk
#log.dirs=/tmp/kafka-logs

host.name=h201
#每節點的地址
log.dirs=/home/hadoop/kafka_2.12-0.10.2.1/kafkalogs
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
broker.id=1
port=9092
zookeeper.connect=h201:2181,h202:2181,h203:2181
########禁用原始的broker.id=0#########

[hadoop@h201 ~]$ scp -r kafka_2.12-0.10.2.1 h202:/home/hadoop/
[hadoop@h202 config]$ vi server.properties
修改
host.name=h202
broker.id=2

[hadoop@h201 ~]$ scp -r kafka_2.12-0.10.2.1 h203:/home/hadoop/
[hadoop@h203 config]$ vi server.properties
修改
host.name=h203
broker.id=3

啟動服務
[hadoop@h201 zookeeper-3.4.5-cdh5.5.2]$ bin/zkServer.sh start
[hadoop@h202 zookeeper-3.4.5-cdh5.5.2]$ bin/zkServer.sh start
[hadoop@h203 zookeeper-3.4.5-cdh5.5.2]$ bin/zkServer.sh start

[hadoop@h201kafka_2.12-0.10.2.1]$ bin/kafka-server-start.sh config/server.properties &
[hadoop@h202kafka_2.12-0.10.2.1]$ bin/kafka-server-start.sh config/server.properties &
[hadoop@h203kafka_2.12-0.10.2.1]$ bin/kafka-server-start.sh config/server.properties &

[hadoop@h201kafka_2.12-0.10.2.1]$jps