zookeeper的安裝分為三種模式:單機模式、集群模式和偽集群模式。 1、單機模式 首先,從Apache官網下載一個Zookeeper穩定版本,本次教程採用的是zookeeper-3.4.9版本。 http://apache.fayea.com/zookeeper/zookeeper-3.4.9/ ...
zookeeper的安裝分為三種模式:單機模式、集群模式和偽集群模式。
1、單機模式
首先,從Apache官網下載一個Zookeeper穩定版本,本次教程採用的是zookeeper-3.4.9版本。
http:
//apache.fayea.com/zookeeper/zookeeper-3.4.9/
然後解壓zookeeper-3.4.9.tar.gz文件到安裝目錄下:
tar -zxvf zookeepre-3.4.9.tar.gz
zookeeper要求Java運行環境,並且需要jdk版本1.6以上。為了以後的操作方便,可以對zookeeper的環境變數進行配置(該步驟可忽略)。方法如下,在/etc/profile文件中加入以下內容:
#Set Zookeeper Environment
export ZOOKEEPER_HOME=/root/zookeeper-3.4.9
export PATH=$ZOOKEEPER_HOME/bin;$ZOOKEEPER_HOME/conf
Zookeeper伺服器包含在單個jar文件中(本環境下為 zookeeper-3.4.9.jar),安裝此服務需要用戶自己創建一個配置文件。預設配置文件路徑為 Zookeeper-3.4.9/conf/目錄下,文件名為zoo.cfg。進入conf/目錄下可以看到一個zoo_sample.cfg文件,可供參考。通過以下代碼在conf目錄下創建zoo.cfg文件:
gedit zoo.cfg
在文件中輸入以下內容並保存:
tickTime=2000dataDir=/home/jxwch/hadoop/data/zookeeper
dataLogDir=/home/jxwch/hadoop/dataLog/zookeeper
clientPort=2181
在這個文件中,各個語句的含義:
tickTime : 伺服器與客戶端之間交互的基本時間單元(ms)
dataDir : 保存zookeeper數據路徑
dataLogDir : 保存zookeeper日誌路徑,當此配置不存在時預設路徑與dataDir一致
clientPort : 客戶端訪問zookeeper時經過伺服器端時的埠號
使用單機模式時需要註意,在這種配置方式下,如果zookeeper伺服器出現故障,zookeeper服務將會停止。
2、集群模式
zookeeper最主要的應用場景是集群,下麵介紹如何在一個集群上部署一個zookeeper。只要集群上的大多數zookeeper服務啟動了,那麼總的zookeeper服務便是可用的。另外,最好使用奇數台伺服器。如歌zookeeper擁有5台伺服器,那麼在最多2台伺服器出現故障後,整個服務還可以正常使用。
之後的操作和單機模式的安裝類似,我們同樣需要Java環境,下載最新版的zookeeper並配置相應的環境變數。不同之處在於每台機器上的conf/zoo.cfg配置文件的參數設置不同,用戶可以參考下麵的配置:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/jxwch/server1/data
dataLogDir=/home/jxwch/server1/dataLog
clientPort=2181
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
maxClientCnxns=60
在這個配置文件中,新出現的語句的含義:
initLimit : 此配置表示允許follower連接並同步到leader的初始化時間,它以tickTime的倍數來表示。當超過設置倍數的tickTime時間,則連接失敗。
syncLimit : Leader伺服器與follower伺服器之間信息同步允許的最大時間間隔,如果超過次間隔,預設follower伺服器與leader伺服器之間斷開鏈接。
maxClientCnxns : 限制連接到zookeeper伺服器客戶端的數量
server.id=host:port:port : 表示了不同的zookeeper伺服器的自身標識,作為集群的一部分,每一臺伺服器應該知道其他伺服器的信息。用戶可以從“server.id=host:port:port” 中讀取到相關信息。在伺服器的data(dataDir參數所指定的目錄)下創建一個文件名為myid的文件,這個文件的內容只有一行,指定的是自身的id值。比如,伺服器“1”應該在myid文件中寫入“1”。這個id必須在集群環境中伺服器標識中是唯一的,且大小在1~255之間。這一樣配置中,zoo1代表第一臺伺服器的IP地址。第一個埠號(port)是從follower連接到leader機器的埠,第二個埠是用來進行leader選舉時所用的埠。所以,在集群配置過程中有三個非常重要的埠:clientPort:2181、port:2888、port:3888。
3、偽集群模式
偽集群模式就是在單機環境下模擬集群的Zookeeper服務。
在zookeeper集群配置文件中,clientPort參數用來設置客戶端連接zookeeper伺服器的埠。server.1=IP1:2888:3888中,IP1指的是組成Zookeeper伺服器的IP地址,2888為組成zookeeper伺服器之間的通信埠,3888為用來選舉leader的埠。由於偽集群模式中,我們使用的是同一臺伺服器,也就是說,需要在單台機器上運行多個zookeeper實例,所以我們必須要保證多個zookeeper實例的配置文件的client埠不能衝突。
下麵簡單介紹一下如何在單台機器上建立偽集群模式。首先將zookeeper-3.4.9.tar.gz分別解壓到server1,server2,server3目錄下:
tar -zxvf zookeeper-3.4.9.tar.gz /home/jxwch/server1
tar -zxvf zookeeper-3.4.9.tar.gz /home/jxwch/server2
tar -zxvf zookeeper-3.4.9.tar.gz /home/jxwch/server3
然後在server1/data/目錄下創建文件myid文件並寫入“1”,同樣在server2/data/,目錄下創建文件myid並寫入“2”,server3進行同樣的操作。
下麵分別展示在server1/conf/、server2/conf/、server3/conf/目錄下的zoo.cfg文件:
server1/conf/zoo.cfg文件
# Server 1
# The number of milliseconds of each tick
# 伺服器與客戶端之間交互的基本時間單元(ms)
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
# 此配置表示允許follower連接並同步到leader的初始化時間,它以tickTime的倍數來表示。當超過設置倍數的tickTime時間,則連接失敗。
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
# Leader伺服器與follower伺服器之間信息同步允許的最大時間間隔,如果超過次間隔,預設follower伺服器與leader伺服器之間斷開鏈接
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# 保存zookeeper數據,日誌路徑
dataDir=/home/jxwch/server1/data
dataLogDir=/home/jxwch/server1/dataLog
# the port at which the clients will connect
# 客戶端與zookeeper相互交互的埠
clientPort=2181
server.1= 127.0.0.1:2888:3888
server.2= 127.0.0.1:2889:3889
server.3= 127.0.0.1:2890:3890
#server.A=B:C:D 其中A是一個數字,代表這是第幾號伺服器;B是伺服器的IP地址;C表示伺服器與群集中的“領導者”交換信息的埠;當領導者失效後,D表示用來執行選舉時伺服器相互通信的埠。
# the maximum number of client connections.
# increase this if you need to handle more clients
# 限制連接到zookeeper伺服器客戶端的數量
maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server2/conf/zoo.cfg文件:
# Server 2
# The number of milliseconds of each tick
# 伺服器與客戶端之間交互的基本時間單元(ms)
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
# zookeeper所能接受的客戶端數量
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
# 伺服器與客戶端之間請求和應答的時間間隔
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# 保存zookeeper數據,日誌路徑
dataDir=/home/jxwch/server2/data
dataLogDir=/home/jxwch/server2/dataLog
# the port at which the clients will connect
# 客戶端與zookeeper相互交互的埠
clientPort=2182
server.1= 127.0.0.1:2888:3888
server.2= 127.0.0.1:2889:3889
server.3= 127.0.0.1:2890:3890
#server.A=B:C:D 其中A是一個數字,代表這是第幾號伺服器;B是伺服器的IP地址;C表示伺服器與群集中的“領導者”交換信息的埠;當領導者失效後,D表示用來執行選舉時伺服器相互通信的埠。
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server3/conf/zoo.cfg文件:
# Server 3
# The number of milliseconds of each tick
# 伺服器與客戶端之間交互的基本時間單元(ms)
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
# zookeeper所能接受的客戶端數量
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
# 伺服器與客戶端之間請求和應答的時間間隔
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# 保存zookeeper數據,日誌路徑
dataDir=/home/jxwch/server3/data
dataLogDir=/home/jxwch/server3/dataLog
# the port at which the clients will connect
# 客戶端與zookeeper相互交互的埠
clientPort=2183
server.1= 127.0.0.1:2888:3888
server.2= 127.0.0.1:2889:3889
server.3= 127.0.0.1:2890:3890
#server.A=B:C:D 其中A是一個數字,代表這是第幾號伺服器;B是伺服器的IP地址;C表示伺服器與群集中的“領導者”交換信息的埠;當領導者失效後,D表示用來執行選舉時伺服器相互通信的埠。
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
從上述三個代碼清單可以發現,除了clientPort不同之外,dataDir和dataLogDir也不同。另外,不要忘記dataDir所對應的目錄中創建的myid文件來指定對應的zookeeper伺服器實例。
4、Zookeeper偽集群模式運行
當上述偽集群環境安裝成功後就可以測試是否安裝成功啦,我們可以嘗嘗鮮:
首先啟動server1伺服器:
cd server1/bin
bash zkServer.sh start
此時出現以下提示信息,表示啟動成功:
打開zookeeper.out文件:
2017-02-23 16:17:46,419 [myid:] - INFO [main:QuorumPeerConfig@124] - Reading configuration from: /home/jxwch/server1/bin/../conf/zoo.cfg
2017-02-23 16:17:46,496 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2017-02-23 16:17:46,496 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2017-02-23 16:17:46,497 [myid:] - INFO [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2017-02-23 16:17:46,497 [myid:] - INFO [main:QuorumPeerConfig@352] - Defaulting to majority quorums
2017-02-23 16:17:46,511 [myid:1] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2017-02-23 16:17:46,511 [myid:1] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2017-02-23 16:17:46,511 [myid:1] - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2017-02-23 16:17:46,525 [myid:1] - INFO [main:QuorumPeerMain@127] - Starting quorum peer
2017-02-23 16:17:46,631 [myid:1] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
2017-02-23 16:17:46,664 [myid:1] - INFO [main:QuorumPeer@1019] - tickTime set to 2000
2017-02-23 16:17:46,664 [myid:1] - INFO [main:QuorumPeer@1039] - minSessionTimeout set to -1
2017-02-23 16:17:46,664 [myid:1] - INFO [main:QuorumPeer@1050] - maxSessionTimeout set to -1
2017-02-23 16:17:46,665 [myid:1] - INFO [main:QuorumPeer@1065] - initLimit set to 10
2017-02-23 16:17:46,771 [myid:1] - INFO [main:FileSnap@83] - Reading snapshot /home/jxwch/server1/data/version-2/snapshot.400000015
2017-02-23 16:17:46,897 [myid:1] - INFO [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:3888
2017-02-23 16:17:46,913 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@774] - LOOKING
2017-02-23 16:17:46,915 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@818] - New election. My id = 1, proposed zxid=0x500000026
2017-02-23 16:17:46,922 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x500000026 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x5 (n.peerEpoch) LOOKING (my state)
2017-02-23 16:17:46,940 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 2 at election address /127.0.0.1:3889
java.net.ConnectException: 拒絕連接
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
at java.lang.Thread.run(Thread.java:745)
2017-02-23 16:17:46,943 [myid:1] - INFO [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2017-02-23 16:17:46,944 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 3 at election address /127.0.0.1:3890
java.net.ConnectException: 拒絕連接
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
at java.lang.Thread.run(Thread.java:745)
2017-02-23 16:17:46,944 [myid:1] - INFO [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
產生上述兩條Waring信息是因為zookeeper服務的每個實例都擁有全局的配置信息,他們在啟動的時候需要隨時隨地的進行leader選舉,此時server1就需要和其他兩個zookeeper實例進行通信,但是,另外兩個zookeeper實例還沒有啟動起來,因此將會產生上述所示的提示信息。當我們用同樣的方式啟動server2和server3後就不會再有這樣的警告信息了。
當三台伺服器均成功啟動後切換至server1/bin/目錄下執行以下命令:
bash zkServer.sh status
終端出現以下提示信息:
說明server1伺服器此時處於follower模式,同樣切換至server2/bin目錄下執行相同的命令,返回如下結果:
說明server2被選舉為leader。
轉自:https://www.cnblogs.com/jxwch/p/6433310.html