Hbase安裝模式介紹 單機模式 1> Hbase不使用HDFS,僅使用本地文件系統 2> ZooKeeper與Hbase運行在同一個JVM中 分散式模式– 偽分散式模式1> 所有進程運行在同一個節點上,不同進程運行在不同的JVM當中2> 比較適合實驗測試– 完全分散式模式1> 進程運行在多個伺服器 ...
Hbase安裝模式介紹
單機模式
1> Hbase不使用HDFS,僅使用本地文件系統
2> ZooKeeper與Hbase運行在同一個JVM中
分散式模式
– 偽分散式模式
1> 所有進程運行在同一個節點上,不同進程運行在不同的JVM當中
2> 比較適合實驗測試
– 完全分散式模式
1> 進程運行在多個伺服器集群中
2> 分散式依賴於HDFS系統,因此佈署Hbase之前一定要有一個正常工作的HDFS集群
Linux環境準備
關閉防火牆和SELinux
# service iptables stop
# chkconfig iptables off
# vim /etc/sysconfig/selinux
SELINUX=disabled
配置主機名及主機名綁定
# vim /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=hbase
# vim /etc/hosts
192.168.244.30 hbase
SSH免密碼登錄
# ssh-keygen
一直按Enter鍵
# ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
安裝jdk
# tar xvf jdk-7u79-linux-x64.tar.gz -C /usr/local/
# vim /etc/profile
export JAVA_HOME=/usr/local/jdk1.7.0_79 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
# source /etc/profile
Hadoop的安裝部署
下載地址:http://hadoop.apache.org/releases.html
在這裡,選擇2.5.2版本
# wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.5.2/hadoop-2.5.2.tar.gz
# tar xvf hadoop-2.5.2.tar.gz -C /usr/local/
# cd /usr/local/hadoop-2.5.2/
# vim etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/local/jdk1.7.0_79
配置HDFS
# mkdir /usr/local/hadoop-2.5.2/data/
# vim etc/hadoop/core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://192.168.244.30:8020</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/usr/local/hadoop-2.5.2/data/tmp</value> </property> </configuration>
# vim etc/hadoop/hdfs-site.xml
<configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration>
配置YARN
# mv etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml
# vim etc/hadoop/mapred-site.xml
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
# vim etc/hadoop/yarn-site.xml
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration>
啟動HDFS
初始化文件系統
# bin/hdfs namenode -format
...
16/09/25 20:33:02 INFO common.Storage: Storage directory /tmp/hadoop-root/dfs/name has been successfully formatted.
...
輸出上述信息即代表文件系統初始化成功
啟動NameNod和DataNode進程
# sbin/start-dfs.sh
16/10/09 19:35:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes wh ere applicableStarting namenodes on [hbase] The authenticity of host 'hbase (192.168.244.30)' can't be established. RSA key fingerprint is 1a:12:f5:e3:5d:e1:2c:5c:8c:56:52:ba:42:1c:ac:ba. Are you sure you want to continue connecting (yes/no)? yes hbase: Warning: Permanently added 'hbase' (RSA) to the list of known hosts. hbase: starting namenode, logging to /usr/local/hadoop-2.5.2/logs/hadoop-root-namenode-hbase.out The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is 1a:12:f5:e3:5d:e1:2c:5c:8c:56:52:ba:42:1c:ac:ba. Are you sure you want to continue connecting (yes/no)? yes localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts. localhost: starting datanode, logging to /usr/local/hadoop-2.5.2/logs/hadoop-root-datanode-hbase.out Starting secondary namenodes [0.0.0.0] The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established. RSA key fingerprint is 1a:12:f5:e3:5d:e1:2c:5c:8c:56:52:ba:42:1c:ac:ba. Are you sure you want to continue connecting (yes/no)? yes 0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts. 0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.5.2/logs/hadoop-root-secondarynamenode-hbase.out 16/10/09 19:35:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes wh ere applicable
啟動YARN
# sbin/start-yarn.sh
starting yarn daemons starting resourcemanager, logging to /usr/local/hadoop-2.5.2/logs/yarn-root-resourcemanager-hbase.out localhost: starting nodemanager, logging to /usr/local/hadoop-2.5.2/logs/yarn-root-nodemanager-hbase.out
通過jps查看各進程是否啟動成功
# jps
2421 NodeManager 2339 ResourceManager 1924 NameNode 2029 DataNode 2170 SecondaryNameNode 2721 Jps
也可通過訪問http://192.168.244.30:50070/查看hdfs是否啟動成功
Hbase的安裝部署
下載地址:http://mirror.bit.edu.cn/apache/hbase/1.2.3/hbase-1.2.3-bin.tar.gz
在這裡,下載的是1.2.3版本
關於hbase和hadoop的版本對應信息,可參考官檔的說明
http://hbase.apache.org/book/configuration.html#basic.prerequisites
# wget http://mirror.bit.edu.cn/apache/hbase/1.2.3/hbase-1.2.3-bin.tar.gz
# tar xvf hbase-1.2.3-bin.tar.gz -C /usr/local/
# cd /usr/local/hbase-1.2.3/
# vim conf/hbase-env.sh
export JAVA_HOME=/usr/local/jdk1.7.0_79
配置Hbase
# mkdir /usr/local/hbase-1.2.3/data
# vim conf/hbase-site.xml
<configuration> <property> <name>hbase.rootdir</name> <value>hdfs://192.168.244.30:8020/hbase</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/usr/local/hbase-1.2.3/data/zookeeper</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> </configuration>
# vim conf/regionservers
192.168.244.30
啟動Hbase
# bin/hbase-daemon.sh start zookeeper
starting zookeeper, logging to /usr/local/hbase-1.2.3/bin/../logs/hbase-root-zookeeper-hbase.out
# bin/hbase-daemon.sh start master
starting master, logging to /usr/local/hbase-1.2.3/bin/../logs/hbase-root-master-hbase.out
# bin/hbase-daemon.sh start regionserver
starting regionserver, logging to /usr/local/hbase-1.2.3/bin/../logs/hbase-root-regionserver-hbase.out
通過jps查看新增的java進程
# jps
2421 NodeManager 2975 HQuorumPeer 3302 HRegionServer 3051 HMaster 2339 ResourceManager 1924 NameNode 2029 DataNode 2170 SecondaryNameNode 3332 Jps
可以看出,新增了HQuorumPeer,HRegionServer和HMaster三個進程。
通過http://192.168.244.30:16030/訪問Hbase的web頁面
至此,Hbase的偽分散式集群搭建完畢~
參考
1. http://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-common/SingleCluster.html
2. http://hbase.apache.org/book.html#quickstart