當在使用greenplum過程中有不當的操作時，可能會出現segment節點宕掉的情況（比如在greenplum運行的過程中停掉其中幾台segment節點的伺服器），通過下麵的方法可以恢復segment。 下麵是現場出現的故障情況： 可以看到有6個節點Failed，有2個節點的Primary和Mir ...
當在使用greenplum過程中有不當的操作時，可能會出現segment節點宕掉的情況（比如在greenplum運行的過程中停掉其中幾台segment節點的伺服器），通過下麵的方法可以恢復segment。
下麵是現場出現的故障情況：
[gpadmin@tj-soc-c04-csfb1 ~]$ gpstate -m
20161010:16:35:54:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-Starting gpstate with args: -m
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1'
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28'
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-Obtaining Segment details from master...
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--Current GPDB mirror list and status
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--Type = Group
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb2 /data1/gpdata/mirror/gpseg0 41000 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb2 /data1/gpdata/mirror/gpseg1 41001 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb3 /data1/gpdata/mirror/gpseg2 41000 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb3 /data1/gpdata/mirror/gpseg3 41001 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:- tj-soc-c04-csfb4 /data1/gpdata/mirror/gpseg4 41000 Acting as Primary Change Tracking
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:- tj-soc-c04-csfb4 /data1/gpdata/mirror/gpseg5 41001 Acting as Primary Change Tracking
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb1 /data1/gpdata/mirror/gpseg6 41000 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb1 /data1/gpdata/mirror/gpseg7 41001 Failed <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-2 segment(s) configured as mirror(s) are acting as primaries
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-6 segment(s) configured as mirror(s) have failed
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-2 mirror segment(s) acting as primaries are in change tracking
可以看到有6個節點Failed，有2個節點的Primary和Mirror交換了。
一、首先需要停掉GP
gpstop -M fast -a 這樣會告訴你有幾個節點DOWN了
二、啟動GP
gpstart 啟動資料庫會忽略DOWN的節點
三、生成一個恢復配置文件
gprecoverseg -o ./recov 會在當前目錄生成一個recov文件，裡面包含了要恢復的節點信息
recov文件內容如下：（註意：這個文件不是手動創建的，而是通過gprecoverseg -o ./recov命令生成的
filespaceOrder=
tj-soc-c04-csfb2:41000:/data1/gpdata/mirror/gpseg0
tj-soc-c04-csfb2:41001:/data1/gpdata/mirror/gpseg1
tj-soc-c04-csfb3:41000:/data1/gpdata/mirror/gpseg2
tj-soc-c04-csfb3:41001:/data1/gpdata/mirror/gpseg3
tj-soc-c04-csfb1:41000:/data1/gpdata/mirror/gpseg6
tj-soc-c04-csfb1:41001:/data1/gpdata/mirror/gpseg7
四、使用恢復配置文件恢復節點
$gprecoverseg -i ./recov
恢復過程中可以用gpstate -m 查看恢復狀態：Resynchronizing（表示正在恢復中），Synchronized（表示恢復完畢）
五、調整Primary和Mirror
上面的情況中有Primary和Mirror兌換的情況，所以需要把他們換回來，可以用下麵的命令
gprecoverseg -r
等待所有的節點都是Synchronized後，segment就恢復好了
