附013.Kubernetes永久存儲Rook部署

一 Rook概述

1.1 Ceph簡介

1.2 Rook簡介

• Rook 提供了捲插件，來擴展了 K8S 的存儲系統，使用 Kubelet 代理程式 Pod 可以掛載 Rook 管理的塊設備和文件系統。
• Rook Operator 負責啟動並監控整個底層存儲系統，例如 Ceph Pod、Ceph OSD 等，同時它還管理 CRD、對象存儲、文件系統。
• Rook Agent 代理部署在 K8S 每個節點上以 Pod 容器運行，每個代理 Pod 都配置一個 Flexvolume 驅動，該驅動主要用來跟 K8S 的捲控制框架集成起來，每個節點上的相關的操作，例如添加存儲設備、掛載、格式化、刪除存儲等操作，都有該代理來完成。

1.3 Rook架構

二 Rook部署

2.1 前期規劃

2.2 獲取YAML

2.3 部署Rook Operator

2.4 配置cluster

  1 apiVersion: ceph.rook.io/v1
  2 kind: CephCluster
  3 metadata:
  4   name: rook-ceph
  5   namespace: rook-ceph
  6 spec:
  7   cephVersion:
  8     image: ceph/ceph:v14.2.4-20190917
  9     allowUnsupported: false
 10   dataDirHostPath: /var/lib/rook
 11   skipUpgradeChecks: false
 12   mon:
 13     count: 3
 14     allowMultiplePerNode: false
 15   dashboard:
 16     enabled: true
 17     ssl: true
 18   monitoring:
 19     enabled: false
 20     rulesNamespace: rook-ceph
 21   network:
 22     hostNetwork: false
 23   rbdMirroring:
 24     workers: 0
 25   placement:				#配置特定節點親和力保證Node作為存儲節點
 26 #    all:
 27 #      nodeAffinity:
 28 #        requiredDuringSchedulingIgnoredDuringExecution:
 29 #          nodeSelectorTerms:
 30 #          - matchExpressions:
 31 #            - key: role
 32 #              operator: In
 33 #              values:
 34 #              - storage-node
 35 #      tolerations:
 36 #      - key: storage-node
 37 #        operator: Exists
 38     mon:
 39       nodeAffinity:
 40         requiredDuringSchedulingIgnoredDuringExecution:
 41           nodeSelectorTerms:
 42           - matchExpressions:
 43             - key: ceph-mon
 44               operator: In
 45               values:
 46               - enabled
 47       tolerations:
 48       - key: ceph-mon
 49         operator: Exists
 50     ods:
 51       nodeAffinity:
 52         requiredDuringSchedulingIgnoredDuringExecution:
 53           nodeSelectorTerms:
 54           - matchExpressions:
 55             - key: ceph-osd
 56               operator: In
 57               values:
 58               - enabled
 59       tolerations:
 60       - key: ceph-osd
 61         operator: Exists
 62     mgr:
 63       nodeAffinity:
 64         requiredDuringSchedulingIgnoredDuringExecution:
 65           nodeSelectorTerms:
 66           - matchExpressions:
 67             - key: ceph-mgr
 68               operator: In
 69               values:
 70               - enabled
 71       tolerations:
 72       - key: ceph-mgr
 73         operator: Exists
 74   annotations:
 75   resources:
 76   removeOSDsIfOutAndSafeToRemove: false
 77   storage:
 78     useAllNodes: false			#關閉使用所有Node
 79     useAllDevices: false		#關閉使用所有設備
 80     deviceFilter: sdb
 81     config:
 82         metadataDevice:
 83         databaseSizeMB: "1024"
 84         journalSizeMB: "1024"
 85     nodes:
 86     - name: "k8snode01"			#指定存儲節點主機
 87       config:
 88         storeType: bluestore	#指定類型為裸磁碟
 89       devices:
 90       - name: "sdb"			    #指定磁碟為sdb
 91     - name: "k8snode02"
 92       config:
 93         storeType: bluestore
 94       devices:
 95       - name: "sdb"
 96     - name: "k8snode03"
 97       config:
 98         storeType: bluestore
 99       devices:
100       - name: "sdb"
101   disruptionManagement:
102     managePodBudgets: false
103     osdMaintenanceTimeout: 30
104     manageMachineDisruptionBudgets: false
105     machineDisruptionBudgetNamespace: openshift-machine-api

2.5 獲取鏡像

2.6 部署cluster

2.7 部署Toolbox

2.8 測試Rook

2.10 複製key和config