使用k8s搭建一个三节点的zookeeper集群
创建zk-pv
首先通过nfs创建三个共享目录
1
| mkdir -p /data/zk/{zk01,zk02,zk03}
|
分别对应三节点zk集群中的三个pod的持久化目录，创建好目录之后编写yaml创建zk-pv.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
| apiVersion: v1
kind: PersistentVolume
metadata:
name: dfs-pv-zk01
namespace: middleware
labels:
app: zk
annotations:
volume.beta.kubernetes.io/storage-class: "jusda"
spec:
capacity:
storage: 50Gi
accessModes:
- ReadWriteMany
nfs:
server: 10.0.15.1
path: /jusdaglobal/zk/zk01
persistentVolumeReclaimPolicy: Retain
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: dfs-pv-zk02
namespace: middleware
labels:
app: zk
annotations:
volume.beta.kubernetes.io/storage-class: "jusda"
spec:
capacity:
storage: 50Gi
accessModes:
- ReadWriteMany
nfs:
server: 10.0.15.1
path: /jusdaglobal/zk/zk02
persistentVolumeReclaimPolicy: Retain
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: dfs-pv-zk03
namespace: middleware
labels:
app: zk
annotations:
volume.beta.kubernetes.io/storage-class: "jusda"
spec:
capacity:
storage: 50Gi
accessModes:
- ReadWriteMany
nfs:
server: 10.0.15.1
path: /jusdaglobal/zk/zk03
persistentVolumeReclaimPolicy: Retain
|
使用如下命令创建zk-pk
1
| kubectl create -f zk-pv.yaml
|
创建ZK集群
我们选择使用statefulset去部署zk集群的三节点,并且使用刚刚创建的pv作为存储设备。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
| apiVersion: v1
kind: Service
metadata:
name: zk-hs
namespace: middleware
labels:
app: zk
spec:
selector:
app: zk
clusterIP: None
ports:
- name: server
port: 2888
- name: leader-election
port: 3888
---
apiVersion: v1
kind: Service
metadata:
name: zk-cs
namespace: middleware
labels:
app: zk
spec:
selector:
app: zk
type: NodePort
ports:
- name: client
port: 2181
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: zk-pdb
namespace: middleware
spec:
selector:
matchLabels:
app: zk
maxUnavailable: 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zk
namespace: middleware
spec:
selector:
matchLabels:
app: zk # has to match .spec.template.metadata.labels
serviceName: "zk-hs"
replicas: 3 # by default is 1
updateStrategy:
type: RollingUpdate
podManagementPolicy: Parallel
template:
metadata:
labels:
app: zk # has to match .spec.selector.matchLabels
spec:
containers:
- name: zk
imagePullPolicy: IfNotPresent
image: 10.0.7.130/middleware/zookeeper:3.4.10
resources:
requests:
memory: "500Mi"
cpu: "0.5"
ports:
- containerPort: 2181
name: client
- containerPort: 2888
name: server
- containerPort: 3888
name: leader-election
command:
- sh
- -c
- "start-zookeeper \
--servers=3 \
--data_dir=/var/lib/zookeeper/data \
--data_log_dir=/var/lib/zookeeper/data/log \
--conf_dir=/opt/zookeeper/conf \
--client_port=2181 \
--election_port=3888 \
--server_port=2888 \
--tick_time=2000 \
--init_limit=10 \
--sync_limit=5 \
--heap=512M \
--max_client_cnxns=60 \
--snap_retain_count=3 \
--purge_interval=12 \
--max_session_timeout=40000 \
--min_session_timeout=4000 \
--log_level=INFO"
readinessProbe:
exec:
command:
- sh
- -c
- "zookeeper-ready 2181"
initialDelaySeconds: 10
timeoutSeconds: 5
livenessProbe:
exec:
command:
- sh
- -c
- "zookeeper-ready 2181"
initialDelaySeconds: 10
timeoutSeconds: 5
volumeMounts:
- name: dfs
mountPath: /var/lib/zookeeper
volumeClaimTemplates:
- metadata:
name: dfs
annotations:
volume.beta.kubernetes.io/storage-class: "jusda"
spec:
accessModes: [ "ReadWriteMany" ]
resources:
requests:
storage: 50Gi
|
使用如下命令创建statefulset
1
| kubectl apply -f zk.yaml
|
验证Zk集群是否启动成功
使用如下命令查看容器
1
| kubectl -n middleware get pods,svc
|
1
2
3
4
5
6
7
8
| NAME READY STATUS RESTARTS AGE
pod/zk-0 1/1 Running 0 15h
pod/zk-1 1/1 Running 0 15h
pod/zk-2 1/1 Running 0 15h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/zk-cs NodePort 10.108.131.44 <none> 2181:31792/TCP 15h
service/zk-hs ClusterIP None <none> 2888/TCP,3888/TCP 15h
|
使用如下命令进入容器
1
| kubectl exec -it zk-1 -n middleware /bin/sh
|
使用如下命令查看所有zk节点的状态
1
| for i in 0 1 2; do kubectl exec zk-$i -n middleware zkServer.sh status; done
|