hyman-海曼工具

一些简单的记录

0%

在k8s上通过helm与动态NFS部署kafka、zookeeper

发表于 更新于 分类于

helm仓库地址https://github.com/helm/charts

添加helm仓库

1
2
3
4
5
6
7
8
9
10
谷歌仓库
helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator

由于国内网络缘故，可添加阿里云，但是zookeeper版本为v2，谷歌为v3
helm repo add incubator https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/charts-incubator/

这里采用谷歌仓库的kafka
[root@public03 ~]# helm search kafka
NAME                     	CHART VERSION	APP VERSION	DESCRIPTION                                                 
incubator/kafka          	0.9.6        	4.1.2      	Apache Kafka is publish-subscribe messaging rethought as ...

修改kafa相关配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#下载
hlem fetch incubator/kafka
#解压
tar zxf kafka-0.9.6.tgz && cd kafka
#修改文件zookeeper
charts/zookeeper/templates/statefulset.yaml
#zookeeper的镜像地址改为registry.cn-hangzhou.aliyuncs.com/appstore/k8szk:v3
最后创建pvc模板修改为以下：
volumeClaimTemplates:
    - metadata:
        name: data
        annotations:
          volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 1Gi

#修改文件kafka
templates/statefulset.yaml
最后创建pvc模板修改为以下：
volumeClaimTemplates:
    - metadata:
        name: data
        annotations:
          volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 1Gi

storageclass查询命令： kubectl get sc --all-namespaces

安装kafka

1
2
3
helm install ./kafka -n kafka --namespace xxx
#查看pvc状态
kubectl get pvc -n xxx

验证

创建测试pod

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: v1
kind: Pod
metadata:
  name: testclient
  namespace: default
spec:
  containers:
  - name: kafka
    image: confluentinc/cp-kafka:4.1.2-2
    command:
    - sh
    - -c
    - "exec tail -f /dev/null"

kubectl apply -f testclient.yaml

列出topics

1
2
3
4
kubectl -n default exec testclient -- /usr/bin/kafka-topics --zookeeper kafka-zookeeper:2181 --list

[root@public03 ~]# kubectl -n default exec testclient -- /usr/bin/kafka-topics --zookeeper kafka-zookeeper:2181 --list
__confluent.support.metrics

创建topic

1
2
3
4
kubectl -n default exec testclient -- /usr/bin/kafka-topics --zookeeper kafka-zookeeper:2181 --topic test1 --create --partitions 1 --replication-factor 1

[root@public03 ~]# kubectl -n default exec testclient -- /usr/bin/kafka-topics --zookeeper kafka-zookeeper:2181 --topic test1 --create --partitions 1 --replication-factor 1
Created topic "test1".

在一个topic监听messages

1
2
3
4
kubectl -n default exec -ti testclient -- /usr/bin/kafka-console-consumer --bootstrap-server kafka:9092 --topic test1 --from-beginning

#获取服务地址
kubectl get svc

在另外一个终端启动交互式消息生产者会话

1
2
3
4
kubectl -n default exec -ti testclient -- /usr/bin/kafka-console-producer --broker-list kafka-headless:9092 --topic test1

#获取服务地址
kubectl get svc