2023年6月30日发(作者：)

kubernetes（15）：k8s有状态服务StatefulSetK8s有状态服务StatefulSet

1 有状态服务RC、Deployment、DaemonSet都是⾯向⽆状态的服务，它们所管理的Pod的IP、名字，启停顺序等都是随机的，⽽StatefulSet是什么？顾名思义，有状态的集合，管理所有有状态的服务，⽐如MySQL、MongoDB集群等。MySQL，cache，这种组件要不要上K8S

2 StatefulSetStatefulSet本质上是Deployment的⼀种变体，在v1.9版本中已成为GA版本，它为了解决有状态服务的问题，它所管理的Pod拥有固定的Pod名称，启停顺序，在StatefulSet中，Pod名字称为⽹络标识(hostname)，还必须要⽤到共享存储。在Deployment中，与之对应的服务是service，⽽在StatefulSet中与之对应的headless service，headless service，即⽆头服务，与service的区别就是它没有Cluster IP，解析它的名称时将返回该Headless Service对应的全部Pod的Endpoint列表。除此之外，StatefulSet在Headless Service的基础上⼜为StatefulSet控制的每个Pod副本创建了⼀个DNS域名，这个域名的格式为：

$(podname).(headless server name)

FQDN： $(podname).(headless server name).

StatefulSet适⽤于具有以下特点的应⽤：具有固定的⽹络标记（主机名）具有持久化存储需要按顺序部署和扩展需要按顺序终⽌及删除需要按顺序滚动更新

3 StatefulSet⽰例3.1 创建storageclass

#nd: PersistentVolumeClaimapiVersion: v1metadata: name: test-claim annotations: /storage-class: "managed-nfs-storage"spec: accessModes: - ReadWriteMany resources: requests: storage: 1Mi

[root@k8s-master storageclass]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 24s[root@k8s-master storageclass]# kubectl apply -f rsistentvolumeclaim/test-claim unchanged[root@k8s-master storageclass]# kubectl get PROVISIONER AGEmanaged-nfs-storage /ifs 13m[root@k8s-master storageclass]# 3.2 创建Nginx-StatefulSet

apiVersion: v1kind: Servicemetadata: name: nginx labels: app: nginxspec: ports: - port: 80 name: web clusterIP: None selector: app: nginx---apiVersion: apps/v1kind: StatefulSetmetadata: name: webspec: selector: matchLabels: app: nginx serviceName: "nginx" replicas: 3 template: metadata: labels: app: nginx spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: nginx ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-nfs-storage" resources: requests: storage: 1Gi

[root@k8s-master statefulset]# kubectl create -f nginx_rvice/nginx /web created[root@k8s-master statefulset]#

3.3 查看sts/pod/pv/pvc/svc[root@k8s-master v1]# kubectl get stsNAME READY AGEweb 3/3 13m[root@k8s-master v1]#[root@k8s-master statefulset]# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEtest-claim Bound default-test-claim-pvc-0ece5a66-dab2-4a13-be51-1a2acdbc45eb 1Mi RWX managed-nfs-storage 24mwww-web-0 Bound default-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8 1Gi RWO managed-nfs-storage 66swww-web-1 Bound default-www-web-1-pvc-39123d42-6712-4a66-b7fe-beae48708aad 1Gi RWO managed-nfs-storage 55swww-web-2 Bound default-www-web-2-pvc-e395f225-ef4a-4645-a2a9-d139590346dc 1Gi RWO managed-nfs-storage 43s[root@k8s-master statefulset]# kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEdefault-test-claim-pvc-0ece5a66-dab2-4a13-be51-1a2acdbc45eb 1Mi RWX Delete Bound default/test-claim managed-nfs-storage 19mdefault-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8 1Gi RWO Delete Bound default/www-web-0 managed-nfs-storage 68sdefault-www-web-1-pvc-39123d42-6712-4a66-b7fe-beae48708aad 1Gi RWO Delete Bound default/www-web-1 managed-nfs-storage 57sdefault-www-web-2-pvc-e395f225-ef4a-4645-a2a9-d139590346dc 1Gi RWO Delete Bound default/www-web-2 managed-nfs-storage 45s[root@k8s-master statefulset]#[root@k8s-master statefulset]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 21mweb-0 1/1 Running 0 2m45sweb-1 1/1 Running 0 2m34sweb-2 1/1 Running 0 2m22s

pod创建过程从web-0到web-2顺序创建StatefulSet创建顺序是从0到N-1，终⽌顺序则是相反。如果需要对StatefulSet扩容，则之前的N个Pod必须已经存在。如果要终⽌⼀个Pod，则它的后序Pod必须全部终⽌。

[root@k8s-master statefulset]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 18mweb-0 1/1 Running 0 18sweb-1 0/1 ContainerCreating 0 7s[root@k8s-master statefulset]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 18mweb-0 1/1 Running 0 24sweb-1 1/1 Running 0 13sweb-2 0/1 Pending 0 1s[root@k8s-master statefulset]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 19mweb-0 1/1 Running 0 33sweb-1 1/1 Running 0 22sweb-2 0/1 ContainerCreating 0 10s[root@k8s-master statefulset]#[root@k8s-master statefulset]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 19mweb-0 1/1 Running 0 52sweb-1 1/1 Running 0 41sweb-2 1/1 Running 0 29s[root@k8s-master statefulset]#

根据volumeClaimTemplates⾃动创建的PVC [root@k8s-master statefulset]# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEtest-claim Bound default-test-claim-pvc-0ece5a66-dab2-4a13-be51-1a2acdbc45eb 1Mi RWX managed-nfs-storage 28mwww-web-0 Bound default-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8 1Gi RWO managed-nfs-storage 5m15swww-web-1 Bound default-www-web-1-pvc-39123d42-6712-4a66-b7fe-beae48708aad 1Gi RWO managed-nfs-storage 5m4swww-web-2 Bound default-www-web-2-pvc-e395f225-ef4a-4645-a2a9-d139590346dc 1Gi RWO managed-nfs-storage 4m52s[root@k8s-master statefulset]#[root@k8s-master statefulset]# cd /data/volumes/[root@k8s-master volumes]# lsv1 v2 v3[root@k8s-master volumes]# tree.├── v1│ ├── default-test-claim-pvc-0ece5a66-dab2-4a13-be51-1a2acdbc45eb│ ├── default-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8│ ├── default-www-web-1-pvc-39123d42-6712-4a66-b7fe-beae48708aad│ └── default-www-web-2-pvc-e395f225-ef4a-4645-a2a9-d139590346dc├── v2└── v37 directories, 0 files

3.4 测试statefulset持久化存储和⾃愈写⼀个⽂件[root@k8s-master v1]# ll总⽤量 0drwxrwxrwx 2 root root 6 9⽉ 5 14:35 default-test-claim-pvc-0ece5a66-dab2-4a13-be51-1a2acdbc45ebdrwxrwxrwx 2 root root 23 9⽉ 5 15:01 default-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8drwxrwxrwx 2 root root 6 9⽉ 5 14:53 default-www-web-1-pvc-39123d42-6712-4a66-b7fe-beae48708aaddrwxrwxrwx 2 root root 6 9⽉ 5 14:54 default-www-web-2-pvc-e395f225-ef4a-4645-a2a9-d139590346dc[root@k8s-master v1]# echo "

test Server

" > default-www-web-0-pvc-fb47aba4-6d37-4f3e-b118-a35d78f4bca8/[root@k8s-master v1]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 29m 10.254.1.103 k8s-node-1 web-0 1/1 Running 0 10m 10.254.1.104 k8s-node-1 web-1 1/1 Running 0 10m 10.254.2.77 k8s-node-2 web-2 1/1 Running 0 10m 10.254.1.105 k8s-node-1 [root@k8s-master v1]# curl 10.254.1.104

test Server

[root@k8s-master v1]#

删除⽂件，pod⾃愈名称不变，只是IP变动了[root@k8s-master v1]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 29m 10.254.1.103 k8s-node-1 web-0 1/1 Running 0 10m 10.254.1.104 k8s-node-1 web-1 1/1 Running 0 10m 10.254.2.77 k8s-node-2 web-2 1/1 Running 0 10m 10.254.1.105 k8s-node-1 [root@k8s-master v1]# kubectl delete pod web-0pod "web-0" deleted[root@k8s-master v1]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 29m 10.254.1.103 k8s-node-1 web-0 0/1 ContainerCreating 0 7s k8s-node-2 web-1 1/1 Running 0 11m 10.254.2.77 k8s-node-2 web-2 1/1 Running 0 10m 10.254.1.105 k8s-node-1 [root@k8s-master v1]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 29m 10.254.1.103 k8s-node-1 web-0 1/1 Running 0 10s 10.254.2.78 k8s-node-2 web-1 1/1 Running 0 11m 10.254.2.77 k8s-node-2 web-2 1/1 Running 0 11m 10.254.1.105 k8s-node-1 [root@k8s-master v1]# curl 10.254.2.78

test Server

3.5 顺序扩容，倒序缩容[root@k8s-master v1]# kubectl get stsNAME READY AGEweb 3/3 13m[root@k8s-master v1]# kubectl scale statefulset web --replicas=/web scaled[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 33mweb-0 1/1 Running 0 3m30sweb-1 1/1 Running 0 14mweb-2 1/1 Running 0 14mweb-3 0/1 ContainerCreating 0 6s[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 33mweb-0 1/1 Running 0 4mweb-1 1/1 Running 0 15mweb-2 1/1 Running 0 14mweb-3 1/1 Running 0 36sweb-4 1/1 Running 0 23sweb-5 0/1 ContainerCreating 0 7s[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 34mweb-0 1/1 Running 0 4m23sweb-1 1/1 Running 0 15mweb-2 1/1 Running 0 15mweb-3 1/1 Running 0 59sweb-4 1/1 Running 0 46sweb-5 1/1 Running 0 30s[root@k8s-master v1]#[root@k8s-master v1]#[root@k8s-master v1]# kubectl scale statefulset web --replicas=/web scaled[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 34mweb-0 1/1 Running 0 4m35sweb-1 1/1 Running 0 15mweb-2 1/1 Running 0 15mweb-3 1/1 Running 0 71sweb-4 1/1 Running 0 58sweb-5 1/1 Terminating 0 42s[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 34mweb-0 1/1 Running 0 4m49sweb-1 1/1 Running 0 15mweb-2 0/1 Terminating 0 15m[root@k8s-master v1]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5558488b74-mkhbs 1/1 Running 0 34mweb-0 1/1 Running 0 4m57sweb-1 1/1 Running 0 16m[root@k8s-master v1]#

3.6 滚动更新-倒序更新我们⽤的是最新的Nginx版本，现在换个1.15版本的[root@k8s-master statefulset]# kubectl get pods -o wide |grep webweb-0 1/1 Running 0 40s 10.254.1.111 k8s-node-1 web-1 1/1 Running 0 56s 10.254.2.81 k8s-node-2 web-2 1/1 Running 0 76s 10.254.1.110 k8s-node-1 [root@k8s-master statefulset]# curl -s -I 10.254.1.111|grep Server:Server: nginx/1.17.3[root@k8s-master statefulset]# curl 10.254.1.111

test Server

[root@k8s-master statefulset]#[root@k8s-master statefulset]# kubectl apply -f nginx_rvice/nginx /web configured[root@k8s-master statefulset]# kubectl get pods -o wide |grep webweb-0 1/1 Running 0 2m22s 10.254.1.111 k8s-node-1 web-1 1/1 Running 0 2m38s 10.254.2.81 k8s-node-2 web-2 0/1 Terminating 0 2m58s 10.254.1.110 k8s-node-1 [root@k8s-master statefulset]# kubectl get pods -o wide |grep webweb-0 1/1 Running 0 2m34s 10.254.1.111 k8s-node-1 web-1 1/1 Running 0 2m50s 10.254.2.81 k8s-node-2 web-2 0/1 ContainerCreating 0 10s k8s-node-1 [root@k8s-master statefulset]# kubectl get pods -o wide |grep webweb-0 0/1 Terminating 0 2m53s k8s-node-1 web-1 1/1 Running 0 13s 10.254.2.82 k8s-node-2 web-2 1/1 Running 0 29s 10.254.1.112 k8s-node-1 [root@k8s-master statefulset]# kubectl get pods -o wide |grep webweb-0 1/1 Running 0 23s 10.254.2.83 k8s-node-2 web-1 1/1 Running 0 37s 10.254.2.82 k8s-node-2 web-2 1/1 Running 0 53s 10.254.1.112 k8s-node-1 [root@k8s-master statefulset]# curl -s -I 10.254.2.83 |grep ServerServer: nginx/1.15.12[root@k8s-master statefulset]# curl 10.254.2.83