2023年6月30日发(作者：)

Kubernetes的负载均衡问题（NginxIngress）Kubernetes关于服务的暴露主要是通过NodePort⽅式，通过绑定minion主机的某个端⼝,然后进⾏pod的请求转发和负载均衡，但这种⽅式下缺陷是Service可能有很多个，如果每个都绑定⼀个node主机端⼝的话，主机需要开放外围⼀堆的端⼝进⾏服务调⽤，管理混乱⽆法应⽤很多公司要求的防⽕墙规则理想的⽅式是通过⼀个外部的负载均衡器，绑定固定的端⼝，⽐如80,然后根据域名或者服务名向后⾯的Service ip转发,Nginx很好的解决了这个需求，但问题是如果有新的服务加⼊，如何去修改Nginx的配置，并且加载这些配置？ Kubernetes给出的⽅案就是Ingress,Ingress包含了两⼤主件Ingress Controller和s解决的是新的服务加⼊后，域名和服务的对应问题，基本上是⼀个ingress的对象，通过yaml进⾏创建和更新进⾏加载。Ingress Controller是将Ingress这种变化⽣成⼀段Nginx的配置，然后将这个配置通过Kubernetes API写到Nginx的Pod中，然后reload.

具体实现如下:1.⽣成⼀个默认的后端，如果遇到解析不到的URL就转发到默认后端页⾯[root@k8s-master ingress]# cat

apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: default-http-backend labels: k8s-app: default-http-backend namespace: kube-systemspec: replicas: 1 template: metadata: labels: k8s-app: default-http-backend spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissable as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: /google_containers/defaultbackend:1.0 livenessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi---apiVersion: v1kind: Servicemetadata: name: default-http-backend namespace: kube-system labels: k8s-app: default-http-backendspec: ports: - port: 80 targetPort: 8080 selector: k8s-app: default-http-backend

2.部署Ingress Controller具体⽂件可以参考官⽅的这⾥贴⼀个我的[root@k8s-master ingress]# cat

apiVersion: v1kind: ReplicationControllermetadata: name: nginx-ingress-lb labels: name: nginx-ingress-lb namespace: kube-systemspec: replicas: 1 template: metadata: labels: name: nginx-ingress-lb annotations: /port: '10254' /scrape: 'true' spec: terminationGracePeriodSeconds: 60 hostNetwork: true containers: - image: /google_containers/nginx-ingress-controller:0.9.0-beta.7 name: nginx-ingress-lb readinessProbe: httpGet: path: /healthz port: 10254 scheme: HTTP livenessProbe: httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 timeoutSeconds: 1 ports: - containerPort: 80 hostPort: 80 - containerPort: 443 hostPort: 443 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: ace - name: KUBERNETES_MASTER value: 192.168.0.105:8080 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --apiserver-host=192.168.0.105:8080曾经出现的问题是，启动后pod总是在CrashLoopBack的状态，通过logs⼀看发现nginx-ingress-controller的启动总是去连接apiserver内部集群ip的443端⼝，导致因为安全问题不让启动，后来在args⾥⾯加⼊- --apiserver-host=192.168.0.105:8080后成功启动.

3.配置ingress 配置如下[root@k8s-master ingress]# cat

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: dashboard-weblogic-ingress namespace: kube-systemspec: rules: - host: http: paths: - path: /console backend: serviceName: helloworldsvc

servicePort: 7001 - path: / backend: serviceName: kubernetes-dashboard servicePort: 80理解如下:host指虚拟出来的域名，具体地址(我理解应该是Ingress-controller那台Pod所在的主机的地址)应该加⼊/etc/hosts中,这样所有去的请求都会发到nginxpath:/console匹配后⾯的应⽤路径servicePort主要是定义服务的时候的端⼝，不是:/ 匹配后⾯dashboard应⽤的路径,以前通过访问master节点8080/ui进⼊dashboard的，但dashboard其实是部署在minion节点中，实际是通过某个路由语句转发过去⽽已，dashboard真实路径如下:⽽yaml⽂件是[root@k8s-master ~]# cat

apiVersion: extensions/v1beta1kind: Deploymentmetadata:# Keep the name in sync with image version and# gce/coreos/kube-manifests/addons/dashboard counterparts name: kubernetes-dashboard-latest namespace: kube-systemspec: replicas: 1 template: metadata: labels: k8s-app: kubernetes-dashboard version: latest /cluster-service: "true" spec: containers: - name: kubernetes-dashboard image: /google_containers/kubernetes-dashboard-amd64:v1.5.1 resources: # keep request = limit to keep this container in guaranteed class limits: cpu: 100m memory: 50Mi requests: cpu: 100m memory: 50Mi ports: - containerPort: 9090 args: - --apiserver-host=192.168.0.105:8080 livenessProbe: httpGet: path: / port: 9090 initialDelaySeconds: 30 timeoutSeconds: 30---kind: Servicemetadata: name: kubernetes-dashboard namespace: kube-system labels: k8s-app: kubernetes-dashboard /cluster-service: "true"spec: selector: k8s-app: kubernetes-dashboard ports: - port: 80 targetPort: 9090所以访问192.168.51.5:9090端⼝就会出现dashboard 4.测试

Ok,⼀切就绪，装逼开始

5.配置TLS SSL访问TLS的配置相当于WebLogic中证书的配置，配置过程如下证书⽣成# ⽣成 CA ⾃签证书mkdir cert && cd certopenssl genrsa -out 2048openssl req -x509 -new -nodes -key -days 10000 -out -subj "/CN=kube-ca"# 编辑 openssl 配置cp /etc/pki/tls/ .vim # 主要修改如下[req]req_extensions = v3_req # 这⾏默认注释关着的 把注释删掉# 下⾯配置是新增的[ v3_req ]basicConstraints = CA:FALSEkeyUsage = nonRepudiation, digitalSignature, keyEnciphermentsubjectAltName = @alt_names[alt_names]DNS.1 = #DNS.2 = # ⽣成证书openssl genrsa -out 2048openssl req -new -key -out -subj "/CN=" -config nssl x509 -req -in -CA -CAkey -CAcreateserial -out -days 365 -extensions v3_req -extfile

需要注意的是DNS需要修改成⾃⼰的host名,然后在配置csr证书请求的时候需要将域名或者访问名带⼊subj,⽐如-subj "/CN="

创建secretkubectl create secret tls ingress-secret --namespace=kube-system --key cert/ --cert cert/

修改Ingress⽂件启⽤证书[root@k8s-master ingress]# cat

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: dashboard-weblogic-ingress namespace: kube-systemspec: tls: - hosts: - secretName: ingress-secret rules: - host: http: paths: - path: /console backend: serviceName: helloworldsvc

servicePort: 7001 - path: / backend: serviceName: kubernetes-dashboard servicePort: 80 测试 然后访问/console,会⾃动转到https页⾯，同时查看证书并加⼊授信列表，可见 访问