2023年6月30日发(作者：)

容器编排系统k8s之Ingress资源　　我们知道在k8s上service是⽤来解决Pod访问问题，它是通过kube-proxy在每个节点上创建iptables规则或ipvs规则，在⽤户请求某个pod时，⽤户的请求会被其service规则所捕获，从⽽实现访问对应pod；对于service来讲，⽤户请求直接在传输层就被捕获转发，效率很⾼效，但这同时也引⼊了⼀个新问题；⽐如我们运⾏的pod对外客户端访问需要https通信，如果使⽤service这种4层调度，那就意味着每个pod上我们要配置证书，这很显然不是我们想要做的；那有没有什么办法做到在⽤户访问pod对应的service时使⽤https，⽽对应pod⾥⼜不⽤https协议呢？答案是有的；⽐如我们可以使⽤nginx来做https会话卸载器；我们只需要在代理上配置证书即可；⼜⽐如我们在k8s上运⾏了各种各样的pod，这些pod的功能每个都不⼀样，有的是专门处理⽤户认证的，有的是专门处理站点主页的，有的专门处理⽀付的等等，⽽这些pod对外都是提供⼀个独有的url，那么这些pod需要怎么才能被集群外部访问到呢？我们知道对于⼀个站点来讲，如果后端有多个server同时提供⼀种服务，我们可以把这些同功能的server定义成⼀个组，然后使⽤nginx代理将不同功能url的访问代理到不同组上即可；这样⼀来就解决了后端多server被负载访问的问题；那么对于k8s上这种同功能的pod怎么归并成⼀个组呢？⽤户访问不同url怎么调度到不同的组上呢？很显然要想实现这些功能，在k8s上应该有⼀个类似nginx⼀样的代理存在；这个代理就叫做ingress 控制器；ingress 控制器和k8s上的其他控制不⼀样，ingress控制器并不能直接运⾏为kube-controller-manager的⼀部分，它类似k8s集群上的coredns，需要在集群上单独部署，本质上就是⼀个pod，我们可以使⽤k8s上的ds或deploy控制器来创建它；ingress controller pod的作⽤主要是引⼊集群外部流量，并实时监控着apiserver上ingress资源的变动，并将其ingress中定义的规则转化为对应ingress控制器对应应⽤程序的专有配置，然后动态的重载或重启对应守护进程来使其配置⽂件⽣效；在k8s上ingress是⼀种标准资源，它本质上就是我们定义的基于dns名称（host）或url路径把请求转发⾄指定service资源的规则；简单讲ingress就是我们⽤来定义代理的配置所创建的资源；ingress控制器就是把对应ingress规则转换为对应ingress控制器中应⽤程序的专有配置，然后重启或重载对应配置⽂件使其⽣效的组件；　　ingress和ingress controller pod的关系　　提⽰：如上图所⽰，ingress就是ingress 控制器pod的代理规则；⽤户请求某个后端pod所提供的服务时，⾸先会通过ingress controller pod把流量引⼊到集群内部，然后ingress controller pod根据ingress定义的规则，把对应ingress规则转化为对应ingress controller pod实现的对应应⽤的配置（ingresscontroller 可以由任何具有七层反向代理功能的服务实现，⽐如nginx,haproxy等等）然后再适配⽤户请求，把对应请求反代到对应service上；⽽对于pod的选择上，ingress控制器可以基于对应service中的标签选择器，直接同pod直接通信，⽆须通过service对象api的再次转发，从⽽省去了⽤户请求到kube-proxy实现的代理开销（本质上ingress controller 也是运⾏为⼀个pod，和其他pod在同⼀⽹段中）；　　ingress controller部署　　在k8s上ingress controller的实现有很多，⽐如基于nginx的，基于haproxy的等等，这⾥以nginx为例；　　下载ingress-nginx包 wget /kubernetes/ingress-nginx/archive/　　解压包，找到对应的部署清单[root@master01 ~]# lltotal 92144-rw------- 1 root root 65586688 Dec 8 15:16 xr-xr-x 2 root root 4096 Dec 21 21:04 manifests-rw-r--r-- 1 root root 28760559 Dec 21 21:02 [root@master01 ~]# tar xf

[root@master01 ~]# ingress-nginx-nginx-0.28.0 manifests [root@master01 ~]# cd ingress-nginx-nginx-0.28.0/[root@master01 ingress-nginx-nginx-0.28.0]# lsbuild docs hack SECURITY_CONTACTS images LICENSE OWNERS testcmd deploy internal Makefile OWNERS_ALIASES rootfs vendor[root@master01 ingress-nginx-nginx-0.28.0]# cd deploy/[root@master01 deploy]# lsaws cloud-generic grafana prometheus static emetal cluster-wide minikube [root@master01 deploy]# cd static/[root@master01 static]# provider [root@master01 static]# pwd/root/ingress-nginx-nginx-0.28.0/deploy/static[root@master01 static]#

　　提⽰：资源配置清单在ingress-nginx-nginx-0.28.0/deploy/static下，名为；　　资源配置清单内容apiVersion: v1kind: Namespacemetadata: name: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: nginx-configuration namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: tcp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: udp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: v1kind: ServiceAccountmetadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: /v1beta1kind: ClusterRolemetadata: name: nginx-ingress-clusterrole labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "" resources: - ingresses/status verbs: - update---apiVersion: /v1beta1kind: Rolemetadata: name: nginx-ingress-role namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "-" # Here: "-" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get---apiVersion: /v1beta1kind: RoleBindingmetadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: Role name: nginx-ingress-rolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: /v1beta1kind: ClusterRoleBindingmetadata: name: nginx-ingress-clusterrole-nisa-binding labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: ClusterRole name: nginx-ingress-clusterrolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-ingress-controller namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: replicas: 1 selector: matchLabels: /name: ingress-nginx /part-of: ingress-nginx template: metadata: labels: /name: ingress-nginx /part-of: ingress-nginx annotations: /port: "10254" /scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: /os: linux containers: - name: nginx-ingress-controller image: /kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix= securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: ace ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown---apiVersion: v1kind: LimitRangemetadata: name: ingress-nginx namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: limits: - default: min: memory: 90Mi cpu: 100m type: ContainerView Code　　提⽰：以上清单主要定义了⼀个名称ingress-nginx的名称空间，在其名称空间下创建了⼏个configmap，最重要的是⽤deployment创建了⼀个ingress-nginx pod；　　这⾥说⼀下，对于ingress-nginx控制器，它本质还是运⾏为⼀个pod，对于pod来说要想接⼊外部访问流量到集群内部来，有三种⽅式，⼀种是使⽤NodePort类型的service；第⼆种是使⽤ds或deploy控制器，在定义pod模板时使⽤hostPort把pod端⼝映射到宿主机⽅式；第三种是定义pod模板时使⽤hostNetwork，直接共享宿主机⽹络名称空间；如下所⽰　　使⽤专有NodePort service来引⼊外部流量　　提⽰：这种使⽤deploy控制管理ingress controller pod，如果在pod模板中没有暴露端⼝，则需要创建⼀个service资源来暴露ingress controller pod的端⼝来引⼊外部流量到集群内部；　　使⽤ds控制器管理ingress controller pod在pod模板中使⽤hostPort⽅式暴露端⼝　　提⽰：使⽤ds控制器能够保证每个节点上只运⾏⼀个ingress controller,所以我们可以把对应ingress controller pod端端⼝通过端⼝映射的⽅式映射到宿主机上的某⼀固定端⼝；　　使⽤ds控制器在pod模板中使⽤hostNetwork⽅式共享宿主机⽹络名称空间　　提⽰：共享宿主机⽹络名称空间，也必须使⽤ds控制器来确保对应每个节点上只能运⾏⼀个ingress controller pod，这样才能确保每个ingress controllerpod能够正常把端⼝暴露出去，以供集群外部客户端访问；　　选择上述其中⼀种⽅式暴露ingress controller pod的端⼝即可；如果选择使⽤ds控制器来暴露端⼝，我们就需要修改其对应资源配置清单中的pod模板，如下所⽰　　使⽤ds控制器来管理ingress controller pod在pod模板中使⽤hostPort⽅式暴露端⼝apiVersion: v1kind: Namespacemetadata: name: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: nginx-configuration namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: tcp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: udp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: v1kind: ServiceAccountmetadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: /v1beta1kind: ClusterRolemetadata: name: nginx-ingress-clusterrole labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "" resources: - ingresses/status verbs: - update---apiVersion: /v1beta1kind: Rolemetadata: name: nginx-ingress-role namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "-" # Here: "-" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get---apiVersion: /v1beta1kind: RoleBindingmetadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: Role name: nginx-ingress-rolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: /v1beta1kind: ClusterRoleBindingmetadata: name: nginx-ingress-clusterrole-nisa-binding labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: ClusterRole name: nginx-ingress-clusterrolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: apps/v1kind: DaemonSetmetadata: name: nginx-ingress-controller namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: selector: matchLabels: /name: ingress-nginx /part-of: ingress-nginx template: metadata: labels: /name: ingress-nginx /part-of: ingress-nginx annotations: /port: "10254" /scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: /os: linux containers: - name: nginx-ingress-controller image: /kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix= securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: ace ports: - name: http containerPort: 80 hostPort: 30080 protocol: TCP - name: https containerPort: 443 hostPort: 30443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown---apiVersion: v1kind: LimitRangemetadata: name: ingress-nginx namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: limits: - default: min: memory: 90Mi cpu: 100m type: ContainerView Code　　提⽰：只需把对应控制器类型更改为DaemonSet，在pod模板中spec字段下把replicas去掉；在字段中加上nodePort字段指定要把容器的端⼝映射到宿主机上某个端⼝；如果暴露的端⼝是⾮标准端⼝，在对应k8s集群外部我们还需要部署反代，⽐如使⽤nginx，haproxy,lvs；　　使⽤ds控制器管理ingress controller pod在ds控制器资源配置中使⽤hostNetworkapiVersion: v1kind: Namespacemetadata: name: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: nginx-configuration namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: tcp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---kind: ConfigMapapiVersion: v1metadata: name: udp-services namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: v1kind: ServiceAccountmetadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginx---apiVersion: /v1beta1kind: ClusterRolemetadata: name: nginx-ingress-clusterrole labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "" resources: - ingresses/status verbs: - update---apiVersion: /v1beta1kind: Rolemetadata: name: nginx-ingress-role namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxrules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "-" # Here: "-" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get---apiVersion: /v1beta1kind: RoleBindingmetadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: Role name: nginx-ingress-rolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: /v1beta1kind: ClusterRoleBindingmetadata: name: nginx-ingress-clusterrole-nisa-binding labels: /name: ingress-nginx /part-of: ingress-nginxroleRef: apiGroup: kind: ClusterRole name: nginx-ingress-clusterrolesubjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx---apiVersion: apps/v1kind: DaemonSetmetadata: name: nginx-ingress-controller namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: selector: matchLabels: /name: ingress-nginx /part-of: ingress-nginx template: metadata: labels: /name: ingress-nginx /part-of: ingress-nginx annotations: /port: "10254" /scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: /os: linux hostNetwork: true containers: - name: nginx-ingress-controller image: /kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix= securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: ace ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown---apiVersion: v1kind: LimitRangemetadata: name: ingress-nginx namespace: ingress-nginx labels: /name: ingress-nginx /part-of: ingress-nginxspec: limits: - default: min: memory: 90Mi cpu: 100m type: ContainerView Code　　提⽰：把对应控制器类型更改外DaemonSet，在pod模板中spec字段下的replicas字段去掉；在字段下加上hostNetwork: true即可；以上两种使⽤ds控制器管理ingress controller pod也可以使⽤node选择器，来筛选在某个节点上创建ingress controller pod；　　使⽤deploy控制器管理ingress controller pod，就直接应⽤即可[root@master01 ~]# kubectl apply -f

namespace/ingress-nginx createdconfigmap/nginx-configuration createdconfigmap/tcp-services createdconfigmap/udp-services createdserviceaccount/nginx-ingress-serviceaccount createdWarning: /v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use /v1 /nginx-ingress-clusterrole createdWarning: /v1beta1 Role is deprecated in v1.17+, unavailable in v1.22+; use /v1 /nginx-ingress-role createdWarning: /v1beta1 RoleBinding is deprecated in v1.17+, unavailable in v1.22+; use /v1 /nginx-ingress-role-nisa-binding createdWarning: /v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use /v1 /nginx-ingress-clusterrole-nisa-binding /nginx-ingress-controller createdlimitrange/ingress-nginx created[root@master01 ~]#

　　查看应⽤资源清单创建的资源对象[root@master01 ~]# kubectl get all -n ingress-nginxNAME READY STATUS RESTARTS AGEpod/nginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 80sNAME READY UP-TO-DATE AVAILABLE /nginx-ingress-controller 1/1 1 1 80sNAME DESIRED CURRENT READY /nginx-ingress-controller-5466cb8999 1 1 1 80s[root@master01 ~]#

　　提⽰：可以看到在ingress-nginx名称空间下创建了⼀个deploy控制器，对应控制器创建了⼀个nginx-ingress-controller控制器pod；但是此pod现在不能被外部客户端访问到，我们需要创建⼀个service来引⼊外部流量到此pod上；　　查看pod标签[root@master01 ~]# kubectl get pod -n ingress-nginx --show-labelsNAME READY STATUS RESTARTS AGE LABELSnginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 4m38s /name=ingress-nginx,/part-of=ingress-nginx,pod-template-hash=5466cb8999[root@master01 ~]#

　　根据上述标签来写⼀个创建ingress-service资源的配置清单[root@master01 ~]# cat iVersion: v1kind: Servicemetadata: name: ingress-nginx-svc namespace: ingress-nginxspec: type: NodePort ports: - port: 80 name: http nodePort: 30080 - port: 443 name: https nodePort: 30443 selector: /name: ingress-nginx /part-of: ingress-nginx[root@master01 ~]#

　　提⽰：以上配置清单主要把满⾜对应标签选择器的pod关联起来；并把对应pod的80和443端⼝分别映射到对应主机上的30080和30443端⼝；　　应⽤配置清单[root@master01 ~]# kubectl apply -f rvice/ingress-nginx-svc created[root@master01 ~]# kubectl get svc -n ingress-nginxNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEingress-nginx-svc NodePort 10.98.4.208 80:30080/TCP,443:30443/TCP 13s[root@master01 ~]#

　　访问集群任意节点ip的30080和30443端⼝，看看是否访问到对应pod？　　提⽰：30080是能够正常访问的，只是它显⽰404，是因为我们没有对应的主页；　　访问30443端⼝　　提⽰：30443是⼀个https端⼝，所以访问必须⽤https协议访问，这⾥提⽰访问页⾯有风险是因为浏览器不信任证书引起的，我们可以点击⾼级，信任证书即可；同样30443端⼝也是返回404，是因为没有主页的原因；两个端⼝能够正常访问，说明我们在k8s上部署的ingress-nginx controller就部署好了；　　ingress资源的使⽤　　在k8s上创建⼀个deploy控制器，让其管理2个 ikubernetes/myapp:v1镜像运⾏的pod，然后再创建⼀个对应的service[root@master01 manifests]# cat iVersion: apps/v1kind: Deploymentmetadata: name: myapp namespace: defaultspec: replicas: 2 selector: matchLabels: app: myapp rel: stable template: metadata: namespace: default labels: app: myapp rel: stable spec: containers: - name: myapp image: ikubernetes/myapp:v1---apiVersion: v1kind: Servicemetadata: name: myapp namespace: defaultspec: selector: app: myapp rel: stable ports: - name: http port: 80 targetPort: 80[root@master01 manifests]#

　　提⽰：⼀个清单中定义多个资源，需要⽤“---”来分割资源；　　应⽤资源清单[root@master01 manifests]# kubectl apply -f /myapp createdservice/myapp created[root@master01 manifests]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESmyapp-6479b786f5-9d4mh 1/1 Running 0 11s 10.244.2.98 myapp-6479b786f5-k252c 1/1 Running 0 11s 10.244.4.20 [root@master01 manifests]# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 443/TCP 4h52mmyapp ClusterIP 10.105.208.218 80/TCP 21s[root@master01 manifests]# kubectl describe svc myappName: myappNamespace: defaultLabels: Annotations: Selector: app=myapp,rel=stableType: ClusterIPIP Families: IP: 10.105.208.218IPs: 10.105.208.218Port: http 80/TCPTargetPort: 80/TCPEndpoints: 10.244.2.98:80,10.244.4.20:80Session Affinity: NoneEvents: [root@master01 manifests]#

　　创建ingress资源来反代以上资源　　⽰例：创建ingress资源[root@master01 manifests]# cat iVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-myapp namespace: default annotations: /: "nginx"spec: rules: - host: http: paths: - path: / backend: serviceName: myapp servicePort: 80[root@master01 manifests]#

　　提⽰：创建ingress资源apiVersion的值要写成extensions/v1beta1，kind为Ingress；对应metadata中的annotations的配置表⽰把ingress资源通知给那个类别的ingress controller，如果k8s集群上有多个类别的ingress controller时，这⼀项特别有⽤；在spec字段主要内嵌了三个字段，rules字段⽤来定义反代规则列表，其值为⼀个对象列表；其中rules字段⾥主要host和http字段；host⽤来指定虚拟主机的fqdn名称，如果不写表⽰匹配任意虚拟主机名称；http是⽤来定义指向后端的http选择器列表；其值为⼀个对象，⾥⾯只有⼀个paths字段，⽤于指定把请求映射到后端的某个路径；其值为⼀个对象列表；对应paths字段中可以定义path，⽤来指定映射后端的路径；backend⽤于指定后端pod的service，其值为⼀个对象；serviceName⽤于指定对应pod的service名称；servicePort⽤于指定后端服务的端⼝；以上配置表⽰把这个虚拟主机的访问全部反代⾄服务名称为myapp端⼝为80的pod上；　　应⽤配置清单[root@master01 manifests]# kubectl apply -f

Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use /v1 ions/ingress-myapp created[root@master01 manifests]# kubectl get ingressNAME CLASS HOSTS ADDRESS PORTS AGEingress-myapp 80 29s[root@master01 manifests]#

　　查看ingress资源的详细信息[root@master01 manifests]# kubectl describe ingress ingress-myappName: ingress-myappNamespace: defaultAddress:

Default backend: default-http-backend:80 ()Rules: Host Path Backends ---- ---- --------

/ myapp:80 (10.244.2.98:80,10.244.4.20:80)Annotations: /: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 81s nginx-ingress-controller Ingress default/ingress-myapp[root@master01 manifests]#

　　提⽰：可以看到对应满⾜service名称为myapp并且其端⼝为80的pod有两个；　　进⼊ingress controller pod⾥，看看对应配置⽂件是否有的配置？[root@master01 manifests]# kubectl get pods -n ingress-nginxNAME READY STATUS RESTARTS AGEnginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 78m[root@master01 manifests]# kubectl exec -it -n ingress-nginx pod/nginx-ingress-controller-5466cb8999-4lsjc -- /bin/sh/etc/nginx $ cd /etc/nginx//etc/nginx $ koi-utf modsecurity owasp-modsecurity-crs uwsgi_t koi-win modules scgi_params win-utffastcgi_params lua scgi_tfastcgi_t t templategeoip t uwsgi_params/etc/nginx $ grep "" ## start server server_name ; ## end server /etc/nginx $

　　提⽰：可以看到在对应ingress-nginx 控制器pod中能够搜索到的配置；说明我们定义的ingress资源已经被ingress-nginx controller 捕获；　　⽤浏览器访问看看是否能够访问到内容？　　提⽰：使⽤访问，需要确保对应域名能够正常解析到k8s集群任意⼀节点上；可以看到访问:30080能够访问到对应pod内容；　　删除ingress代理规则[root@master01 manifests]# kubectl delete -f

Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use /v1 ions "ingress-myapp" deleted[root@master01 manifests]# kubectl get ingress

No resources found in default namespace.[root@master01 manifests]#

　　⽰例：配置基于url路径进⾏流量分发[root@master01 manifests]# cat iVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-myapp namespace: default annotations: /: "nginx" /rewrite-target: /spec: rules: - host: http: paths: - path: /bbs backend: serviceName: myapp servicePort: 80 - path: /blog backend: serviceName: myapp servicePort: 80[root@master01 manifests]#

　　提⽰：以上配置表⽰把/bbs反代到service名称为myapp并且端⼝为80的pod上；把/blog反代到ervice名称为myapp并且端⼝为80的pod上；我这⾥是因为k8s上只有这⼀种应⽤，⽣成环境中按照对应的业务逻辑来反代对应url到对应pod上即可；　　应⽤配置清单[root@master01 manifests]# kubectl apply -f

Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use /v1 ions/ingress-myapp created[root@master01 manifests]# kubectl get ingressNAME CLASS HOSTS ADDRESS PORTS AGEingress-myapp 80 5s[root@master01 manifests]# kubectl describe ingress ingress-myappName: ingress-myappNamespace: defaultAddress:

Default backend: default-http-backend:80 ()Rules: Host Path Backends ---- ---- --------

/bbs myapp:80 (10.244.2.98:80,10.244.4.20:80) /blog myapp:80 (10.244.2.98:80,10.244.4.20:80)Annotations: /rewrite-target: / /: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 30s nginx-ingress-controller Ingress default/ingress-myapp[root@master01 manifests]#

　　提⽰：可以看到对应ingress上就有两个url分别指向后端service名称为myapp端⼝为80的pod上；　　访问对应url，看看是否访问到内容？　　提⽰：这⾥访问不到内容的原因是对应pod内部并没有对应url的页⾯；　　进⼊ingress controller pod内部，查看是否有对应配置？　　提⽰：可以看到对应在ingress中定义的配置，都转为对应该ingress controller pod中的配置，说明我们定义基于url分发流量的ingress没有问题；　　⽰例：定义ingress规则基于主机名称的虚拟主机[root@master01 manifests]# cat iVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-myapp namespace: default annotations: /: "nginx" /rewrite-target: /spec: rules: - host: http: paths: - path:

backend: serviceName: myapp servicePort: 80 - host: http: paths: - path:

backend: serviceName: myapp servicePort: 80[root@master01 manifests]#

　　提⽰：以上配置表⽰把这个虚拟主机名称的访问流量分发⾄service名称为myapp端⼝为80的pod上；把的流量分发⾄⾄service名称为myapp端⼝为80的pod上；⽣成环境按照对应的service名称来分发即可；　　应⽤配置清单[root@master01 manifests]# kubectl apply -f

Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use /v1 ions/ingress-myapp created[root@master01 manifests]# kubectl get ingressNAME CLASS HOSTS ADDRESS PORTS AGEingress-myapp , 80 16s[root@master01 manifests]# kubectl describe ingress ingress-myappName: ingress-myappNamespace: defaultAddress:

Default backend: default-http-backend:80 ()Rules: Host Path Backends ---- ---- --------

myapp:80 (10.244.2.98:80,10.244.4.20:80)

myapp:80 (10.244.2.98:80,10.244.4.20:80)Annotations: /rewrite-target: / /: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 32s nginx-ingress-controller Ingress default/ingress-myapp[root@master01 manifests]#

　　验证配置信息　　访问对应虚拟主机，看看是否能够访问对应pod？　　提⽰：可以看到两个虚拟主机名称都可以正常访问到，对应也做了调度；　　⽰例：创建tls类型的ingress资源　　创建证书[root@master01 manifests]# openssl genrsa -out 2048Generating RSA private key, 2048 bit +++........+++e is 65537 (0x10001)[root@master01 manifests]# openssl req -x509 -key -out -subj /C=CN/ST=SiChuan/L=GuangYuan/O=Test/CN= -days 3650

[root@master01 manifests]#

　　提⽰：以上两条命令创建了⼀个名为的私钥和⼀个⾃签名证书，其名为；　　创建Secret资源[root@master01 manifests]# kubectl create secret tls www-myapp-com-ingress-secret --cert=./ --key=./

secret/www-myapp-com-ingress-secret created[root@master01 manifests]# kubectl get secretNAME TYPE DATA AGEdefault-token-xvd4c /service-account-token 3 13dwww-myapp-com-ingress-secret /tls 2 21s[root@master01 manifests]#

　　提⽰：在ingress控制器上配置https主机时，不能直接使⽤私钥和证书⽂件，⽽是需要使⽤secret资源对象来传递相关数据；　　定义tls类型ingress资源清单[root@master01 manifests]# cat iVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-myapp-tls namespace: default annotations: /: "nginx"spec: tls: - hosts: - secretName: www-myapp-com-ingress-secret rules: - host: http: paths: - path: /

backend: serviceName: myapp servicePort: 80[root@master01 manifests]#

　　提⽰：定义tls类型ingress资源清单，需要在spec字段下⽤tls字段来指定对应主机名称，以及secret资源对象的名称；　　应⽤资源清单[root@master01 manifests]# kubectl apply -f rning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use /v1 ions/ingress-myapp-tls created[root@master01 manifests]# kubectl get ingressNAME CLASS HOSTS ADDRESS PORTS AGEingress-myapp , 80 31mingress-myapp-tls 80, 443 8s[root@master01 manifests]# kubectl describe ingress ingress-myapp-tlsName: ingress-myapp-tlsNamespace: defaultAddress:

Default backend: default-http-backend:80 ()TLS: www-myapp-com-ingress-secret terminates es: Host Path Backends ---- ---- -------- / myapp:80 (10.244.2.98:80,10.244.4.20:80)Annotations: /: nginxEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 26s nginx-ingress-controller Ingress default/ingress-myapp-tls[root@master01 manifests]#

　　验证：访问对应虚拟主机名称，看看对应的https端⼝是否能够正常访问到内容？　　提⽰：可以看到使⽤https协议访问对应的30443端⼝能够正常访问到对应后端pod提供的内容；