TKE集群如何手动创建apiserver内网和公网访问入口

TKE集群的apiserver访问，可以在控制台开启，但是没法通过公网ip去访问，只能是自定义域名，如果想要用公网ip去访问，则需要进行一定的改动。

下面我们说下如何不在控制台开启apiserver的内网或者公网访问，手动配置TKE集群apiserver的公网和内网访问入口。

1. 申请eip

因为clb已经域名化，这里不支持申请ip类型的clb，我们可以先申请一个eip，然后将eip绑定给clb。

2. 申请公网clb

这里弹性公网ip选择第一步创建的eip，如果要限制公网的访问，直接在公网clb绑定安全组，参考文档

3. 申请内网clb

内网clb没有限制，可以用于内网访问apiserver

4. 部署kubernetes-proxy

将下面的yaml apply到集群，如果你的集群之前有控制台开启公网或者内网访问，需要先关闭，否则会有报错rbac资源已存在

代码语言：yaml复制

apiVersion: v1
kind: Namespace
metadata:
  name: apiserver-proxy
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: kubernetes-proxy
  name: kubernetes-proxy
  namespace: apiserver-proxy
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-proxy
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        k8s-app: kubernetes-proxy
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node.kubernetes.io/instance-type
                operator: NotIn
                values:
                - external
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: k8s-app
                  operator: In
                  values:
                  - kubernetes-proxy
              topologyKey: topology.kubernetes.io/zone
            weight: 100
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: k8s-app
                  operator: In
                  values:
                  - kubernetes-proxy
              topologyKey: kubernetes.io/hostname
            weight: 50
      containers:
      - args:
        - ./apiserver-proxy
        - start
        - --CN=kubernetes-proxy
        - --port=443
        - --to=https://169.254.128.162:60002
        - --internalIp=xx.xx.xx.xx
        - --externalIp=xx.xx.xx.xx
        - --domains=tke.test
        image: ccrs.tencentyun/tkeimages/apiserver-proxy:v1.4.5
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 60
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: kubernetes-proxy
        ports:
        - containerPort: 443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 60
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: 100m
            memory: 128Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: kubernetes-proxy
      serviceAccountName: kubernetes-proxy
      terminationGracePeriodSeconds: 30  
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kubernetes-proxy
  namespace: apiserver-proxy
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubernetes-proxy
rules:
- apiGroups:
  - certificates.k8s.io
  resources:
  - '*'
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - users
  - groups
  - serviceaccounts
  - uids
  - userextras
  verbs:
  - impersonate
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-proxy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-proxy
subjects:
- kind: ServiceAccount
  name: kubernetes-proxy
  namespace: apiserver-proxy
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/tke-existed-lbid: lb-xxxxxx
  name: kubernetes-intranet
  namespace: apiserver-proxy
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    k8s-app: kubernetes-proxy
  sessionAffinity: None
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/tke-existed-lbid: lb-xxxxxx
  name: kubernetes-extranet
  namespace: apiserver-proxy
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    k8s-app: kubernetes-proxy
  sessionAffinity: None
  type: LoadBalancer

这里有几个字段说明下： 1. deployment的启动参数

• ---to=https://169.254.128.162:60002

这个ip和端口是vpc的一个clb，会通过这个访问到托管的master，可以在default的kubernetes这个service的endpoint获取

• ---internalIp=172.16.20.65

第3步申请内网clb的vip

• ---externalIp=106.54.234.49

第1步申请eip

• ---domains=tke.test

自定义域名配置，需要自行做域名解析

2. service注解配置

这里会创建2个service，都是LoadBalancer类型，kubernetes-intranet和kubernetes-extranet，并且都是通过引用已有clb的方式关联clb。

yaml都会配置这个注解service.kubernetes.io/tke-existed-lbid

kubernetes-intranet：配置的是内网clb的实例id

kubernetes-extranet：配置的是公网clb的实例id

代码语言：txt复制

service.kubernetes.io/tke-existed-lbid: lb-xxxxxx

5. 调云api接口获取kubeconfig

如果不是控制台开启的apiserver访问，控制台是不提供kubeconfig的，但是我们可以通过接口DescribeClusterKubeconfig

获取，提取response的Kubeconfig字段即可，这个时候kubeconfig的server地址还是默认域名，暂时无法访问。

6. 测试clb的vip和自定义域名访问apiserver

替换第5步获取kubeconfig的server地址，可以用内网clb的vip，eip，也可以是域名，下面我们测试下

测试域名，公网和内网ip都正常访问，手动配置访问入口成功。