2018-07-25

[KUBERNETES] nginx 컨테이너를 활용한 ingress load balancer 구성

1. 테스트용 웹 백엔드 생성

back-end-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: deploy-test-1
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: web-front-end
        department: group1
    spec:
      containers:
      - name: m-client-web
        image: xxx/client:latest
        env:
        - name: PORT_ARGS
          value: "--port=80"
        ports:
        - containerPort: 80
          name: web-port
          protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: deploy-test-2
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: web-front-end
        department: group2
    spec:
      containers:
      - name: tiny-webserver-2
        image: xxx/node-api:latest
        env:
        - name: PORT_ARGS
          value: "--port=3501"
        ports:
        - containerPort: 3501
          name: web-port
          protocol: TCP

back-end-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: backend-svc-1
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: web-port
  selector:
    app: web-front-end
    department: group1
---
apiVersion: v1
kind: Service
metadata:
  name: backend-svc-2
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: web-port
  selector:
    app: web-front-end
    department: group2

실행

1 2	kubectl apply -f back-end-deployment.yaml kubectl apply -f back-end-service.yaml

2. 테스트용 ingress용 백엔드 생성

default-back-end-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi

default-back-end-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: default-http-backend

실행

kubectl apply -f default-back-end-deployment.yaml
kubectl apply -f default-back-end-service.yaml

4. Nginx Ingress Controller 설정값 저장을 위한 Configmap 등록

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-ingress-controller-conf
labels:
  app: nginx-ingress-lb
  group: lb
data:
# for VTS page of the Nginx load balancer
enable-vts-status: 'true'
enable-sticky-sessions: 'true'

5. Nginx Ingress Controller 생성

SA, ClusterRole, ClusterRoleBinding

# Add: SA, ClusterRole, ClusterRoleBinding
# Name Space: default
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress
  namespace: default

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: system:ingress
rules:
- apiGroups:
  - ""
  resources: ["configmaps","secrets","endpoints","events","services"]
  verbs: ["list","watch","create","update","delete","get"]
- apiGroups:
  - ""
  - "extensions"
  resources: ["services","nodes","ingresses","pods","ingresses/status"]
  verbs: ["list","watch","create","update","delete","get"]

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    # Could be like 'kube-system' ...
    namespace: default

deployment

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  # Could be like 'kube-system' ...
  namespace: default
spec:
  replicas: 2
  revisionHistoryLimit: 3
  template:
    metadata:
      labels:
        app: nginx-ingress-lb
    spec:
      serviceAccountName: ingress
      terminationGracePeriodSeconds: 60
      containers:
        - name: nginx-ingress-controller
          image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.2
          imagePullPolicy: IfNotPresent
          readinessProbe:
            httpGet:
              path: /healthz
              port: 18080
              scheme: HTTP
          livenessProbe:
            httpGet:
              path: /healthz
              port: 18080
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 5
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-ingress-controller-conf
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - containerPort: 443
            - containerPort: 18080

5. Ingress Rule 디플로이

default-back-end-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
namespace: default
spec:
rules:
- host: [svc1 도메인]
  http:
    paths:
    - backend:
        serviceName: backend-svc-1
        servicePort: 80
- host: [svc2 도메인]
  http:
    paths:
    - backend:
        serviceName: backend-svc-2
        servicePort: 80
- host: [통합 도메인]
  http:
    paths:
    - path: /svc1
      backend:
        serviceName: backend-svc-1
        servicePort: 80
    - path: /svc2
      backend:
        serviceName: backend-svc-2
        servicePort: 80
    - path: /nginx_status
      backend:
        serviceName: nginx-ingress-nodeport
        servicePort: 18080

참고링크

Ingress Load Balancer 개념도