[K8S] ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž์˜ EKS ๊ฒฝํ—˜๊ธฐ 4๏ธโƒฃ

2024-06-02T13:06:12.000Z

์ง€๋‚œ ํฌ์ŠคํŠธ์—์„œ๋Š” ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์ƒ์„ฑํ•˜๊ณ , ๊ฐ€์žฅ ๊ธฐ๋ณธ์ ์ธ ๋ฆฌ์†Œ์Šค ์œ ํ˜•์ธ Deployment์™€ Service๋ฅผ ์ •์˜ํ•œ ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์„ ์ž‘์„ฑํ•ด๋ณด์•˜๋‹ค.

์ด๋ฒˆ์—๋Š” istio์™€ ๊ด€๋ จ๋œ ๋ฆฌ์†Œ์Šค์ธ Gateway, VirtualService๋ฅผ ํฌํ•จํ•œ ๋‚˜๋จธ์ง€ ์ž‘์—…์„ ์ „๋ถ€ ๋งˆ๋ฌด๋ฆฌํ•˜์—ฌ, ์ตœ์ข…์ ์œผ๋กœ ๋ฐฐํฌ ์ž๋™ํ™”๋ฅผ ๋‹ฌ์„ฑํ•ด๋ณด์ž.

๐Ÿšจ ๊ฒฝ๊ณ , ์ „๋ฌธ์„ฑ์ด ๋†’์ง€ ์•Š์€ ๊ธ€์ž…๋‹ˆ๋‹ค!

์‹ค์ œ devOps ๊ฐœ๋ฐœ์ž ๋ถ„๋“ค ํ˜น์€ ํด๋ผ์šฐ๋“œ ์„œ๋น„์Šค์— ์ผ๊ฐ€๊ฒฌ์ด ์žˆ์œผ์‹  ๋ถ„๋“ค์€ ์ œ ๊ธ€์„ ๋ณด๊ณ  ์˜๋ฌธ์„ ํ’ˆ๋Š” ๋ถ€๋ถ„์ด ํ•œ๋‘ ๊ณณ์ด ์•„๋‹ˆ์‹ค ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  • โ€œ์™œ externalDNS๋ฅผ ์‚ฌ์šฉํ•˜์ง€?โ€
  • โ€œ์‚ฌ์ด๋“œ์นด ์ธ์ ์…˜๋„ ๋ชจ๋ฅด๋Š”๋ฐ istio๋Š” ์™œ ์‚ฌ์šฉํ•˜๋Š” ๊ฑฐ์ง€?โ€
  • โ€œvpc๊ฐ€ ๋ญ”์ง€๋Š” ์•Œ๊ณ  ๊ธ€์„ ์“ฐ๋Š”๊ฑด๊ฐ€ ์ด์‚ฌ๋žŒ?โ€

์—ฌ๋Ÿฌ๋ถ„๋“ค์˜ ์˜๊ฒฌ์ด ์ „๋ถ€ ๋งž์Šต๋‹ˆ๋‹ค. ์ €๋Š” ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž์ด๋ฉฐ, ๋‹จ์ˆœํžˆ ํšŒ์‚ฌ์˜ k8s ์ธํ”„๋ผ๊ฐ€ ์–ด๋–ค ๋ฐฉ์‹์œผ๋กœ ๋™์ž‘ํ•˜๋Š” ์ง€ ๋„ˆ๋ฌด๋‚˜๋„ ๊ถ๊ธˆํ•˜์—ฌ, ํšŒ์‚ฌ์—์„œ ๊ตฌ์ถ•ํ•œ ๋ฐฉ์‹์„ ๊ธฐ์ค€์œผ๋กœ ์„ค๋ช…์„ ํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.


์ œ ์ˆ˜์ค€์—์„œ ์„ค๋ช…์„ ํ•  ์ˆ˜ ์žˆ๋Š” ๋ถ€๋ถ„(์™œ spot ์ธ์Šคํ„ด์Šค๋ฅผ ์‚ฌ์šฉํ–ˆ๋Š”๊ฐ€?)์€ ์„ค๋ช…ํ•˜๊ฒ ์ง€๋งŒ, ๋„ˆ๋ฌด deepํ•ด์ง€๋Š” ๋ถ€๋ถ„๋“ค(ex. istio๋Š” ๋ฌด์—‡์ด๋ฉฐ, ์™œ ์‚ฌ์šฉํ•˜๋Š”๊ฐ€?)์€ ์ œ ์ˆ˜์ค€์„ ๋ฒ—์–ด๋‚˜๋Š” ์ฃผ์ œ๊ฐ€ ๋˜์–ด์„œ ๋ถ€๋“์ด ์ƒ๋žตํ•˜๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ๐Ÿ™‡

ํŠธ๋ž˜ํ”ฝ ํ๋ฆ„ ์ดํ•ดํ•˜๊ธฐ

์ด๋ฏธ์ง€ ์ถœ์ฒ˜ : aws-eks-istio-lab

์œ„ ์ด๋ฏธ์ง€๋Š” EKS ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด์—์„œ ํŠธ๋ž˜ํ”ฝ์ด ์–ด๋–ป๊ฒŒ ํ๋ฅด๋Š” ์ง€๋ฅผ ๋ณด์—ฌ์ค€๋‹ค.

  1. ์œ ์ €๊ฐ€ ๋„๋ฉ”์ธ์— ์ ‘๊ทผํ•œ๋‹ค.
  2. AWS DNS ์„œ๋น„์Šค์ธ Route53์ด ๋„๋ฉ”์ธ์„ ๋กœ๋“œ๋ฐธ๋Ÿฐ์„œ์ธ ALB๋กœ ๋ผ์šฐํŒ…ํ•œ๋‹ค.
  3. ALB๋Š” ํŠธ๋ž˜ํ”ฝ์„ EKS ํด๋Ÿฌ์Šคํ„ฐ์˜ istio-ingressgateway๋กœ ์ „๋‹ฌํ•œ๋‹ค.
  4. istio-ingressgateway๋Š” ํŠธ๋ž˜ํ”ฝ์„ Gateway๋กœ ์ „๋‹ฌํ•œ๋‹ค.
  5. Gateway๋Š” ํŠธ๋ž˜ํ”ฝ์„ VirtualService๋กœ ์ „๋‹ฌํ•œ๋‹ค.
  6. VirtualService๋Š” ํŠธ๋ž˜ํ”ฝ์„ Service๋กœ ์ „๋‹ฌํ•œ๋‹ค.
  7. Service๋Š” ํ•ด๋‹น ํŠธ๋ž˜ํ”ฝ์„ Pod๋กœ ์ „๋‹ฌํ•œ๋‹ค.
  8. Pod๋Š” ์ตœ์ข…์ ์œผ๋กœ ์‚ฌ์šฉ์ž์—๊ฒŒ ํ•ด๋‹น ํŽ˜์ด์ง€๋ฅผ ์‘๋‹ตํ•œ๋‹ค.

์œ„์™€ ๊ฐ™์€ ํ๋ฆ„์œผ๋กœ ํŠธ๋ž˜ํ”ฝ์ด ์ฒ˜๋ฆฌ๋˜๋„๋ก ํ•˜๊ธฐ ์œ„ํ•ด, ๋‚˜๋Š” ๋‹ค์Œ ์ˆœ์„œ๋กœ ์ž‘์—…์„ ์ง„ํ–‰ํ•  ๊ฒƒ์ด๋‹ค.

  1. ALB ์„ค์น˜
  2. Istio ์„ค์น˜
  3. ExternalDNS ๋ฐฐํฌ
  4. Ingress ๋ฐฐํฌ
  5. Gateway, VirtualService ๋ฐฐํฌ

1๏ธโƒฃ ALB ์„ค์น˜

ALB๋Š” AWS์—์„œ ์ œ๊ณตํ•˜๋Š” ๋กœ๋“œ๋ฐธ๋Ÿฐ์„œ ์„œ๋น„์Šค๋กœ, ํด๋Ÿฌ์Šคํ„ฐ ์™ธ๋ถ€์—์„œ ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด๋ถ€๋กœ ํŠธ๋ž˜ํ”ฝ์„ ์ „๋‹ฌํ•˜๋Š” ์—ญํ• ์„ ํ•œ๋‹ค.

์„ค์น˜ํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ์—ฌ๋Ÿฌ๊ฐ€์ง€๊ฐ€ ์žˆ๊ฒ ์ง€๋งŒ, AWS์—์„œ ์ดˆ์‹ฌ์ž๋Š” helm์„ ์‚ฌ์šฉํ•ด์„œ ์„ค์น˜ํ•˜๋Š” ๊ฒƒ์„ ๊ถŒ์žฅํ•˜๋ฏ€๋กœ, ์ด๋ฅผ ๋”ฐ๋ผํ•˜๊ธฐ๋กœ ํ–ˆ๋‹ค.

๐Ÿค” helm์ด๋ž€?



Node.js์—๋Š” NPM์ด ์žˆ๋“ฏ์ด, Kubernetes์—๋Š” Helm์ด๋ผ๋Š” ํŒจํ‚ค์ง€ ๋งค๋‹ˆ์ €๊ฐ€ ์žˆ๋‹ค. ๋”ฐ๋ผ์„œ NPM์˜ npm install ๋ช…๋ น์–ด์™€ ๋น„์Šทํ•˜๊ฒŒ helm install ๋ช…๋ น์–ด๋กœ ํด๋Ÿฌ์Šคํ„ฐ์— ํ•„์š”ํ•œ ์˜์กด์„ฑ์„ ์‰ฝ๊ฒŒ ์„ค์น˜ํ•  ์ˆ˜ ์žˆ๋‹ค.


macOS์—์„œ๋Š” brew install helm์œผ๋กœ ๊ฐ„๋‹จํžˆ ์„ค์น˜ํ•  ์ˆ˜ ์žˆ๋‹ค.

1. Service Account์— ๋Œ€ํ•œ IAM ์—ญํ•  ์ƒ์„ฑ

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.2/docs/install/iam_policy.json

์œ„ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•ด์„œ IAM ์ •์ฑ… ํŒŒ์ผ์ธ iam_policy.json์„ ํ˜„์žฌ ์ž‘์—… ๋””๋ ‰ํ† ๋ฆฌ๋กœ ๋‹ค์šด๋กœ๋“œํ•œ๋‹ค.

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \ # 1๏ธโƒฃ ์ •์ฑ… ์ด๋ฆ„
    --policy-document file://iam_policy.json           # 2๏ธโƒฃ ์ •์ฑ… ํŒŒ์ผ

์ดํ›„ ์œ„ aws ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ IAM ์ •์ฑ…์„ ์ƒ์„ฑํ•˜์ž.

์ •์ƒ์ ์œผ๋กœ ์ƒ์„ฑ๋œ IAM ์ •์ฑ… AWSLoadBalancerControllerIAMPolicy

IAM ์ •์ฑ…์„ ์ƒ์„ฑํ–ˆ๋‹ค๋ฉด ์ด์ œ eksctl ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•ด์„œ IAM ์—ญํ• ์„ ์ƒ์„ฑํ•œ๋‹ค.

eksctl create iamserviceaccount \
  --cluster=greenhead-cluster \ # ํด๋Ÿฌ์Šคํ„ฐ ์ด๋ฆ„
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::971490215356:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve # ๋ช…๋ น์–ด ์‹คํ–‰ ํ›„ ๋ณ„๋„ ์Šน์ธ๊ณผ์ • ์—†์ด ๋ฐ”๋กœ ์„œ๋น„์Šค ๊ณ„์ • ์‚ฌ์šฉ

โ€”attach-policy-arn์—์„œ 971490215356์€ ๋‚ด AWS ๊ณ„์ • ID๋ฅผ ์˜๋ฏธํ•œ๋‹ค.

2. AWS Load Balancer Controller ์„ค์น˜

# 1๏ธโƒฃ eks-charts ๋ ˆํฌ์ง€ํ† ๋ฆฌ๋ฅผ ๋กœ์ปฌ์— ์ถ”๊ฐ€
$ helm repo add eks https://aws.github.io/eks-charts
# npm๊ณผ ๋‹ฌ๋ฆฌ, helm์—์„œ๋Š” ๋ฆฌ์†Œ์Šค๋ฅผ ์„ค์น˜ํ•˜๋ ค๋ฉด ์šฐ์„  ์„ค์น˜ํ•˜๋ ค๋Š” ๋ ˆํฌ์ง€ํ† ๋ฆฌ๋ฅผ ๋กœ์ปฌ์— ๋“ฑ๋กํ•ด์ค˜์•ผ ํ•œ๋‹ค.

# 2๏ธโƒฃ ๋“ฑ๋กํ•œ ๋กœ์ปฌ ๋ ˆํฌ์ง€ํ† ๋ฆฌ ์ตœ์‹ ํ™”
$ helm repo update eks

# 3๏ธโƒฃ helm์œผ๋กœ ALB ์„ค์น˜
$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \ # ๋„ค์ž„์ŠคํŽ˜์ด์Šค ๋ช…์‹œ
  --set clusterName=greenhead-cluster \
  --set serviceAccount.create=false \ # ์ „ ๋‹จ๊ณ„์—์„œ ์„œ๋น„์Šค ๊ณ„์ •์„ ์ด๋ฏธ ์ƒ์„ฑํ•˜์˜€์œผ๋ฏ€๋กœ ์ƒ๋žต
  --set serviceAccount.name=aws-load-balancer-controller

์œ„ ๋ช…๋ น์–ด๋“ค์ด ์ „๋ถ€ ์ •์ƒ์ ์œผ๋กœ ์‹คํ–‰๋˜์—ˆ๋‹ค๋ฉด, ์•„๋ž˜ ๋ช…๋ น์–ด๋ฅผ ํ†ตํ•ด ์ปจํŠธ๋กค๋Ÿฌ๊ฐ€ ์„ค์น˜๋˜์—ˆ๋Š” ์ง€ ํ™•์ธํ•˜์ž.

$ kubectl get deployment -n kube-system aws-load-balancer-controller

# ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅ๋˜๋ฉด ์ •์ƒ์ ์œผ๋กœ ์„ค์น˜๋œ ๊ฒƒ์ด๋‹ค.
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           47d

2๏ธโƒฃ Istio ์„ค์น˜

Google Cloud์—์„œ ์„ค๋ช…ํ•˜๋Š” Istio์˜ ์ •์˜๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

โ€œIstio๋Š” ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋„คํŠธ์›Œํฌ ๊ธฐ๋Šฅ์„ ์œ ์—ฐํ•˜๊ณ  ์‰ฝ๊ฒŒ ์ž๋™ํ™”ํ•  ์ˆ˜ ์žˆ๋Š” ํˆฌ๋ช…ํ•œ ์–ธ์–ด ๋…๋ฆฝ์  ๋ฐฉ๋ฒ•์„ ์ œ๊ณตํ•˜๋Š” ํ˜„๋Œ€ํ™”๋œ ์„œ๋น„์Šค ๋„คํŠธ์›Œํ‚น ๋ ˆ์ด์–ด์ธ ์„œ๋น„์Šค ๋ฉ”์‹œ์ž…๋‹ˆ๋‹ค.โ€

ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž์ธ ๋‚˜๋กœ์„œ๋Š” ๋„๋ฌด์ง€ ํ•ด๋‹น ๋ฌธ์žฅ์„ ์ดํ•ดํ•  ์ˆ˜ ์—†์—ˆ๋‹คโ€ฆ

Istio๊ฐ€ ๋ฌด์—‡์ธ์ง€ ์ •ํ™•ํžˆ ์ดํ•ดํ•˜๋ ค๋ฉด ์ ์ง€ ์•Š์€ ์‹œ๊ฐ„์„ ์จ์•ผ ํ•  ๊ฒƒ ๊ฐ™๋‹ค๊ณ  ๋А๊ผˆ๊ธฐ์—, ๋‚˜๋Š” ์•ž์˜ ํŠธ๋ž˜ํ”ฝ ํ๋ฆ„ ์ดํ•ดํ•˜๊ธฐ ์„น์…˜์—์„œ istio๊ฐ€ ์–ด๋–ป๊ฒŒ ์‚ฌ์šฉ๋˜์—ˆ๋Š” ์ง€๋ฅผ ์ฐธ๊ณ ํ•˜์—ฌ istio๋ฅผ โ€œํด๋Ÿฌ์Šคํ„ฐ ์™ธ๋ถ€์—์„œ ๋‚ด๋ถ€๋กœ ๋“ค์–ด์˜ค๋Š” ํŠธ๋ž˜ํ”ฝ์„ ๊ด€๋ฆฌํ•˜๋Š” ๋„๊ตฌโ€๋กœ ๊ฐ„๋‹จํžˆ ์ƒ๊ฐํ•˜๊ธฐ๋กœ ํ•˜์˜€๋‹ค.

๋” ์ •ํ™•ํ•œ ๋‚ด์šฉ์€ Istio ๊ณต์‹ ๋ฌธ์„œ์—์„œ ์„ค๋ช…ํ•˜๋Š” ๋‚ด์šฉ์„ ์ฐธ๊ณ ํ•˜์žโ€ฆ

ํด๋Ÿฌ์Šคํ„ฐ์— Istio๋ฅผ ์„ค์น˜ํ•˜๊ธฐ ์œ„ํ•ด ๋‚˜๋Š” CLI ๋„๊ตฌ์ธ istioctl`์„ ์‚ฌ์šฉํ•  ๊ฒƒ์ด๋‹ค. Homebrew๋กœ ๊ฐ„๋‹จํžˆ ์„ค์น˜ํ•  ์ˆ˜ ์žˆ๋‹ค.

๋‹ค์Œ istioctl ๋ช…๋ น์–ด๋ฅผ ์ž…๋ ฅํ•˜์—ฌ Istio ํด๋Ÿฌ์Šคํ„ฐ์— ์„ค์น˜ํ•œ๋‹ค.

istioctl install --set profile=default # istio-system ๋„ค์ž„์ŠคํŽ˜์ด์Šค์— ์„ค์น˜๋œ๋‹ค.
๐Ÿค” profile ์†์„ฑ์ด ์˜๋ฏธํ•˜๋Š” ๊ฒƒ์€?

profile ์†์„ฑ์„ ๋ช…์‹œํ•จ์œผ๋กœ์จ ํŠน์ • istio์˜ ๊ตฌ์„ฑ ์š”์†Œ๋ฅผ ํฌํ•จ์‹œํ‚ค๊ฑฐ๋‚˜ ์ œ์™ธํ•˜์—ฌ istio๋ฅผ ์„ค์น˜ํ•  ์ˆ˜ ์žˆ๋‹ค. istio์˜ ์ฃผ์š” ๊ตฌ์„ฑ ์š”์†Œ ์ค‘ ๊ฐ€์žฅ ์ค‘์š”ํ•œ 3๊ฐ€์ง€๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.


  • istio-egressgateway: ํด๋Ÿฌ์Šคํ„ฐ ์™ธ๋ถ€ -> ๋‚ด๋ถ€๋กœ ๋“ค์–ด์˜ค๋Š” ํŠธ๋ž˜ํ”ฝ ๊ด€๋ฆฌ
  • istio-ingressgateway: ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด๋ถ€ -> ์™ธ๋ถ€๋กœ ๋“ค์–ด์˜ค๋Š” ํŠธ๋ž˜ํ”ฝ ๊ด€๋ฆฌ
  • istiod: istio ํ•ต์‹ฌ ๊ตฌ์„ฑ์š”์†Œ

๋‚˜์˜ ํด๋Ÿฌ์Šคํ„ฐ๊ฐ€ ์ •์ƒ ๋™์ž‘ํ•˜๋ ค๋ฉด istio-ingressgateway, istiod๊ฐ€ ํ•„์š”ํ•˜๋ฏ€๋กœ profile์„ default๋กœ ํ•˜์—ฌ istio๋ฅผ ์„ค์น˜ํ•œ๋‹ค.

istioctl์˜ profile ๊ด€๋ จํ•˜์—ฌ ์ข€ ๋” ์ž์„ธํ•œ ๋‚ด์šฉ์€ ์—ฌ๊ธฐ ํฌ์ŠคํŠธ์—์„œ ํ•œ๊ตญ์ธ ๊ฐœ๋ฐœ์ž๋ถ„๊ป˜์„œ ์ดํ•ดํ•˜๊ธฐ ์‰ฝ๊ฒŒ ์ž‘์„ฑํ•ด ์ฃผ์…จ๋‹ค.

์ด๋•Œ ์„ค์น˜๋œ istio-ingressgateway์˜ Type์€ Loadbalancer์ธ๋ฐ, ์ด ๊ฒฝ์šฐ ์ž๋™์œผ๋กœ ์™ธ๋ถ€ ๋กœ๋“œ๋ฐธ๋Ÿฐ์„œ๋ฅผ ์ƒ์„ฑํ•˜๊ฒŒ ๋œ๋‹ค. ๋‚˜์˜ ๊ฒฝ์šฐ ์•ž์—์„œ ์„ค์น˜ํ•œ ALB๊ฐ€ ๊ด€๋ฆฌํ•˜๋Š” ๋กœ๋“œ๋ฐธ๋Ÿฐ์„œ๋ฅผ ์‚ฌ์šฉํ•  ๊ฒƒ์ด๋ฏ€๋กœ, istio-ingressgateway์˜ Type์„ NodePort๋กœ ๋ณ€๊ฒฝํ•˜์—ฌ ๋ถˆํ•„์š”ํ•˜๊ฒŒ ๋กœ๋“œ๋ฐธ๋Ÿฐ์„œ๊ฐ€ ์ƒ์„ฑ๋˜์ง€ ์•Š๋„๋ก ํ•  ํ•„์š”๊ฐ€ ์žˆ๋‹ค.

kubectl edit -n istio-system service istio-ingressgateway

์œ„ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜๋ฉด istio-ingressgateway์˜ ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์„ ์“ฐ๊ธฐ ๋ชจ๋“œ๋กœ ์—ด ์ˆ˜ ์žˆ๋‹ค. ์•„๋ž˜์™€ ๊ฐ™์ด ์ˆ˜์ •ํ•˜์ž.

spec:
  ...
  type: NodePort # Loadbalancer๋กœ ๋˜์–ด ์žˆ์„ ํ…๋ฐ, NodePort๋กœ ์ˆ˜์ •ํ•˜๊ธฐ
  ...

3๏ธโƒฃ ExternalDNS ๋ฐฐํฌ

ExternalDNS๋Š” ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด์—์„œ DNS ๋ ˆ์ฝ”๋“œ๋ฅผ ์ž๋™์œผ๋กœ ์ƒ์„ฑํ•˜๊ณ  ์—…๋ฐ์ดํŠธํ•˜๋Š” ๋„๊ตฌ์ด๋‹ค. ์ด๋ฅผ ํ†ตํ•ด ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋ฆฌ์†Œ์Šค์ธ Ingress ํ˜น์€ Service์˜ IP ์ฃผ์†Œ ๋˜๋Š” LoadBalancer ์ฃผ์†Œ๋ฅผ Route 53 ๋“ฑ์˜ DNS ์ œ๊ณต์ž์— ์ž๋™์œผ๋กœ ๋“ฑ๋กํ•˜์—ฌ, ๋„๋ฉ”์ธ ์ด๋ฆ„์„ ํ†ตํ•ด ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด ์„œ๋น„์Šค์— ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•œ๋‹ค.

์„ธ๋ถ€์ ์ธ ๋‹จ๊ณ„๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์œผ๋ฉฐ, ExternalDNS์˜ ๊ณต์‹ ํŠœํ† ๋ฆฌ์–ผ์„ ์ฐธ๊ณ ํ•˜์—ฌ ์ง„ํ–‰ํ•˜์˜€๋‹ค.

  1. IAM ์ •์ฑ… ์ƒ์„ฑ
  2. IAM ์—ญํ•  ์ƒ์„ฑ
  3. ExternalDNS ๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ ๋ฐ ๋ฐฐํฌ
โš ๏ธ ๋„๋ฉ”์ธ ๊ด€๋ จ๋œ ์ž‘์—…์€ ์ „๋ถ€ ์™„๋ฃŒ๋œ ์ƒํƒœ๋ผ ๊ฐ€์ •ํ•œ๋‹ค.

๋‚˜์˜ ๊ฒฝ์šฐ greenhead.blog ๋„๋ฉ”์ธ์ด ์ด๋ฏธ Route 53์— ๋“ฑ๋ก๋˜์–ด ์žˆ๊ณ , HTTPS ํ†ต์‹ ์„ ์œ„ํ•œ SSL ์ธ์ฆ์„œ๋ฅผ ACM์„ ํ†ตํ•ด ์ƒ์„ฑ๋œ ์ƒํƒœ์ด๋‹ค.

1. IAM ์ •์ฑ… ์ƒ์„ฑ

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "route53:ChangeResourceRecordSets"
      ],
      "Resource": [
        "arn:aws:route53:::hostedzone/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ListHostedZones",
        "route53:ListResourceRecordSets",
        "route53:ListTagsForResource"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

์œ„ JSON ์ฝ”๋“œ๋ฅผ policy.json์ด๋ผ๋Š” ์ด๋ฆ„์œผ๋กœ ์ €์žฅํ•œ๋‹ค. (์ด๋ฆ„์€ ์ƒ๊ด€์—†๋‹ค)

aws iam create-policy --policy-name "AllowExternalDNSUpdates" --policy-document file://policy.json

์ดํ›„ ์œ„ aws ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ AllowExternalDNSUpdates๋ผ๋Š” ์ด๋ฆ„์˜ IAM ์ •์ฑ…์„ ์ƒ์„ฑํ•˜๋ฉด ๋œ๋‹ค.

2. IAM ์—ญํ•  ์ƒ์„ฑ

eksctl ๋ช…๋ น์–ด๋ฅผ ํ†ตํ•ด IAM ์—ญํ• ์„ ์ƒ์„ฑํ•œ๋‹ค.

eksctl create iamserviceaccount \
  --name external-dns-green-blog-co \
  --namespace external-dns \
  --cluster greenhead-cluster \
  --attach-policy-arn arn:aws:iam::971490215356:policy/AllowExternalDNSUpdates \ # ์ „ ๋‹จ๊ณ„์—์„œ ์ƒ์„ฑํ•œ IAM ์ •์ฑ…์— ๋Œ€ํ•œ ARN ์ž…๋ ฅ
  --approve

3. ExternalDNS ๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ ๋ฐ ๋ฐฐํฌ

์ด์ œ ExternalDNS ๊ณต์‹ ๋ฌธ์„œ์˜ โ€œManifest (for cluster with RABC enalbed)โ€์— ์ž‘์„ฑ๋œ ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์„ ๊ธฐ์ค€์œผ๋กœ ExternalDNS ๊ตฌ์„ฑ์„ ์ž‘์„ฑํ•˜์—ฌ ํด๋Ÿฌ์Šคํ„ฐ์— ๋ฐฐํฌํ•˜๋ฉด ๋œ๋‹ค.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: external-dns-green-blog-co
rules:
  - apiGroups: [""]
    resources: ["services","endpoints","pods"]
    verbs: ["get","watch","list"]
  - apiGroups: ["extensions","networking.k8s.io"]
    resources: ["ingresses"]
    verbs: ["get","watch","list"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["list"]
  - apiGroups: ["networking.istio.io"]
    resources: ["gateways", "virtualservices"]
    verbs: ["get","watch","list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: external-dns-green-blog-co
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns-green-blog-co
subjects:
  - kind: ServiceAccount
    name: external-dns-green-blog-co
    namespace: external-dns
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns-green-blog-co
  namespace: external-dns
spec:
  selector:
    matchLabels:
      app: external-dns-green-blog-co
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: external-dns-green-blog-co
    spec:
      serviceAccountName: external-dns-green-blog-co
      containers:
        - name: external-dns-green-blog-co
          image: registry.k8s.io/external-dns/external-dns:v0.14.0
          args:
            - --source=service
            - --source=ingress
            - --source=istio-gateway
            - --source=istio-virtualservice
            - --domain-filter=green-blog.co # ๋„๋ฉ”์ธ ์ด๋ฆ„
            - --provider=aws
            - --policy=upsert-only
            - --aws-zone-type=public
            - --registry=txt
            - --txt-owner-id=Z06833101WGBT1HIY3N6H # ํ˜ธ์ŠคํŒ… ์˜์—ญ ID
๐Ÿ“ ๊ฐ€์žฅ ์•„๋ž˜ --txt-owner-id๋Š” Route 53์˜ ํ˜ธ์ŠคํŒ… ์˜์—ญ ID ๊ฐ’์„ ์‚ฌ์šฉํ•œ๋‹ค.

์ž‘์„ฑํ•œ yaml ํŒŒ์ผ์„ external-dns.yaml ์ด๋ผ๋Š” ์ด๋ฆ„์œผ๋กœ ์ €์žฅํ•˜๊ณ , kubectl ๋ช…๋ น์–ด๋ฅผ ํ†ตํ•ด ํด๋Ÿฌ์Šคํ„ฐ์— ๋ฐฐํฌํ•˜๋ฉด ๋œ๋‹ค.

kubectl apply -f external-dns.yaml

4๏ธโƒฃ Ingress ๋ฐฐํฌ

Ingress๋Š” ์ด์ „ ํฌ์ŠคํŠธ์—์„œ ๋‹ค๋ฃฌ Deployment, Service์™€ ๊ฐ™์€ ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋นŒํŠธ์ธ ๋ฆฌ์†Œ์Šค ์ค‘ ํ•˜๋‚˜๋กœ, ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ์™ธ๋ถ€์—์„œ ๋‚ด๋ถ€ ์„œ๋น„์Šค๋กœ์˜ HTTP(S) ํŠธ๋ž˜ํ”ฝ์„ ๋ผ์šฐํŒ…ํ•˜๋Š” ์—ญํ• ์„ ํ•œ๋‹ค.

๋‹ค๋งŒ Ingress๋Š” ํŠธ๋ž˜ํ”ฝ์„ ์–ด๋–ป๊ฒŒ ๋ผ์šฐํŒ…ํ•  ์ง€์— ๋Œ€ํ•œ ๊ทœ์น™์„ ์ •์˜ํ•˜๊ธฐ๋งŒ ํ•  ๋ฟ, ์‹ค์ œ๋กœ ํŠธ๋ž˜ํ”ฝ์„ ์ฒ˜๋ฆฌํ•˜๋ ค๋ฉด Ingress Controller๋ผ๋Š” ๊ฒƒ์ด ํ•„์š”ํ•˜๋‹ค. Nginx, Traefik ๋“ฑ ๋‹ค์–‘ํ•œ Ingress Controller๊ฐ€ ์กด์žฌํ•œ๋‹ค๊ณ  ํ•˜๋ฉฐ, ์ „ ๋‹จ๊ณ„์—์„œ ์„ค์น˜ํ•œ istio-ingressgateway ๋˜ํ•œ Ingress Controller์ด๋‹ค.

์ด์ œ Ingress ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์„ ์ž‘์„ฑํ•ด๋ณด์ž.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: istio-ingress-green-blog-co
  namespace: istio-system # istiod, istio-ingressgateway๊ฐ€ ์„ค์น˜๋œ ๋„ค์ž„์ŠคํŽ˜์ด์Šค๋กœ ๋งž์ถ”๊ธฐ
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing # ALB๋ฅผ ์™ธ๋ถ€์— ๋…ธ์ถœ์‹œ์ผœ ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋„๋ก ์„ค์ •
    alb.ingress.kubernetes.io/certificate-arn: "[green-blog.co์— ๋Œ€ํ•œ SSL ์ธ์ฆ์„œ์˜ ARN์„ ์—ฌ๊ธฐ์— ์ž…๋ ฅ]"
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    external-dns.alpha.kubernetes.io/hostname: "green-blog.co" # ๋„๋ฉ”์ธ ์ด๋ฆ„
spec:
  ingressClassName: alb
  rules:
    - http: # http๋Š” https๋กœ ๋ฆฌ๋‹ค์ด๋ ‰์…˜ ๋˜๋ฏ€๋กœ, http ํ•„๋“œ๋งŒ ์ž‘์„ฑํ•œ๋‹ค.
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ssl-redirect
                port:
                  name: use-annotation
          - path: /
            pathType: Prefix
            backend:
              service:
                name: istio-ingressgateway
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: istio-ingressgateway
                port:
                  number: 443

์œ„ Ingress ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๊ทœ์น™์„ ์ •์˜ํ•˜๊ณ  ์žˆ๋‹ค.

  • ALB๋ฅผ ์ธํ„ฐ๋„ท์— ๋…ธ์ถœ์‹œ์ผœ ์™ธ๋ถ€์—์„œ ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋„๋ก ํ—ˆ์šฉ
  • green-blog.co ๋„๋ฉ”์ธ์— ๋Œ€ํ•œ SSL ์ธ์ฆ์„œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ HTTPS๋ฅผ ์„ค์ •
  • HTTP๋ฅผ HTTPS๋กœ ๋ฆฌ๋‹ค์ด๋ ‰ํŠธ
  • green-blog.co ๋„๋ฉ”์ธ์— ๋Œ€ํ•œ ExternalDNS ์„ค์ • ์ •์˜
  • ๊ฒฝ๋กœ์— ๋”ฐ๋ผ ์š”์ฒญ์„ istio-ingressgateway ์„œ๋น„์Šค๋กœ ๋ผ์šฐํŒ… (์ดํ›„ ๋‹ค๋ฃฐ Gateway ์„ค์ •๊ณผ ๋ฐ€์ ‘ํ•œ ๊ด€๋ จ)

์ด ์„ค์ •์„ ํ†ตํ•ด ์™ธ๋ถ€์—์„œ green-blog.co ๋„๋ฉ”์ธ์œผ๋กœ ๋“ค์–ด์˜ค๋Š” ์š”์ฒญ์ด ALB๋ฅผ ํ†ตํ•ด Istio Ingress Gateway๋กœ ์ „๋‹ฌ๋˜๋ฉฐ, HTTP ์š”์ฒญ์€ HTTPS๋กœ ๋ฆฌ๋‹ค์ด๋ ‰ํŠธ๋œ๋‹ค.

5๏ธโƒฃ Gateway, VirtualService ๋ฐฐํฌ

Gateway์™€ VirtualService๋Š” Deployment, Service์™€ ๊ฐ™์€ ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋นŒํŠธ์ธ ๋ฆฌ์†Œ์Šค๊ฐ€ ์•„๋‹ˆ๋ผ Istio์—์„œ ์ œ๊ณตํ•˜๋Š” ์จ๋“œํŒŒํ‹ฐ ๋ฆฌ์†Œ์Šค์ด๋‹ค.

๊ฐ๊ฐ์€ ๋‹ค์Œ์˜ ์—ญํ• ์„ ์ˆ˜ํ–‰ํ•œ๋‹ค.

  • Gateway: ์™ธ๋ถ€ ํŠธ๋ž˜ํ”ฝ์„ ์ „๋‹ฌ๋ฐ›์•„ ์ด๋ฅผ VirtualService๋กœ ์ „๋‹ฌ
  • VirtualService: Gateway๋กœ๋ถ€ํ„ฐ ์ „๋‹ฌ๋ฐ›์€ ํŠธ๋ž˜ํ”ฝ์„ ๋‚ด๋ถ€ Service๋กœ ์ „๋‹ฌ

๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ(๋ฐฐํฌ) ์ˆœ์„œ๋Š” Gateway โžก๏ธ VirtualService ์ˆœ์ด๋‹ค.

1. Gateway ๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ ๋ฐ ๋ฐฐํฌ

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: green-blog-co-gateway
  namespace: green-blog-co
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "green-blog.co"

์œ„์™€ ๊ฐ™์ด ์ž‘์„ฑ ํ›„ ํด๋Ÿฌ์Šคํ„ฐ์— Gateway๋ฅผ ๋ฐฐํฌํ•˜๋ฉด ๋กœ๋“œ ๋ฐธ๋Ÿฐ์„œ๊ฐ€ ์ƒ์„ฑ๋œ๋‹ค.

์ƒ์„ฑ๋œ ๋กœ๋“œ ๋ฐธ๋Ÿฐ์„œ์˜ DNS ์ด๋ฆ„์€ ์ดํ›„ VirtualService๋ฅผ ์ž‘์„ฑํ•  ๋•Œ ํ•„์š”ํ•˜๋‹ค.

2. VirtualService ๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ ๋ฐ ๋ฐฐํฌ

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: green-blog-co-virtual-service
  namespace: green-blog-co
  annotations: # ์ „ ๋‹จ๊ณ„์—์„œ ์ƒ์„ฑ๋œ ๋กœ๋“œ ๋ฐธ๋Ÿฐ์„œ์˜ DNS ์ด๋ฆ„์„ ์•„๋ž˜ ํ•„๋“œ์— ์ž‘์„ฑํ•œ๋‹ค.
    external-dns.alpha.kubernetes.io/target: "k8s-istiosys-istioing-5a5c0fc93a-784634683.ap-northeast-2.elb.amazonaws.com"
spec:
  hosts:
    - "green-blog.co"
  gateways:
    - green-blog-co-gateway
  http:
    - match:
      - uri:
          prefix: /
      route:
        - destination:
            host: green-blog-co-service
            port:
               number: 3000 # green-blog-co-service์˜ ์—ด๋ ค์žˆ๋Š” ํฌํŠธ ๋ฒˆํ˜ธ

๋!

์ด๋ฅผ ํ†ตํ•ด ์›น ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์„ Github Actions + ECR + EKS๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐฐํฌํ•œ ๋‚˜์˜ ๊ฒฝํ—˜์„ ์ „๋ถ€ ์ •๋ฆฌํ•˜์˜€๋‹ค.

ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž๊ฐ€ ์‹ค๋ฌด์—์„œ ์‹ค์ œ๋กœ ์ธํ”„๋ผ ์„ค์ •์„ ๋‹ค๋ฃจ๊ฒŒ ๋˜๋Š” ๊ฒฝ์šฐ๋„ ๋“œ๋ฌผ ํ…Œ์ง€๋งŒ, ์ด ๊ฒฝํ—˜์„ ํ†ตํ•ด์„œ ์‚ฌ๋‚ด ์›น ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์ด ์–ด๋–ป๊ฒŒ ๋ฐฐํฌ๋˜๋Š” ์ง€์— ๋Œ€ํ•œ ์ „์ฒด์ ์ธ ํ๋ฆ„์„ ๊นŠ๊ฒŒ ์ดํ•ดํ•  ์ˆ˜ ์žˆ์—ˆ๊ธฐ ๋•Œ๋ฌธ์— ๋ฐฐ์šฐ๋ฉด์„œ ์ •๋ง ์ฆ๊ฑฐ์› ๋‹ค.

์‚ฌ์‹ค ์—ฌ๊ธฐ์„œ ๋๋‚ด๊ธฐ์—” ์•ฝ๊ฐ„ ์•„์‰ฌ์šด ๋ถ€๋ถ„์€ ์žˆ๋‹ค. ์ด ์ •๋„๋งŒ์œผ๋กœ๋Š” ์™„์ „ํ•œ ๋ฐฐํฌ ์ž๋™ํ™”๊ฐ€ ๋˜์ง€ ์•Š๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค. ์ง„์ •ํ•œ Push To Deploy๋ฅผ ๊ตฌํ˜„ํ•˜๊ธฐ ์œ„ํ•ด์„œ๋Š” ๋งค๋‹ˆํŽ˜์ŠคํŠธ ํŒŒ์ผ์˜ ๋ณ€๊ฒฝ์„ ๊ฐ์ง€ํ•˜์—ฌ ์ด๋ฅผ ํด๋Ÿฌ์Šคํ„ฐ์— ์ ์šฉํ•˜๋Š” ArgoCD๋ผ๋Š” ๋„๊ตฌ๋„ ๋‹ค๋ค„์•ผ ํ•˜๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค.

ArgoCD๋ฅผ ๋‹ค๋ค„๋ณธ ๊ฒฝํ—˜์€ ์–ธ์  ๊ฐ€ ์—ฌ์œ ๊ฐ€ ๋˜๋ฉด ์ถ”๊ฐ€๋กœ ์ž‘์„ฑํ•ด๋ณด๊ธฐ๋กœ ํ•˜๊ฒ ๋‹ค.. ๐Ÿซก


Greenhead
Written by@Greenhead
ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž ์œค๋…น๋‘์ž…๋‹ˆ๋‹ค :)

GitHub